its been 158 mins

Hey everyone!

I've been learning Python and I'm really interested in getting into machine learning and deep learning. The problem is that whenever I try to figure out how to get started, I end up finding a huge amount of mathematics, statistics, linear algebra, calculus, and theory that I'm apparently supposed to learn first.

I understand that math is important if you want to properly understand how these models work, and I'm not trying to avoid learning it forever. However, I don't want to spend months studying theory before I can actually start building things and experimenting with code.

I'd much rather learn by doing projects, making mistakes, seeing what happens when I change something, and gradually learning the theory behind it as I need it.

Is it realistic to start building ML projects with Python while learning the math along the way?

How much math do I actually need to know before getting started with things like neural networks and deep learning?

Would you recommend starting with scikit-learn and traditional ML, or jumping straight into PyTorch or TensorFlow?

Are there any resources that teach you by making you build actual projects instead of spending most of the time on theory?

I'm not looking for a way to completely skip the fundamentals. I just want to avoid getting stuck in tutorial hell or spending so much time preparing to learn that I never actually get to build anything.

I'd really appreciate any advice, especially from people who learned this way themselves. Thanks!

I went into this thinking: GPT-2 is only 124M parameters. How much memory could training it possibly need?

Turns out, quite a lot lol.

I traced the second training step of my from-scratch GPT-2 in PyTorch. Setup: one A40, fp32 AdamW, batch 8, 1,024 tokens per sequence.

Peak allocated memory: 17,054 MiB. Weights + Adam + masks and persistent overhead at that point: about 1,494 MiB. I was counting what the model owns and missing almost everything training makes.

The bit that surprised me most: four vocab-sized tensors take up 6,282 MiB at the peak. My block hooks only saw 13,913 MiB; the allocator trace caught the extra spike inside log-softmax backward.

I also tested batches in fresh processes. 20 worked, 21 didn't. At batch 20, peak allocated was 40,375 MiB, while peak reserved was 44,948 MiB. So just dividing the card's capacity by the tensor budget wasn't enough.

I put the whole thing into my new blog post, with the maths, a stacked memory chart, an interactive step-by-step scrubber, and a batch/context calculator. Code and raw CSV are there too. The calculator is an estimate, not a guarantee that a batch fits.

Post: https://angry-kratos.github.io/posts/2026-10-training-memory/ Code: https://github.com/angry-kratos/angry-kratos.github.io/tree/main/code/training-memory

I'm learning the maths behind ML systems by building and measuring them. If you spot something off in the accounting, tell me. That's the whole point of putting the experiments out there.

AI explanations lose most people at the vocabulary, not the ideas. Here are five terms, translated:

• **Prompt** — what you type in. That is it.
• **Model** — the program doing the answering, trained on huge amounts of text.
• **Training data** — the text it learned from. It shapes what the model is good (and bad) at.
• **Hallucination** — when the model confidently invents a fact. Always verify names, dates, and numbers.
• **Token** — roughly, a chunk of a word. It is how usage gets measured, and why long conversations cost more.

If these five make sense, you already understand more than you think — the rest is practice, not theory.

Most of us start learning AI with real intent: a great course, the book everyone recommends. Then life interrupts. A hard week at work, a move, a busy month. Coming back means remembering where you were and finding the next step, and that friction is often enough to stall us. Meanwhile the field keeps moving, and every week away feels like falling further behind.

What changed it for me was rethinking what I studied, not just how often. Some ideas don't add to what you know; they rewire it. Education researchers Jan Meyer and Ray Land called these threshold concepts in 2003. Their classic example is opportunity cost: once it clicks, nothing is ever free again, and every hour and every dollar carries the price of what you gave up for it. If you code, you've felt this with recursion. One day it's a magic trick; the next, you see it in trees, parsers, and every divide-and-conquer problem. You can't unlearn a threshold concept, and it quietly reorganizes everything you learn after it.

That's the strategy, and it works for any field: find its handful of thresholds, learn them roughly in the order they were discovered, and go deep enough that they change how you see the rest. New facts stop piling up and start snapping onto a frame.

Here are eight thresholds that greatly help explain AI, roughly in their historical order, each with a surprising fact and a free resource:

1. Behavior learned from data, not programmed. In 1958, Rosenblatt's perceptron, a single artificial neuron, taught itself to tell left from right card marks after about 50 tries. 60 years too soon.
2. Representations and embeddings. In 2012, AlexNet learned its own features and hit 15% top-5 error on ImageNet, against 26% for the runner-up built on hand-picked ones. Jay Alammar's Illustrated Word2vec
3. Next-token prediction. A network trained only to predict the next character of Amazon reviews grew a neuron that tracked sentiment, and nobody had taught it that. The only task of the neural net was to predict tokens in a sequence of text, but in doing so it learned how to identify and represent underlying concepts. At the scale of internet data, the exact same task produced grammar, facts, and reasoning, the foundation of today’s models.  Karpathy's "Let's build GPT"
4. Attention. The 2017 transformer dropped word-by-word reading and reached top-tier translation after about 12 hours on one 8-GPU machine. This enabled the transformer, training in parallel on GPUs, and the speed up and scale that followed. Jay Alammar's Illustrated Transformer
5. Evals. How do you know a change actually made a general model better? The trap is old: Goodhart’s law warns that when a measure becomes a target, it stops being a good measure, and models are very good at hitting targets. Hugging Face's Evaluation Guidebook
6. Post-training and RL fine-tuning. People preferred a 1.3B InstructGPT model over the 175B GPT-3. How you train after pretraining can matter more than size. This enables ChatGPT-style assistants and reasoning models trained on answers that can be checked. Nathan Lambert's RLHF book
7. Harnesses and agents. On SWE-bench Lite, GPT-4 scored 2.7% with one harness and 28.3% with another. Same model, almost 10× better. This enables agents that plan, act, and retry, and it explains why the same model does better in one product than another. Anthropic's "Building effective agents"
8. Recursive self-improvement. DeepMind's AlphaEvolve sped up a key piece of Gemini's own training code by 23%. DeepMind's AlphaEvolve post

These eight are a small subset. There are of course many more threshold concepts in AI, and which ones matter next depends on what you’re exploring, at what granularity and where you’re starting from. The bigger point here is the approach and orientation.

💡 My biggest lightbulb moment came from working out by hand how a neural net could handle a fixed goal, like deciding whether a piece of text fits a topic. I started with a neural version of bag-of-words, one weight per word, then realized I could encode arbitrary rules in the parameters. After that, "learned from data instead of programmed" stopped feeling abstract.

The full write-up has more of the history behind each idea, the "spiral" approach to going deeper with each pass, and a study prompt that turns these into a plan with any LLM: Feynman's wobbling plate and 8 core ideas behind modern AI

Disclosure: I'm building Zairai, an AI coach built on these learning science principles, delivering bite-sized interactive sessions that feel like real work and compound strategically: you step into the shoes of the people who made these discoveries and measure for yourself, so the foundations that you're missing click and everything built on them makes more sense. It's free to start and launching soon. Everything above works without it.

What was the threshold concept that made AI click for you?

Title: How do you introduce yourself in an AI Engineer interview?

Hey everyone,

I feel like my self-introduction isn't that good during AI Engineer interviews.

I have around 3 years of IT experience, but I sometimes struggle to explain my experience properly.

If you're working as an AI Engineer, GenAI Engineer, or LLM Engineer, could you share how you introduce yourself in interviews?

Even a simple example would really help me improve mine.

Thanks in advance!

Hi everyone. I’m a hands-on AI engineering leader with a PhD in computer science and experience teaching in academia and mentoring working professionals. My work involves both leading teams and building AI systems—from architecture and implementation to evaluation and deployment.

Based on that experience, I’ve been developing Nextia Learning, a free platform covering the different skills involved in AI engineering.

The motivation is straightforward: AI engineering requires knowledge across several areas, and knowing what to learn—and in what order—can be difficult.

Understanding models matters. So does knowing how to prepare data, build APIs, work with databases, evaluate outputs, manage permissions, deploy applications, and investigate failures. With LLMs and agents, you also need to think about retrieval, tool access, unpredictable outputs, latency, and cost.

You don’t need to master everything before you start. But understanding how these pieces fit together helps you build better systems and identify where you need to develop your skills.

That’s what I wanted to reflect in this platform.

Courses covering different aspects of AI engineering

The curriculum brings together:

• Foundations: Python, Git, SQL, mathematics, and statistics.
• Models and applications: machine learning, LLMs, RAG, agents, and MCP.
• Software engineering: APIs, databases, authentication, permissions, and frontend considerations.
• Quality and safety: evaluation, testing AI applications, security, and privacy.
• Deployment and operations: Docker, cloud deployment, CI/CD, observability, scaling, and cost.
• Leadership and governance: planning AI products, leading teams, and making decisions about risk and accountability.

The selection reflects the range of problems and decisions I’ve encountered while building AI applications. My aim is to help learners develop skills across these areas and understand how to use them together.

Learning paths are a central part of it

Having a collection of courses doesn’t necessarily tell you where to begin.

A software developer moving into LLM applications has different needs from a new graduate learning machine learning, a domain expert applying AI, or someone leading an AI team.

The learning paths organize courses around those goals, with prerequisites and a suggested order. They help you see what to learn next, how it builds on what you already know, and where it fits into the work you want to do.

Lessons are text-first, written in simple English, and include worked examples, guided practice, common mistakes, and checks for understanding. There’s also an interview preparation section.

A few things to be transparent about

• All courses and certificates are free. You can read lessons without an account. Signing in lets you enroll and save your progress.
• The content is AI-assisted and human-reviewed. I design the outlines and instructional approach, guide the examples, and review the courses. This is disclosed on the site.
• Your data isn’t sold or used for marketing. Lesson reminders are optional, off by default, and you control their frequency.
• The platform is evolving. Some courses are in pilot or marked coming soon. Feedback and corrections are welcome.

You can explore the courses and learning paths here: https://learning.nextia-ai.com/

For those learning or working in AI engineering: which skills were missing from your initial learning but turned out to be important in your work? I’d also welcome feedback on whether the learning paths make sense for your background and goals.

Hi everyone, I'm an undergraduate Geophysics student, working on my final-year thesis. My work involves subsurface imaging, mostly with Python. I'm comfortable with programming, but I'm an absolute beginner when it comes to machine learning and deep learning.

I'd like to apply Implicit Neural Representations at one stage of my workflow, but I don't want to just copy code without understanding what's happening under the hood. So I'm looking to learn the fundamentals properly, from zero. Could you recommend: Books or courses for learning ML/deep learning fundamentals and a suggested learning path: what to study first, and what I can safely skip for now.

Thanks in advance

Hello, newb here looking for recommendations for a personal project where I would like to analyze the players' movements from the Mobile Legends Bang Bang (MLBB) Mini-map. From my limited research on Sports Game Analyses – I have learnt that the researchers mostly apply Computer Vision technologies, RNN, and CNN for conducting their analyses. However, contrary to their projects, videogame Mini-maps are simpler without much random events — which means replicating their techniques are a bit overkill.

P.s, a MLBB match typically lasts around 15 minutes and I need all player-movements analyzed for my project. Thanks! **Pic was taken from Vcgamers not mine.

I've been working on computer vision projects and one lesson I've learned is that a high overall metric doesn't always tell the full story.

A few things I now check during evaluation:

- Per-class precision and recall

- Confusion matrices and difficult examples

- Performance on validation data that wasn't used for training

- Whether improvements hold up on real-world images

I'm curious how others here approach error analysis. What's the first thing you inspect when a model's validation metrics look good but its real-world predictions are disappointing?

New learners often stall trying to get access to the biggest, newest model, as if that is what stands between them and understanding. In my experience the model choice barely matters for learning. A smaller, older, freely available model is more than good enough to teach you data handling, training loops, evaluation, and why your results are bad, which is where the real learning is. The bottleneck is never the model, it is your understanding of the problem and the data. Spend the energy you would waste chasing access on actually running experiments with whatever you have. Good enough and in your hands beats the latest and out of reach every time for a student.

Advice I keep getting is to build intuition for gradients before drowning in formulas, but nobody says what the intuition actually is. Right now a gradient is just a thing I compute. For those who have the intuitive grasp, how do you picture it? Is it the slope direction, a compass pointing uphill, something else? I am trying to get past treating it as a symbol I manipulate and actually feel what it represents. Concrete pictures welcome, I learn way better from those than from the notation.

Hi everyone,

I’m a self-taught software engineer with a couple of years of industry experience. I was recently made redundant, so I have a good chunk of free time on my hands before I dive heavily back into the job hunt (which, honestly, feels a bit daunting right now).

Lately, I’ve become really interested in Machine Learning and want to start learning it. However, I’ve been reading a lot of discouraging posts lately. It seems like even people with CS Bachelor's degrees are struggling to land MLE roles because they are competing against a massive pool of Masters and PhD graduates.

Given that I don't have an academic background or official credentials in ML, am I wasting my time trying to break into this field?

I’m absolutely willing to put in the work to learn, but I want to be realistic about my employment prospects. For those working in the industry: is there a viable path for an experienced, self-taught dev to pivot into ML/AI roles, or should I focus my energy elsewhere?

Thanks in advance for any insights or brutal honesty.

I started learning CUDA a while back to work on inference and understand FlashAttention. I read a lot of blogs on various topics, but the order was always confusing, so I wrote up the order that I'd pick if I started over.

What it covers:
- H100 architecture
- CUDA software mental model
- Getting started with Matrix Multiplication and building up to a very optimized solution
- Understanding how all of these add to optimization constraints
- An introduction to reading and understanding PTX(the compiler code)

It's meant to be a starter overview that will get you confident enough to read deeper, niche resources. I'd appreciate any feedback or corrections. Cheers!

Link to blog

Hi,

I am trying to transition from a Data Analyst to an AI Engineer role. I recently followed Andrej Karpathy’s youtube video to build a character level LLM.

I feel like i have a rough understanding of how llms work and now i want to move on to building projects that will help me understand other fundamentals.

I am thinking of building a murder mystery game where AI generates the scanerio, the characters as well as any evidence. The players are required to chat with the suspects and solve the case. The game will then give them feedback on their deduction abilities.

I plan to build more projects in the future but i want to start with something that is simplistic enough in terms of concept and scope.

Do you think this is a good project to learn the fundamentals of AI Engineering? Will it attract attention from Hiring Managers and Recruiters?

Any AI Engineers, if you have better project ideas please share!

Thanks you!

We wanted to understand how well a model will adhere to your fonts and colors and preserve your images when making edits.

Take a look at the complete writeup here: https://cocolabs.ai/blog/introducing-vybbench

Our HuggingFace dataset is here: https://huggingface.co/datasets/cocolabs/vybbench

It's a 50 hour course for beginners in AI/ML (obviously it's fully free). Starts out with linear algebra, derivatives, statistics and then linear regression, neural networks and backpropagation etc...

kinoml.tech

Researchers at Xint found that OpenAI's compaction feature can be used to copy private model reasoning into ordinary plain text output. Compaction shortens long conversations to manage context limits. The model's internal chain-of-thought, which the service is designed to keep hidden from callers, became readable through a routine operational function. Deployment-time guardrails did not catch it.

The structural problem here extends beyond this one feature. Any system function that touches conversation state or generates output is a potential path for data that was never meant to reach callers. Compaction is one example. Summarization, caching, and tool-call routing are others. Policies applied at configuration time govern expected flows. They do not automatically cover every execution path the runtime can take.

How are teams actually enforcing output policies across non-primary paths like these? Has anyone done a systematic audit of the secondary features that produce or transform output in their deployments — and found surprises?

First I felt that such large logic, implementation and intelligence just reduced to such small lines of code made learning such vast concepts meaning less since it's just reduced to like 3 lines of code

Later realised this is the beauty of learning ie learn how the backend works and implement it flawlessly through frontend

Am I the only one who feels this way though?

An exercise to help build the right mental model for Python data. - Solution - Explanation - More exercises

The “Solution” link uses 𝐦𝐞𝐦𝐨𝐫𝐲_𝐠𝐫𝐚𝐩𝐡 to visualize execution and reveals what’s actually happening.