1 00:00:00,080 --> 00:00:02,580 So welcome to the Deep Dive. Our mission today 2 00:00:02,580 --> 00:00:05,099 is to act as your shortcut to becoming well -informed 3 00:00:05,099 --> 00:00:07,820 about a truly groundbreaking paper. It's a highly 4 00:00:07,820 --> 00:00:10,800 technical paper from the ICML 2026 conference. 5 00:00:11,179 --> 00:00:13,419 Yeah, the paper is titled, Learning to Discover 6 00:00:13,419 --> 00:00:16,059 at Test Time, and it's authored by this incredible 7 00:00:16,059 --> 00:00:17,839 collaborative team of researchers. Oh, it's a 8 00:00:17,839 --> 00:00:20,719 massive team. Spanning Stanford, UC San Diego, 9 00:00:21,039 --> 00:00:23,940 Together AI, the Astera Institute, and NVIDIA. 10 00:00:24,179 --> 00:00:26,399 Right, including lead authors like Mert Uxagonal, 11 00:00:26,780 --> 00:00:29,420 Daniel Cochea, and Xinhua Li. I mean, it's a 12 00:00:29,420 --> 00:00:31,359 heavy -heading lineup. It really is. And the 13 00:00:31,359 --> 00:00:33,020 scope of what they have attempted to prove here 14 00:00:33,020 --> 00:00:36,020 is just incredibly ambitious. They are essentially 15 00:00:36,020 --> 00:00:38,280 asking if we can build a system that acts more 16 00:00:38,280 --> 00:00:42,060 like a real scientist in a lab, right? Yes. Adapting 17 00:00:42,060 --> 00:00:44,240 and learning from its immediate environment rather 18 00:00:44,240 --> 00:00:47,079 than just acting like a very smart encyclopedia. 19 00:00:47,539 --> 00:00:50,259 So here's the hook for you, the listener. How 20 00:00:50,259 --> 00:00:53,399 does an AI discover a completely new state -of 21 00:00:53,399 --> 00:00:55,750 -the -art solution to a scientific problem? It's 22 00:00:55,750 --> 00:00:58,009 a million -dollar question. Literally. To date, 23 00:00:58,090 --> 00:01:00,109 we have largely relied on taking what we call 24 00:01:00,109 --> 00:01:02,570 frozen language models and just prompting them 25 00:01:02,570 --> 00:01:04,829 over and over. But what if the model could actually 26 00:01:04,829 --> 00:01:07,769 continue to train while it takes the test? Yeah, 27 00:01:07,769 --> 00:01:09,590 what if it could learn specifically from its 28 00:01:09,590 --> 00:01:12,530 own failures and its own partial successes on 29 00:01:12,530 --> 00:01:15,870 that exact specific problem? In real time. To 30 00:01:15,870 --> 00:01:18,090 really understand how they pulled this off and 31 00:01:18,090 --> 00:01:20,989 why it matters, we are going to meticulously 32 00:01:20,989 --> 00:01:23,810 break down this entire paper. Chapter by chapter. 33 00:01:23,930 --> 00:01:26,409 Exactly. We're going to explore the foundational 34 00:01:26,409 --> 00:01:28,650 mathematical concepts they tackled. We'll look 35 00:01:28,650 --> 00:01:31,269 at some intense GPU engineering results too. 36 00:01:31,530 --> 00:01:33,950 And we'll dive into how this applies to biological 37 00:01:33,950 --> 00:01:37,109 benchmarks. But most importantly, we will unpack 38 00:01:37,109 --> 00:01:40,170 the exact mechanics of how this method actually 39 00:01:40,170 --> 00:01:42,209 works under the hood. Right. The method they 40 00:01:42,209 --> 00:01:45,709 call TTT discover or test time training to discover. 41 00:01:46,329 --> 00:01:48,409 So let's start with a foundational philosophy 42 00:01:48,409 --> 00:01:50,790 here. I want to use an analogy from the authors 43 00:01:50,790 --> 00:01:54,590 that I think perfectly sets the stage. Think 44 00:01:54,590 --> 00:01:56,950 back to tackling a really hard programming assignment 45 00:01:56,950 --> 00:02:00,590 in school. Oh, those are the worst. Yeah. If 46 00:02:00,590 --> 00:02:02,590 you just try guessing the solution based on the 47 00:02:02,590 --> 00:02:04,890 textbook, which represents your initial training 48 00:02:04,890 --> 00:02:07,489 data, you're probably going to fail. The problem 49 00:02:07,489 --> 00:02:10,810 is just too complex. Exactly. It requires multiple 50 00:02:10,810 --> 00:02:13,870 steps, unexpected bugs pop up, and the textbook 51 00:02:13,870 --> 00:02:16,490 just doesn't cover this specific edge case. So 52 00:02:16,490 --> 00:02:18,909 what do you do? You have to try. You have to 53 00:02:18,909 --> 00:02:20,819 fail. you have to look at the error messages. 54 00:02:21,180 --> 00:02:23,000 Right. You learn from that specific failure and 55 00:02:23,000 --> 00:02:25,419 you actually update your own internal understanding 56 00:02:25,419 --> 00:02:27,639 of the concepts right then and there. That iterative 57 00:02:27,639 --> 00:02:29,800 process is the core of human problem solving. 58 00:02:30,639 --> 00:02:32,680 But to understand why that represents a shift 59 00:02:32,680 --> 00:02:35,580 in AI, we need to clarify what a frozen model 60 00:02:35,580 --> 00:02:38,219 is. Let's define that for everyone. When an AI 61 00:02:38,219 --> 00:02:40,960 company spends tens of millions of dollars and 62 00:02:40,960 --> 00:02:43,879 months of computing time training a massive large 63 00:02:43,879 --> 00:02:46,639 language model, they eventually have to stop. 64 00:02:46,759 --> 00:02:49,629 They can't train it forever. Right. At that point, 65 00:02:50,030 --> 00:02:52,490 the internal parameters, the billions of weights 66 00:02:52,490 --> 00:02:54,990 and biases that define the network are locked 67 00:02:54,990 --> 00:02:57,569 in place. They remain fixed. So when you use 68 00:02:57,569 --> 00:02:59,590 a standard chatbot, you are interacting with 69 00:02:59,590 --> 00:03:02,610 a frozen model. Yes. Its brain is essentially 70 00:03:02,610 --> 00:03:05,270 frozen in time. It can't learn anything fundamentally 71 00:03:05,270 --> 00:03:07,469 new about the structure of reality once that 72 00:03:07,469 --> 00:03:09,629 training run ends, right? Well, I wouldn't go 73 00:03:09,629 --> 00:03:11,590 quite that far. And we have to be very careful 74 00:03:11,590 --> 00:03:13,870 with that framing. Oh, really? Yeah. It's a common 75 00:03:13,870 --> 00:03:16,849 misconception that a frozen model is stuck. and 76 00:03:16,849 --> 00:03:19,770 cannot produce anything novel, or that its next 77 00:03:19,770 --> 00:03:22,150 attempt just ignores previous results. But if 78 00:03:22,150 --> 00:03:25,270 the weights are frozen, how does it adapt? Through 79 00:03:25,270 --> 00:03:28,069 what we call in -context learning. OK, what does 80 00:03:28,069 --> 00:03:30,849 that mean in practice? By using large context 81 00:03:30,849 --> 00:03:33,990 windows and providing external tools like calculators 82 00:03:33,990 --> 00:03:36,710 or web search, giving the model feedback from 83 00:03:36,710 --> 00:03:40,270 a code compiler. Oh, so the prompt itself changes. 84 00:03:40,569 --> 00:03:43,590 Exactly. Frozen models can and absolutely do 85 00:03:43,590 --> 00:03:46,539 produce novel breakthrough solutions. Their internal 86 00:03:46,539 --> 00:03:49,979 weights remain fixed, but their prompt gets updated 87 00:03:49,979 --> 00:03:52,219 with the new context. So if it writes a piece 88 00:03:52,219 --> 00:03:54,500 of code and the code fails, the error message 89 00:03:54,500 --> 00:03:56,979 just goes right into the prompt? Right. The model 90 00:03:56,979 --> 00:03:59,240 reads the prompt, understands the error based 91 00:03:59,240 --> 00:04:01,780 on its preexisting knowledge, and generates a 92 00:04:01,780 --> 00:04:04,939 new attempt. Ah, I see. So it's not like a frozen 93 00:04:04,939 --> 00:04:07,439 model has zero probability of generating something 94 00:04:07,439 --> 00:04:09,659 it doesn't already have memorized. Not at all. 95 00:04:09,780 --> 00:04:12,120 It can synthesize new ideas within the prompt 96 00:04:12,120 --> 00:04:15,219 space. It's vital to clarify that a frozen model 97 00:04:15,219 --> 00:04:18,379 is highly capable of generating something new. 98 00:04:19,120 --> 00:04:22,480 So the TTT discover authors aren't saying frozen 99 00:04:22,480 --> 00:04:25,399 models are useless for discovery. Exactly. They 100 00:04:25,399 --> 00:04:27,439 are presenting experimental advantages of their 101 00:04:27,439 --> 00:04:30,500 new method, not establishing some mathematical 102 00:04:30,500 --> 00:04:33,199 impossibility theorem about frozen models. In 103 00:04:33,199 --> 00:04:35,540 fact, later on, we'll see the paper's own purely 104 00:04:35,540 --> 00:04:38,220 frozen baseline model achieve some remarkable 105 00:04:38,220 --> 00:04:41,000 stuff on its own. It did. That's a great distinction. 106 00:04:41,420 --> 00:04:43,639 The frozen model is like a genius locked in a 107 00:04:43,639 --> 00:04:45,939 room. You slip notes under the door to give it 108 00:04:45,939 --> 00:04:47,980 feedback, and it sends brilliant answers vibe. 109 00:04:48,519 --> 00:04:50,660 But its underlying personality and deep structural 110 00:04:50,660 --> 00:04:53,800 knowledge isn't changing. Right. So if frozen 111 00:04:53,800 --> 00:04:56,279 models are still highly capable, what exactly 112 00:04:56,279 --> 00:04:58,920 is the TTT Discover approach doing differently? 113 00:04:59,459 --> 00:05:02,300 TTT Discover takes reinforcement learning, or 114 00:05:02,300 --> 00:05:05,180 RL, and runs it directly inside the environment 115 00:05:05,180 --> 00:05:07,139 defined by the single test problem you want to 116 00:05:07,139 --> 00:05:09,860 solve. Instead of just updating the text in the 117 00:05:09,860 --> 00:05:12,160 prompt window? Yes. It is literally executing 118 00:05:12,160 --> 00:05:14,300 gradient updates on the model's internal weights 119 00:05:14,300 --> 00:05:16,720 based on its experience with this one specific 120 00:05:16,720 --> 00:05:19,720 problem. Okay, let's slow down and mechanicalize 121 00:05:19,720 --> 00:05:21,980 that. Walk me through the actual loop. I give 122 00:05:21,980 --> 00:05:25,160 the AI a math problem. What is step one? Step 123 00:05:25,160 --> 00:05:28,389 one. The model generates an action. In the context 124 00:05:28,389 --> 00:05:30,470 of scientific discovery, that action is typically 125 00:05:30,470 --> 00:05:33,089 writing a piece of Python code. Or a heuristic 126 00:05:33,089 --> 00:05:35,970 algorithm or a mathematical proof, right? Exactly. 127 00:05:36,569 --> 00:05:39,910 So it writes the code. Then what? Step two, the 128 00:05:39,910 --> 00:05:42,889 environment executes that code, like a Python 129 00:05:42,889 --> 00:05:45,170 interpreter actually runs the script. Okay. And 130 00:05:45,170 --> 00:05:47,870 what does that execution do? It produces a new 131 00:05:47,870 --> 00:05:51,230 state. And crucially, it produces a continuous 132 00:05:51,230 --> 00:05:53,470 reward signal based on how well the code performed. 133 00:05:53,709 --> 00:05:56,769 What kind of reward? Like, did it compile? Yeah, 134 00:05:56,810 --> 00:05:59,250 did it compile? Did it get closer to the mathematical 135 00:05:59,250 --> 00:06:01,730 bound we are searching for? Did it run faster 136 00:06:01,730 --> 00:06:03,769 than the previous attempt? So it basically gets 137 00:06:03,769 --> 00:06:07,649 a score? Yes, a highly specific numerical score. 138 00:06:08,170 --> 00:06:10,750 Then comes step three, which is the big differentiator. 139 00:06:11,389 --> 00:06:14,610 OK, lay it on me. TTT Discover takes that specific 140 00:06:14,610 --> 00:06:18,089 experience, the state, the action, and the reward, 141 00:06:18,389 --> 00:06:21,029 and it performs a gradient update. It actually 142 00:06:21,029 --> 00:06:23,899 adjusts its own weights? It mathematically adjusts 143 00:06:23,899 --> 00:06:26,040 the internal weights of the neural network to 144 00:06:26,040 --> 00:06:28,339 make the model more likely to generate high reward 145 00:06:28,339 --> 00:06:32,639 code in the future. Wow. It is literally rewiring 146 00:06:32,639 --> 00:06:35,160 its own brain to become an expert at the one 147 00:06:35,160 --> 00:06:37,720 specific test it is currently taking. That's 148 00:06:37,720 --> 00:06:40,000 exactly what it's doing. Hold on. Updating a 149 00:06:40,000 --> 00:06:43,259 massive language model sounds incredibly computationally 150 00:06:43,259 --> 00:06:46,180 expensive. It usually is. Right. If I have a 151 00:06:46,180 --> 00:06:48,660 120 billion parameter model and I'm updating 152 00:06:48,660 --> 00:06:51,589 its weights. every time it tries to write a line 153 00:06:51,589 --> 00:06:54,009 of code, wouldn't that cost millions of dollars? 154 00:06:54,370 --> 00:06:56,250 If they were doing full parameter fine -tuning, 155 00:06:56,610 --> 00:06:58,970 yes. It would be prohibitively expensive and 156 00:06:58,970 --> 00:07:01,889 incredibly slow. So how are they doing it? We 157 00:07:01,889 --> 00:07:03,910 need to look at the specific computational cost 158 00:07:03,910 --> 00:07:06,689 and scale they report in the paper because it's 159 00:07:06,689 --> 00:07:09,410 remarkably efficient. The implementation uses 160 00:07:09,410 --> 00:07:13,250 a finite budget. How finite? Exactly 50 training 161 00:07:13,250 --> 00:07:17,870 steps. Just 50? Yes, just 50. That seems tiny. 162 00:07:18,089 --> 00:07:21,610 At each step, they generate 512 rollouts or attempts 163 00:07:21,610 --> 00:07:24,649 from the model. OK, so that's a fair amount of 164 00:07:24,649 --> 00:07:27,269 attempts, but still, updating the whole model? 165 00:07:27,850 --> 00:07:30,089 No, crucially, they don't update the whole model. 166 00:07:30,629 --> 00:07:33,970 They use a technique called Rank32Laura. Which 167 00:07:33,970 --> 00:07:36,610 stands for low rank adaptation, right? Exactly. 168 00:07:37,050 --> 00:07:42,550 Applied to the OpenAI GPT -OSS -120B model. Exactly. 169 00:07:42,819 --> 00:07:45,240 I've heard of Lora, but let's translate that 170 00:07:45,240 --> 00:07:47,959 for the listener. What is it actually doing physically 171 00:07:47,959 --> 00:07:50,899 to the model? Think of it this way. Imagine you 172 00:07:50,899 --> 00:07:53,800 have a massive comprehensive textbook. That's 173 00:07:53,800 --> 00:07:56,839 your 120 billion parameter frozen model. Okay, 174 00:07:57,040 --> 00:07:59,920 heavy textbook. Instead of erasing and rewriting 175 00:07:59,920 --> 00:08:02,360 the printed pages of the textbook, which takes 176 00:08:02,360 --> 00:08:05,439 forever, Lora is like slipping a few transparent 177 00:08:05,439 --> 00:08:08,899 sheets of plastic over the pages. Oh, and you 178 00:08:08,899 --> 00:08:11,339 write your new specific notes on the plastic. 179 00:08:11,560 --> 00:08:14,129 Exactly. You freeze the main model's weights 180 00:08:14,129 --> 00:08:17,149 entirely, and you just inject a very small set 181 00:08:17,149 --> 00:08:20,089 of trainable matrices, the transparent plastic, 182 00:08:20,649 --> 00:08:23,170 into the network. So when the signal passes through 183 00:08:23,170 --> 00:08:26,649 the brain, it hits the massive frozen knowledge 184 00:08:26,649 --> 00:08:29,310 base, but then gets slightly tweaked by these 185 00:08:29,310 --> 00:08:33,309 small, highly specialized LoRa layers. Yep, layers 186 00:08:33,309 --> 00:08:35,950 that are rapidly learning how to pass this specific 187 00:08:35,950 --> 00:08:38,990 test. That is a perfect way to visualize it. 188 00:08:39,360 --> 00:08:42,039 I imagine this drastically reduces the compute 189 00:08:42,039 --> 00:08:45,460 required. It does. In total, over the entire 190 00:08:45,460 --> 00:08:49,820 discovery process, they are doing 25 ,600 rollouts. 191 00:08:50,039 --> 00:08:53,659 That's 50 steps multiplied by 512 rollouts per 192 00:08:53,659 --> 00:08:56,639 step. Right. We're not talking about thousands 193 00:08:56,639 --> 00:08:59,799 of gradient updates to hundreds of billions of 194 00:08:59,799 --> 00:09:01,960 parameters. And what does that actually cost 195 00:09:01,960 --> 00:09:04,679 in cold hard cash? Because cheap in AI research 196 00:09:04,679 --> 00:09:07,809 can still mean like more than my mortgage. The 197 00:09:07,809 --> 00:09:11,149 research cost they report was roughly $500 per 198 00:09:11,149 --> 00:09:15,970 problem run. Wait, $500? Yes, using the Tinker 199 00:09:15,970 --> 00:09:19,929 platform. That is wild. Now I have to emphasize, 200 00:09:20,370 --> 00:09:23,029 this is a reported research cost for this specific 201 00:09:23,029 --> 00:09:25,350 experimental setup. Right, it's not consumer 202 00:09:25,350 --> 00:09:28,690 pricing. Exactly. It is not some universal guaranteed 203 00:09:28,690 --> 00:09:31,250 retail price for generating scientific breakthroughs. 204 00:09:31,570 --> 00:09:33,870 You can't just hand a server $500 and expect 205 00:09:33,870 --> 00:09:37,210 a Nobel Prize. Sure, sure. But what it proves 206 00:09:37,210 --> 00:09:40,070 is that the concept is computationally accessible. 207 00:09:40,210 --> 00:09:43,230 It's not limited to some massive cluster running 208 00:09:43,230 --> 00:09:46,549 for six months. Exactly. $500 to potentially 209 00:09:46,549 --> 00:09:48,850 find a new state -of -the -art solution to a 210 00:09:48,850 --> 00:09:51,730 math problem that has stumped humans for decades? 211 00:09:52,070 --> 00:09:55,269 It's mind -blowing. But here is where my mind 212 00:09:55,269 --> 00:09:58,649 goes immediately. If they are just using reinforcement 213 00:09:58,649 --> 00:10:01,110 learning to update these LoRa weights, why not 214 00:10:01,110 --> 00:10:04,730 just plug in standard RL algorithms? Like PPO 215 00:10:04,730 --> 00:10:07,450 or GRPO? Yeah, the algorithms that companies 216 00:10:07,450 --> 00:10:09,889 already use to train chat models to be helpful 217 00:10:09,889 --> 00:10:12,149 and safe. Why not just use those? That touches 218 00:10:12,149 --> 00:10:14,529 on the core theoretical contribution of the paper. 219 00:10:15,110 --> 00:10:17,269 This is where we transition from merely what 220 00:10:17,269 --> 00:10:20,649 the system does computationally to how it fundamentally 221 00:10:20,649 --> 00:10:24,230 alters the mathematics. How so? Standard RL algorithms 222 00:10:24,230 --> 00:10:27,990 like PPO are designed with a very specific overarching 223 00:10:27,990 --> 00:10:31,769 goal. They aim to maximize the expected reward. 224 00:10:32,289 --> 00:10:34,789 Expected reward meaning? Like the average outcome. 225 00:10:34,940 --> 00:10:37,679 Essentially, yes. In Standard RL, you are looking 226 00:10:37,679 --> 00:10:40,440 for a policy, a way of behaving, that performs 227 00:10:40,440 --> 00:10:44,139 robustly and reliably across many, many different 228 00:10:44,139 --> 00:10:46,279 states and deployments. You want the model to 229 00:10:46,279 --> 00:10:50,419 be consistently good. Right. OK. Let me use an 230 00:10:50,419 --> 00:10:53,080 analogy here to see if I'm tracking. Maximizing 231 00:10:53,080 --> 00:10:55,840 expected reward is kind of like a major restaurant 232 00:10:55,840 --> 00:10:58,840 franchise trying to design a menu. OK. I follow. 233 00:10:59,049 --> 00:11:00,950 They want to make sure that every single dish 234 00:11:00,950 --> 00:11:04,370 on that massive menu is a safe, consistent B+. 235 00:11:04,370 --> 00:11:06,570 You want every customer to have a decent meal. 236 00:11:06,710 --> 00:11:08,889 Right. Whether they order the burger, the salad, 237 00:11:09,029 --> 00:11:12,470 or the pasta. Exactly. You heavily penalize a 238 00:11:12,470 --> 00:11:14,730 recipe that sometimes produces a masterpiece, 239 00:11:14,909 --> 00:11:17,429 but often starts a fire in the kitchen. That's 240 00:11:17,429 --> 00:11:20,429 a great analogy. Standard RL wants to raise the 241 00:11:20,429 --> 00:11:23,759 floor of performance. Right. But scientific discovery 242 00:11:23,759 --> 00:11:25,960 doesn't care about the average. It doesn't care 243 00:11:25,960 --> 00:11:28,440 about raising the floor. No, it doesn't. It only 244 00:11:28,440 --> 00:11:31,220 cares about finding one single Michelin star 245 00:11:31,220 --> 00:11:34,200 dish. You just need the one absolute best solution 246 00:11:34,200 --> 00:11:36,460 that beats the current state of the art. That 247 00:11:36,460 --> 00:11:38,659 is the goal of discovery. It cares if the other 248 00:11:38,659 --> 00:11:42,539 25 ,599 attempts were terrible burnt messes in 249 00:11:42,539 --> 00:11:46,220 the kitchen. If attempt number 25 ,600 solves 250 00:11:46,220 --> 00:11:49,259 the math problem, you win the Nobel Prize. You're 251 00:11:49,259 --> 00:11:51,740 touching on a crucial distinction, though I must 252 00:11:51,879 --> 00:11:54,860 Carefully note that optimizing for expected reward 253 00:11:54,860 --> 00:11:57,919 doesn't inherently minimize variance in all mathematical 254 00:11:57,919 --> 00:12:01,259 setups. Oh, it doesn't. No, and it doesn't always 255 00:12:01,259 --> 00:12:03,879 strictly penalize risk. It's simply an average 256 00:12:03,879 --> 00:12:08,539 objective. However. However. The authors correctly 257 00:12:08,539 --> 00:12:10,799 identify that in the context of scientific search 258 00:12:10,799 --> 00:12:13,720 spaces, maximizing average performance is often 259 00:12:13,720 --> 00:12:16,559 very poorly aligned with finding one exceptional 260 00:12:16,559 --> 00:12:19,309 outlier solution. Makes sense. If your model 261 00:12:19,309 --> 00:12:21,269 spends all its computational budget learning 262 00:12:21,269 --> 00:12:24,769 how to safely generate mediocre, compilable code, 263 00:12:25,009 --> 00:12:27,190 it might never take the wild, chaotic leap required 264 00:12:27,190 --> 00:12:29,929 to find a genuine breakthrough. So how does TTT 265 00:12:29,929 --> 00:12:32,090 Discover fix that? How do they fundamentally 266 00:12:32,090 --> 00:12:34,309 change the math to tell the AI, forget being 267 00:12:34,309 --> 00:12:37,110 safe, go exclusively for the Michelin star? They 268 00:12:37,110 --> 00:12:39,509 introduce what they call an entropic objective. 269 00:12:39,750 --> 00:12:43,730 Entropic objective. OK. How does that work? Conceptually, 270 00:12:44,129 --> 00:12:46,789 the math works by exponentially reweighting the 271 00:12:46,789 --> 00:12:50,149 rewards based on a tuning parameter called beta. 272 00:12:51,049 --> 00:12:54,590 Exponentially reweighting. OK, so normally if 273 00:12:54,590 --> 00:12:57,730 attempt A scores a 5 and attempt B scores a 10, 274 00:12:57,950 --> 00:13:01,710 attempt B is twice as good. Right. How does exponential 275 00:13:01,710 --> 00:13:03,870 reweighting change that? Instead of treating 276 00:13:03,870 --> 00:13:06,309 the rewards linearly, where a 10 is twice as 277 00:13:06,309 --> 00:13:09,470 good as a 5, the exponential function heavily 278 00:13:09,470 --> 00:13:12,899 skews the focus. Excuse it, how? Under an entropic 279 00:13:12,899 --> 00:13:16,320 objective with a high beta, that 10 isn't just 280 00:13:16,320 --> 00:13:19,139 twice as good. It might be treated as a thousand 281 00:13:19,139 --> 00:13:21,200 times more important than the five. Oh, wow. 282 00:13:22,039 --> 00:13:24,720 The authors introduce a mechanism called adaptive 283 00:13:24,720 --> 00:13:28,240 beta. The system doesn't use a fixed exponential 284 00:13:28,240 --> 00:13:31,399 multiplier. It changes. Yes. It sets the beta 285 00:13:31,399 --> 00:13:34,100 parameter adaptively for each initial state. 286 00:13:34,220 --> 00:13:36,460 And it does this by strictly constraining something 287 00:13:36,460 --> 00:13:40,830 called the KL divergence. KL divergence. I feel 288 00:13:40,830 --> 00:13:43,350 like I see this term everywhere in AI papers. 289 00:13:43,850 --> 00:13:45,830 Let's translate that for the listener. What is 290 00:13:45,830 --> 00:13:49,590 it actually measuring? Simply put, callback Liebler 291 00:13:49,590 --> 00:13:52,990 divergence, or KL divergence, is a mathematical 292 00:13:52,990 --> 00:13:55,509 measure of how much one probability distribution 293 00:13:55,509 --> 00:13:57,950 differs from another reference distribution. 294 00:13:58,490 --> 00:14:01,389 But does that guarantee it won't break? No, and 295 00:14:01,389 --> 00:14:03,929 we need a strict qualification here. This KL 296 00:14:03,929 --> 00:14:07,649 constraint is not some magic theorem. that guarantees 297 00:14:07,649 --> 00:14:10,730 monotonic improvement. Meaning it doesn't guarantee 298 00:14:10,730 --> 00:14:12,669 the model gets strictly better every single step. 299 00:14:12,870 --> 00:14:15,190 Right. It doesn't guarantee complete stability, 300 00:14:15,190 --> 00:14:17,889 and it certainly doesn't guarantee perfect exploration 301 00:14:17,889 --> 00:14:21,190 of the problem space. But remember, the model 302 00:14:21,190 --> 00:14:24,490 is generating 512 rollouts per step. That's a 303 00:14:24,490 --> 00:14:27,309 lot of attempts. It is. It's building a massive 304 00:14:27,309 --> 00:14:30,090 tree of possible solutions. How does it decide 305 00:14:30,090 --> 00:14:32,289 which of those previous partial solutions it 306 00:14:32,289 --> 00:14:34,070 should actually build upon for the next step? 307 00:14:34,370 --> 00:14:36,450 It can't explore all of them, obviously. Right. 308 00:14:36,490 --> 00:14:39,009 Does it just pick the one with the highest score 309 00:14:39,009 --> 00:14:41,490 and keep going? If it only picked the highest 310 00:14:41,490 --> 00:14:43,990 score, it would get trapped in local optima. 311 00:14:44,309 --> 00:14:47,090 Local optima? It means it would climb the nearest 312 00:14:47,090 --> 00:14:49,830 small hill and never realize there's a massive 313 00:14:49,830 --> 00:14:52,409 mountain right next to it. Oh, it would get stuck 314 00:14:52,409 --> 00:14:55,129 on a decent idea and ignore the great ones. Right. 315 00:14:56,009 --> 00:14:59,289 To solve this, TTT Discover uses a search subroutine 316 00:14:59,289 --> 00:15:02,450 inspired by a well -known algorithm called PUCT. 317 00:15:03,049 --> 00:15:06,620 PUCT. That stands for predictor upper confidence 318 00:15:06,620 --> 00:15:09,940 bound applied to trees, right? Yes. It sounds 319 00:15:09,940 --> 00:15:12,759 like something from AlphaGo. Good catch. It is 320 00:15:12,759 --> 00:15:15,039 heavily related to the Monte Carlo tree search 321 00:15:15,039 --> 00:15:18,000 used in systems like AlphaGo. Oh, cool. It is 322 00:15:18,000 --> 00:15:20,200 a state selection rule designed to carefully 323 00:15:20,200 --> 00:15:23,500 balance exploiting known good solutions and exploring 324 00:15:23,500 --> 00:15:25,799 new uncertain ones. So how does the math work 325 00:15:25,799 --> 00:15:28,419 for that? The rule in this paper breaks down 326 00:15:28,419 --> 00:15:31,059 into three specific mathematical components. 327 00:15:31,299 --> 00:15:33,299 Let's break them down. What's the first component? 328 00:15:33,610 --> 00:15:36,509 First, it looks at the maximum reward among a 329 00:15:36,509 --> 00:15:40,220 state's children. OK, meaning what? Basically, 330 00:15:40,379 --> 00:15:42,940 it asks, when we explored down this specific 331 00:15:42,940 --> 00:15:45,379 branch of code previously, what was the absolute 332 00:15:45,379 --> 00:15:48,720 best score we eventually hit? Oh, so it looks 333 00:15:48,720 --> 00:15:50,799 at the empirical ceiling of that path. Exactly. 334 00:15:51,299 --> 00:15:54,039 OK, that's the exploitation part. It favors paths 335 00:15:54,039 --> 00:15:55,820 that have already proven they can hit high scores. 336 00:15:56,200 --> 00:15:59,559 What's the second component? Second, it uses 337 00:15:59,559 --> 00:16:02,179 a prior that is proportional to the state's reward 338 00:16:02,179 --> 00:16:06,320 rank. in the replay buffer. Meaning, if a state 339 00:16:06,320 --> 00:16:09,299 itself scored really high, it gets a boost? Exactly. 340 00:16:09,820 --> 00:16:12,500 Regardless of the children, if a specific state 341 00:16:12,500 --> 00:16:15,100 scored very highly compared to everything else 342 00:16:15,100 --> 00:16:17,460 the model has ever tried, it gets a boost in 343 00:16:17,460 --> 00:16:20,399 probability of being selected again. High reward 344 00:16:20,399 --> 00:16:23,720 states get favored. Right. Makes sense. And the 345 00:16:23,720 --> 00:16:26,220 third, because right now it still just sounds 346 00:16:26,220 --> 00:16:28,700 like it's favoring the winners. The third is 347 00:16:28,700 --> 00:16:32,409 the crucial counterbalance. Okay. It's an exploration 348 00:16:32,409 --> 00:16:35,110 bonus based on expansion counts. How does that 349 00:16:35,110 --> 00:16:37,789 work? The formula heavily boosts the selection 350 00:16:37,789 --> 00:16:40,429 probability of states that have a low visitation 351 00:16:40,429 --> 00:16:44,649 count. Oh. So the exploration bonus forces the 352 00:16:44,649 --> 00:16:47,409 system to look at paths it has neglected. Exactly. 353 00:16:47,830 --> 00:16:50,309 It's the algorithm saying, hey, you've completely 354 00:16:50,309 --> 00:16:52,870 ignored this weird, mediocre piece of code for 355 00:16:52,870 --> 00:16:55,230 10 straight steps. Maybe there's a breakthrough 356 00:16:55,230 --> 00:16:57,610 hidden behind it. Go expand it once just to see. 357 00:16:57,970 --> 00:17:01,269 Precisely. It encourages visiting under -visited 358 00:17:01,269 --> 00:17:03,710 states to prevent the system from getting tunnel 359 00:17:03,710 --> 00:17:06,450 vision. That's really smart. Now, we must ensure 360 00:17:06,450 --> 00:17:08,730 it is entirely clear to the listener that this 361 00:17:08,730 --> 00:17:11,809 is a mathematical heuristic rule. It encourages 362 00:17:11,809 --> 00:17:15,049 exploration, but it does not guarantee exploration 363 00:17:15,049 --> 00:17:17,369 of the entire expanding tree of possibilities. 364 00:17:18,009 --> 00:17:19,809 Because the tree is just too big, right? Right. 365 00:17:20,309 --> 00:17:23,210 With only 50 steps and 512 rollouts, the tree 366 00:17:23,210 --> 00:17:26,529 of possible code combinations is infinitely larger 367 00:17:26,529 --> 00:17:28,839 than the compute budget. So it's just making 368 00:17:28,839 --> 00:17:32,140 smart bets. Exactly. And importantly, there is 369 00:17:32,140 --> 00:17:34,559 no separate smaller neural network predicting 370 00:17:34,559 --> 00:17:37,299 this prior, which is how some other systems work. 371 00:17:37,539 --> 00:17:40,200 Oh, it's just a raw mathematical calculation. 372 00:17:40,480 --> 00:17:43,920 Yes. In TTT Discover, it is strictly a mathematical 373 00:17:43,920 --> 00:17:47,240 rule calculated directly from the observed rewards 374 00:17:47,240 --> 00:17:49,960 and the visitation counts in the buffer. It's 375 00:17:49,960 --> 00:17:52,240 elegant and computationally lightweight. Very 376 00:17:52,240 --> 00:17:54,819 lightweight. OK, so let's summarize the theoretical 377 00:17:54,819 --> 00:17:57,230 engine we've built here. We have the entropic 378 00:17:57,230 --> 00:18:00,009 objective acting as a massive gravitational pull 379 00:18:00,009 --> 00:18:02,990 toward extreme successes. Now, we need to connect 380 00:18:02,990 --> 00:18:05,589 this beautiful theoretical engine to concrete, 381 00:18:05,869 --> 00:18:08,509 verifiable results. And the paper starts in the 382 00:18:08,509 --> 00:18:12,140 purest domain possible. Mathematics. Okay, so 383 00:18:12,140 --> 00:18:14,119 let's talk about the first mathematical challenge 384 00:18:14,119 --> 00:18:17,480 they threw at TTT Discover. Erdos's minimum overlap 385 00:18:17,480 --> 00:18:20,319 problem. A classic problem. First off, who is 386 00:18:20,319 --> 00:18:23,319 Erdos and why is this problem important? Paul 387 00:18:23,319 --> 00:18:25,920 Erdos was one of the most prolific and eccentric 388 00:18:25,920 --> 00:18:28,539 mathematicians of the 20th century. He was a 389 00:18:28,539 --> 00:18:30,859 wanderer who essentially lived out of a suitcase, 390 00:18:31,359 --> 00:18:33,720 traveling from university to university, collaborating 391 00:18:33,720 --> 00:18:36,660 on thousands of papers. Sounds like a fascinating 392 00:18:36,660 --> 00:18:40,119 guy. He really was. This specific problem is 393 00:18:40,119 --> 00:18:43,440 a classic in combinatorial number theory, originally 394 00:18:43,440 --> 00:18:47,759 posed back in 1955. Combinatorial number theory. 395 00:18:48,279 --> 00:18:51,099 So lots of counting and arranging integers. Humans 396 00:18:51,099 --> 00:18:52,779 have been chipping away at this for decades. 397 00:18:53,039 --> 00:18:56,539 Have previous AI agents tackled this? Yes. A 398 00:18:56,539 --> 00:18:58,940 previous evolutionary search AI called Alpha 399 00:18:58,940 --> 00:19:01,619 Evolve recently tackled it. And how did they 400 00:19:01,619 --> 00:19:06,220 do? They found an improved upper bound of 0 .380924. 401 00:19:06,900 --> 00:19:09,339 And they proved this bound by having the AI construct 402 00:19:09,339 --> 00:19:12,640 a 95 -piece symmetric step function. OK, so that 403 00:19:12,640 --> 00:19:15,259 was the record to beat. What did TTT Discover 404 00:19:15,259 --> 00:19:18,970 do? TTT Discover achieved an even tighter, improved 405 00:19:18,970 --> 00:19:24,170 upper bound of 0 .380876. Now, to a lay person, 406 00:19:24,309 --> 00:19:28,230 moving the decimal from .3809 to .3808 sounds 407 00:19:28,230 --> 00:19:31,069 like a tiny, meaningless difference. It does 408 00:19:31,069 --> 00:19:34,589 sound small. But in asymptotic mathematics, moving 409 00:19:34,589 --> 00:19:36,650 that decimal point at all is a massive breakthrough, 410 00:19:36,869 --> 00:19:39,509 right? Oh, absolutely. It's a huge deal. But 411 00:19:39,509 --> 00:19:41,920 here is the crazy part of the paper. It didn't 412 00:19:41,920 --> 00:19:44,599 just tweak the previous answer. It achieved this 413 00:19:44,599 --> 00:19:49,160 new bound by constructing a 600 -piece asymmetric 414 00:19:49,160 --> 00:19:52,980 step function. Yes, 600 pieces. That just sounds 415 00:19:52,980 --> 00:19:55,900 so weird and specific. What does that actually 416 00:19:55,900 --> 00:19:59,299 mean visually? Think about a step function as 417 00:19:59,299 --> 00:20:02,160 a staircase on a graph. It's a piecewise constant 418 00:20:02,160 --> 00:20:05,000 function. The value is flat, then it jumps up 419 00:20:05,000 --> 00:20:07,440 or down, then it's flat again. Right, like literal 420 00:20:07,440 --> 00:20:10,430 stairs. Alpha Evolve's best solution was a staircase 421 00:20:10,430 --> 00:20:13,750 with 95 distinct stairs, and it was symmetric. 422 00:20:14,390 --> 00:20:16,769 Meaning, if you folded the graph in half, the 423 00:20:16,769 --> 00:20:18,869 left side perfectly mirrored the right side. 424 00:20:19,509 --> 00:20:22,130 But as an expert, I have to apply a crucial, 425 00:20:22,390 --> 00:20:24,369 rigorous limit to our excitement here. Okay, 426 00:20:24,410 --> 00:20:26,849 what's the limit? The fact that the AI found 427 00:20:26,849 --> 00:20:29,509 an asymmetric solution does not prove that the 428 00:20:29,509 --> 00:20:32,430 exact, absolute mathematical optimum to the Erdos 429 00:20:32,430 --> 00:20:35,519 problem is asymmetric. Wait, really? It just 430 00:20:35,519 --> 00:20:37,079 found a better one. It didn't prove it's the 431 00:20:37,079 --> 00:20:40,609 best one. Correct. We have a new best bound, 432 00:20:40,990 --> 00:20:44,210 but we do not have a proof of optimality. Ah, 433 00:20:44,210 --> 00:20:46,349 okay. Furthermore, this result does not prove 434 00:20:46,349 --> 00:20:48,569 that all symmetric constructions are inherently 435 00:20:48,569 --> 00:20:51,130 inferior. Oh, so there could still be a better 436 00:20:51,130 --> 00:20:54,450 symmetric one out there. Exactly. It is entirely 437 00:20:54,450 --> 00:20:57,710 possible that a 10 ,000 -piece symmetric function 438 00:20:57,710 --> 00:21:01,250 exists that beats this one. It simply means this 439 00:21:01,430 --> 00:21:05,210 specific 600 piece asymmetric function is the 440 00:21:05,210 --> 00:21:08,509 best bound humanity or machine kind has discovered 441 00:21:08,509 --> 00:21:11,349 so far. Fair enough. We have to separate empirical 442 00:21:11,349 --> 00:21:14,329 results from mathematical guarantees. Yes. And 443 00:21:14,329 --> 00:21:16,890 we also need to do a quick context check on the 444 00:21:16,890 --> 00:21:18,650 frozen models here, right? Because earlier we 445 00:21:18,650 --> 00:21:21,910 said frozen models aren't useless. Yes. This 446 00:21:21,910 --> 00:21:24,049 is a very important point for the listeners keeping 447 00:21:24,049 --> 00:21:27,950 score. The paper includes a purely frozen baseline. 448 00:21:28,250 --> 00:21:32,069 Which they call best of 25 ,600, right? Right. 449 00:21:32,329 --> 00:21:35,569 This baseline just prompted the frozen 120 billion 450 00:21:35,569 --> 00:21:39,829 parameter model 25 ,600 times without ever updating 451 00:21:39,829 --> 00:21:41,950 its internal weights. And how did the frozen 452 00:21:41,950 --> 00:21:45,029 model do? That frozen baseline also discovered 453 00:21:45,029 --> 00:21:47,049 a construction that improved upon the previous 454 00:21:47,049 --> 00:21:50,470 Alpha Evolve record. Wow. So even without the 455 00:21:50,470 --> 00:21:52,630 fancy test time training, the base LLM was smart 456 00:21:52,630 --> 00:21:54,750 enough to push the boundaries of combinatorics 457 00:21:54,750 --> 00:21:58,430 just by rolling the dice 25 ,000 times. Exactly. 458 00:21:58,789 --> 00:22:00,789 Frozen models are highly capable mathematical 459 00:22:00,789 --> 00:22:03,269 explorers in their own right, assuming you have 460 00:22:03,269 --> 00:22:06,009 enough compute to sample them extensively. TTT 461 00:22:06,009 --> 00:22:08,990 Discover just provided the focus learning required 462 00:22:08,990 --> 00:22:12,049 to push the frontier even further than the frozen 463 00:22:12,049 --> 00:22:13,950 model could reach. That's the perfect way to 464 00:22:13,950 --> 00:22:16,849 summarize it. OK, let's look at the next math 465 00:22:16,849 --> 00:22:19,769 problem they threw at the system. Auto -correlation 466 00:22:19,769 --> 00:22:22,190 inequalities. They tackled two of these, AC1 467 00:22:22,190 --> 00:22:25,880 and AC2. First, what is AC1? These problems also 468 00:22:25,880 --> 00:22:29,380 relate to additive combinatorics. AC1 involves 469 00:22:29,380 --> 00:22:32,619 finding the tightest upper bound for a specific 470 00:22:32,619 --> 00:22:36,299 convolution inequality. Convolution. I know that 471 00:22:36,299 --> 00:22:38,420 from image processing, like blurring an image. 472 00:22:38,839 --> 00:22:41,420 What does it mean in pure math? A convolution 473 00:22:41,420 --> 00:22:44,000 is a mathematical operation on two functions 474 00:22:44,000 --> 00:22:46,640 that produces a third function. It expresses 475 00:22:46,640 --> 00:22:48,819 how the shape of one function is modified by 476 00:22:48,819 --> 00:22:51,799 the other. Okay, I'm with you. In this specific 477 00:22:51,799 --> 00:22:53,799 inequality, mathematicians are trying to bound 478 00:22:53,799 --> 00:22:56,400 how large the integral of this convolved function 479 00:22:56,400 --> 00:22:59,220 can get under certain constraints. And how did 480 00:22:59,220 --> 00:23:01,920 TTT -Discover do? The previous state -of -the 481 00:23:01,920 --> 00:23:04,500 -art upper bound was pushed down by TTT -Discover 482 00:23:04,500 --> 00:23:09,200 to a new record of 1 .50286. And how did it do 483 00:23:09,200 --> 00:23:12,579 that? Because the Erdos -1 was a 600 -piece step 484 00:23:12,579 --> 00:23:15,319 function, what kind of monster did it build for 485 00:23:15,319 --> 00:23:19,910 AC -1? It generated a massive 30 ,000 piece step 486 00:23:19,910 --> 00:23:24,710 function. 30 ,000 pieces. How does an AI even 487 00:23:24,710 --> 00:23:27,349 write that? It didn't just randomly guess 30 488 00:23:27,349 --> 00:23:29,390 ,000 different decimal numbers and hope they 489 00:23:29,390 --> 00:23:31,910 worked, right? No, definitely not. And this is 490 00:23:31,910 --> 00:23:34,309 where we see the model acting not just as a guesser, 491 00:23:34,309 --> 00:23:36,869 but as a brilliant algorithm designer. Oh, it 492 00:23:36,869 --> 00:23:39,910 wrote an algorithm to solve it. To find these 493 00:23:39,910 --> 00:23:42,710 step functions, the AI writes Python code to 494 00:23:42,710 --> 00:23:44,730 optimize the function using a technique called 495 00:23:44,730 --> 00:23:47,460 linear programming. Okay, linear programming. 496 00:23:48,349 --> 00:23:51,369 But a 30 ,000 piece function creates an optimization 497 00:23:51,369 --> 00:23:53,930 problem that is far too large to solve naively. 498 00:23:54,289 --> 00:23:56,630 If you just hand a standard solver a matrix with 499 00:23:56,630 --> 00:23:59,170 30 ,000 variables and millions of constraints... 500 00:23:59,170 --> 00:24:01,589 What happens? The computer will run out of memory 501 00:24:01,589 --> 00:24:05,250 or take years to compute. So how did the AI bypass 502 00:24:05,250 --> 00:24:08,750 the memory limit? The AI learned through its 503 00:24:08,750 --> 00:24:11,230 test time training loop to focus its linear programming 504 00:24:11,230 --> 00:24:14,150 optimization only on the constraints that were 505 00:24:14,150 --> 00:24:16,819 close to being tight. What does tight mean in 506 00:24:16,819 --> 00:24:19,859 this context? Imagine you are trying to pack 507 00:24:19,859 --> 00:24:23,470 a suitcase. Most of the clothes fit easily. They 508 00:24:23,470 --> 00:24:25,529 aren't pushing against the zipper. Those are 509 00:24:25,529 --> 00:24:28,690 loose constraints. OK. But the shoes and the 510 00:24:28,690 --> 00:24:31,329 hairdryer are pushing hard against the fabric, 511 00:24:31,730 --> 00:24:33,890 threatening to pop the zipper. Those are tight 512 00:24:33,890 --> 00:24:37,089 constraints. I love that analogy. So in math, 513 00:24:37,690 --> 00:24:40,329 tight constraints are the specific rules that 514 00:24:40,329 --> 00:24:42,609 are actively preventing the score from getting 515 00:24:42,609 --> 00:24:45,849 better. Exactly. It identified the active bottlenecks 516 00:24:45,849 --> 00:24:48,349 in the math and only spent compute on those, 517 00:24:48,829 --> 00:24:51,269 which allowed it to scale its function up to 518 00:24:51,269 --> 00:24:53,829 30 ,000 pieces. It wrote an efficient algorithm 519 00:24:53,829 --> 00:24:56,569 to solve the math problem. But they didn't win 520 00:24:56,569 --> 00:24:58,569 every single challenge, did they? What happened 521 00:24:58,569 --> 00:25:03,009 with the AC2 problem? AC2 is another autocorrelation 522 00:25:03,009 --> 00:25:05,509 inequality, but the goal here is finding a lower 523 00:25:05,509 --> 00:25:07,589 bound rather than an upper bound. And how did 524 00:25:07,589 --> 00:25:10,230 it do? TTT Discover reached a lower bound of 525 00:25:10,230 --> 00:25:13,490 0 .959. Which is an impressive number, but... 526 00:25:13,490 --> 00:25:15,329 But we must adhere strictly to the facts of the 527 00:25:15,329 --> 00:25:17,829 paper. It must be clearly stated that this did 528 00:25:17,829 --> 00:25:19,690 not beat the previous state -of -the -art lower 529 00:25:19,690 --> 00:25:22,630 bound. Oh, really? The previous record was 0 530 00:25:22,630 --> 00:25:26,990 .961, achieved by AlphaVolv2. So it lost to the 531 00:25:26,990 --> 00:25:31,049 older AI on this specific problem. Yes. TTT Discover 532 00:25:31,049 --> 00:25:33,470 produced a highly capable result. It got very 533 00:25:33,470 --> 00:25:36,609 close, but it did not set a new record for AC2. 534 00:25:36,789 --> 00:25:39,069 You know, I actually love that. It is incredibly 535 00:25:39,069 --> 00:25:41,089 refreshing to see a research paper publish the 536 00:25:41,089 --> 00:25:43,430 near misses and the losses alongside the massive 537 00:25:43,430 --> 00:25:45,410 wins. It shows they aren't just cherry picking 538 00:25:45,410 --> 00:25:47,470 the one time the algorithm worked and hiding 539 00:25:47,470 --> 00:25:50,450 the rest. Exactly. It gives us a realistic picture 540 00:25:50,450 --> 00:25:52,789 of the technology's current limits. It is the 541 00:25:52,789 --> 00:25:55,390 hallmark of rigorous science. Okay, so we have 542 00:25:55,390 --> 00:25:58,880 seen TTT discover, tackle, abstract math. Now 543 00:25:58,880 --> 00:26:02,119 I want to transition from abstract integers stretching 544 00:26:02,119 --> 00:26:06,000 toward infinity to the literal physical silicon 545 00:26:06,000 --> 00:26:09,400 hardware that runs AI in the first place. GPU 546 00:26:09,400 --> 00:26:12,839 engineering. Yes. If this model is so smart at 547 00:26:12,839 --> 00:26:15,480 optimizing math, can it optimize the very code 548 00:26:15,480 --> 00:26:18,619 that makes its own computations possible? Let's 549 00:26:18,619 --> 00:26:21,900 dive into GPU kernel engineering. This is a fiercely 550 00:26:21,900 --> 00:26:24,180 competitive and critical domain. What exactly 551 00:26:24,180 --> 00:26:27,430 is a GPU kernel? A GPU kernel is the low -level 552 00:26:27,430 --> 00:26:30,109 code that directly commands the graphics processing 553 00:26:30,109 --> 00:26:33,230 unit. It dictates exactly how memory is accessed, 554 00:26:33,309 --> 00:26:36,390 how calculations are batched. And how the thousands 555 00:26:36,390 --> 00:26:38,269 of tiny cores on the chip communicate with each 556 00:26:38,269 --> 00:26:41,769 other, right? Exactly. It is the physical computational 557 00:26:41,769 --> 00:26:45,230 foundation of all modern AI. Exactly. And to 558 00:26:45,230 --> 00:26:47,569 test this, the authors entered TTT Discover into 559 00:26:47,569 --> 00:26:50,589 the GPU mode competitions. Specifically, they 560 00:26:50,589 --> 00:26:53,829 targeted the Trimol challenge and the MLA decode 561 00:26:53,829 --> 00:26:56,500 challenge, right? So the goal of the GPU mode 562 00:26:56,500 --> 00:26:58,400 competition is very simple, right? Write the 563 00:26:58,400 --> 00:27:01,420 absolute fastest possible GPU kernel for this 564 00:27:01,420 --> 00:27:04,960 specific matrix operation. So how did TTT Discover 565 00:27:04,960 --> 00:27:07,700 approach this? What was its breakthrough? The 566 00:27:07,700 --> 00:27:09,720 AI, through trial and error, identified that 567 00:27:09,720 --> 00:27:12,859 the raw calculation speed, the actual multiplication 568 00:27:12,859 --> 00:27:15,839 of the numbers, wasn't the problem. Really? Then 569 00:27:15,839 --> 00:27:18,759 what was? The bottleneck was heavy memory input 570 00:27:18,759 --> 00:27:21,819 -output, specifically caused by frequent element 571 00:27:21,819 --> 00:27:25,069 -wise operations. Okay, translation time. Element 572 00:27:25,069 --> 00:27:27,470 -wise operations, that's when you have two lists 573 00:27:27,470 --> 00:27:30,750 of numbers, say 1, 2, 3, and 4, 5, 6. And you 574 00:27:30,750 --> 00:27:32,890 just add or multiply them straight across, right? 575 00:27:33,109 --> 00:27:35,089 Yes. The first number with the first number, 576 00:27:35,170 --> 00:27:37,130 the second with the second. It's the simplest 577 00:27:37,130 --> 00:27:40,250 math possible. But why is doing on a GPU incredibly 578 00:27:40,250 --> 00:27:43,289 expensive? Because the GPU has to constantly 579 00:27:43,289 --> 00:27:45,950 read data from the massive slow main memory, 580 00:27:46,069 --> 00:27:48,750 the HBM, bring it onto the chip, do the simple 581 00:27:48,750 --> 00:27:51,150 addition, and then write the answer all the way 582 00:27:51,150 --> 00:27:53,950 back to main memory. Oh, I see. It's like needing 583 00:27:53,950 --> 00:27:56,430 to chop 10 onions for a recipe, but you only 584 00:27:56,430 --> 00:27:59,009 carry one onion from the pantry to the cutting 585 00:27:59,009 --> 00:28:03,049 board at a time. That's hilarious. But accurate. 586 00:28:03,349 --> 00:28:05,289 The chopping takes two seconds, but the walking 587 00:28:05,289 --> 00:28:07,809 back and forth takes an hour. The memory bandwidth 588 00:28:07,809 --> 00:28:10,630 is the bottleneck, not the compute. Perfect analogy. 589 00:28:11,519 --> 00:28:13,980 So what did the AI do to fix the grocery trips? 590 00:28:14,720 --> 00:28:17,180 It leveraged a programming language called Triton 591 00:28:17,180 --> 00:28:20,180 to perform what is known as kernel fusion. Kernel 592 00:28:20,180 --> 00:28:22,839 fusion. Marrying the operations together. Right. 593 00:28:22,940 --> 00:28:25,240 Kernel Fusion means the code tells the GPU to 594 00:28:25,240 --> 00:28:28,000 load a large chunk of data into this super fast 595 00:28:28,000 --> 00:28:31,660 on -chip SRAM cache just once. Just once. Then 596 00:28:31,660 --> 00:28:34,839 it performs all the necessary element -wise operations, 597 00:28:35,119 --> 00:28:37,180 multiplications, and additions while the data 598 00:28:37,180 --> 00:28:40,869 is sitting in that ultra -fast cache. Oh. And 599 00:28:40,869 --> 00:28:43,410 then it writes it back. Only after everything 600 00:28:43,410 --> 00:28:45,690 is completely finished does it write the final 601 00:28:45,690 --> 00:28:48,230 result back to the slow main memory. It brought 602 00:28:48,230 --> 00:28:50,089 the whole bag of onions to the cutting board. 603 00:28:50,950 --> 00:28:54,009 Exactly. Additionally, The AI discovered that 604 00:28:54,009 --> 00:28:56,789 it could utilize mixed precision. Mixed precision. 605 00:28:57,269 --> 00:28:59,849 What does that mean? It chose to store intermediate 606 00:28:59,849 --> 00:29:02,210 activations, the temporary scratchpad numbers, 607 00:29:02,650 --> 00:29:06,049 in a 16 -bit floating point format, or FP16. 608 00:29:06,230 --> 00:29:08,769 Instead of the standard 32 -bit floats? Correct. 609 00:29:09,289 --> 00:29:11,329 By having the size of the numbers it was storing, 610 00:29:11,809 --> 00:29:13,730 it literally had the memory bandwidth required 611 00:29:13,730 --> 00:29:16,130 to move them around. Further alleviating the 612 00:29:16,130 --> 00:29:18,069 bottleneck without destroying the accuracy of 613 00:29:18,069 --> 00:29:20,809 the final answer. That is so clever. Very clever. 614 00:29:21,150 --> 00:29:23,769 Okay, so it fused the kernels and shrunk the 615 00:29:23,769 --> 00:29:27,109 data size. What were the actual results? How 616 00:29:27,109 --> 00:29:29,390 fast was the AI's code compared to human engineers? 617 00:29:29,710 --> 00:29:31,849 We need to detail the exact hardware testing 618 00:29:31,849 --> 00:29:33,890 here to be scientifically precise. Okay, give 619 00:29:33,890 --> 00:29:36,250 me the specs. The training process, where the 620 00:29:36,250 --> 00:29:38,809 AI was actually running the code, checking the 621 00:29:38,809 --> 00:29:41,569 latency and updating its weights, was done exclusively 622 00:29:41,569 --> 00:29:44,869 using an NVIDIA H100 GPU. The state -of -the 623 00:29:44,869 --> 00:29:48,250 -art chip. Yes. However, to prove the code wasn't 624 00:29:48,250 --> 00:29:50,819 just hallucinating a trick specific to one chip, 625 00:29:51,440 --> 00:29:53,579 the final evaluation of the generated kernels 626 00:29:53,579 --> 00:29:56,359 covered multiple architectures. Which ones? The 627 00:29:56,359 --> 00:30:00,240 H100, the older A100, the newer B200, and even 628 00:30:00,240 --> 00:30:03,279 the AMD MI300X. And what happened when they ran 629 00:30:03,279 --> 00:30:06,940 it on the older A100 chip? On the A100, the model 630 00:30:06,940 --> 00:30:09,319 achieved a roughly two -fold speedup over the 631 00:30:09,319 --> 00:30:11,400 best human submission on the competition leaderboard. 632 00:30:11,599 --> 00:30:13,359 Double the speed. It was double the speed of 633 00:30:13,359 --> 00:30:15,579 the best human code. That's incredible. It is 634 00:30:15,579 --> 00:30:17,680 impressive. But let's keep our feet on the ground 635 00:30:17,680 --> 00:30:20,109 here. OK, reality check. What does this actually 636 00:30:20,109 --> 00:30:22,450 mean for the industry? Is all AI about to get 637 00:30:22,450 --> 00:30:25,349 twice as fast overnight? No. And I must firmly 638 00:30:25,349 --> 00:30:27,690 qualify this result. The twofold speed up is 639 00:30:27,690 --> 00:30:30,769 highly task and hardware specific. So it's not 640 00:30:30,769 --> 00:30:34,170 a magic bullet. We absolutely cannot imply that 641 00:30:34,170 --> 00:30:36,690 this single module speed up is going to exponentially 642 00:30:36,690 --> 00:30:40,029 accelerate all downstream AI workloads everywhere 643 00:30:40,029 --> 00:30:43,690 in the world. It is a specific win on a specific 644 00:30:43,690 --> 00:30:47,309 mathematical operation. That makes sense. Furthermore, 645 00:30:47,500 --> 00:30:49,819 we must acknowledge that the official remote 646 00:30:49,819 --> 00:30:52,599 evaluation environment for the competition doesn't 647 00:30:52,599 --> 00:30:55,059 eliminate every single source of hardware timing 648 00:30:55,059 --> 00:30:58,480 variation. Right. Cloud GPUs can have a tiny 649 00:30:58,480 --> 00:31:00,720 bit of noise depending on temperature or what 650 00:31:00,720 --> 00:31:02,819 else is running on the rack. Exactly. There's 651 00:31:02,819 --> 00:31:05,019 always a tiny bit of noise. But nevertheless, 652 00:31:05,500 --> 00:31:08,640 across multiple runs, it consistently beat the 653 00:31:08,640 --> 00:31:11,480 best human submissions. But wait, this brings 654 00:31:11,480 --> 00:31:14,279 up a huge question. If we know that AI is capable 655 00:31:14,279 --> 00:31:17,609 of writing good kernel code, Why go through the 656 00:31:17,609 --> 00:31:19,710 test time training loop at all? Exactly. Why 657 00:31:19,710 --> 00:31:21,829 go through this expensive test time training 658 00:31:21,829 --> 00:31:24,369 loop? Why not just take all the best GPU kernels 659 00:31:24,369 --> 00:31:27,150 in the world, build a massive dataset, and pre 660 00:31:27,150 --> 00:31:29,490 -train the model on that? That is a fantastic 661 00:31:29,490 --> 00:31:32,130 question. Just let it memorize how to write fast 662 00:31:32,130 --> 00:31:34,730 code, and then it can just do it instantly without 663 00:31:34,730 --> 00:31:37,390 needing 50 steps of trial and error. And it is 664 00:31:37,390 --> 00:31:39,630 exactly what the authors investigated in Appendix 665 00:31:39,630 --> 00:31:41,950 H of the paper. Oh, they did test that. They 666 00:31:41,950 --> 00:31:44,210 established what they called a general post -training 667 00:31:44,210 --> 00:31:46,630 baseline. And they took the base model and they 668 00:31:46,630 --> 00:31:49,109 fine -tuned it extensively on kernel bench, which 669 00:31:49,109 --> 00:31:52,210 is a large comprehensive data set of GPU kernel 670 00:31:52,210 --> 00:31:54,769 optimization problems and solutions. Okay, so 671 00:31:54,769 --> 00:31:57,630 they built a GPU expert model. What happened 672 00:31:57,630 --> 00:31:59,789 when they entered that model into the Trimel 673 00:31:59,789 --> 00:32:01,910 challenge? Post -training definitely improved 674 00:32:01,910 --> 00:32:04,309 the model's baseline capabilities. It improved 675 00:32:04,309 --> 00:32:06,690 its ability to write valid, compilable code without 676 00:32:06,690 --> 00:32:09,710 syntax errors. But did it win? Crucially, its 677 00:32:09,710 --> 00:32:11,849 absolute best runtime still failed to beat the 678 00:32:11,849 --> 00:32:14,480 top human submissions. OK, we've seen abstract 679 00:32:14,480 --> 00:32:17,400 math and we've seen low -level hardware optimization. 680 00:32:18,099 --> 00:32:20,339 Now I want to see how this applies to industrial 681 00:32:20,339 --> 00:32:23,720 algorithms and messy real -world data. Let's 682 00:32:23,720 --> 00:32:26,759 talk about algorithm design and biological benchmarks. 683 00:32:27,220 --> 00:32:29,980 Let's start with algorithm design. To test this, 684 00:32:30,240 --> 00:32:33,160 they evaluated the model on past at -coder heuristic 685 00:32:33,160 --> 00:32:36,440 contests. What are those? These are famous, highly 686 00:32:36,440 --> 00:32:38,339 competitive algorithm engineering challenges 687 00:32:38,339 --> 00:32:41,180 where programmers write scripts to solve complex 688 00:32:41,180 --> 00:32:43,859 logistical puzzles. They looked at two specific 689 00:32:43,859 --> 00:32:46,180 contests, right? What were the challenges? The 690 00:32:46,180 --> 00:32:50,480 first was the AHC 039 contest. This was a computational 691 00:32:50,480 --> 00:32:52,720 geometry problem about designing the optimal 692 00:32:52,720 --> 00:32:55,779 layout for fishing nets in a grid. Fishing nets? 693 00:32:56,339 --> 00:32:58,619 Yeah, trying to maximize the catch while carefully 694 00:32:58,619 --> 00:33:00,920 avoiding penalties for overlapping or inefficient 695 00:33:00,920 --> 00:33:03,579 shapes. Okay, interesting. And the second one? 696 00:33:03,809 --> 00:33:07,190 The second was the AHC058 contest, which was 697 00:33:07,190 --> 00:33:09,490 a complex scheduling and resource allocation 698 00:33:09,490 --> 00:33:11,890 problem involving upgrading production capacity 699 00:33:11,890 --> 00:33:14,650 over time to meet shifting demands. Heavy logistics. 700 00:33:15,190 --> 00:33:19,369 And how did the TTT Discover algorithm fare against 701 00:33:19,369 --> 00:33:23,230 the humans who competed? In both cases, the final 702 00:33:23,230 --> 00:33:25,650 algorithm generated by TTT Discover would have 703 00:33:25,650 --> 00:33:28,130 scored first place on the leaderboard. First 704 00:33:28,130 --> 00:33:30,529 place? But wait, let's be careful with the phrasing 705 00:33:30,529 --> 00:33:34,380 here. You said would have. Yes. crucial detail 706 00:33:34,380 --> 00:33:37,140 we must state clearly. These comparisons concern 707 00:33:37,140 --> 00:33:40,299 past contests evaluated retrospectively. Oh, 708 00:33:40,400 --> 00:33:43,480 so this wasn't a live competition. Right. This 709 00:33:43,480 --> 00:33:46,119 was not a live ongoing competition win where 710 00:33:46,119 --> 00:33:48,480 the AI was competing in real time under the same 711 00:33:48,480 --> 00:33:51,279 time constraints against humans. The researchers 712 00:33:51,279 --> 00:33:54,539 took the historical data, ran the AI, and compared 713 00:33:54,539 --> 00:33:56,579 its final score to the historical leaderboard. 714 00:33:56,730 --> 00:33:59,230 Still, beating the historical winning scores 715 00:33:59,230 --> 00:34:01,970 generated by top human engineers is a massive 716 00:34:01,970 --> 00:34:04,789 validation. It is. But how did it compare to 717 00:34:04,789 --> 00:34:07,329 other AIs, like OpenEvolve, which is built for 718 00:34:07,329 --> 00:34:09,789 this kind of thing? TTT Discover beat the AI 719 00:34:09,789 --> 00:34:12,690 baselines as well, setting new high scores. However, 720 00:34:13,050 --> 00:34:15,110 in the interest of a rigorous fairness check, 721 00:34:15,750 --> 00:34:17,750 the authors explicitly noted a limitation in 722 00:34:17,750 --> 00:34:20,349 their comparison to OpenEvolve. What was the 723 00:34:20,349 --> 00:34:23,570 limitation? OpenEvolve relies heavily on prompting. 724 00:34:23,650 --> 00:34:26,530 And as it iterated on the algorithms, it suffered 725 00:34:26,530 --> 00:34:30,250 from context window truncation. Context window 726 00:34:30,250 --> 00:34:33,530 truncation. Meaning the prompt just got too long 727 00:34:33,530 --> 00:34:37,150 for the AI's memory. Essentially, yes. Yet evolutionary 728 00:34:37,150 --> 00:34:39,429 prompts containing the history of the code grew 729 00:34:39,429 --> 00:34:42,309 so large that they exceeded the model's maximum 730 00:34:42,309 --> 00:34:44,550 memory capacity. So the beginning of the prompt 731 00:34:44,550 --> 00:34:48,150 got cut off. Exactly. This severely limited open 732 00:34:48,150 --> 00:34:50,989 evolves performance in the later stages. So while 733 00:34:50,989 --> 00:34:54,289 TTT Discover 1, it wasn't a perfectly level playing 734 00:34:54,289 --> 00:34:56,989 field against that specific baseline due to memory 735 00:34:56,989 --> 00:35:00,309 limits. OK. Let's move to the messiest data of 736 00:35:00,309 --> 00:35:04,150 all. Biology. The paper highlights single cell 737 00:35:04,150 --> 00:35:07,150 RNA sequencing. This sounds incredibly complex 738 00:35:07,150 --> 00:35:09,409 and intimidating. Let's break it down. It is 739 00:35:09,409 --> 00:35:12,030 a complex field. They tested the model on the 740 00:35:12,030 --> 00:35:15,170 open problems denoising benchmark. Okay, what 741 00:35:15,170 --> 00:35:18,710 is single cell RNA sequencing? It is a biological 742 00:35:18,710 --> 00:35:21,469 protocol that isolates individual cells and measures 743 00:35:21,469 --> 00:35:24,590 their gene expression by tagging messenger RNA 744 00:35:24,590 --> 00:35:27,449 with unique chemical identifiers. So it's basically 745 00:35:27,449 --> 00:35:29,949 taking a census of exactly what genes are turned 746 00:35:29,949 --> 00:35:33,349 on inside a single specific cell. Yes. But the 747 00:35:33,349 --> 00:35:35,530 physical and chemical reality of doing this at 748 00:35:35,530 --> 00:35:38,329 a microscopic level means the resulting data 749 00:35:38,329 --> 00:35:41,130 is incredibly noisy and filled with errors. Why 750 00:35:41,130 --> 00:35:44,409 is there so much error? Specifically? It suffers 751 00:35:44,409 --> 00:35:47,050 from a massive issue called technical dropout. 752 00:35:47,789 --> 00:35:50,949 What drops out? The signal. Sometimes a gene 753 00:35:50,949 --> 00:35:53,690 is actively expressing in a cell. It is turned 754 00:35:53,690 --> 00:35:56,530 on. But the delicate chemical sequencing process 755 00:35:56,530 --> 00:35:59,150 just completely misses it. So the data is wrong. 756 00:35:59,489 --> 00:36:01,530 The resulting data matrix that the biologists 757 00:36:01,530 --> 00:36:04,050 look at is full of false zeros. It looks like 758 00:36:04,050 --> 00:36:05,670 the gene is turned off, but it's actually just 759 00:36:05,670 --> 00:36:08,409 a measurement error. Oh, wow. It's like taking 760 00:36:08,409 --> 00:36:10,429 a picture of a starry night, but your camera 761 00:36:10,429 --> 00:36:12,469 sensor is broken, so half the stars just show 762 00:36:12,469 --> 00:36:15,010 up as black pixels. That is an excellent analogy. 763 00:36:15,510 --> 00:36:18,090 You need a denoising algorithm to look at the 764 00:36:18,090 --> 00:36:20,590 surrounding pixels and guess which black spots 765 00:36:20,590 --> 00:36:24,170 are actually supposed to be stars. Exactly. So 766 00:36:24,170 --> 00:36:27,489 in the TTT Discover environment, the state the 767 00:36:27,489 --> 00:36:30,230 AI is generating is a data processing algorithm 768 00:36:30,230 --> 00:36:32,590 written in Python. And what's the reward signal? 769 00:36:32,849 --> 00:36:35,869 The reward signal is the mean squared error or 770 00:36:35,869 --> 00:36:39,789 MSE score of how well that Python algorithm denoises 771 00:36:39,789 --> 00:36:42,630 the biological data compared to a known high 772 00:36:42,630 --> 00:36:45,369 quality benchmark data set. How did they set 773 00:36:45,369 --> 00:36:47,849 up the scientific method for this? Did they just 774 00:36:47,849 --> 00:36:49,949 let it memorize the answers? No, they followed 775 00:36:49,949 --> 00:36:53,190 strict machine learning protocols. The TTT discover 776 00:36:53,190 --> 00:36:55,929 algorithm was optimized, meaning it underwent 777 00:36:55,929 --> 00:36:58,690 its 50 steps of trial, error and wait updates 778 00:36:58,690 --> 00:37:02,170 using one specific data set known as the pancreas 779 00:37:02,170 --> 00:37:05,280 data set. Okay. And then what? After it finished 780 00:37:05,280 --> 00:37:08,619 training, that final algorithm was frozen and 781 00:37:08,619 --> 00:37:11,559 then evaluated on two entirely held out unseen 782 00:37:11,559 --> 00:37:15,579 datasets. The PBMC dataset and the Tabula dataset. 783 00:37:15,980 --> 00:37:19,179 And it won. It beat the existing denoising tools 784 00:37:19,179 --> 00:37:22,659 on the unseen data. Yes. It significantly outperformed 785 00:37:22,659 --> 00:37:25,659 the baseline. To do this, the algorithm it wrote 786 00:37:25,659 --> 00:37:28,380 added techniques like gene -adaptive transform 787 00:37:28,380 --> 00:37:31,769 ensembling and low -rank SVD refinement. Okay, 788 00:37:31,829 --> 00:37:34,909 we need to unpack those terms. Transform ensembling. 789 00:37:35,309 --> 00:37:37,849 Imagine looking at that noisy starry night photo. 790 00:37:38,590 --> 00:37:40,869 Transform ensembling is like the algorithm deciding 791 00:37:40,869 --> 00:37:43,730 to rotate the photo, flip it, adjust the contrast, 792 00:37:44,150 --> 00:37:46,110 make predictions on all those different versions. 793 00:37:46,630 --> 00:37:49,030 And then average the predictions together to 794 00:37:49,030 --> 00:37:52,130 get a more robust answer. Exactly. It looks at 795 00:37:52,130 --> 00:37:55,869 the data from multiple mathematical angles. And 796 00:37:55,869 --> 00:37:59,489 SVD refinement. Singular value decomposition. 797 00:38:00,050 --> 00:38:03,329 SVD is a complex linear algebra technique, but 798 00:38:03,329 --> 00:38:05,710 conceptually, it's a way of compressing data 799 00:38:05,710 --> 00:38:08,010 to find the most important underlying patterns 800 00:38:08,010 --> 00:38:10,809 while throwing away the random static. Oh, I 801 00:38:10,809 --> 00:38:13,269 see. The AI essentially wrote a script that said, 802 00:38:13,769 --> 00:38:16,369 find the core mathematical shape of this cell's 803 00:38:16,369 --> 00:38:18,969 behavior and use that shape to fill in the missing 804 00:38:18,969 --> 00:38:21,849 zeros rather than just guessing randomly. And 805 00:38:21,849 --> 00:38:23,510 this was better than what humans were using? 806 00:38:23,789 --> 00:38:26,150 This approach improved upon the well -known magic 807 00:38:26,150 --> 00:38:28,469 algorithm that was the previous standard in the 808 00:38:28,469 --> 00:38:31,639 field. This result is evidence of the algorithm's 809 00:38:31,639 --> 00:38:34,860 benchmark performance. It strictly does not prove 810 00:38:34,860 --> 00:38:37,280 broad biological validity for downstream medical 811 00:38:37,280 --> 00:38:39,699 tasks. But it fixed the data. We absolutely cannot 812 00:38:39,699 --> 00:38:42,000 claim the model does or doesn't know what a cell 813 00:38:42,000 --> 00:38:45,340 is. It doesn't have a PhD in biology. It is a 814 00:38:45,340 --> 00:38:48,099 language model optimizing Python code to minimize 815 00:38:48,099 --> 00:38:51,340 a mathematical loss function called mean squared 816 00:38:51,340 --> 00:38:54,579 error. Right. It just sees a matrix of numbers 817 00:38:54,579 --> 00:38:57,800 and a score that goes up or down. Exactly. Better 818 00:38:57,800 --> 00:39:00,860 numerical scores on a mean squared error benchmark 819 00:39:00,860 --> 00:39:03,619 do not establish biological truth by themselves. 820 00:39:05,079 --> 00:39:08,079 Sometimes an algorithm can get a better MSE score 821 00:39:08,079 --> 00:39:11,079 by overly smoothing the data, which might look 822 00:39:11,079 --> 00:39:13,539 mathematically clean, but could actually erase 823 00:39:13,539 --> 00:39:16,760 a rare vital biological signal that a real doctor 824 00:39:16,760 --> 00:39:21,119 is looking for. A biologist would still need 825 00:39:21,119 --> 00:39:23,539 to take this AI -generated algorithm, run it 826 00:39:23,539 --> 00:39:26,179 on real samples, and validate its findings in 827 00:39:26,179 --> 00:39:28,780 a wet lab before we can say it has discovered 828 00:39:28,780 --> 00:39:31,579 biological truth. That is a very important distinction 829 00:39:31,579 --> 00:39:33,619 to keep this grounded. It's a brilliant math 830 00:39:33,619 --> 00:39:36,659 machine doing complex math on a biological data 831 00:39:36,659 --> 00:39:39,079 set. It didn't cure a disease, it just smoothed 832 00:39:39,079 --> 00:39:41,039 out a matrix better than the last script did. 833 00:39:41,280 --> 00:39:43,980 Exactly. We must separate the optimization of 834 00:39:43,980 --> 00:39:46,820 a proxy metric from the reality of the physical 835 00:39:46,820 --> 00:39:49,489 sciences. Okay, so before we conclude our deep 836 00:39:49,489 --> 00:39:51,309 dive, a good scientist always takes the engine 837 00:39:51,309 --> 00:39:53,789 apart to see which pieces actually matter. We 838 00:39:53,789 --> 00:39:56,230 need to look at the ablation studies in the paper. 839 00:39:56,449 --> 00:39:59,050 Yes, the ablations. Ablation is basically the 840 00:39:59,050 --> 00:40:00,889 process of turning off different parts of the 841 00:40:00,889 --> 00:40:03,349 AI system one by one to see what breaks, right? 842 00:40:03,530 --> 00:40:05,510 It tells you what is actually doing the heavy 843 00:40:05,510 --> 00:40:08,170 lifting. Precisely. Let's look at the ablation 844 00:40:08,170 --> 00:40:11,710 realities. First, what happens if you turn off 845 00:40:11,710 --> 00:40:14,750 the entropic objective? The thing that aggressively 846 00:40:14,750 --> 00:40:17,489 chases the Michelin star outliers. Right. and 847 00:40:17,489 --> 00:40:19,590 replace it with the standard expected reward 848 00:40:19,590 --> 00:40:22,289 objective that typical RL uses. Does the whole 849 00:40:22,289 --> 00:40:25,110 thing crash? It doesn't crash, but the improvement 850 00:40:25,110 --> 00:40:28,269 is just much, much slower. It eventually made 851 00:40:28,269 --> 00:40:31,090 progress, but the Entropic objective was clearly 852 00:40:31,090 --> 00:40:33,929 the primary driver for rapid breakthrough discovery 853 00:40:33,929 --> 00:40:36,909 within the tight 50 -step budget. What happens 854 00:40:36,909 --> 00:40:38,929 if you turn off the weight updates entirely, 855 00:40:39,110 --> 00:40:41,190 like if you freeze the model, but you let it 856 00:40:41,190 --> 00:40:43,929 keep the PUCT search algorithm and the buffer 857 00:40:43,929 --> 00:40:47,960 of past states? In this specific setup, it stagnates. 858 00:40:48,239 --> 00:40:50,719 Really? Even with the smart search? Without the 859 00:40:50,719 --> 00:40:52,760 gradient updates altering the internal weights, 860 00:40:53,179 --> 00:40:55,139 the search process just couldn't break through 861 00:40:55,139 --> 00:40:58,500 the performance ceiling. It got stuck rearranging 862 00:40:58,500 --> 00:41:01,380 its existing ideas rather than synthesizing a 863 00:41:01,380 --> 00:41:03,820 fundamentally better one. Now, what about that 864 00:41:03,820 --> 00:41:06,900 complex PUCT search rule, the one bouncing the 865 00:41:06,900 --> 00:41:09,840 child rewards, the historical rank, and the exploration 866 00:41:09,840 --> 00:41:12,739 bonus for unvisited states? What if we remove 867 00:41:12,739 --> 00:41:15,150 it? Yeah, what if you replace that intricate 868 00:41:15,150 --> 00:41:17,650 math with something really simple like epsilon 869 00:41:17,650 --> 00:41:20,989 greedy reuse? Epsilon greedy is a classic algorithm. 870 00:41:21,610 --> 00:41:24,469 It basically means 90 % of the time just pick 871 00:41:24,469 --> 00:41:26,610 the absolute best state from the buffer, but 872 00:41:26,610 --> 00:41:29,250 10 % of the time pick a completely random state 873 00:41:29,250 --> 00:41:31,329 just to see what happens. Right, it's the most 874 00:41:31,329 --> 00:41:34,090 basic form of exploration. Does using that break 875 00:41:34,090 --> 00:41:37,469 the TTT discover system? This is a fascinating 876 00:41:37,469 --> 00:41:39,710 correction to a common assumption. You might 877 00:41:39,710 --> 00:41:42,829 think losing the fancy PUCT rule would destroy 878 00:41:42,829 --> 00:41:45,349 the system's efficiency. I assumed it would. 879 00:41:45,590 --> 00:41:48,070 But the paper clarifies that simple epsilon greedy 880 00:41:48,070 --> 00:41:50,989 reuse actually works reasonably well. Really? 881 00:41:51,309 --> 00:41:53,489 Just occasionally picking random things works. 882 00:41:53,869 --> 00:41:56,510 Yes. Especially if the model happens to stumble 883 00:41:56,510 --> 00:42:00,269 on a lucky kernel or a good heuristic early on 884 00:42:00,269 --> 00:42:02,710 in the search process. Oh, because if it finds 885 00:42:02,710 --> 00:42:06,309 a good path early... Epsilon Greedy just exploits 886 00:42:06,309 --> 00:42:09,289 it heavily. Exactly. And the weight updates do 887 00:42:09,289 --> 00:42:11,809 the rest. It is certainly not a catastrophic 888 00:42:11,809 --> 00:42:15,369 failure to use a simpler search heuristic. So 889 00:42:15,369 --> 00:42:18,329 the search rule helps, but it's not the secret 890 00:42:18,329 --> 00:42:22,829 sauce? The PUCT rule is more mathematically robust 891 00:42:22,829 --> 00:42:25,690 across many runs, but the learning objective 892 00:42:25,690 --> 00:42:29,190 is the true star of the show. That's funny. Sometimes 893 00:42:29,190 --> 00:42:31,469 simple random exploration is good enough if you 894 00:42:31,469 --> 00:42:34,420 have a powerful learning engine behind it. But 895 00:42:34,420 --> 00:42:37,519 here's the big question, one I have been wondering 896 00:42:37,519 --> 00:42:41,239 this whole time. Catastrophic forgetting. Ah, 897 00:42:41,599 --> 00:42:44,780 the bane of continual learning. Right. If the 898 00:42:44,780 --> 00:42:47,219 model spends 50 steps aggressively obsessing 899 00:42:47,219 --> 00:42:50,159 over a GPU kernel. Literally changing its internal 900 00:42:50,159 --> 00:42:53,260 weights to become the world's absolute best Trimal 901 00:42:53,260 --> 00:42:56,380 GPU coder. Right, focusing exclusively on memory 902 00:42:56,380 --> 00:42:58,559 bandwidth. Does it forget how to do basic math? 903 00:42:58,800 --> 00:43:01,099 Does it overwrite its general knowledge base 904 00:43:01,099 --> 00:43:03,800 to make room for the GPU stuff? It's a very valid 905 00:43:03,800 --> 00:43:06,579 concern. When neural networks learn new things, 906 00:43:06,840 --> 00:43:09,400 they tend to catastrophically overwrite old connections. 907 00:43:09,579 --> 00:43:12,019 So did they test for it? The researchers ran 908 00:43:12,019 --> 00:43:14,380 the exact experiment to test this. They took 909 00:43:14,380 --> 00:43:17,159 a TTT Discover checkpoint that had been heavily 910 00:43:17,159 --> 00:43:19,300 trained and updated on the ERDOS math problem. 911 00:43:19,760 --> 00:43:22,059 Okay. And they took another checkpoint that had 912 00:43:22,059 --> 00:43:24,119 been fully trained on the Trimol GPU problem. 913 00:43:24,539 --> 00:43:27,159 Then they cross -evaluated them. They tested 914 00:43:27,159 --> 00:43:30,199 the GPU optimized model on the AC1 and AC2 math 915 00:43:30,199 --> 00:43:33,480 tasks to see if its math skills had degraded. 916 00:43:33,639 --> 00:43:38,000 Exactly. And? Did the GPU model forget how to 917 00:43:38,000 --> 00:43:42,110 do math? The result was that no significant catastrophic 918 00:43:42,110 --> 00:43:45,059 forgetting was observed. Wow. The models retain 919 00:43:45,059 --> 00:43:47,079 their baseline capabilities on the alternative 920 00:43:47,079 --> 00:43:50,619 tasks despite being heavily optimized for a single 921 00:43:50,619 --> 00:43:53,559 unrelated problem. Okay, speaking of the 120 922 00:43:53,559 --> 00:43:56,860 billion parameter behemoth that requires massive 923 00:43:56,860 --> 00:44:00,099 data center GPUs to run, does this TTT discover 924 00:44:00,099 --> 00:44:03,460 method only work on those massive models? The 925 00:44:03,460 --> 00:44:05,599 authors tested much smaller base models to answer 926 00:44:05,599 --> 00:44:09,150 that exactly. They tested QUIN 3 8B and QUIN 927 00:44:09,150 --> 00:44:12,909 3 4B models on both the air -dose math task and 928 00:44:12,909 --> 00:44:16,550 the Trimol GPU task. An 8 billion parameter model 929 00:44:16,550 --> 00:44:19,449 is tiny. You could run that on a high -end laptop. 930 00:44:19,730 --> 00:44:23,260 How did it do? The results were mixed, but highly 931 00:44:23,260 --> 00:44:26,159 encouraging. Even the tiny 8B model, when equipped 932 00:44:26,159 --> 00:44:29,400 with TTT Discover, matched or beat some previous 933 00:44:29,400 --> 00:44:31,519 state -of -the -art AI methods in the combinatorial 934 00:44:31,519 --> 00:44:33,960 math problems. That is incredible efficiency. 935 00:44:34,760 --> 00:44:37,139 However, we have to note the limits. The performance 936 00:44:37,139 --> 00:44:39,260 of the smaller model severely degrades on the 937 00:44:39,260 --> 00:44:42,380 complex coding tasks like the Trimol GPU kernel. 938 00:44:42,699 --> 00:44:45,619 Why is that? It seems that writing optimal GPU 939 00:44:45,619 --> 00:44:48,000 code requires a much higher baseline of general 940 00:44:48,000 --> 00:44:50,760 coding intelligence and syntax understanding 941 00:44:50,760 --> 00:44:52,780 that the smaller models just don't possess yet. 942 00:44:53,119 --> 00:44:55,519 Ah, I get it. You can't test time train a model 943 00:44:55,519 --> 00:44:57,960 to be a genius if its baseline is too low to 944 00:44:57,960 --> 00:45:00,320 understand the feedback. Exactly. You can't teach 945 00:45:00,320 --> 00:45:02,940 me quantum physics in 50 steps if I don't know 946 00:45:02,940 --> 00:45:06,309 algebra yet. That makes sense. Okay, one last 947 00:45:06,309 --> 00:45:09,309 point on fairness, the cost -matched baseline. 948 00:45:09,550 --> 00:45:12,250 This is an important one. If TTT Discover is 949 00:45:12,250 --> 00:45:14,909 taking extra compute to calculate gradients and 950 00:45:14,909 --> 00:45:16,909 update weights, is it really fair to compare 951 00:45:16,909 --> 00:45:18,610 it to a frozen model that just generates the 952 00:45:18,610 --> 00:45:21,110 same number of rollouts? That was a major concern. 953 00:45:21,409 --> 00:45:24,869 Right, because TTT Discover used 25 ,600 rollouts 954 00:45:24,869 --> 00:45:27,309 plus the heavy computational cost of backpropagation 955 00:45:27,309 --> 00:45:30,449 and training, the frozen baseline just used 25 956 00:45:30,449 --> 00:45:34,010 ,600 rollouts. That's not equal dollars spent. 957 00:45:34,250 --> 00:45:36,849 The authors address this potential critique directly. 958 00:45:37,809 --> 00:45:40,510 So they compared TTT Discover against a best 959 00:45:40,510 --> 00:45:45,550 of 58 ,880 frozen baseline. Exactly. And did 960 00:45:45,550 --> 00:45:47,630 giving the frozen model double the attempts help 961 00:45:47,630 --> 00:45:50,889 it win? No. Scaling the frozen sampling beyond 962 00:45:50,889 --> 00:45:55,190 25 ,600 yielded little to no improvement. Okay, 963 00:45:55,269 --> 00:45:57,590 let's zoom out and summarize the incredible breadth 964 00:45:57,590 --> 00:46:01,610 of this paper. We have seen an AI bound the convolutions 965 00:46:01,610 --> 00:46:05,110 of 30 ,000 -piece step functions by identifying 966 00:46:05,110 --> 00:46:07,650 active mathematical bottlenecks. We have seen 967 00:46:07,650 --> 00:46:10,630 it leverage Triton to fuse GPU memory operations 968 00:46:10,630 --> 00:46:12,909 and double the speed of human engineers on an 969 00:46:12,909 --> 00:46:16,130 A100 chip. We've seen it write logistics algorithms 970 00:46:16,130 --> 00:46:19,289 to design phishing nets and Python code to denoise 971 00:46:19,289 --> 00:46:22,250 complex RNA sequencing data. And it did all of 972 00:46:22,250 --> 00:46:25,110 this not by memorizing a textbook, but by continuously 973 00:46:25,110 --> 00:46:28,670 updating its own brain during the test. all for 974 00:46:28,670 --> 00:46:32,190 roughly $500 of compute per problem. It is a 975 00:46:32,190 --> 00:46:34,829 profound demonstration of test time reinforcement 976 00:46:34,829 --> 00:46:37,269 learning applied successfully to verifiable, 977 00:46:37,789 --> 00:46:40,110 continuous reward scientific problems. It is 978 00:46:40,110 --> 00:46:42,989 a major step forward. But I want to leave you, 979 00:46:43,050 --> 00:46:45,510 the listener, with an open, lingering question. 980 00:46:45,869 --> 00:46:48,230 This is derived directly from the source's future 981 00:46:48,230 --> 00:46:50,889 work section and the very nature of continual 982 00:46:50,889 --> 00:46:52,429 learning we discussed. That's a big question. 983 00:46:52,670 --> 00:46:54,869 We know the model didn't suffer catastrophic 984 00:46:54,869 --> 00:46:58,230 forgetting, but what... if anything, transfers 985 00:46:58,230 --> 00:47:02,289 to a new problem. This is the absolute frontier 986 00:47:02,289 --> 00:47:05,269 of artificial intelligence research. Great. If 987 00:47:05,269 --> 00:47:07,829 you use test time training on an LLM to discover 988 00:47:07,829 --> 00:47:11,269 a new Erdos bound and it succeeds, did it actually 989 00:47:11,269 --> 00:47:14,570 learn a meta skill about mathematical exploration? 990 00:47:15,090 --> 00:47:18,250 Does a model that just optimized a GPU kernel 991 00:47:18,250 --> 00:47:21,889 become intrinsically better at general algorithm 992 00:47:21,889 --> 00:47:24,380 design? Yeah, like if I handed a completely different 993 00:47:24,380 --> 00:47:26,679 math problem the next day, does it start from 994 00:47:26,679 --> 00:47:29,780 a higher baseline because it learned how to discover? 995 00:47:30,079 --> 00:47:33,119 Or is all that learned value completely locked 996 00:47:33,119 --> 00:47:36,219 away in the specific context of that one single 997 00:47:36,219 --> 00:47:39,860 problem? That is the open question. We must distinguish 998 00:47:39,860 --> 00:47:42,599 what has actually been proven here today. We 999 00:47:42,599 --> 00:47:45,440 have an improved solution for specific tasks. 1000 00:47:45,639 --> 00:47:48,320 We have an improved mathematical bound. And we 1001 00:47:48,320 --> 00:47:51,539 have a general algorithm, TTT Discover, for finding 1002 00:47:51,539 --> 00:47:55,119 them. But the proof of whether the AI can truly 1003 00:47:55,119 --> 00:47:58,000 internalize the abstract art of discovery itself, 1004 00:47:58,639 --> 00:48:01,460 whether it can carry the intuition from one breakthrough 1005 00:48:01,460 --> 00:48:04,219 into a completely unrelated field the way a human 1006 00:48:04,219 --> 00:48:07,559 scientist does, remains an open, unproven frontier. 1007 00:48:08,280 --> 00:48:10,960 The paper does not conclusively settle that question. 1008 00:48:11,440 --> 00:48:13,400 Thank you so much for joining us on this deep 1009 00:48:13,400 --> 00:48:16,039 dive into the mechanics of discovery. Keep questioning 1010 00:48:16,039 --> 00:48:18,639 the results, keep exploring the mechanisms, and 1011 00:48:18,639 --> 00:48:19,860 we will see you next time.