Discussion about this post

User's avatar
Nathan Lambert's avatar

I appreciate you putting this out there, both as a way to pull people who are confused in and as a way for me in so deep in the weeds to see how people at different levels of abstraction see how AI works. That being said, I think you make a few crucial details that will set people up for confusion with where AI is heading.

Analogizing pretraining to cramming doesn't really work. I think this is the core of the critique I have. Pretraining, even though I hate to say this, is far more akin on order of magnitude to the lessons a advanced biological creature has gained through evolution. Cramming is like finetuning. Pretraining is at such a vast scale that studying doesn't begin to cover it.

This, sets the rest of the analogy up for challenge. All *models* are doing a closed book exam. Normal models AND reasoning models. They are just weights drawing on knowledge.

What reasoning models are doing is showing us how inference time compute can improve performance. Here, RL has served as a platform where models can learn to regularly check their work. This is a new behavior that has a lot of value. It's more like taking a closed book exam and just being smarter. You have a tool that you can use to break down the exam problems.

Inference time compute is a general thing. Inference can be spent on many problems.

The real reason this analogy faces challenge is because true, open book exam analogies are coming. These are the new agents, the manus, the deep research, the like where a model that could just give an answer now has an ability to go and get its own information. People need to know that models alone are closed books, but systems and apps are turning it open and will enable such far more powerful experiences.

Cheers Kevin! Hope this helps.

Expand full comment
Tao's avatar
Mar 23Edited

Have been using the car analogy: A model (including reasoning models) is like the engine of a car providing the raw power (some engines are better built than others); Inferencing is like the transmission, channeling that power to make the wheels spin forward/backward and at the right speed; Prompting is like a person driving the car—gasing the pedal, steering the wheel, setting the cruise control. Agents are new animals—not only are they trying to replace people by taking control of the wheel, but they also have antennas (thus make the car open to the outer world) and know (like your mom) what music you like to listen to. But humans still need to be human—they just need to say where they want to go, not give turn-by-turn directions (now or to be). Does the analogy make sense?

Expand full comment
2 more comments...

No posts