Discussion about this post

User's avatar
Ljubomir Josifovski's avatar

Great write up - thanks.

One thing to add - it's immense joy, all these open source models, and the entire ecosystem that sprung around in short time. With many 1000s of enthusiasts in their homes, that would not have been part of this revolution otherwise. Reminds me of the prior IT revolutions: the 1st the PC revolution in the 1990s that put a computer on every desk (first; then in everyones pocket latter), and of the 2nd Internet revolution in the 2000s that connected (by now) almost 8B people 24/7 at almost zero costs in a global village. That too had bottom-up ecosystems of enthusiasts and tinkerers, learning by doing.

Now checking my disk what OSS AI LLM stuff I use daily. Software - llama.cpp, mlx, pytorch, LMStudio, Jan. Models - new default as of last night https://huggingface.co/unsloth/dots.llm1.inst-GGUF. Previous favourite was Qwen3-30B-A6B-16-Extreme-128k-context-Q6_K-GGUF, and few other MoEs variants of Qwen3-30B-A3B. With runner up OpenBuddy-R1-0528-Distill-Qwen3-32B-Preview2-QAT.Q8_0.gguf - but that one is non-MoE so slower for me on a MBP with lots of VRAM but lacking Nvidia gpu.

Looking forward, anticipating GGUF quants waiting to try https://huggingface.co/MiniMaxAI/MiniMax-M1-80k and https://huggingface.co/moonshotai/Kimi-VL-A3B-Thinking-2506. All Chinese. Only non-China labs I see I got are Jan-nano-128K, and Mistral magistral but they release small one not even the medium ones - se meh.

Expand full comment
Nathan Lambert's avatar

Take my word on this matter too!

Expand full comment

No posts