On running LLMs locally

So you want infinite intelligence without paying a dollar? That's possible if you run an intelligent model on your laptop.

Right now, the only open model that is intelligent enough and can run on consumer-grade hardware is Gemma 4 31B IT. There are smaller Gemma 4 models, but they are not good for tasks that require reasoning and precision. There are intelligent and open models from other companies, but they don’t run on consumer-grade hardware. Gemma 4 31B IT is the only option worth considering.

But what does consumer-grade even mean here? Which laptop do you need to buy to run Gemma 4 31B IT?

The cheapest laptop/PC that can run Gemma 4 31B IT with a long context window and other concurrent apps is the Mac Mini Pro or MacBook Pro with at least 64GB RAM. Search for their prices in your location; this is the cheapest option.

To conclude: If you are an average person like me with an average laptop, you'll have to wait a few more months or years for a more lightweight model. The current intelligent models aren't light enough.

Until then, you can get a free Gemma 4 31B IT API key from aistudio.google.com. The free tier is generous enough that you can’t use all the free tokens -- unless you use the API for programming-related tasks.