Local AI vs. Cloud AI: When Each One Makes Sense

There's no one answer that fits every business or every hobbyist. Anyone who says local AI is always better is probably selling you something. So is anyone who says cloud is always better.

Technology is a tool. It isn't good or bad on its own. The person using it decides whether it works. Here's how I'd decide.

Where Local Makes Sense

Local here means the model runs on your machine. Three reasons I'd pick it:

You keep the data. Client proposals, pricing sheets, and personal writing never leave your computer.

The cost is predictable. Local AI starts with a hardware purchase. After that, it's electricity. No per-token pricing. No surprise overages. No tier upgrades.

Nothing changes on you. A cloud provider can change pricing, rotate models, or shut down without warning. A local model behaves the same today, next month, and next year.

Where Cloud Makes Sense

Cloud still wins in a few places, and I won't pretend otherwise.

It's faster. A dedicated GPU cluster will beat any consumer machine.

The quality ceiling is higher. The biggest models don't fit on consumer hardware yet.

If you've got one headline or mission sentence where every word has to land, cloud is worth the trip. Use it as a spot tool for that final polish.

Four Real Examples

A shop owner sends 30 vendor emails a day through a cloud model. At a few cents a prompt, that's about $12 a week. More than $600 a year. Local.

A contractor is pricing a roughly $200,000 renovation. The scope of work and the numbers are private. Local.

A veteran is drafting a small business grant application. That's personal writing and sensitive details. Local.

You need one headline or mission sentence to land on the first try. Cloud, for that one job.

What People On X Are Showing Off (And What You Actually Need)

Scroll X for local AI and you'll see some big machines. Some of them cost $6,000 or more and carry 128GB of memory or beyond. Some of the posts are useful. Some are a flex. Most small business owners and veterans don't need that kind of machine, so let me sort out what's worth paying attention to.

I haven't tested any of these claims myself. These are other people's reports, and I'm sharing them as such.

Speed is improving fast. One user, @adrgrondin, reports a 27B model doing an agentic task at about 120 tokens per second on an M5 Max MacBook Pro in LM Studio. He says it ran around 20 tokens per second at launch a little over a month earlier, so about 6x faster now. Local keeps getting quicker, and that matters more than any single benchmark.

Hardware can hold its value. Another user, @jun_song, reports selling a 5 month old M5 Max MacBook with 128GB for $8k after buying it for $6k with AppleCare+, after running local AI on it around the clock. He says people told him subscriptions were cheaper than hardware. That's one person's result, and it isn't a promise. It does show why the used market is worth a look.

Memory is the limit. One user, @mfranz_on, posted about how hard a laptop with 24GB of RAM struggled to run a top model. That matches what I've said all along: memory is the bottleneck, and the model has to fit with room to spare.

The top end gets expensive. @jmacftw says that to fully cut the cord from cloud AI, you need around 256GB of memory and roughly $10K or more. Maybe so, if your goal is to never touch the cloud. Mine isn't, and yours probably isn't either.

Both sides have a point. @UnspokenNames argued that a DGX Spark is slow and you could rent cloud GPUs for years for the same money. That's a fair counterpoint. And @Youssofal_ says a 128GB M5 Max Mac gets 85GB or more used on an average work day. Heavy users really do need heavy machines.

Here's what I take from all of it. The loudest posts are about the top of the market. Your work probably lives lower down.

What Local Actually Costs

Memory is the bottleneck. A 35B model needs about 21GB, plus roughly 8GB of headroom for everything else.

Memory prices are also up. A 32GB DDR5 kit now runs $390 to $440. That's why a used Mac is the practical path right now. A used MacBook Pro with 36GB runs about $1,200 to $2,000, and it's the sweet spot for 14B to 35B models.

Here's the math from that shop owner. About $600 a year in cloud prompts, or a one-time machine that does everything else too.

Common Mistakes

Buying more machine than the work needs. If your tasks are drafting, summarizing, and research, you don't need a flagship.

Ignoring headroom. The model isn't the only thing in memory. Your system and apps need room too.

Expecting local to beat the biggest cloud models. It won't, at least not yet. Local is great for daily work. Cloud still has the higher ceiling.

A Simple Way To Decide

Answer yes or no:

1. Does my work include client, pricing, or personal info I don't want on someone else's servers?
2. Am I using AI often enough that per-prompt costs add up?
3. Do I need the tool to behave the same next month?

The more yes answers, the more local makes sense.

Leaning local? Start with a used 36GB MacBook Pro and an app like Ollama or LM Studio. Both keep your data on your machine.

Leaning cloud? Watch the pricing. Per-prompt costs add up, tiers change, and models change without your say.

One more question: is this a one-off where polish matters more than cost or privacy? If yes, use cloud for that one job.

Pick Your Setup

Here's the quick version by situation.

A few prompts a week, nothing private: stay on cloud. No hardware needed.

Private work and a tight budget: the budget tier, a used M1 MacBook Air with 16GB at $400 to $800. It runs 7B to 14B models for simple tasks and drafting.

Daily use and you're serious about it: the mid-range tier, a used MacBook Pro with 36GB at $1,200 to $2,000. This is the sweet spot.

Heavy daily use and bigger models: the serious tier, a used M2 Max or M3 Max with 64GB to 96GB at $2,000 to $3,000 or more.

Need one great sentence: use cloud for that one job, whatever else you run.

Who Shouldn't Bother

If you send a few prompts a week and none of it is private, local probably isn't worth it. Stay on cloud. You'd be buying hardware to solve a problem you don't have.

The Bottom Line

You don't have to pick a side. Local for daily work and private stuff. Cloud for the occasional polish pass. That isn't a compromise. It's using the right tool for the job.

For the full hardware tiers, read The Case for Local AI.

Here's my question: what's one job you'd keep off someone else's servers?