XT.PT Local AI → This story
Filed

Updated 07:45
Reporting
Prelo
Verified by Roger Morais
4 min · 710 words
News Local AI

Framework stops pointing at 128GB

Which open-weight models fit in each Strix Halo memory tier, why mixture-of-experts wins on this hardware, and the one model whose lookup table breaks the unified-memory assumption.

Filed07 Sep 2026, 09:45 UTC Length4 min · 710 words ReportingPrelo
frame.work

When the Framework Desktop launched, the 128GB configuration was the one the local-AI crowd talked about, and Framework's own blog said so. A September 2 post by Donato Capitella on that blog now argues the opposite case for the two smaller tiers, and the reason is not marketing. It is memory prices.

"Fast forward to September 2026, and memory supply constraints across the industry have pushed up prices, including for the Framework Desktop. Fortunately, this has coincided with the release of new, smaller open-weight models that are well suited to the 32GB and 64GB configurations, making the less expensive versions more capable and helping to offset the impact of higher memory prices." The product page underlines the pressure from the other side: as of this reading, the 128GB option is listed as "coming soon".

The three boxes

Framework's desktop page lists three configurations. The 32GB model uses the Ryzen AI Max 385 with an 8-core, 16-thread CPU. The 64GB and 128GB models use the Ryzen AI Max+ 395 with 16 cores and 32 threads. All three use LPDDR5x-8000 soldered to the mainboard, so the tier you buy is the tier you keep. The post refers to the platform by AMD's codename, Strix Halo.

The constraint that shapes every recommendation in the post is bandwidth, not capacity. "Strix Halo can hold large models in unified memory, but its 256GB/s memory bandwidth limits how quickly the GPU can read the weights. This is why mixture-of-experts models are often a better choice when speed matters." The worked example is a pair of Qwen models: "Qwen3.6-35B-A3B is a good example. It has 35 billion parameters in total but uses only about 3 billion for each token, while the dense Qwen3.8-27B uses all 27 billion. That is why Qwen3.6 can run much faster even though its GGUF is larger."

What fits where

The recommendations are dated ("as of August 2026") and mostly use Unsloth's Dynamic GGUF quantizations.

  • 32GB: "Qwen3.8-27B with the Unsloth Dynamic four-bit quantization." The author says its 17.6GB file "leaves enough memory for the operating system, the runtime, and a useful context window", and offers Qwen3.6-35B-A3B for simpler tasks where speed matters.
  • 64GB: the same two models "at higher precision", or Qwen3.5-122B-A10B, whose "52.5GB Q3 file fits, but leaves less memory for context than the smaller models." The post frames this tier as a choice among "a larger model, higher precision, more context, or more than one model loaded at once."
  • 128GB: "DeepSeek V4 Flash 0731 with the 90.9GB Unsloth Dynamic two-bit GGUF", with Inkling-Small as the multimodal alternative at 82.3GB for two-bit or 107GB and 119GB for three-bit.

One model shows why unified memory is not simply a bigger GPU. Qwen3.8-Flash-Next carries "a 51-billion-parameter n-gram embedding table" that Qwen designed to live in main memory or on disk. On a discrete GPU that table sits in system RAM. "On Strix Halo, the GPU and system RAM are the same unified pool, so keeping the table in main memory does not directly solve the 64GB fit." Support landed in upstream llama.cpp on August 27, and the author says local results are pending.

How the numbers were taken

The benchmark conditions are stated: "concurrency one, a 2,048-token prompt, and 128 generated tokens. The figures are measured at the start of context." The engines compared are upstream llama.cpp with either the ROCm or Vulkan backend, plus Strix Halo-specific forks, one of which "includes Q4_0_ROCMI4, which targets a four-bit matrix-multiplication path specific to Strix Halo. It trades some precision for speed". The result tables themselves are published as images, so they are not reproduced here; the author says the same models will be tracked with growing context depth on a separate benchmarks site.

The conclusion reverses the launch-era assumption without disowning the top tier: "Although the 128GB configuration is still one of the best options, it is no longer the only configuration worth considering for local AI". For a reader deciding between a 32GB box now and a 128GB box that is not currently orderable, that is the useful sentence.

Primary sources: Choosing a Framework Desktop for Local AI: 32GB, 64GB, and 128GB, Framework Desktop product page, read 2026-09-04.

Corrections and source documents: contact the desk
Read next →
Read next
Silicon · 4 min

The carbon that wasn't in the chip

Hot Chips · 4 min

Intel's Hot Chips slate: 16 channels at 12800 MT/s, and UCIe in an entry-level laptop chip