Edge0-145B Brings a 145B Open Model to One GPU: What You Actually Need to Run It
Edge0's new 145B open-weight model runs on one consumer NVIDIA GPU with about 10B active parameters per token, but today's release needs CUDA 13 on Linux or Windows, not a Mac.
Edge0 has released Edge0-145B, a 145-billion-parameter open-weight model that the company says can run on a single personal computer rather than a cloud server. A smaller sibling, Edge0-120B, shipped alongside it, together with the inference engine needed to run both. The company announced the release on November 12 and posted the weights and code on November 13.
The headline claim is bold: frontier-class coding and agent skills on a laptop, with about 29GB of peak memory and more than 80 tokens per second of output. The release files tell a more specific story. Today the two big models are built for Linux or Windows machines with an NVIDIA GPU and CUDA 13, and the company's own model card notes that its Apple measurements "do not establish macOS release availability". If you want to try it this week, the GPU in your machine matters far more than the word "laptop".
Key facts at a glance
- Size: 145 billion total parameters, with about 10.12 billion active for each token, according to the model card.
- Design: a "latent" mixture-of-experts model with 48 layers and 512 experts per layer, of which 10 routed experts plus 1 shared expert work on each token.
- Context: up to 256K tokens, with the key-value cache stored in 8-bit integers.
- Formats: two builds, Q3_K at 40.81 GiB and NVFP4 at 47.79 GiB. Both were trained natively at 4 bits rather than compressed afterwards.
- Base: both models are derived from Qwen3.8-Flash-Next, so Qwen's licence terms apply on top of Edge0's own.
- Hardware today: an NVIDIA GPU of compute capability 7.5 or newer (RTX 20, 30, 40 or 50 series) with at least 16 GiB of video memory, plus 20 GiB of system memory and the CUDA Toolkit 13.
Background: why big local models have been rare
Until recently, models meant to run on phones and laptops stayed small. Edge0's announcement points out that Apple's on-device foundation model has roughly 3 billion parameters and that Google's Gemini Nano came in 1.8B and 3.25B versions. Even after heavy compression, a 35B model can need around 20GB of memory, which is more than most phones can spare.
Edge0 has been pushing that ceiling up in steps. It released Edge0-35B for phones on September 10, and the company says that model runs at more than 37 tokens per second in about 2.5GB of memory on an iPhone 18 Pro Max. On November 4 it replaced its older Python framework and four separate platform apps with a single C++ engine called edge0-infer. The 145B and 120B models are the next step up, aimed at desktops and laptops rather than phones.
How it fits a 145B model into consumer hardware
The trick is that most of the model never sits in fast memory at once. Edge0 describes four design choices that work together.
- Shared matrices across experts. Each group of 64 experts shares two large matrices, and each expert keeps only a small set of its own parameters. That cuts the amount of data stored and moved without reducing the number of experts.
- Built for local use from the start. The company says it trained the model in its on-device form from day one, instead of shrinking a cloud model afterwards.
- Three storage tiers. Frequently used "hot" experts stay on the GPU. An expert library of roughly 40GB sits in system memory and is pulled onto the GPU when needed. A 51.2-billion-parameter n-gram embedding table, stored in FP8 and about 48GB in size, stays on the SSD and is read on demand.
- CPU and GPU together. The GPU computes cached experts while the CPU handles the rest. A 1.26B "draft" head proposes several tokens at once for speculative decoding, so each expensive weight read produces more output.
In practice, that means your SSD speed and system memory become part of the performance picture. The model card offers low-memory switches such as --mmap-experts, which reads uncached experts straight from the SSD, and --resident-budget-gib 20, which caps the host-memory expert cache at 20 GiB. Both trade speed for a smaller memory footprint.
What the speed numbers really say
Edge0 published throughput for several machines. The most flattering figures come from a desktop card, not a laptop.
| Machine and build | 16K prefill (tok/s) | Chat decode (tok/s) | Agent decode (tok/s) |
|---|---|---|---|
| RTX 5090 D 24GB, Q3_K | 3,554.9 | 99.7 | 121.2 |
| RTX 4090 24GB, Q3_K | 3,179.4 | 69.9 | 91.7 |
| RTX Spark N1X 60GiB, NVFP4 | 922.4 | 40.1 | 46.0 |
| Apple M2 Ultra (Metal), Q3_K | 800.3 | 60.9 | 57.0 |
| Capped at 16GiB VRAM + 20GiB RAM, Q3_K | 335 | 36.8 | 44.9 |
Three points stand out. First, the "121 tokens per second" headline is on an RTX 5090 D with 24GB of video memory, a high-end desktop graphics card. Second, the constrained row is the same RTX 5090 D artificially limited to 16GiB of video memory and 20GiB of system memory. Edge0 calls these test budgets, not minimum requirements, but they are the closest guide to what a mid-range machine might see: under 40 tokens per second in chat. Third, the Apple result was measured with Metal, yet the release does not currently ship a macOS build for the 145B and 120B models. The GitHub platform table lists macOS support only for the older 35B model.
The launch post also quotes about 120W of typical power draw. That is plausible for a gaming laptop plugged into the wall, but not for long sessions on battery.
The benchmark claims
On Edge0's own tests, Edge0-145B scored 64.6 on the DeepSWE 1.1 coding benchmark, ahead of Qwen3.8-Flash-Next at 58.7 and DeepSeek-V4-Flash-0731 at 54.4. It scored 86.52 on Terminal-Bench 2.1, 90.4 on GPQA-Diamond and 80.6 on the IFBench instruction-following test. Across six benchmarks it averaged 72.8, against 71.7 for Qwen3.8-Flash-Next.
Two caveats matter. The Edge0, Qwen, DeepSeek and Claude results all come from Edge0's internal evaluations, and the company says full reproduction instructions will arrive with its technical report. And the gains are uneven: on the hard HLE reasoning test, Edge0-145B scored 35.4, slightly below the Qwen model it was derived from at 35.9. The fair reading is that Edge0 kept most of its parent model's ability while cutting the memory needed to run it. Edge0 puts that at 29GB measured peak, against an estimated 70GB for a standard 4-bit deployment of Qwen3.8-Flash-Next.
The licence is not fully open
The engine code is Apache-2.0, and the Strata engine it builds on keeps its MIT licence. The model weights are a different matter. They fall under the Edge0 Community Model License 1.0, and the Qwen Community License also still applies.
According to the model card, the Edge0 licence allows personal use, research, modification, noncommercial sharing, internal use and commercial use of the model's outputs. Selling or bundling the model, running a commercial model service, or offering an independent commercial coding or office assistant built on it requires a separate written licence from Edge0. Startups that plan to ship a product on top of these weights should read the licence file before writing code.
What to do if you want to try it
- Check your GPU first. You need an NVIDIA card of the RTX 20 series or newer with at least 16 GiB of video memory, and the CUDA Toolkit 13 installed. Without one, the 145B and 120B models will not run with today's release.
- Plan for disk and memory. The weights alone are 40.81 GiB (Q3_K) or 47.79 GiB (NVFP4). Put them on a fast SSD, because some data is read from disk during generation.
- Start with the Q3_K build. It is smaller and was faster than NVFP4 on the RTX 5090 D in Edge0's tests.
- Use the local server.
edge0-infer serveexposes OpenAI-compatible and Anthropic Messages-compatible endpoints on your own machine, so existing tools and coding agents can often point at it with a changed base URL. - Test on your own work. Run a handful of real tasks from your codebase before trusting the benchmark table, and compare against the cloud model you use today.
- Mac users should wait. Apple Silicon results exist, but the 145B release does not yet ship for macOS.
What it means
Edge0-145B is a meaningful step for local AI. A model of this size running at usable speed on one consumer GPU, with weights you can download, changes the trade-off for developers who want privacy, offline access or predictable costs. Edge0 says local inference adds no per-token cloud fees, although hardware and electricity still cost money.
The marketing gets ahead of the release notes, though. Today, "runs on a laptop" means a high-end Windows or Linux machine with a strong NVIDIA GPU, not any recent notebook. The benchmarks are the company's own, and the licence limits commercial use of the weights. For developers with the right hardware, it is well worth an afternoon of testing. For everyone else, it is a clear sign of where local models are heading over the next year.