A Reddit homelab builder ran Moonshot AI's 2.8-trillion-parameter Kimi K3 AI model using discontinued second-gen Intel Optane PMem 200 memory with a 40-core 3rd-gen Xeon Scalable CPU and no GPU for inference. The builder chose Optane because it costs "a fraction" of equivalent DRAM, which has seen prices skyrocket. The project follows an earlier build by a separate user who ran a 1-trillion-parameter model on first-gen Optane, and highlights critical thermal management challenges that can damage neighboring memory components without proper cooling.
Reddit Builder Achieves 0.85 Tokens Per Second with CPU-Only Inference
The homelab project, shared on Reddit's r/homelab community, achieved 0.85 tokens per second with 1-bit quantization and 0.75 tokens per second with 3-bit quantization. These speeds compare to roughly 4 tokens per second achieved in an earlier build that used first-gen Optane in Memory Mode with an RTX 3060 handling part of the smaller 1-trillion-parameter K2.5 model. One commenter estimated that a 50,000-token task would take approximately 13 hours at about 1 token per second, though another noted that overnight batch processing does not prioritize speed.
The builder used App Direct Mode, which exposes Optane as a separate, persistent region that software maps directly. This approach can give "more control over where the model's tensors live," according to the first build's creator, u/APFrisco. The alternative Memory Mode has Optane act as system memory with DRAM as its cache.
Optane PMem 200 Generates Extreme Heat Requiring Active DRAM Cooling
The builder issued a critical thermal warning, stating that Optane DIMMs get "extremely hot" with sustained inference and can endanger neighboring DRAM. The builder's advice is to "treat active DIMM cooling as mandatory," using dual 12W blower fans aimed at the modules to control temperatures. Intel rates each Optane PMem 200 module at 12W or 15W regardless of capacity, meaning four sticks generate up to 60W of heat placed directly next to system memory.
Intel specifies that the modules require forced airflow of 2.4 to 2.7 meters per second for cooling, conditions typically found in data centers rather than home systems. Throttling occurs at 83 degrees Celsius, with a thermal trip shutdown at 85 degrees Celsius. By comparison, typical DDR4 server memory is rated for 85 degrees Celsius and up to 95 degrees in extended range operation.
Second-Gen Optane Delivers 32% More Bandwidth Than First Generation
Intel launched its Optane Persistent Memory 200 Series (PMem 200) in 2020 for 3rd-gen Xeon Scalable CPUs in 128GB, 256GB, and 512GB capacities. On two-socket Xeon platforms, it runs at a higher effective MT/s rate than first-generation PMem 100 and can deliver an average of up to 32% more memory bandwidth.
Moonshot AI recommends data center inference engines to run Kimi K3, and even the smallest community quantization needs at least 610GB of RAM. Using secondhand Optane is one of the few ways to reach that capacity at home, as the builder notes DRAM prices have climbed significantly.
Builder Plans GPU Integration and DRAM Tier Optimization
The builder stated that the performance numbers given are pre-tuning, meaning without tuning thread counts or NUMA affinity, leaving room for improvement. The builder plans to add GPUs and to load parts of the model into DRAM as a faster tier alongside the Optane. One commenter asked about using Memory Mode with DRAM caching for the most-used experts. The builder indicated that "more benchmarks to come."
FAQ
What hardware did the Reddit builder use to run the 2.8-trillion-parameter AI model?
The builder used second-gen Intel Optane PMem 200 memory with a 40-core 3rd-gen Xeon Scalable CPU, running the model on CPU only without any GPU for inference.
Why does Optane memory require active cooling in this setup?
Optane PMem 200 modules generate 12W to 15W of heat each, creating up to 60W of total heat that can damage neighboring DRAM without proper airflow. The builder warned that Optane can "burn out" nearby memory and used dual 12W blower fans to prevent thermal damage.
How does second-gen Optane performance compare to first-gen Optane for AI inference?
Second-gen Optane PMem 200 delivers up to 32% more memory bandwidth than first-gen PMem 100, though the current build achieved 0.85 tokens per second compared to roughly 4 tokens per second in the earlier first-gen build that also used an RTX 3060 GPU and a smaller model.
