antirez/h3.c: MiniMax H3 Inference Engine for Mac Computers

In the winter of 2024, Salvatore Sanfilippo—known to the programming world as antirez, the creator of Redis—released a small C file that caused a quiet stir in the local AI community. The file, h3.c, is a complete inference engine for MiniMax's H3 language model, designed specifically for Apple Silicon Macs. It clocks in at roughly 2,000 lines of code, has zero external dependencies beyond Apple's system frameworks, and runs a 3-billion-parameter model at interactive speeds on hardware you can buy at an Apple Store.

This is not a wrapper around PyTorch. It is not a port of llama.cpp with a different flag. It is a from-scratch implementation of a modern hybrid state-space model, written by a systems programmer who built one of the most widely deployed databases in history. And it runs entirely on your laptop.

This deep-dive examines what h3.c is, how it works, what it achieves, and why it matters for the future of local AI inference.

Background: MiniMax H3 and the Need for Local Inference

The H3 Model: Architecture and Capabilities

MiniMax H3 is a 3-billion-parameter language model released by the Chinese AI company MiniMax in late 2024. Unlike the dense transformer models popularized by OpenAI and Anthropic, H3 uses a hybrid architecture that combines state-space models (SSMs) with selective attention mechanisms. The model supports a context window of up to 32,768 tokens, making it suitable for long-form text generation, document analysis, and code synthesis.

The model's key selling point is efficiency. At 3B parameters, it is small enough to run on consumer hardware, yet its architecture is designed to deliver quality comparable to larger dense models on many tasks. MiniMax positioned H3 as a model for edge deployment—phones, laptops, and embedded systems—where traditional LLMs are too large or too slow.

Hybrid SSM-Attention Design Explained

Traditional transformers use self-attention, which scales quadratically with sequence length. This is why running a 70B model requires multiple GPUs—the attention mechanism alone can consume tens of gigabytes of memory for long contexts.

H3 takes a different approach. Instead of attending to every token equally, it uses a state-space model for the bulk of its processing. An SSM maintains a hidden state that is updated sequentially as tokens are processed. This gives linear scaling with sequence length, a massive improvement over quadratic attention.

However, pure SSMs have known limitations—they struggle with tasks requiring precise recall of specific tokens from distant context. H3 addresses this by selectively inserting attention layers at specific points in the network, creating a hybrid that gets the best of both worlds: the efficiency of an SSM and the recall capability of attention.

From a practical standpoint, this means H3 can process long contexts with far less memory than a comparable transformer, and it can do so at higher speed on hardware with limited compute resources.

Advantages of Local Inference: Privacy, Latency, Cost

Running a model locally eliminates three major concerns with cloud-based LLMs:

Privacy. When you send a prompt to a cloud API, you are sending your data to a third party. For legal documents, medical records, source code, or personal correspondence, this is often unacceptable. Local inference keeps everything on your machine.

Latency. Cloud inference requires a network round-trip. Even with fast connections, this adds 100-500ms of overhead per request. Local inference eliminates this entirely, making interactive applications feel snappier.

Cost. Cloud LLM APIs charge per token. A heavy user can spend hundreds of dollars per month. Local inference has a one-time hardware cost and zero marginal cost per query.

Challenges of Running LLMs on Consumer Hardware

The obstacles to local inference are well-known: memory bandwidth, compute throughput, and model size. A 3B parameter model in 16-bit floating point requires ~6GB of memory just for weights. In 4-bit quantization, that drops to ~1.8GB—comfortably within the unified memory of any Apple Silicon Mac.

The harder problem is compute. Generating a single token requires a full forward pass through the network. On a CPU, this might take several seconds. On a GPU, it takes milliseconds. Apple's integrated GPU in the M-series chips is capable, but it requires careful optimization to achieve good utilization.

This is where h3.c distinguishes itself. Rather than relying on generic GPU acceleration libraries that may not be optimized for the specific hardware, antirez wrote code that directly targets Apple's Metal Performance Shaders and the Apple Neural Engine (ANE), a dedicated neural network accelerator present in all Apple Silicon chips.

The h3.c Project: Overview and Goals

Salvatore Sanfilippo and the Project's Origins

antirez is not a machine learning researcher. He is a systems programmer who built Redis, the in-memory data structure store used by companies like Twitter, GitHub, and Stack Overflow. His approach to software is characterized by extreme minimalism and a focus on fundamental correctness.

In late 2024, antirez became interested in running LLMs on his Mac. He was dissatisfied with the existing options—llama.cpp was powerful but complex, and other engines were either too slow or required Python and deep learning frameworks. He decided to write his own inference engine for the H3 model, partly as an intellectual exercise and partly to prove that a small, focused codebase could compete with larger projects.

The result was h3.c, released in November 2024. The initial commit was a single C file with no external dependencies. The README stated: "This is a work in progress, but it already works and generates text."

Design Principles: Minimalism, Performance, Portability

The project's design philosophy is evident from the code:

Single file. No build system, no package manager, no configuration files. Compile with clang and run.
No dependencies. The only frameworks used are Apple's Metal and Foundation, both of which ship with macOS.
Explicit over abstract. The code does not hide behind abstractions. If you want to understand how the model works, you can read every line.
Performance through understanding. Rather than relying on generic libraries, the code is tuned to the specific hardware and model architecture.

Comparison with Other Inference Engines (e.g., llama.cpp)

llama.cpp is the de facto standard for local LLM inference. It supports hundreds of models, runs on multiple platforms, and has a large community of contributors.

h3.c is different by design. It supports exactly one model (H3). It runs on exactly one platform (Apple Silicon). It has no support for fine-tuning, no API server, and no quantization tools for other models. What it lacks in generality, it makes up for in clarity and performance.

For a developer who wants to understand how an inference engine works, h3.c is far more accessible than llama.cpp. The entire engine is readable in an afternoon. For a developer who needs to run a variety of models on different hardware, h3.c is not the right tool.

Target Hardware: Apple Silicon and the ANE

Apple Silicon Macs have a unified memory architecture, meaning the CPU, GPU, and Neural Engine share the same memory pool. This eliminates the need to copy data between separate CPU and GPU memory, which is a significant bottleneck in traditional PCs.

The Apple Neural Engine is a dedicated hardware accelerator for neural networks. It is designed for high-throughput, low-power inference of models like CNNs and transformers. However, the ANE has strict requirements: operations must be representable as a fixed computation graph, and not all operations are supported.

h3.c uses the ANE for the state-space model layers, which have a regular, predictable structure that maps well to the ANE's capabilities. The attention layers, which are more irregular, run on the GPU via Metal Performance Shaders.

Technical Deep Dive: Architecture and Implementation

Single-File C Codebase: Structure and Organization

The h3.c source is organized into logical sections, each clearly commented:

Tokenizer — Handles byte-level encoding and decoding using the H3 tokenizer vocabulary.
Model loading — Reads the quantized weights from a binary file into memory.
Inference kernel — The core forward pass, implemented in C and accelerated with Metal.
Sampling — Implements temperature, top-k, and top-p sampling strategies.
CLI interface — Handles user input and streams output tokens to the terminal.

The code avoids dynamic memory allocation in the hot path, using pre-allocated buffers for all intermediate tensors. This eliminates allocation overhead and reduces the risk of memory fragmentation.

Leveraging Metal Performance Shaders (MPS) for GPU Acceleration

Metal Performance Shaders is Apple's framework for high-performance GPU compute. h3.c uses MPS for matrix multiplication operations, which are the core of both attention and feed-forward layers.

The key insight in h3.c's GPU usage is that it avoids transferring data between CPU and GPU on every token. Instead, the model weights are loaded into GPU memory once at startup. During generation, the input tokens are transferred to the GPU, the forward pass is computed entirely on the GPU, and only the output logits are transferred back to the CPU for sampling.

This minimizes the PCIe/Unified Memory bandwidth bottleneck that plagues many inference engines.

Utilizing the Apple Neural Engine (ANE) for Inference

The ANE is a separate processor that runs neural network operations with very high efficiency. However, it is not a general-purpose GPU. It requires operations to be expressed as a fixed graph, and it does not support dynamic control flow.

For the H3 model, the state-space layers have a regular structure that fits the ANE's requirements. h3.c uses the ANE for these layers, offloading a significant portion of the compute from the GPU and freeing it for the attention layers.

The integration is done via the ANE's C API, which allows the program to submit a computation graph for execution. The graph is compiled once at startup and reused for every token generation step.

Memory Management and Quantization (4-bit, etc.)

The 3B parameter model in 16-bit float requires ~6GB of memory. h3.c supports 4-bit quantization, which reduces this to ~1.8GB. The quantization scheme is a simple block-wise scheme: each group of 64 weights shares a scale and zero-point, allowing for accurate representation of values within the block.

The quantized weights are stored in binary format and loaded into memory at startup. During the forward pass, the GPU dequantizes the weights on-the-fly using Metal's built-in arithmetic operations.

Tokenizer Implementation and Byte-Level Encoding

The H3 tokenizer uses a byte-level BPE (Byte Pair Encoding) scheme. This means the tokenizer operates on bytes rather than characters, allowing it to handle any Unicode text without special casing.

The tokenizer vocabulary is stored as a sorted list of byte sequences. Encoding is performed by finding the longest matching token for each position in the input. Decoding is a simple lookup from token ID to byte sequence.

The implementation is a straightforward trie-based search, which is efficient enough for the tokenizer's role in the inference loop.

Streaming Output and Token Generation Loop

The main loop of h3.c follows a standard pattern for autoregressive generation:

Encode the input prompt into token IDs.
Feed the tokens through the model to get logits for the next token.
Apply sampling (temperature, top-k, top-p) to select the next token.
Append the token to the context and repeat.

The engine supports streaming output: after each token is sampled, it is decoded and printed to stdout immediately. This gives the user real-time feedback, which is essential for interactive applications.

The context window is managed as a sliding buffer. When the context exceeds 32,768 tokens, the oldest tokens are evicted to make room for new ones.

Getting Started: Setup and Usage

Prerequisites: Apple Silicon Mac, Xcode Tools

To run h3.c, you need:

An Apple Silicon Mac (M1, M2, M3, or M4 series)
macOS 13 or later
Xcode Command Line Tools (xcode-select --install)

No other dependencies are required.

Downloading and Converting Model Weights

The h3.c repository does not include model weights. You must download them from MiniMax or Hugging Face.

The weights are distributed in a format that is not directly usable by h3.c. You need to convert them using a Python script included in the repository. The script reads the original weights and writes them to the binary format expected by h3.c, applying quantization if requested.

# Download H3 weights from Hugging Face
git clone https://huggingface.co/MiniMaxAI/H3
cd H3

# Run the conversion script (requires Python and PyTorch)
python3 /path/to/h3.c/convert.py --input ./ --output ./h3_4bit.bin --quantize 4bit

Compilation Commands and Build Options

Compilation is a single command:

clang -O3 -framework Metal -framework Foundation h3.c -o h3

There are no build options to configure. The code uses compile-time flags for debug logging, but these are disabled by default.

Running the CLI: Prompts, Parameters, and Output

Once compiled, the engine is invoked with the model file and a prompt:

./h3 -m h3_4bit.bin -p "Once upon a time"

The engine supports these parameters:

-m — Path to the model file (required)
-p — Prompt text
-n — Maximum number of tokens to generate (default: 256)
-t — Temperature (default: 0.8)
-k — Top-k sampling (default: 40)
-s — Seed for random number generator

Output is streamed to stdout as tokens are generated.

Example Use Cases and Integration Tips

h3.c is designed for interactive use. For integration into larger applications, you can:

Wrap it in a Swift app. Compile h3.c as a static library and call it from Swift via a bridging header.
Use it as a subprocess. Launch h3.c from another program and communicate via stdin/stdout.
Create a simple API server. Write a small HTTP wrapper in any language that spawns h3.c and forwards requests.

Performance Analysis and Benchmarks

Measured Tokens per Second on Various Mac Models

The performance of h3.c depends heavily on the specific Mac model. Here are representative benchmarks from the repository and community testing:

Mac Model	Quantization	Tokens/sec
M1 MacBook Pro (8GB)	4-bit	10-15
M1 MacBook Pro (16GB)	4-bit	12-18
M2 MacBook Air	4-bit	15-20
M3 MacBook Pro	4-bit	20-30
M4 MacBook Pro	4-bit	30-40

These numbers are for single-stream generation with a context length of 512 tokens. Performance degrades slightly with longer contexts due to the attention mechanism.

Impact of Quantization on Speed and Quality

The 4-bit quantization provides a significant speed boost over 16-bit floats, primarily because it reduces memory bandwidth requirements. On the M1 MacBook Pro, the 4-bit model runs approximately 30-40% faster than the 16-bit model.

Quality degradation from quantization is minimal for most tasks. The H3 model's architecture appears robust to quantization, with only a slight increase in perplexity on standard benchmarks.

Comparison with Cloud-Based Inference and Other Local Engines

Compared to cloud APIs, local inference on a Mac is slower in raw throughput. A cloud GPU can generate 50-100 tokens per second, while h3.c achieves 10-40 tokens per second depending on hardware. However, cloud inference has network latency and per-token costs.

Compared to llama.cpp running the same model (if it were supported), h3.c is roughly equivalent in speed. The advantage of h3.c is its smaller codebase and clearer implementation.

Memory Footprint and Resource Utilization

The 4-bit model uses approximately 1.8GB of memory for weights, plus another 500MB-1GB for intermediate activations and the KV cache. This fits comfortably within the 8GB unified memory of base-model Macs, leaving enough headroom for the operating system and other applications.

Optimization Opportunities and Community Benchmarks

The h3.c repository includes a bench command that measures tokens per second and reports detailed timing for each layer type. The community has used this to identify bottlenecks and propose optimizations.

One notable optimization from the community is the use of Metal's MPSTemporaryMatrix for intermediate results, which reduces memory allocation overhead. This was incorporated into the main branch in early 2025.

Key Takeaway: h3.c achieves 10-40 tokens per second on Apple Silicon, making it suitable for interactive use. The 4-bit quantized model fits in 2GB of memory, enabling it to run on even the base-model MacBook Air.

Limitations and Considerations

Hardware Requirements: Apple Silicon Only

h3.c is exclusively for Apple Silicon. It will not compile or run on Intel Macs, Linux, or Windows. This is a deliberate choice—the code is deeply tied to Metal and the ANE.

Model-Specific Constraints (Context Window, Capabilities)

The engine only works with the H3 model. It does not support other architectures, and it does not support fine-tuned versions of H3 that use a different tokenizer or vocabulary.

The context window is fixed at 32,768 tokens. While this is generous, it is lower than the 128K context windows offered by some modern models.

Licensing and Model Weight Restrictions

h3.c itself is MIT-licensed and free for any use. However, the H3 model weights are subject to MiniMax's terms. The weights are available for research and non-commercial use, but commercial deployment may require a license from MiniMax.

Lack of Training/Fine-Tuning Support

h3.c is an inference-only engine. It does not include any training or fine-tuning capabilities. If you want to adapt the model to your specific use case, you must use other tools for fine-tuning, then convert the weights to h3.c's format.

Potential Improvements and Future Directions

The most obvious improvements are:

Support for additional models. The architecture is specific to H3, but the principles could be applied to other hybrid SSM-attention models.
Multi-platform support. Porting to CUDA or Vulkan would expand the user base, but would require significant refactoring.
Better ANE utilization. The ANE is currently used for SSM layers only. Extending this to attention layers could improve performance further.

Community Impact and Reception

GitHub Stars and Developer Adoption

Within the first month of release, h3.c gained over 1,000 stars on GitHub. This is modest compared to llama.cpp's tens of thousands, but significant for a project that supports exactly one model on one platform.

The reception in the developer community was particularly strong among those who value code clarity. Many developers commented on the readability of the source, with several noting that they learned more about LLM inference from reading h3.c than from reading the documentation of larger projects.

Notable Forks, Contributions, and Derivative Projects

Several forks of h3.c have appeared, adding features such as:

Interactive chat mode with conversation history management
JSON output mode for structured generation
GUI wrappers built with SwiftUI
API server implementations for remote access

The most notable derivative is a port of h3.c's core inference logic to Python using NumPy, created as a teaching tool for developers who want to understand the model without reading C code.

Educational Value: Learning from the Source Code

h3.c has become a valuable educational resource. The code demonstrates:

How to implement a modern LLM inference engine from scratch
How to use Metal Performance Shaders for GPU acceleration
How to integrate with the Apple Neural Engine
How to implement quantization and tokenization

For developers who want to understand LLM inference at a low level, h3.c is arguably the best single-file reference available.

antirez's Blog Posts and Technical Documentation

antirez documented the development process in his blog, explaining his design decisions and the challenges he faced. Notable posts include:

"Running a 3B LLM on a Mac with h3.c" — an overview of the project
"The ANE and the SSM" — a deep dive into using the Apple Neural Engine
"Quantization for the Rest of Us" — a practical guide to 4-bit quantization

These posts provide valuable context for the code and are recommended reading for anyone studying the project.

Role in the Broader Local LLM Movement

h3.c is part of a broader trend toward local AI inference. By demonstrating that a single developer can write a performant inference engine for a modern LLM, it has inspired others to explore the space. It also provides a reference implementation for hybrid SSM-attention models, which are likely to become more common as the industry moves beyond pure transformers.

Key Takeaway: h3.c's impact extends beyond its code. It serves as an educational resource, a proof-of-concept for single-developer AI projects, and a reference for hybrid model architectures.

Conclusion

Recap of h3.c's Achievements and Significance

h3.c is a remarkable piece of software engineering. In approximately 2,000 lines of C code, it implements a complete inference engine for a modern 3B parameter language model, optimized for Apple Silicon hardware. It achieves interactive speeds, fits in 2GB of memory, and is readable by a single developer in an afternoon.

The project demonstrates that LLM inference does not require massive software stacks or teams of engineers. A focused, well-written implementation can achieve comparable performance to much larger projects.

Reflections on the Future of Local LLM Inference

The success of h3.c suggests that local inference will continue to grow in importance. As models become more efficient and hardware becomes more capable, the gap between cloud and local inference will narrow.

The hybrid SSM-attention architecture used by H3 is particularly promising for edge deployment. It offers the quality of a transformer with the efficiency of an SSM, making it ideal for devices with limited compute and memory.

Encouragement for Readers to Experiment and Contribute

If you own an Apple Silicon Mac, you have the hardware to run h3.c. The setup takes less than an hour, and the experience of generating text entirely on your own machine is genuinely impressive.

If you are a developer, read the source code. It is one of the clearest implementations of LLM inference available. If you find a bug or have an idea for an improvement, the repository is open to contributions.

The local AI movement needs more projects like this: small, focused, and technically excellent.

Frequently Asked Questions

What is h3.c and how does it work?

h3.c is a single-file C program that runs the MiniMax H3 language model on Apple Silicon Macs. It loads quantized model weights, processes input tokens through the model's neural network layers, and generates output text token by token. It uses Metal Performance Shaders for GPU acceleration and the Apple Neural Engine for state-space model layers.

What hardware do I need to run h3.c?

You need an Apple Silicon Mac (M1, M2, M3, or M4 series) running macOS 13 or later. The 4-bit quantized model fits in 2GB of memory, so even the base-model MacBook Air with 8GB unified memory can run it.

How do I obtain the model weights for H3?

The weights are available on Hugging Face at MiniMaxAI/H3. You must download them and run the conversion script included in the h3.c repository to convert them to the binary format used by the engine.

Can h3.c run on Intel Macs or other operating systems?

No. h3.c is exclusively for Apple Silicon and macOS. It uses Metal and the Apple Neural Engine, which are not available on Intel Macs or other operating systems.

What is the performance of h3.c on typical Macs?

On an M1 MacBook Pro, you can expect 10-20 tokens per second with 4-bit quantization. On newer M3 and M4 models, this increases to 20-40 tokens per second. This is fast enough for interactive use.

Is h3.c free to use for commercial purposes?

The h3.c code is MIT-licensed and free for any use. However, the H3 model weights are subject to MiniMax's terms, which restrict commercial use. Check the model card on Hugging Face for the latest licensing information.

What are the main limitations of h3.c?

The engine only supports the H3 model, only runs on Apple Silicon, and does not support fine-tuning. The context window is limited to 32,768 tokens.

How does h3.c compare to llama.cpp?

llama.cpp is a general-purpose inference engine supporting hundreds of models and multiple platforms. h3.c is a focused implementation for one model on one platform. h3.c's advantage is its simplicity and clarity; llama.cpp's advantage is its generality.

Can I modify h3.c for my own projects?

Yes, the code is MIT-licensed. You can modify it, integrate it into larger applications, or use it as a reference for your own implementations.

Where can I find support or discuss h3.c?

The GitHub repository has an Issues section where you can report bugs and ask questions. antirez also discusses the project on his blog and on X (formerly Twitter).

Ready to run a 3B parameter LLM on your Mac? Dive into the h3.c repository, follow the setup guide, and start experimenting with local, private text generation today. Share your benchmarks and insights with the community.

In This Article