How LLMs are built, stored, squeezed and changed.
From "what is a tensor" to EXL3, NVFP4, KV cache, MoE routing, LoRA and abliteration. Every module opens a real model from Hugging Face (Qwen, Ornith, DeepSeek, Kimi, gpt-oss) and reads its own files, and every number is either measured by the scripts in this repo or linked to its source.
Basics
What a model is made of, in plain numbers.
Map and how to use this
Where does each topic sit, and in what order should I read?
- Reading order
- Glossary and search
- How sources and measured numbers are marked
Foundations
What are vectors, matrices and tensors, and what does training actually change?
- Numbers, vectors, matrices, tensors
- Matrix multiply as the core operation
- Loss, gradients, backprop, optimizers (SGD, Adam)
- Parameters vs activations
- Why GPUs: FLOPs and memory bandwidth
Inside the model
How text becomes probabilities, layer by layer.
Tokens and tokenizers
How does text turn into numbers the model can read?
- BPE and byte-level BPE
- Vocab size and the embedding table
- Special tokens and chat templates
- Why token counts differ between models
- tokenizer.json vs tokenizer_config.json
The transformer
What happens inside one layer?
- Embeddings
- Self-attention: Q, K, V
- MHA, MQA, GQA, MLA
- MLP / FFN and SwiGLU
- RMSNorm, residual stream
- +3 more
Inference, KV cache and context
Why is the first token slow, and what eats memory in long chats?
- Prefill vs decode
- KV cache: what is stored and how big it gets
- Context length, RoPE scaling (YaRN), sliding windows
- KV cache quantization
- Batching, paged attention
- +2 more
Files, formats, quantization
What you download from Hugging Face and how people shrink it.
Anatomy of a model repo
What are all those files on a Hugging Face model page?
- config.json line by line
- Tokenizer files and chat template
- safetensors: header, tensors, shards, index
- Counting parameters from the tensor list
- GGUF, MLX and EXL3 repos
- +1 more
Number formats
What is FP32, FP16, BF16, FP8, MXFP4, NVFP4, and why does it matter?
- Sign, exponent, mantissa
- FP32 vs FP16 vs BF16
- FP8 E4M3 and E5M2
- INT8, INT4
- Block formats: MXFP4, NVFP4
- +2 more
Quantization
How do people shrink a model, and how do they prove it still works?
- Round-to-nearest, scales, zero points
- Group size and block scales
- Outliers and sensitive tensors
- Mixed precision: what stays high
- GPTQ, AWQ, imatrix
- +5 more
Architectures
Real open models taken apart through their own configs.
Mixture of Experts
What are experts, and why does a 30B-A3B run like a 3B?
- Router and top-k
- Shared experts
- Total vs active parameters
- Load balancing
- Expert offloading
- +1 more
Architectures compared
How is DeepSeek built differently from Qwen, Kimi or gpt-oss?
- Llama / Qwen3 dense
- Qwen3 MoE and Qwen3-Next
- Qwen3.5 and Ornith (hybrid linear attention)
- DeepSeek-V3 (MLA, shared expert, MTP)
- Kimi K2
- +3 more
Training and changing models
Pretraining, post-training and modifying weights.
Pretraining
How is a model trained from zero, and what does it cost?
- Data: crawl, filtering, dedup, mixes
- Training a tokenizer
- Compute: 6ND rule
- Scaling laws (Chinchilla)
- Mixed-precision training (BF16, FP8)
- +2 more
Post-training
How does a base model become a chat or coding agent, in theory and in practice?
- SFT and chat templates
- Full fine-tune vs LoRA vs QLoRA
- DPO and preference data
- RL: PPO, GRPO, verifiable rewards
- Distillation from a teacher
- +4 more
Modifying models
How do merges, abliteration and steering change a model without training?
- Merges: linear, SLERP, TIES, DARE (mergekit)
- Abliteration: finding and removing the refusal direction
- Activation steering
- Expert pruning and layer pruning
- Upscaling (frankenmerges)
Running and judging
Serving engines, evals and labs for your Mac.
Recipes
What is a recipe on Hugging Face, and how do I read one?
- Quant recipes (llm-compressor, EXL3, GGUF)
- Merge recipes
- Training recipes
- Serving recipes (vLLM, llama.cpp)
Evaluation
How do I tell if a model is actually good at my work?
- Benchmarks and what they measure
- Harnesses: Terminal-Bench, SWE-bench, BFCL
- Contamination
- Practical task evals (pagoda, todo app)
- Agent reliability: tool-call failures, retries
- +1 more
Serving engines
vLLM, SGLang, llama.cpp, MLX, ExLlamaV3: which one when?
- vLLM and SGLang
- llama.cpp and GGUF
- MLX / oMLX on Apple Silicon
- ExLlamaV3 and TabbyAPI
- OpenAI-compatible APIs
- +1 more
Labs for your Mac
Which scripts can I run myself to see all of this?
- Read a safetensors header
- Measure top-1 / KLD of a quant
- Quantize with llama.cpp and MLX
- LoRA fine-tune with MLX-LM
- Abliterate a small model
- +1 more