Ephraim Ohana Open Weight Models and Distilled Models
Open-weight models are AI models whose trained parameters are published publicly, so anyone can download, run, fine-tune, and inspect them without relying on a vendor's API.
Distilled models are smaller models trained to reproduce the behavior of a larger "teacher" model, delivering much of its capability at a fraction of the size, cost, and compute.
Available Open-Weight Models
| Model | Organization | Parameters | License | Size (4-bit) | Download |
|---|---|---|---|---|---|
| Llama 3.1 8B Instruct | Meta | 8B | Llama 3.1 Community License | ~5 GB | Hugging Face |
| Llama 3.1 70B Instruct | Meta | 70B | Llama 3.1 Community License | ~40–45 GB | Hugging Face |
| Llama 3.2 3B Instruct | Meta | 3B | Llama 3.2 Community License | ~2 GB | Hugging Face |
| Mistral 7B Instruct v0.3 | Mistral AI | 7B | Apache 2.0 | ~4–5 GB | Hugging Face |
| Qwen2.5 3B Instruct | Alibaba | 3B | Apache 2.0 | ~2 GB | Hugging Face |
| Gemma 3 4B IT | 4B | Gemma License | ~2.5–3 GB | Hugging Face | |
| Phi-3 Mini 4K Instruct | Microsoft | 3.8B | MIT | ~2–3 GB | Hugging Face |
| OLMo 2 7B Instruct | Allen Institute for AI | 7B | Apache 2.0 | ~4–5 GB | Hugging Face |
| DeepSeek-R1-Distill-Llama-8B | DeepSeek | 8B | MIT (Llama 3.1 terms apply to base) | ~5 GB | Hugging Face |
| GLM-4 9B Chat | Zhipu AI | 9B | Apache 2.0 | ~5 GB | Hugging Face |
Key Research & Whitepapers
| Title | Author / Publisher | Date | Type | Link |
|---|---|---|---|---|
| Distilling the Knowledge in a Neural Network | Hinton, Vinyals & Dean (Google) | 2015 | Foundational Paper | arXiv 1503.02531 |
| The Llama 3 Herd of Models | Meta AI | 2024 | Technical Report | arXiv 2407.21783 |
| DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning | DeepSeek-AI | 2025 | Technical Report | arXiv 2501.12948 |
| AI Model Risk Management Framework | Ephraim Ohana Network | 2026 | Framework | Read on this site |
Further Technical Reading on Distillation
- PyTorch: Knowledge Distillation Tutorial — official walkthrough of implementing teacher/student training in PyTorch.
- Hugging Face: Knowledge Distillation for Computer Vision — Transformers documentation on distilling a fine-tuned model into a smaller student.
- Hugging Face: DistilBERT Model Documentation — reference documentation for one of the most widely deployed distilled models.
- Hugging Face Computer Vision Course: Knowledge Distillation with Vision Transformers — applied walkthrough covering DeiT-style distillation.