Boards / Jetson Orin NX 16GB
Jetson Orin NX 16GB (original mode): LLM benchmarks
Orin NX 16GB module in its original power mode.
At a glance
- Measured runs7
- Models tested7
- Fastest run59.5 tok/s
- SmolLM2 1.7B, INT4, MLC · source ↗
- Sources1
Where to buy
- Jetson Orin NX 16GB module: price not verified yet Check price (affiliate slot)
Every run on this board
| Model | Quant | Runtime | Gen tok/s | Prompt tok/s | TTFT ms | Source |
|---|---|---|---|---|---|---|
| SmolLM2 1.7B | INT4 | MLC | 59.5 | – | – | source ↗ 2025-01-16 |
| Llama 3.2 3B | INT4 | MLC | 40.4 | – | – | source ↗ 2025-01-16 |
| Phi-3.5 3.8B | INT4 | MLC | 35.9 | – | – | source ↗ 2025-01-16 |
| Gemma 2 2B | INT4 | MLC | 31.6 | – | – | source ↗ 2025-01-16 |
| Qwen2.5 7B | INT4 | MLC | 20.8 | – | – | source ↗ 2025-01-16 |
| Llama 3.1 8B | INT4 | MLC | 20.5 | – | – | source ↗ 2025-01-16 |
| Gemma 2 9B | INT4 | MLC | 10.56 | – | – | source ↗ 2025-01-16 |
Can the Jetson Orin NX 16GB run…
Can it run Gemma 2 2B?Can it run Gemma 2 9B?Can it run Llama 3.1 8B?Can it run Llama 3.2 3B?Can it run Phi-3.5 3.8B?Can it run Qwen2.5 7B?Can it run SmolLM2 1.7B?
Have this board? Submit your own run →