Boards / Jetson Orin NX 8GB
Jetson Orin NX 8GB (original mode): LLM benchmarks
Orin NX 8GB module in its original power mode.
At a glance
- Measured runs7
- Models tested7
- Fastest run51.5 tok/s
- SmolLM2 1.7B, INT4, MLC · source ↗
- Sources1
Where to buy
- Jetson Orin NX 8GB module: price not verified yet Check price (affiliate slot)
Every run on this board
| Model | Quant | Runtime | Gen tok/s | Prompt tok/s | TTFT ms | Source |
|---|---|---|---|---|---|---|
| SmolLM2 1.7B | INT4 | MLC | 51.5 | – | – | source ↗ 2025-01-16 |
| Llama 3.2 3B | INT4 | MLC | 34.5 | – | – | source ↗ 2025-01-16 |
| Phi-3.5 3.8B | INT4 | MLC | 30.8 | – | – | source ↗ 2025-01-16 |
| Gemma 2 2B | INT4 | MLC | 26.6 | – | – | source ↗ 2025-01-16 |
| Qwen2.5 7B | INT4 | MLC | 17.1 | – | – | source ↗ 2025-01-16 |
| Llama 3.1 8B | INT4 | MLC | 15.9 | – | – | source ↗ 2025-01-16 |
| Gemma 2 9B | INT4 | MLC | 8.8 | – | – | source ↗ 2025-01-16 |
Can the Jetson Orin NX 8GB run…
Can it run Gemma 2 2B?Can it run Gemma 2 9B?Can it run Llama 3.1 8B?Can it run Llama 3.2 3B?Can it run Phi-3.5 3.8B?Can it run Qwen2.5 7B?Can it run SmolLM2 1.7B?
Have this board? Submit your own run →