GPU

NVIDIA H200 NVL

Edit@1 months ago

Intergrated Memory(VRAM)
Capacity

141 GB

(HBM3e )

Bandwidth

4800 GB/s

685 Token/s

Vector Compute
FP64
30.16 T
FP32
60.32 T
FP16
X
BF16
X
INT32
NI
INT8
X

NVIDIA H200 NVL General-Purpose Float-Point performance (Vector Performance / Scalar Performance)

FP64: 30.16 TFLOPS

FP32: 60.32 TFLOPS

Matirx Compute
FP64
60.32 T
120.64 T
FP32
X
FP16
965.10 T
1930.20 T
FP8
1930.20 T
3860.40 T
TF32
482.55 T
965.10 T
BF16
965.10 T
1930.20 T
INT16
X
INT8
1930.20 T
3860.40 T
INT4
X

NVIDIA H200 NVL AI performance (Tensor Performance / Matrix Performance)

FP64: 60.32 TFLOPS, with sparsity: 120.64 TFLOPS

FP16: 965.10 TFLOPS, with sparsity: 1930.20 TFLOPS

FP8: 1930.20 TFLOPS, with sparsity: 3860.40 TFLOPS

TF32: 482.55 TFLOPS, with sparsity: 965.10 TFLOPS

BF16: 965.10 TFLOPS, with sparsity: 1930.20 TFLOPS

INT8: 1930.20 TOPS, with sparsity: 3860.40 TOPS

Hardware Specs
NVIDIA H200 NVL is a 5nm chip, has 80000 million transistors, launched by NVIDIA at 2024. It has 141 GB built-in(On-Board/On-Chip) memory with bandwidth up to 4800 GB/s. It has 16896 general-purpose ALUs(CUDA cores/Shader cores) and 528 matrix cores(Tensor cores) .
Process Node
5 nm
Launch Year
2024
Vector(CUDA) Cores
16896
Matrix(Tensor) Cores
528
Core Frequency
1365 ~ 1785 MHz
Cache
50MB

Comment without registration

Share your experience with NVIDIA H200 NVL / Found an Error? Help Us Improve!