GPUUnlock

cuBLAS FP16 GEMM, FP32 accumulate

A library GEMM rather than an issue-rate loop, so it measures what a real workload gets at several M.N.K sizes.

How this path is measured

cuBLAS FP16-in/FP32-accumulate GEMM at M.N.K 1024, 2048, 4096 and 8192, one timed run per size. The Turing rows are the nlzy/turing-unlock README table, which states TFLOPS directly.

Governed by the SM speed-select fuse: yes

What the unlock changes on this path

21 rows over 8 cards. 21 improved by more than 5%, 0 did not; the largest gain is GeForce RTX 4080 at 1.94×.

ModeStockUnlockedGainCard
cuBLAS f16/f32 4096³ · cublasGemmEx, CUBLAS_COMPUTE_32F100.7194.9 TFLOP/s1.94×GeForce RTX 4080 · AD103 · 76 SMs
cuBLAS f16/f32 8192³83.4162.1 TFLOP/s1.94×GeForce RTX 4070 Ti · AD104 · 60 SMs
cuBLAS f16/f32 4096³98.7190.7 TFLOP/s1.93×GeForce RTX 4080 SUPER · AD103 · 80 SMs
cuBLAS f16/f32 4096³79.8154.0 TFLOP/s1.93×GeForce RTX 4070 Ti · AD104 · 60 SMs
cuBLAS f16/f32 8192³75.7144.4 TFLOP/s1.91×GeForce RTX 4070 SUPER · AD104 · 56 SMs
cuBLAS f16/f32 8192³ · power-limited at 250 W100.9177.7 TFLOP/s1.76×GeForce RTX 4080 SUPER · AD103 · 80 SMs
cuBLAS f16/f32 4096³73.6122.9 TFLOP/s1.67×GeForce RTX 4070 SUPER · AD104 · 56 SMs
f16/f32 2048³ · README row 2048: 55.7 (97.9% of a 56.9 TF/s peak) -> 91.4 (160.6%)55.791.4 TFLOP/s1.64×GeForce RTX 2080 Ti · TU102 · 68 SMs
cuBLAS f16/f32 8192³ · power-limited at 250 W, not issue-limited102.5164.9 TFLOP/s1.61×GeForce RTX 4080 · AD103 · 76 SMs
cuBLAS f16/f32 4096³ · PyTorch cuBLAS, median of 3 reps of 30 calls30.046.5 TFLOP/s1.55×GeForce RTX 4060 · AD107 · 24 SMs
f16/f32 4096³ · README row 4096: 53.7 (94.4% of a 56.9 TF/s peak) -> 81.7 (143.6%)53.781.7 TFLOP/s1.52×GeForce RTX 2080 Ti · TU102 · 68 SMs
cuBLAS f16/f32 4096³64.096.3 TFLOP/s1.50×GeForce RTX 4070 (AD103) · AD103 · 46 SMs
cuBLAS f16/f32 1024³51.977.8 TFLOP/s1.50×GeForce RTX 4070 Ti · AD104 · 60 SMs
f16/f32 8192³ · README row 8192: 52.8 (92.8% of a 56.9 TF/s peak) -> 77.9 (136.9%)52.877.9 TFLOP/s1.48×GeForce RTX 2080 Ti · TU102 · 68 SMs
cuBLAS f16/f32 8192³ · power-limited at 200 W64.493.6 TFLOP/s1.45×GeForce RTX 4070 (AD103) · AD103 · 46 SMs
cuBLAS f16/f32 1024³ · not tensor-bound at this shape49.170.1 TFLOP/s1.43×GeForce RTX 4070 (AD103) · AD103 · 46 SMs
cuBLAS f16/f32 4096³ · 70 W cap active in most samples, so large shapes are power-limited15.822.5 TFLOP/s1.42×GeForce RTX 3050 6GB · GA107 · 18 SMs
cuBLAS f16/f32 8192³31.444.2 TFLOP/s1.41×GeForce RTX 4060 · AD107 · 24 SMs
cuBLAS f16/f32 1024³ · not tensor-bound at this shape49.367.5 TFLOP/s1.37×GeForce RTX 4070 SUPER · AD104 · 56 SMs
f16/f32 1024³ · README row 1024: 50.3 (88.4% of a 56.9 TF/s peak) -> 68.3 (120.0%)50.368.3 TFLOP/s1.36×GeForce RTX 2080 Ti · TU102 · 68 SMs
cuBLAS f16/f32 8192³ · power-limited at 70 W16.119.7 TFLOP/s1.22×GeForce RTX 3050 6GB · GA107 · 18 SMs

The smallest ratio in this set is GeForce RTX 3050 6GB at 1.22×.

Reading the numbers

A 1.00× row is not a failed unlock. It says that particular path was never rate-limited, so the override had nothing to restore. If you benchmark your own card and see 1.00× on the mode you care about, the unlock may well have applied correctly and simply does not help that workload.

Other measurement paths