Tools: ncu 2025.5 (--set basic --print-units base), nsys 2025.5
Hardware: NVIDIA RTX A4000 (GA104), 16 GB GDDR6, 48 SMs, sm_86,
448 GB/s peak DRAM. CUDA 13.1, driver 580.82.07.
Profile dirs: profiles/a4000/{ncu,nsys}/{ver}_{matrix}.{ncu-rep,nsys-rep}.
Per-kernel breakouts under profiles/a4000/per_kernel/<kernel>/REPORT.md,
generated by python3 scripts/per_kernel_report.py --base profiles/a4000.
Tile width T was auto-picked from free GPU memory at run time (src/common/auto_tile.h). On the A4000 the picker chose T=1024 for every (matrix, version, single- stream) combo and T=512 for V5 streams=2 on pct20stif. The 3050 Laptop bench used T=512 for sstmodel/benzene and T=256 for pct20stif V4..V6.
| ver | step1 (row DFT) | step2 (col cuFFT) | other | step1+step2+other | wall_ms |
|---|---|---|---|---|---|
| V0 | scatter:3.21 | - | 0.28 | 3.49 | 943 |
| V1 | naive:0.68 | 1.95 | 0 | 2.63 | 16 |
| V2 | tile:0.79 | 2.13 | 0.68 | 3.60 | 17 |
| V3 | iter:0.35 | 2.12 | 0.81 | 3.27 | 18 |
| V4 | iter_u16:0.34 | 1.46 | 1.99 | 3.79 | 17 |
| V5 | iter_u16:2.82 | 3.05 | 0.14 | overlap | 27 |
| V6 | iter_u16:0.18 | 0.73 | 1.10 | 2.01 | 16 |
| ver | step1 | step2 | other | step1+step2+other | wall_ms |
|---|---|---|---|---|---|
| V0 | scatter:67.06 | - | 1.42 | 68.48 | 94 |
| V1 | naive:11.17 | 36.13 | 0 | 47.30 | 66 |
| V2 | tile:13.23 | 39.11 | 3.33 | 55.67 | 70 |
| V3 | iter:4.06 | 39.08 | 3.45 | 46.59 | 62 |
| V4 | iter_u16:3.88 | 34.94 | 9.75 | 48.57 | 63 |
| V5 | iter_u16:45.45 | 53.00 | 0.14 | overlap | 81 |
| V6 | iter_u16:2.09 | 19.42 | 5.25 | 26.76 | 41 |
| ver | step1 | step2 | other | step1+step2+other | wall_ms |
|---|---|---|---|---|---|
| V2 | tile:876.2 | 2437.2 | 138.9 | 3452.3 | 3474 |
| V3 | iter:846.4 | 2434.7 | 142.0 | 3423.2 | 3444 |
| V4 | iter_u16:898.3 | 1311.8 | 378.0 | 2588.1 | 2613 |
| V5 | iter_u16:2676.7 | 2684.9 | 0.3 | overlap | 2745 |
| V6 | iter_u16:452.4 | 655.9 | 189.4 | 1297.6 | 1322 |
ncu Duration values include profiling replay overhead and run ~1.5-2x
slower than the bench step1/step2 numbers above.
Kernel: sparse_row_dft_naive -> sparse_row_dft_iter (benzene)
| metric | V1 naive | V3 iter | delta |
|---|---|---|---|
| Duration / launch | 28.5 ms | 1.06 ms | 27x kernel speedup |
| Compute (SM) Throughput | 99.4 % | 76.4 % | bound moves off compute |
| Memory Throughput | 12.9 % | 48.5 % | W-table reads dominate |
| L1/TEX Cache Throughput | 12.9 % | 48.8 % | W-table accesses |
| L2 Cache Throughput | 3.4 % | 13.8 % | W partly L2-resident |
| Achieved Occupancy | 97.3 % | 96.9 % | unchanged |
| Registers / Thread | 38 | 40 | +2 (acc_re/im[PER_THREAD]) |
Kernel: sparse_row_dft_tile -> sparse_row_dft_iter (pct20stif)
| metric | V2 tile | V3 iter | delta |
|---|---|---|---|
| Duration / launch | 39.3 ms | 37.1 ms | 6 % |
| Compute (SM) Throughput | 99.7 % | 23.8 % | -76 pp |
| Memory Throughput | 12.9 % | 77.6 % | +65 pp |
| L1/TEX Cache Throughput | 12.9 % | 99.9 % | saturated |
| L2 Cache Throughput | 1.6 % | 77.6 % | +76 pp |
| Achieved Occupancy | 98.6 % | 94.3 % | -4 pp |
The V3 kernel still saturates L1/TEX (99.9 %), so the iterative twiddle buys only ~6 % at the kernel level on pct20stif. Same outcome as laptop.
vs laptop: laptop reported V2->V3 occupancy 97.9 % -> 65.8 % because the laptop bench forced T=256 (block=64). On A4000, auto-tile picks T=1024 (block=256), so occupancy stays at 94 %. The higher occupancy does not translate into wall time because L1/TEX is the binding bottleneck on both cards, not occupancy.
Kernel: cufft::multi_bluestein_fft (benzene)
| metric | V3 strided | V4 unit | delta |
|---|---|---|---|
| Duration / launch | 0.975 ms | 0.764 ms | 1.28x faster |
| Compute (SM) Throughput | 41.3 % | 50.1 % | +8.8 pp |
| Memory Throughput | 84.7 % | 91.5 % | +6.8 pp |
| DRAM Throughput | 77.0 % | 91.5 % | +14.5 pp (near peak) |
Kernel: cufft::multi_bluestein_fft (pct20stif)
| metric | V3 strided | V4 unit | delta |
|---|---|---|---|
| Duration / launch | 10.50 ms | 5.08 ms | 2.07x faster |
| Compute (SM) Throughput | 26.2 % | 38.2 % | +12.0 pp |
| Memory Throughput | 72.8 % | 86.8 % | +14.0 pp |
| DRAM Throughput | 70.6 % | 86.8 % | +16.2 pp |
vs laptop: same direction and similar magnitude on benzene (laptop: 1.49x faster, DRAM 77 -> 92 %). On pct20stif the A4000 reaches only 86.8 % DRAM under V4 unit-stride versus the laptop's 90.6 %. The A4000's wider memory subsystem doesn't fully saturate on this Bluestein plan (52488-pt batched 1D); a small but real headroom remains. Wall saving on pct20stif is still ~830 ms (V3->V4: 3444 -> 2613) and the optimization is unambiguously the largest single contributor.
| matrix | duration | mem throughput | SM throughput |
|---|---|---|---|
| sstmodel | 0.16 ms | 91.6 % | 17.2 % |
| benzene | 0.40 ms | 93.0 % | 17.0 % |
| pct20stif | 2.54 ms | 92.4 % | 16.8 % |
DRAM-bandwidth-limited at near-peak with low SM use, exactly as on laptop. vs laptop: per-launch time on pct20stif is 2.54 ms (A4000) vs 1.18 ms (laptop); the A4000 launch processes 4x the data because T=1024 vs T=256, so per element it is 1.86x faster (47 ps/elem vs 88 ps/elem) - close to the 2.3x DRAM-bandwidth ratio.
V6 does not change any kernel; it just runs each kernel half as many
times. Per-launch ncu metrics for sparse_row_dft_iter,
transpose_kernel, tile_stats, cufft::multi_bluestein_fft are
identical between V4 and V6 (within 0.1 pp). The wall-time saving is
purely from launch count.
| matrix | V4 launches | V6 launches | ratio |
|---|---|---|---|
| sstmodel | 4 tiles | 2 tiles | 0.50x |
| benzene | 9 tiles | 5 tiles | 0.56x |
| pct20stif | 52 tiles | 26 tiles | 0.50x |
(Tile counts are smaller than the laptop's 7 / 17 / 205 because A4000 uses T=1024 vs the laptop's T=512/256, so each tile covers 2-4x the columns. The V4/V6 launch ratio is unchanged at ~0.5x.)
Per-launch ncu metrics for V5 cuFFT match V4 exactly (Duration, DRAM, occupancy all within 0.1 pp). The wall difference is purely a stream- overlap effect that nsys timeline shows but ncu's per-kernel view cannot.
| matrix | mode | V4 wall | V5 wall | V5 vs V4 |
|---|---|---|---|---|
| pct20stif | no-keep | 2613 | 2745 | +5 % regression |
| pct20stif | --keep | 22 164 | 22 108 | basically tied |
| benzene | no-keep | 63 | 81 | +29 % regression |
| benzene | --keep | 555 | 561 | basically tied |
On the laptop V5 was a clear regression in compute-only mode (extra plan workspace, kernel durations staggered up to 3.6x on later tiles). On A4000, with 16 GB, the per-launch staggering is gone: pct20stif row-DFT Duration spans 36.9-37.1 ms on V4 (n=4) and 36.9-37.1 ms on V5 (n=3), a 0.4-0.5 % spread across all profiled launches. The remaining no-keep regression is just stream-launch overhead with nothing to overlap.
With --keep-output, the V5/V4 advantage that the laptop showed on
benzene (V5 saved ~90 ms of the ~976 ms wall) does not carry over to
the A4000: V4 keep = 555 ms, V5 keep = 561 ms, basically tied. Same
on sstmodel (V4 102 ms vs V5 109 ms). On pct20stif, V5 squeaks ahead
of V4 by 56 ms out of ~22.1 s. The streams still overlap, but the
A4000's per-tile work is fast enough that the D2H of one tile no longer
hides enough latency.
| kernel | bottleneck (A4000) | indicator |
|---|---|---|
sparse_row_dft_naive (V1) |
Compute (sincos+mod) | SM 99.4 %, mem 13 % |
sparse_row_dft_tile (V2) |
Compute (sincos+mod) | SM 99.7 %, mem 13 % |
sparse_row_dft_iter (V3+, benzene) |
Compute (mul-add chain) | SM 76 %, mem 49 % |
sparse_row_dft_iter (V3+, pct20stif) |
L1/TEX (W scatter) | L1/TEX 99.9 %, SM 24 %, mem 78 % |
transpose_kernel (V4+) |
DRAM (~92 %) | mem 92 %, SM 17 % |
tile_stats |
Compute (atomic-free reduction) | SM 86 %, mem 19 % |
cufft::multi_bluestein_fft (V2/V3) |
Memory non-coalesced | mem 73-85 %, L1/TEX 43-62 % |
cufft::multi_bluestein_fft (V4+, pct20stif) |
DRAM at ~87 % | mem 87 %, SM 38 % |
cufft::multi_bluestein_fft (V4+, benzene) |
DRAM at ~92 % | mem 92 %, SM 50 % |
For pct20stif the wall went 3474 -> 2613 -> 1322 ms (V2 -> V4 -> V6). Decomposing by bench stage timings:
V2 -> V4 saving: 3474 - 2613 = 861 ms
step2 (col cufft) : 2437.2 -> 1311.8 = -1125 ms unit-stride layout
step1 (sparse row) : 876.2 -> 898.3 = + 22 ms noise (uint16 colIdx)
other (transpose+st) : 138.9 -> 378.0 = + 239 ms transpose cost
Net : - 864 ms matches wall
V4 -> V6 saving: 2613 - 1322 = 1291 ms
step2 (col cufft) : 1311.8 -> 655.9 = - 656 ms half the launches
step1 (sparse row) : 898.3 -> 452.4 = - 446 ms half the launches
other (transpose+st) : 378.0 -> 189.4 = - 189 ms half the launches
Net : - 1291 ms matches wall
vs laptop: same shape (cuFFT layout fix dominates V2->V4, symmetry halves all three kernels for V4->V6). Absolute savings shrink ~3.5x because the A4000 baselines are all 2.4-2.8x lower to start with.
# Fresh profiles (A4000):
scripts/profile_all.sh # nsys + ncu
python3 scripts/extract_metrics.py --base profiles/a4000 # ncu_metrics.csv (and nsys CSV when nsys version matches)
python3 scripts/per_kernel_report.py --base profiles/a4000 # per_kernel/<kernel>/REPORT.mdTables in this file were built from the resulting
profiles/a4000/reports/ncu_metrics.csv and the bench CSV at
results/a4000/bench_nokeep.csv.
The per-kernel cross-version drilldowns live under
profiles/a4000/per_kernel/ (one REPORT.md per
kernel).
A wide raw export of one ncu report:
ncu --import profiles/a4000/ncu/v4_pct20stif.ncu-rep --csv --page raw \
> /tmp/v4_pct20stif_a4000_all.csv