v0.3.3 — GPU Kernel Profiler

Overhead Validation — v0.3.3 (lite mode, MI355X, Apr 2026)

All models measured at conc=1 and conc=64. Symbol audit: 0 unresolved kernel symbols across all traces.

Standard Benchmark (ISL=1024, OSL=1024)

Decode-Focused Benchmark (ISL=128, OSL=2048)

More prompts per run (conc×15, min 30) for higher statistical confidence on decode throughput.

Top 10 Kernels per Model — Prefill vs Decode

Kernel hotspots broken down by inference phase. Classification: MoE _ps_ suffix = prefill, paged_attention/sampling = decode, NCCL collectives shown separately. Rank 0 trace, conc=64, ISL/OSL=1k/1k. See Hot Trace Viewer for interactive visualization.

RTL_MODE Comparison (DeepSeek-R1 TP=8)

* full mode requires ROCm 7.13+ with ROCR fix 559d48b1.

Version Regression (DeepSeek-R1 TP=8, conc=64)

Tutorial — Profiling Prefill vs Decode with roctx Markers

RTL includes a built-in roctx shim. Add markers to your code to analyze GPU kernel hotspots per inference phase.

Step 1: Add roctx markers

import ctypes, torch

lib = ctypes.CDLL(None)  # RTL injects roctx via LD_PRELOAD
roctx_push = lib.roctxRangePushA
roctx_push.argtypes = [ctypes.c_char_p]
roctx_push.restype = ctypes.c_int
roctx_pop = lib.roctxRangePop
roctx_pop.restype = ctypes.c_int

# Prefill
roctx_push(b"prefill")
output = model.prefill(input_tokens)
torch.cuda.synchronize()
roctx_pop()

# Decode
roctx_push(b"decode")
for step in range(max_tokens):
    token = model.decode_step()
torch.cuda.synchronize()
roctx_pop()

Step 2: Run with RTL

rtl trace -o trace.db python3 my_model.py

That's it. rtl trace auto-sets LD_PRELOAD so roctx markers just work.

Step 3: Analyze

rtl summary trace.db

Or query by roctx region in SQLite:

SELECT s.string AS kernel, COUNT(*) AS calls,
       ROUND(AVG(o.end - o.start) / 1e3, 1) AS avg_us
FROM rocpd_op o JOIN rocpd_string s ON o.description_id = s.id
WHERE o.gpuId >= 0
  AND o.start BETWEEN <prefill_start> AND <prefill_end>
GROUP BY s.string ORDER BY SUM(o.end - o.start) DESC LIMIT 10;

Example output

roctx markers:
  prefill: 24.3ms
  decode:  54.9ms

=== prefill (74 ops, 0.7ms GPU time) ===
  Cijk_Ailk_Bljk (MT64x64x256)    calls=12   avg=16.1us  27.7%  # Attention GEMM (large batch)
  Cijk_Ailk_Bljk (MT64x128x128)   calls=12   avg=13.2us  22.7%  # FFN up-project
  Cijk_Ailk_Bljk (MT64x16x512)    calls=13   avg=12.0us  22.5%  # FFN down-project
  vectorized_elementwise (gelu)    calls=12   avg=5.7us    9.9%  # Activation

=== decode (6336 ops, 36.9ms GPU time) ===
  Cijk_Ailk_Bljk (MT256x16x384)   calls=1600 avg=8.0us   34.6%  # Skinny GEMM (batch=1)
  ScaleAlphaVec_PostGSU8_VW1       calls=1600 avg=4.5us   19.3%  # Post-GEMM scale
  Cijk_Ailk_Bljk (MT64x16x128)    calls=768  avg=9.1us   18.9%  # FFN GEMM

How to view Perfetto traces

1. Download a .json.gz trace file from the tables above
2. Open ui.perfetto.dev
3. Ctrl+O → select file — roctx regions appear as labeled spans on the timeline