Two ways to get a kernel

Everything sass2mlir does starts from the same input: a 128-bit-per-instruction SASS stream for one kernel. There are two ways to get that stream out of real software:

• Offline — the cubin is already on disk (embedded in an app binary or built by nvcc). Extract it with cuobjdump -xelf all, then run sass2mlir harvest on the file.
• Live — the kernel only exists inside a running process (JIT-compiled, loaded through the driver, built by a framework you don’t control). sass2mlir intercept runs the process under a capture shim that intercepts cubins in flight and captures every kernel as it loads.

Both paths produce the same artifacts — a .code byte stream and a lifted .mlir module per kernel — so everything after capture is identical.

Rule of thumb: if you can point at a file, harvest it offline — it’s reproducible and scriptable. Reach for intercept when the cubin only exists in flight: PyTorch extensions, cuDNN/cuBLAS JIT paths, anything loaded through cuModuleLoadData* at runtime.

Harvesting a cubin from disk

Start from a real application binary. Here it’s the flash_attn Python extension, which embeds its cubins in the .so:

cuobjdump -xelf all \
  /usr/local/lib/python3.12/site-packages/flash_attn_2_cuda.cpython-312-x86_64-linux-gnu.so
# Extracting ELF file    1: flash_attn_2_cuda.1.sm_90.cubin
# Extracting ELF file    2: flash_attn_2_cuda.2.sm_80.cubin

The sm_90 cubin is the one that runs on an H100. Harvest it — sass2mlir harvest parses the cubin ELF and writes the raw instruction stream plus the lifted module for every kernel it contains:

sass2mlir harvest flash_attn_2_cuda.1.sm_90.cubin -o out/
# harvest: sm_90 flash_fwd_kernel: 1042 instructions
# harvest: sm_90 flash_fwd_splitkv_combine_kernel: 218 instructions

ls out/
# flash_fwd_kernel.code                    flash_fwd_kernel.mlir
# flash_fwd_splitkv_combine_kernel.code    flash_fwd_splitkv_combine_kernel.mlir

To lift a single .code stream on its own — or to re-lift after editing bytes out of band — point lift at it directly. --arch asserts the SM the bytes were built for; the tool refuses to decode against the wrong tables:

sass2mlir lift out/flash_fwd_kernel.code --arch sm_90 -o fwd.mlir
# lift: sm_90 flash_fwd_kernel: 1042 instructions, 14 blocks

The interesting part of the module — one predicated FFMA, one global load, one conditional branch:

module {
  sass.func "flash_fwd_kernel" {
    %0 = builtin.unrealized_conversion_cast to !sass.reg     // kernel live-in values
    ...
    // global load: row of Q/K tile, predicated on the bounds check
    %312 = sass.LDG_0x381.E.128 guard(%48) [%306]
             : (!sass.reg) -> (!sass.reg)

    // fused multiply-add: predicated, FTZ modifier on the opcode name
    %4706 = sass.FFMA_0x223.FTZ guard(%4380) -%4550, %4464, %4551
              : (!sass.reg, !sass.reg, !sass.reg) -> (!sass.reg)
    ...
    // conditional branch over the masked tail; target is a recovered block
    sass.BRA_0x942 guard(%4712) ^bb7(%4783, %4751 : !sass.reg, !sass.reg) : () -> ()
    ...
    sass.exit
  }
}

One op per instruction, named sass.<MNEMONIC>_0x<opcode>; SSA values for registers; guard(%p) inline; block arguments at branch targets are the SSA φ-nodes. Every op also carries its original 128-bit encoding as a $raw attribute — elided here — which is what the trip back to bytes is anchored on.

Capturing from a running process

When the cubin only exists inside a process, run it under sass2mlir intercept. The intercept shim hooks the driver’s module-load path, gates on the CUDA ELF header, and captures every kernel as it loads — raw cubin, .code stream, lifted .mlir, and one run.csv row per launch:

sass2mlir intercept --out ./captures -- python bench_flash.py
# intercept: captured flash_fwd_kernel (sm_90, 1042 instructions)
# intercept: captured flash_fwd_splitkv_combine_kernel (sm_90, 218 instructions)

ls captures/
# flash_fwd_kernel.cubin                   flash_fwd_kernel.code
# flash_fwd_kernel.mlir                    flash_fwd_splitkv_combine_kernel.cubin
# flash_fwd_splitkv_combine_kernel.code    flash_fwd_splitkv_combine_kernel.mlir
# run.csv

run.csv is what the offline path can’t give you: the launch configuration each kernel actually ran under. That grid/block/shared-memory triple is what an occupancy or scheduling analysis is measured against later.

kernel,grid,block,smem
flash_fwd_kernel,"(12,8,1)","(128,1,1)",99328
flash_fwd_kernel,"(24,8,1)","(128,1,1)",99328
flash_fwd_splitkv_combine_kernel,"(12,8,1)","(128,1,1)",0

Two flags control the volume. --filter is a function glob — capture only the kernels you care about:

sass2mlir intercept --out ./captures --filter '*flash*' -- python bench_flash.py
# intercept: captured flash_fwd_kernel (sm_90, 1042 instructions)
# intercept: skipped elementwise_kernel (filter)

And --no-lift writes bytes only — the .cubin and .code but no .mlir — for the case where you want a fast capture pass over a long run and will lift selectively afterwards with sass2mlir lift:

sass2mlir intercept --out ./captures --no-lift -- python bench_flash.py

When the lifter doesn't recognize an instruction

sass2mlir models semantics only for instructions it has positively identified from the probe tables. Anything it does not recognize — a hidden instruction, an undocumented opcode, an arch-specific form with no registered layout — is not guessed at and not dropped. It lifts to the generic sass.inst carrier: the original 128-bit encoding in $raw, and nothing more.

// A recognized instruction — a structured, per-mnemonic op:
%r = sass.LDC_0xb82 : () -> (!sass.reg) {
       operands = [#sass.reg_ref<R, dst, ...>, #sass.constmem<...>],
       raw = #sass.raw<0xdf00ff017b82, 0xfe20000000800>, ... }

// An unrecognized (hidden / undocumented) instruction — generic carrier, raw only:
%r = sass.inst : () -> (...) {
       mnemonic = "…", opcode_val = 0x… : ui16, operands = [],
       raw = #sass.raw<0x…, 0x…>, ... }

The consequence for lowering: an op whose $raw is intact emits its bytes verbatim. A structured op carries semantic fields and $raw; a sass.inst carrier carries only $raw. Either way, an instruction no pass has touched lowers in place, byte-for-byte — the unknown instruction is preserved exactly, never reverse-engineered.

You can prove that on the kernel you just lifted. Lower the module back to bytes and diff against the stream harvest wrote:

sass2mlir lower fwd.mlir -o fwd.rebuilt.code
# lower: flash_fwd_kernel: 1042 instructions re-encoded

cmp out/flash_fwd_kernel.code fwd.rebuilt.code && echo byte-identical
# byte-identical

cmp exits 0: every one of the 1042 instructions — including any sass.inst carriers in the stream — came back as the exact bytes it went in as. This is the same lower(lift(bytes)) == bytes identity the framework holds at corpus scale (478,211,236 instructions, 0 mismatches), checked here on one kernel with standard tools. The per-module equivalent is sass2mlir roundtrip, which runs the same check and exits non-zero on the first mismatched instruction.

From bytes to structure

You now have a lifted module with recovered control flow and SSA — the raw material every other example starts from.