Example · security

Kernel blacklisting: the GPU antivirus pattern

Malicious GPU code doesn’t ship with source. A compromised library, a supply-chain poisoned wheel, a co-tenant on a shared GPU — what you actually receive is a cubin, and the code that executes is whatever ptxas emitted. That is the layer sass2mlir reads, so it’s the layer where a signature check can run.

The reference attack class is the Membar+Load cross-MIG side channel from “Behind Bars: A Side-Channel Attack on NVIDIA MIG Cache Partitioning Using Memory Barriers” (USENIX Security 2026): a sender in one MIG instance modulates MEMBAR traffic and a receiver in another times LD.STRONG.GPU to recover the bits — straight through MIG’s physical L2 partitioning. The signature lives entirely at the instruction level, in the post-ptxas cubin.

This walkthrough builds the countermeasure as a pass plugin run by sass2mlir apply: match the signature in the lifted IR, and neutralize the function by replacing its body with a single EXIT — the kernel still launches, returns instantly, and the host application never knows anything happened.

Step 1

See the attack in the IR

Capture the suspect process’s kernels the usual way — sass2mlir intercept on a live process, or sass2mlir harvest on a cubin you already have:

sass2mlir intercept --out ./captures -- ./suspect_app
sass2mlir harvest captures/infer.cubin -o out/

Lifted, the sender kernel’s loop is unmistakable. A tight block that issues memory barriers against the GPU scope in a counted loop, with no memory consumer of anything the barrier orders — the modulation primitive itself:

^bb1(%i: !sass.reg):                              // the modulation loop
  sass.MEMBAR_0x118.GPU : () -> ()                // the channel's "1" (or skipped for "0")
  %c = sass.CS2R_0x1a4 SR_CLOCKLO : () -> (!sass.reg)
  ...
  %i2 = sass.IADD3_0x1cc %i, 1, RZ : (!sass.reg) -> (!sass.reg)
  %p  = sass.ISETP_0x5f2.LT.AND %i2, %n, PT : (!sass.reg, !sass.reg) -> (!sass.pred)
  sass.BRA_0x942 guard(%p) ^bb1(%i2) : () -> ()   // counted loop back-edge

The receiver is the mirror image: a loop of LD.STRONG.GPU (a strongly-ordered load that bypasses the usual relaxation) interleaved with CS2R SR_CLOCKLO reads — a timing loop, and equally recognizable.

What makes this a signature rather than a heuristic: MEMBAR at GPU scope inside a counted hot loop with no ordered dataflow consumer is not something a legitimate GEMM, attention, or inference kernel emits — ptxas inserts barriers at well-understood points (memory fences around flag stores, reductions), never as the loop body itself. The pass matches op kind, loop structure from the recovered CFG, and the absence of a dependent consumer — all three directly queryable on the dialect.

Step 2

The pass: match, then replace with EXIT

The plugin is the same ABI as writing a pass — the interesting part is what it does on a match. Not a careful rewrite: delete everything. The function’s blocks are replaced by a single entry block containing one unconditional sass.exit:

- bb0: LDC R2, c[0x0][0x210]      // attack setup: channel buffer
-      ISETP P0, ...  BRA ^bb1    // loop init
- bb1: MEMBAR.GPU                 // modulation loop (the signature)
-      CS2R R4, SR_CLOCKLO  ...   // timing
-      BRA @P0 ^bb1
+ bb0: EXIT                       // neutralized: every thread returns instantly

Three properties make this the right neutralization:

  • It’s one instruction to encode. Only the EXIT is re-encoded structurally by nvisa_encode (with a valid control word); every other op in the function is dropped, so there’s nothing left to get wrong. crosscheck on the result audits exactly one instruction.
  • The ABI is untouched. Same kernel name, same parameter buffer, same const-bank layout — the patched cubin drops into the original kernel slot via the same apply patch path as any other pass. Launch config, grid geometry, host code: all unchanged.
  • Every thread exits at the first instruction. No predication games, no guard to satisfy — EXIT at the entry offset ends each thread before the attack’s first load issues. The kernel reports success and completes in nanoseconds.
Neutralize vs refuse-to-launch

There are two enforcement postures, and they compose:

  • Neutralize (this example) rewrites the cubin, so the process keeps running — the right choice for a managed fleet where you want the workload alive and observable while the malicious function does nothing, plus a telemetry hook in the interposer logging that a blacklisted kernel launched at all.
  • Refuse to launch needs no byte editing at all: the intercept shim already sees every cubin at load time, so a signature match there can fail the load before any execution. Harsher — the host application sees an error — but nothing of the attack ever runs, even once.

The pass below is the neutralizer; the refuser is the same signature matcher on the capture path instead of the apply path.

Step 3

Audit first, neutralize second

Like every shipped pass, blacklist defaults to report mode — which is itself the audit tool: sweep a whole corpus of cubins and list any function matching a known-attack signature, changing nothing.

# blacklist.toml
[[pass]]
name = "blacklist"
sm   = ["sm_90*", "sm_87"]
fn   = ["*"]            # audit every function in the cubin
mode = "report"
sass2mlir apply infer.cubin --passes passes/ --config blacklist.toml -q
# blacklist: MATCH infer_kernel — MEMBAR.GPU counted loop, no ordered consumer (bb1, 14 insns)
# blacklist: 1/6 functions matched

Then flip the matched function to mode = "apply" — scoped to exactly that kernel, so everything else in the cubin stays byte-identical:

[[pass]]
name = "blacklist"
sm   = ["sm_90*"]
fn   = ["infer_kernel"]
mode = "apply"
sass2mlir apply infer.cubin --passes passes/ --config blacklist.toml \
  --keep-mlir kept/ -o infer.patched.cubin
# blacklist: infer_kernel — body replaced with EXIT (1 op re-encoded)
# patch: .text rewritten in slot infer_kernel, .nv.info unchanged
Step 4

Verify the neutered kernel

Three checks, cheapest first:

# 1. The one re-encoded instruction is structurally sound:
sass2mlir crosscheck kept/infer_kernel.after.mlir
# crosscheck: 1/1 instructions re-encode byte-identical (0 mismatches)

# 2. NVIDIA's own disassembler agrees on what the kernel now is:
cuobjdump -sass infer.patched.cubin | grep -A2 infer_kernel
#   infer_kernel:
#     /*0000*/  EXIT ;

3. The channel is closed at runtime. The behavioral oracle for this attack class is the receiver’s timing distribution: with the sender neutralized, LD.STRONG.GPU timings in the co-tenant instance stop correlating with the sender’s modulation — the covert channel’s bit error rate collapses to chance. That’s the end-to-end gate, run the same way the attack paper measures the open channel.

And the negative control: the five other functions in the cubin compare byte-identical against the original — the pass named one function, so only one function changed.

Why this layer

Why blacklisting works at SASS

  • The attacker can’t hide behind the source. Whatever the original CUDA or Triton looked like, the modulation primitive must appear as MEMBAR at GPU scope in the machine code to work — obfuscating the source changes nothing about the signature.
  • The IR has the structure a byte-grep lacks. DeepGEMM-style hex patching can’t express “a counted loop whose body is a barrier with no ordered consumer.” Lifted, that’s a three-line query over op kinds, CFG loop structure, and def-use — and it survives recompilation, register reassignment, and instruction reordering by a future ptxas.
  • Signatures are per-architecture data, not code. The signature table ships per-SM (sm_90* and sm_87 above), riding the same probe-derived opcode tables as everything else — a new architecture gets signatures the day its tables land.

The same pattern extends past this one channel: any attack whose primitive is an identifiable instruction-level idiom — a known side-channel’s timing loop, a banned instruction in a hardened deployment, a fuzzer-found miscompare kernel in CI — is a signature row and a mode flag away.