Three questions every schedule must answer
A pass that changes instructions must answer three separate questions: is the encoding legal for this exact form, are the dependencies accounted for, and does the resulting program execute correctly? Round-trip identity, a clean scoreboard, and plausible disassembly each check part of that contract. None replaces the other checks.
This chapter explains the controls exposed by the current implementation and the gaps a pass author must handle. The concrete encoding observations below concern inspected SM86, SM90, and SM120 descriptions; the experimental scalar scheduler has a narrower Hopper admission policy.
Credit: redplait’s work is part of the foundation
Our scheduling work builds on redplait’s denvdis machine descriptions and blog analyses of NVIDIA SASS control fields and dependency models. The investigation uses his named scheduling states, form-specific joint encoding tables, and separate result, input-hold, output, and resource constraints. These informed our representation, exposed missing field preservation, and supplied constraints for the bounded scheduling policy.
The retained upstream snapshot is revision 22bc333a4e2a5283b1a56ef7cd222fb91d5d84cd. Relevant sources are:
- SASS assembler: explains why batch and issue fields must select a valid row together.
- nvdisasm SASS parser: motivates preserving fields beyond ordinary assembly text.
- SASS latency tables and instruction reordering: examines the dependency model and the difficulty of matching compiler scheduling.
- Pinned SM90 encoding description and dependency/resource description, alongside the corresponding SM86 and SM120 descriptions.
sass2mlir combines that foundation with independent instruction-word witnesses, native-library dependency observations, physical completion tracking, compiler checks, and numerical GPU experiments. Model distances are hypotheses and planning constraints; they are not universal measured cycle guarantees.
Try the Encoding Lab to compare raw words and inspect fields alongside these examples. Its browser decoder and MLIR projection are experimental; changing a bit never executes GPU code or proves schedule legality.
1. Physical slices are not independent controls
The IR records a schedule resembling this illustrative attribute:
#sass.sched<stall = 1, yield = true,
wbar = 7, rbar = 7, watdb = 2,
reuse = 0, rsv = 0, pm = 0>The names expose physical slices. Their interpretation depends on the instruction form.
| Field or view | What a pass must account for |
|---|---|
stall + yield | Together represent the five-bit USCHED issue/group code. The field named yield is its upper bit. |
reuse + batch + USCHED | Participate in a form-specific joint opex table; copying one slice can change another meaning. |
wbar, rbar, watdb | Describe scoreboard posts and waits. Their dependencies must remain valid when instructions move or disappear. |
pm | A separate two-bit slice at absolute bits 102–103, named PMN/PM1/PM2/PM3 in the inspected descriptions. Its full hardware effect is unresolved here. |
| Encoder ownership | Some operand extensions overlap the nominal control window. Regeneration must respect the matched form’s owned bits. |
PM is separate from the ordinary @P guard. Its name does not justify treating it as a predicate SSA value. Opaque bits and unrecognized combinations require preservation or refusal.
W1 and W1EG have different codes
The inspected descriptions name these USCHED families:
| Low five-bit code | Name | Consequence |
|---|---|---|
| 0 | DRAIN | A distinct named issue state. |
| 1–15 | W1EG–W15EG | End-group family. |
| 17–27 | W1–W11 | Continuation family. |
| 16, 28–31 | Absent from inspected tables | A field-width check alone can accept an undefined code. |
For example, stall=1, yield=false gives code 1, W1EG. Setting yield=true gives code 17, W1. Setting stall=15, yield=true produces code 31, which is absent from those tables. Treating yield as an independent “switch warps” flag misses this coupling. These names distinguish issue states; they do not by themselves establish elapsed delays or warp-switch behavior.
2. Select the instruction form before the joint code
The opex key gathers bits 109:105 and 124:122. Its table combines issue/group, batch, and form-dependent reuse information. redplait’s assembler article demonstrates the practical consequence: individually acceptable field values can still have no valid joint row.
In our retained SM90 witnesses, the same opex=0x31 has two meanings:
| Instruction form | Interpretation in the matched table |
|---|---|
S2R | Batch start + W1 |
Immediate ISETP | Source-A reuse + batch NOP + W1 |
Even the full opcode can be insufficient. SM90 F2FP opcode 0x23e spans several tables. A PACK_AB form accepts 0x31; an inspected TF32/PACK_B form rejects the same code, and accepts 0x11 after the reuse slice is cleared. This is an encoding diagnostic, with deliberately different arithmetic, rather than an optimization example.
Pass rule: retain the matched form and its encoding obligations. A same-form register edit must preserve the remaining state. A form-changing edit needs an admitted translation; otherwise reject it. A mnemonic-wide switch or a range check cannot supply a general joint codec. That codec remains incomplete in the current implementation.
3. Separate readiness from completion
The machine has several kinds of outstanding work. The current scheduling code separates their obligations:
| Obligation | Why ordinary SSA is insufficient | Implementation landmarks |
|---|---|---|
| Fixed-pipeline readiness | A producer’s result may not be ready for the next instruction even without a scoreboard post. | lib/pepesilvia/schedule_machine.cpp, schedule_policy.cpp |
| Physical RAW / WAR / WAW | Reusing a register can destroy a pending read or result; different destinations alone do not imply independence. | lib/pepesilvia/schedule.cpp, schedule_machine.cpp |
| Variable-latency completion | Wait masks must cover pending accesses; a slot cannot be reused while live. | lib/pepesilvia/schedule.cpp |
| Matrix lifetimes | HGMMA may retain register or shared-memory operands after their apparent last SSA use. | lib/pepesilvia/schedule_mma.cpp |
| Cross-warp TMA / publication | Completion, expected byte counts, stage generations, and ownership must agree across producer and consumers. | Attention program protocol and scheduling passes in passes/ |
The generic remapping path repairs controls and branches after supported layout edits. The generated-scalar policy chooses new issue codes in admitted windows. Attention reconstruction additionally constructs matrix and transfer protocols. Keeping one of these models consistent does not establish the other obligations.
In the attention experiments, completion-only scheduling passed static register and scoreboard checks but failed numerical execution. Adding low-field delay budgets also failed. These counterexamples show why “the model accepted it” cannot be the final correctness argument.
4. Bound the scheduler’s claims
The experimental native-sm90 scalar policy admits exact, unguarded, no-reuse forms of FADD.FTZ, FMUL.FTZ, FFMA.FTZ, max-form FMNMX.FTZ, and F2FP.F16.F32.PACK_AB, with further operand restrictions. It plans nominal issue coordinates from physical dependencies and pipe classes. Its admitted windows stay within one basic block and bounded completion fences.
The model distinguishes FMAI and FXU result distances, anti/output dependencies, and resource initiation. The implemented 4/5/6 nominal result distances are policy inputs informed by the machine descriptions and native witnesses. A broader native scan found a consecutive FMNMX.FTZ pair using W4EG, refuting an earlier universal minimum inferred from fewer kernels. Native observations can falsify an overly broad rule without proving a replacement safe in another context.
No delay credit crosses an unsupported word or block boundary. An ordinary full wait does not complete every matrix, TMA, or cross-thread protocol. The measured full attention schedule has execution evidence; that evidence belongs to its exact program and tested workloads.
5. What a good pass must leave behind
Before changing a schedule, retain the original word, architecture, exact form, physical effects, and reason the edit is admitted. After mutation, rebuild affected scheduling projections and check:
- Encoding: the final word has an admitted joint code and preserves encoder-owned operand bits, PM, and other untouched state.
- Dependencies: physical register holds, result readiness, scoreboard reuse, CFG joins, and asynchronous stage ownership remain accounted for.
- Reconstruction: displaced branch targets and affected cubin metadata refer to the final instruction layout.
- Execution: numerical tests and relevant sanitizers accept the resulting program before any timing result is admitted.
Cases requiring refusal include undefined USCHED combinations, changing a form while keeping its old reuse code, deleting a required wait without a supported repair, reusing a live scoreboard slot, and overwriting an asynchronous operand. Unsupported repairs should refuse publication rather than silently default fields.
Continue with the first-pass tutorial, allocation and lifetimes, or FlashAttention reconstruction. Repository evidence and implementation notes live in docs/probes/redplait-scheduling.md, docs/probes/schedule-policy-audit.md, and docs/flash-attention/psir-boundary-scheduling.md.
The compiler contracts connect these scheduling obligations to value preservation, allocation, memory coverage, and publication.