AscendC passes

-ascendc-allocate-tensor

Assign memory addresses to LocalTensorAutoOp placeholders for static UB allocation

Converts LocalTensorAutoOp placeholders to LocalTensorV3Op with concrete memory addresses and tile sizes. This pass performs static memory allocation for on-chip buffers by assigning consecutive aligned addresses to tensors based on their normalized memory position (A1/L1, A2/L0A, B2/L0B, CO1/L0C, VECCALC/UB).

Position normalization: B1→A1, VECIN/VECOUT→VECCALC. Addresses are aligned to ubBlockSize (256 bytes). After allocation, LocalTensorAutoOp is replaced with LocalTensorV3Op containing position, address, and tile size.

-ascendc-compute-memory-consumption

Calculate total memory consumption per buffer location and attach to module

Analyzes all LocalTensorV3Op operations and computes total memory usage for each hardware buffer location: L1, L0A, L0B, L0C, and UB. The results are stored as a dictionary attribute memoryConsumed on the module, with keys like “L1”, “L0A”, etc. and values in bytes.

This pass is useful for verifying that kernel memory usage stays within hardware limits.

-ascendc-declare-py-struct

Insert emitasc.declare_py_struct operations for all Python struct types used in the module

Collects all emitasc.py_struct types referenced in function arguments, block arguments, and operation results, then inserts emitasc.declare_py_struct operations at the module beginning to declare them for C++ code emission.

The pass deduplicates struct types while preserving the order of first occurrence. Nested struct types (structs containing other structs) are also declared.

-ascendc-define-cube-only

Define ASCENDC_CUBE_ONLY macro for cube-only (matrix multiplication) kernels

Inserts #define ASCENDC_CUBE_ONLY verbatim at module start and sets matmul_cube_only attribute. This macro enables cube-only optimizations in the Ascend C runtime for kernels that only perform matrix multiplication without vector operations.

-ascendc-detect-enable-debug

Detect debug utility usage (printf, dump_tensor) and set enable_debug attribute

Scans the module for PrintfOp or DumpTensorOp operations. If any are found, sets the enable_debug unit attribute on the module to enable debug runtime support during kernel execution.

-ascendc-detect-kernel-type

Classify kernel as vector, cube, or mixed based on operation types present

Analyzes operations in the module to determine kernel type and sets kernel_type string attribute:

  • “vector”: Only vector operations present (no MmadOp or RegistMatmulObjOp)

  • “cube”: Only matrix multiplication operations present (no VectorOp)

  • “mixed”: Both vector and cube operations present

This classification affects synchronization strategy and runtime configuration.

-ascendc-erase-sync

Remove TQueBind synchronization operations and replace deque_tensor with allocated tensors

Removes intra-core synchronization infrastructure for static allocation mode:

  • Erases TQueBindEnqueTensorOp, SetFlagOp, WaitFlagOp, PipeBarrierOp

  • Replaces TQueBindDequeTensorOp with the tensor from corresponding TQueBindAllocTensorOp

This pass is used when tensors are statically allocated and queue-based synchronization is not needed.

-ascendc-fill-asc-operands

Fill default mask, repeat_times, and repeat_params operands for L0/L2 Ascend C operations

Populates default operands for vector operations based on tensor shape:

  • L0 operations: Fills mask, repeat_times, and repeat_params (stride values)

  • L2 operations: Fills cal_count with number of elements

Default stride values: block_stride=1, repeat_stride=8. Mask is computed based on element type size (32-bit: high=0, low=0xFFFFFFFF; 16-bit: high=low=0xFFFFFFFF). Repeat times calculated as ceil(numElements / numPerRepeat).

-ascendc-fixup-mmad-acc-params

Fix accumulator initialization for sequential mmad operations using the same destination tensor

Handles matrix multiplication accumulation where multiple mmad operations write to the same destination tensor. Creates a runtime boolean variable to track whether this is the first mmad to the destination:

  • First mmad: cmatrixInitVal=true (initialize accumulator)

  • Subsequent mmads: cmatrixInitVal=false (accumulate into existing values)

Only processes mmad operations that have cmatrixInitVal parameter in their mmad_params.

Example transformation:

// Before: mmad with fixed cmatrixInitVal in loop
%acc = ascendc.local_tensor_auto co1() : <16x16xf32>
scf.for %i = 0 to %n {
  %params = emitasc.init_struct !ascendc.mmad_params(
    "cmatrixInitVal" = %c1  // always true (reinit each iteration)
  )
  ascendc.mmad %acc, %a, %b, %params
}

// After: runtime variable controls initialization
%acc = ascendc.local_tensor_auto co1() : <16x16xf32>
%var = emitasc.variable true, memref<1xi1>  // init flag
scf.for %i = 0 to %n {
  %init_val = memref.load %var[%c0]
  %params = emitasc.init_struct !ascendc.mmad_params(
    "cmatrixInitVal" = %init_val  // true first iter, false after
  )
  ascendc.mmad %acc, %a, %b, %params
  %false = arith.constant false
  memref.store %false, %var[%c0]  // set false for next iter
}

-ascendc-fuse-bufid-sync

Remove redundant get_buf/rls_buf synchronization between consecutive operations with matching bufId

Optimizes BufId-based synchronization by removing redundant get_buf/rls_buf operations:

  • For consecutive operations with the same bufId attribute and same pipeline type:

    • Remove get_buf before middle operations (only first op needs it)

    • Remove rls_buf after middle operations (only last op needs it)

  • Pipeline types tracked: PIPE_V (vector), PIPE_MTE2 (GM→UB load), PIPE_MTE3 (UB→GM store)

After optimization, removes the bufId attribute from all operations. Used on Ascend 910_95 hardware.

-ascendc-generate-boilerplate

Insert required C++ include headers based on operations used in the kernel

Adds necessary Ascend C header includes at module start:

  • Always includes kernel_operator.h (core runtime)

  • If ListTensorDescOp or ListTensorDescV2Op present: includes kernel_operator_list_tensor_intf.h

  • If RegistMatmulObjOp present: includes lib/matmul_intf.h

  • If TensorDescOp present: includes kernel_operator_list_tensor_intf.h

-ascendc-hoist-que-bind

Move queue and buffer initialization operations to the function entry block

Hoists TQueBind-related initialization ops from loops to function root:

  • QueueOp, QueBindOp, TBufOp declarations

  • TPipeInitQueueOp, TPipeInitBufferOp initialization

  • TBufGetTensorOp tensor retrieval

This optimization ensures queue/buffer setup happens once at kernel entry rather than repeatedly in loops.

-ascendc-hoist-ub-allocation

Move LocalTensorAutoOp allocations from loops to the function entry block

Hoists LocalTensorAutoOp tensor allocations to the function root to reduce repeated allocations in loops. When exclude-in-out=true: only hoists tensors without input or output flags (pure temporary tensors). When exclude-in-out=false: hoists all tensors regardless of input/output status.

Input/output tensors are those connected to GM via DataCopyOp with gm_ubuf or ubuf_gm direction.

Options

-exclude-in-out : Keep input/output tensors inside loops, only hoist temporary tensors

-ascendc-input-output-tensor

Mark tensors as input/output based on GM data copy direction and handle input-only to output copies

Analyzes DataCopyOp to set input/output flags on LocalTensorAutoOp:

  • gm_ubuf direction → tensor is input (loaded from GM)

  • ubuf_gm direction → tensor is output (stored to GM)

Also handles special case: if an input-only tensor is used for ubuf_gm copy, creates a separate output tensor and inserts DataCopyL2Op to copy input tensor to output tensor before the GM store.

Example transformation:

// Before: unmarked tensors
%0 = ascendc.local_tensor_auto vecin() : <64xf32>
ascendc.data_copy_l2 %0, %gm_in, %c64  // gm→ub (input)
%1 = ascendc.local_tensor_auto vecout() : <64xf32>
ascendc.data_copy_l2 %gm_out, %1, %c64  // ub→gm (output)

// After: tensors marked with input/output flags
%0 = ascendc.local_tensor_auto vecin() input : <64xf32>
%1 = ascendc.local_tensor_auto vecout() output : <64xf32>

-ascendc-insert-bias-bufid-sync

Insert get_buf/rls_buf synchronization around MmadOp for bias tensors copied to BT

Inserts buffer synchronization (get_buf/rls_buf) around MmadOp operations that use bias tensors in C2 position (bias tensor memory).

This pass ensures proper synchronization between bias tensor access and matrix multiplication operations on Ascend 910_95 hardware. The pass:

  1. Finds LocalTensorV3Op in C2 position with a bufId attribute

  2. Extracts the bufId value from this tensor

  3. For each MmadOp with cmatrixSource parameter:

    • Inserts get_buf pipe_m, <bufId> before the MmadOp

    • Inserts rls_buf pipe_m, <bufId> after the MmadOp

Important: This pass assumes there is only one bias tensor with bufId per function. The UnifyBiasTensor pass consolidates all C2 tensors into a single tensor, ensuring only one bufId exists.

-ascendc-insert-bufid-sync

Insert get_buf/rls_buf synchronization around operations for BufId-based tracking on Ascend 910_95

Inserts BufId synchronization for Ascend 910_95 hardware using buffer ID tracking:

  1. Assigns unique bufId to each tensor allocation (LocalTensorV3Op, TBufGetTensorOp)

  2. For each operation using a tensor, inserts get_buf before and rls_buf after

  3. Tracks pipeline type per operation: PIPE_MTE1 (L1 load), PIPE_MTE2 (GM load), PIPE_MTE3 (GM store), PIPE_FIX (fixpipe), PIPE_S (scalar), PIPE_M (matmul), PIPE_V (vector)

  4. For scf.for loops (except vec_scope_loop or parallel), inserts MTE3-MTE2/MTE3-MTE1 sync at loop end

Sets bufId array attribute on operations for downstream FuseBufIdSync optimization.

-ascendc-insert-sync

Insert TQueBind enqueue/dequeue synchronization and scalar get/set_value barriers

Inserts intra-core synchronization for TQueBind-managed tensors and scalar operations:

  1. Enqueue tensors: After each OpWithDst, inserts TQueBindEnqueTensorOp or PipeBarrierOp(pipe_v)

  2. Dequeue tensors: For enqueued tensors, inserts TQueBindDequeTensorOp before first user

  3. Get/Set value sync: Wraps get_value/set_value with V_S/S_V event sync (fetch_event_id, set_flag, wait_flag)

  4. Loop handling: Special sync placement for set_value inside single-op loops

  5. Canonicalization: Adds PipeBarrierOp(pipe_all) at function end and runs canonicalization

TQueBind synchronization enables double-buffering and overlapping compute with data transfer.

Example: enqueue/dequeue synchronization:

// Before: tensor used directly after allocation
%tensor = ascendc.que_bind.alloc_tensor %queue
ascendc.add_l2 %dst, %tensor, %tensor, %count

// After: tensor dequeued before use, enqueued after compute
%tensor = ascendc.que_bind.alloc_tensor %queue
%dequeued = ascendc.que_bind.deque_tensor %queue
ascendc.add_l2 %dst, %dequeued, %dequeued, %count
ascendc.que_bind.enque_tensor %queue, %dst

Example: scalar get/set_value with V_S/S_V sync:

// Before: direct scalar access
%val = ascendc.local_tensor.get_value %tensor, %offset

// After: wrapped with event sync
%event = ascendc.pipe.fetch_event_id %pipe, v_s
ascendc.set_flag v_s, %event
ascendc.wait_flag v_s, %event
%val = ascendc.local_tensor.get_value %tensor, %offset
%event2 = ascendc.pipe.fetch_event_id %pipe, s_v
ascendc.set_flag s_v, %event2
ascendc.wait_flag s_v, %event2

-ascendc-legalize-kernel-args

Attach kernel argument attributes and insert ffts_addr handling for multi-core kernels

Processes kernel entry functions (marked with ascendc.global attribute):

  1. Marks all existing arguments as Explicit kernel arguments

  2. If setFftsAddr=true: adds ffts_addr memref argument and calls SetFftsBaseAddrOp

  3. If matmul present and not cube-only: inserts AscendIsAICOp check with FftsCrossCoreSyncOp for AIC mode

Kernel argument attributes (emitasc.kernel_arg) control how arguments are handled by the runtime.

Options

-set-ffts-addr : Append ffts_addr kernel argument and call set_ffts_base_addr for cross-core sync

-ascendc-lower-to-l0

Lower L2 vector operations to L0 intrinsics with default repeat parameters

Converts L2-level vector operations to L0-level hardware intrinsics:

  • Unary L2→L0: abs_l2→abs_l0, exp_l2→exp_l0, etc. with mask, repeat_times=0, empty repeat_params

  • Binary L2→L0: add_l2→add_l0, mul_l2→mul_l0, etc. with similar parameters

  • Special ops: duplicate_l2→duplicate_l0, cast_l2→cast_l0, compare_l2→compare_l0

  • Scalar ops: adds_l2→adds_l0, muls_l2→muls_l0

L0 operations provide direct hardware control but require explicit parameters; this pass provides defaults. L2 operations are simpler API with automatic parameter calculation (done by FillAscOperands).

-ascendc-materialize-tensor

Convert LocalTensorAutoOp placeholders to concrete TBuf or TQueBind allocations

Materializes tensor allocation placeholders into concrete Ascend C buffer objects:

  • TQueBind path (for input/output tensors when alwaysBuf=false): Creates QueueOp + TPipeInitQueueOp + TQueBindAllocTensorOp + TQueBindFreeTensorOp at function end

  • TBuf path (for temporary tensors or when alwaysBuf=true): Creates TBufOp + TPipeInitBufferOp + TBufGetTensorOp

Input tensors use VECIN queue position, output tensors use VECOUT. Temporary tensors use VECCALC TBuf. Buffer size computed from tensor shape (static or dynamic via shape operands).

Example transformation (TQueBind path for input tensor):

// Before: placeholder
%0 = ascendc.local_tensor_auto vecin() input : <64xf32>

// After: queue-based allocation
%pipe = ascendc.pipe
%queue = ascendc.queue : <vecin, 1>
ascendc.pipe.init_queue %pipe, %queue, %c1, %c256
%0 = ascendc.que_bind.alloc_tensor %queue : !ascendc.queue<vecin, 1>, !ascendc.local_tensor<64xf32>
// ... at function end:
ascendc.que_bind.free_tensor %queue, %0

Example transformation (TBuf path for temporary tensor):

// Before: placeholder
%0 = ascendc.local_tensor_auto veccalc() : <64xf32>

// After: buffer-based allocation
%pipe = ascendc.pipe
%tbuf = ascendc.tbuf : <veccalc>
ascendc.pipe.init_buffer %pipe, %tbuf, %c256
%0 = ascendc.tbuf.get_tensor %tbuf : !ascendc.tbuf<veccalc>, !ascendc.local_tensor<64xf32>

Options

-always-buf : Use TBuf for all tensors; otherwise use TQueBind for input/output tensors

-ascendc-noop

Placeholder pass that performs no transformations

A no-operation pass useful for testing, pipeline debugging, or as a placeholder in pass schedules. The pass walks the function but performs no modifications to the IR.

-ascendc-privatize-func

Mark non-kernel functions as private and kernel functions as public for emission

Adjusts function visibility based on kernel status:

  • Functions without ascendc.global attribute → set as private (internal helper functions)

  • Functions with ascendc.global and body → set as public (kernel entry points)

  • Functions with ascendc.global but no body (declarations) → remain as-is

This ensures only kernel entry points are exported while helper functions remain internal.

-ascendc-reuse-tensor-allocation

Reuse freed on-chip allocations based on unroll_iter lifetime analysis

Reuses freed on-chip buffer allocations by inserting LocalTensorReinterpretCastOp. Covers VECCALC (UB), A1 (L1), A2 (L0A), B2 (L0B), and CO1 (L0C) positions. Reuse is restricted to tensors of the same position, since each position maps to a distinct hardware buffer.

Reuse decisions are based on asctile.unroll_iter attribute: if top tensor’s max unroll_iter is less than bottom tensor’s min unroll_iter, their lifetimes don’t overlap and reuse is valid.

Input/output tensors cannot be reused across different unroll_iters because each iteration loads/stores different data. Reusing would cause data races. Input/output tensors can only be reused with tensors that have the same unroll_iter or no unroll_iter at all.

For in/out tensors: all users share the same unroll_iter (guaranteed by upstream passes), so max == min. Two in/out tensors at different iterations can reuse each other. Two in/out tensors at the same iteration overlap and cannot reuse.

For tensors without unroll_iter: fall back to position-based ordering within their scope.

For cube positions (A1, A2, B2), allocation size is computed with cube-block alignment to match the downstream AllocateTensor pass.

When reusing: larger tensor wraps smaller via reinterpret_cast, or smaller moved before larger and cast.

-ascendc-reuse-ub-allocation

Reuse freed UB allocations via reinterpret_cast to reduce memory consumption

Analyzes tensor lifetimes and reuses freed UB allocations by inserting LocalTensorReinterpretCastOp:

  1. For each block, collects LocalTensorAutoOp tensors and their last users

  2. Sorts tensors by last user position (earlier-freed tensors can be reused first)

  3. Checks reusability: same block, non-overlapping lifetimes, static shape, VECCALC position

  4. When reusing: larger tensor wraps smaller via reinterpret_cast, or smaller moved before larger and cast

Special handling for control flow: tensors used across different regions (if/else, while before/after) can be reused. Input/output tensors in same loop are not reused unless reuse-in-out=true.

Example: two tensors with non-overlapping lifetimes reused:

// Before: separate allocations for sequential tensors
%0 = ascendc.local_tensor_auto veccalc() : <333xi32>
ascendc.data_copy_l2 %0, %gm, %c1
%1 = ascendc.local_tensor_auto veccalc() : <333xi32>
ascendc.data_copy_l2 %1, %gm, %c1

// After: single allocation with reinterpret_cast
%0 = ascendc.local_tensor_auto veccalc() : <333xi32>
%1 = ascendc.reinterpret_cast %0 : !ascendc.local_tensor<333xi32> to !ascendc.local_tensor<333xi32>
ascendc.data_copy_l2 %0, %gm, %c1
ascendc.data_copy_l2 %1, %gm, %c1

Example: tensors in different loops can be reused:

// Before: output tensor in loop 1, input tensor in loop 2
scf.for { %output = ascendc.local_tensor_auto veccalc() output }
scf.for { %input = ascendc.local_tensor_auto veccalc() input }

// After: output reused as input across different loops
%output = ascendc.local_tensor_auto veccalc() output
%input = ascendc.reinterpret_cast %output
scf.for { use %output }
scf.for { use %input }

Options

-reuse-in-out : Allow reuse between input/output tensors even in same loop

-ascendc-unify-bias-tensor

Unify C2 (bias) tensor with identical type and size, setting addr to 0

Consolidates all LocalTensorV3Op operations in C2 position (bias tensor memory) into a single tensor.

When multiple C2 tensors exist (typically created during loop unrolling or tiling), this pass:

  1. Keeps the first encountered C2 tensor

  2. Replaces all uses of subsequent C2 tensors with the first tensor

  3. Erases the duplicate tensor operations

  4. Sets the address of the unified tensor to 0

This optimization ensures a single bias tensor allocation per function, which is required for correct bufId-based synchronization in InsertBiasBufIdSync pass.

-ascendc-unify-pipe

Replace multiple PipeOp instances with a single unified pipe at function entry

Consolidates multiple PipeOp declarations into a single pipe object at function entry. All uses of individual pipes are replaced with the unified pipe, and original PipeOp instances are erased.

This ensures consistent pipe management across the kernel and reduces redundant TPipe object creation.

-ascendc-verify-sync

Validate TQueBind synchronization correctness and emit warnings for mismatches

Verifies proper pairing of TQueBind operations and emits warnings for incorrect usage:

  • alloc_tensor without corresponding free_tensor

  • free_tensor for already-freed tensor or without alloc_tensor

  • enque_tensor without corresponding deque_tensor

  • deque_tensor without enque_tensor in queue

  • Unexpected tensor uses between enque_tensor and deque_tensor

This is a verification-only pass that helps detect synchronization bugs during development.