Runtime API

JIT decorator

@asc2.jit(fn: Callable[[P], T]) → JITFunction[P, T]

@asc2.jit(**options) → Callable[[Callable[[P], T]], JITFunction[P, T]]

Instantiate a JIT function using the default options. This should be used as a decorator for the kernel function:

@asc2.jit
def kernel(x, y):
    ...

JIT options may be provided as keyword arguments to be applied to the decorated kernel function. See asc.CodegenOptions, asc.CompileOptions, asc.LaunchOptions for the details.

JIT options

Attributes from the following classes can be used as keyword arguments in asc2.jit decorator.

Tip

The user can mix options from different sections. For example, to disable CodegenOptions.capture_exceptions and set CompileOptions.opt_level to 2, the kernel function may be decorated as the following:

@asc2.jit(capture_exceptions=False, opt_level=2)
def kernel(x, y):
    ...

class asc.CodegenOptions(capture_exceptions: bool = True, ir_multithreading: bool = True, custom_builtins: ~asc.codegen.function_visitor.CustomBuiltins = <factory>)

Code generation and AST traversal options

capture_exceptions: bool = True: Capture all exceptions raised by functions called by code generator internally and show the corresponding location in the user function code being traversed that caused an exception instead. Usually, it must always be enabled, but may be disabled for the language debugging purposes.

ir_multithreading: bool = True: Enable parallel processing for the current IR context if possible. Usually, it must always be enabled, but may be disabled for the IR debugging purposes.

class asc.CompileOptions(debug: bool = False, strip_loc: bool = False, verify_sync: bool = False, print_ir_before_all: bool = False, run_passes: bool = True, kernel_type: KernelType | None = None, auto_sync: bool | None = True, auto_sync_log: str | None = '', matmul_cube_only: bool = False, always_compile: bool = False, bisheng_options: Tuple[str, ...] | None = None, opt_level: int = 3, run_asc2_passes: bool = False, densify_load_store: bool = False, insert_sync: bool | None = None, reuse_alloc: Literal[0, 1, 2] = 0, static_alloc: bool | None = None, vf_fusion: bool = False, vf_vec_len: int | None = None)

Binary compilation and IR transformation options

always_compile: bool = False: Always run full compilation pipeline instead of fetching a cached objects from the prevoius runs. This option may be useful for testing and parallel execution.

bisheng_options: Tuple[str, ...] | None = None

Append extra arguments to the bisheng command line used to produce an object file for the kernel. Please, run bisheng --help to get the list of options supported by the compiler.

Note: To pass a single argument, use ("-O3",) (with a trailing comma). Without the comma, ("-O3") is just a string, not a tuple.

opt_level: int = 3: Optimization level for the Bisheng compiler. Supported values are 1, 2, 3. Typically, this parameter affects the -O argument of the command line for the compiler.

run_asc2_passes: bool = False: Enable PyAsc2 compilation pipeline. This option is enabled automatically when @asc2.jit decorator is used.

densify_load_store: bool = False: Densify asc2.copy_in and asc2.copy_out statements by grouping them together. This feature cannot be enabled at the same time as reuse_alloc.

Warning

This is an experimental feature. It might or might not cause functional or performance regressions.

insert_sync: bool | None = None: Insert synchronization instructions automatically. This feature is enabled by default, which is usually a must, but may be disabled for the debugging purposes.

reuse_alloc: Literal[0, 1, 2] = 0

Try to reduce the on-chip memory usage by replacing the tensors with those allocated earlier but became unused. Having this feature enabled may help to avoid memory overflow but may introduce performance regressions.

Value	Effect
`0`	Disable the feature (default)
`1`	Enable the feature, use a legacy implementation (recommended)
`2`	Enable the feature, use an experimental implementation

This feature cannot be enabled at the same time as densify_load_store.

static_alloc: bool | None = None

Perform static allocation for tiles instead of relying on Ascend C TPipe backend. The static allocation feature may help to reduce an overhead caused by scalar code.

This feature is enabled by default on supported platforms (such as Ascend950PR_9599).

vf_fusion: bool = False: Fuse groups of consecutive vector operations into VF blocks using Ascend C MicroAPI. This feature may help to eliminate unnecessary memory transfers and improve data locality.

vf_vec_len: int | None = None: Vector register length for the C310 architecture. This option is only available on supported platforms (such as Ascend950PR_9599). The default value is 256.

class asc.LaunchOptions(core_num: int | None = None, stream: c_void_p | None = None)

Kernel launch and device runtime options.

These options can also be used as positional arguments in [ brackets ] when launch JIT function:

@asc2.jit
def kernel(x_ptr, y_ptr):
    ...

def launch(x, y, core_num = 16):
    kernel[core_num](x, y)

core_num: int | None = None: Number of active execution blocks (AI cores) used to launch the kernel. By default, all cores available on the current platform will be used.

Platform configuration

`asc2.Backend`	Execution backend for kernel compilation and execution.
`asc2.Platform`	Ascend NPU platform types.

asc2.set_platform(backend: Backend | str, soc_version: Platform | str | None = None, device_id: int | None = None, check=True) → None

Configure the execution platform and backend for kernel execution.

Sets up the runtime environment by specifying the backend (Model simulator or NPU hardware) and optionally the SoC version and device ID. For the Model backend, a default platform is used if none is specified. For the NPU backend, the actual hardware platform must match the specified soc_version.

Parameters:

backend – Execution backend type, either Backend.Model for simulator or Backend.NPU for hardware execution. Can be specified as a Backend enum or string (“Model” or “NPU”).
soc_version – Target SoC platform version. Can be specified as a Platform enum or string. Required for Model backend; must match actual hardware for NPU backend. Defaults to Ascend910B1 for Model backend if not specified.
device_id – Device ID to use for execution. If specified, sets the device for subsequent kernel executions. Uses 0-th device otherwise.
check – Whether to verify runtime library availability. If True, raises an error if the library is not available.

Raises:

ValueError – If the backend type is unknown, or if the specified soc_version does not match the actual hardware platform when using NPU backend.
RuntimeError – If check is True and the runtime library is not available.

Example

>>> asc2.set_platform(asc2.Backend.Model, asc2.Platform.Ascend910B1)
>>> asc2.set_platform("NPU", device_id=0)
>>> asc2.set_platform(asc2.Backend.Model, "Ascend910B3", check=False)