Runtime API

JIT decorator

@asc.experimental.asctile.jit(fn: Callable[[P], T]) → JITFunction[P, T]

@asc.experimental.asctile.jit(**options) → Callable[[Callable[[P], T]], JITFunction[P, T]]

Instantiate a JIT function using the default options. This should be used as a decorator for the kernel function:

@asctile.jit
def kernel(x, y):
    ...

JIT options may be provided as keyword arguments to be applied to the decorated kernel function. See asc.CodegenOptions, asctile.CompileOptions, asc.LaunchOptions for the details.

JIT options

Attributes from the following classes can be used as keyword arguments in asc.experimental.asctile.jit decorator.

Tip

The user can mix options from different sections. For example, to disable CodegenOptions.capture_exceptions and set CompileOptions.opt_level to 2, the kernel function may be decorated as the following:

@asctile.jit(capture_exceptions=False, opt_level=2)
def kernel(x, y):
    ...

class asc.CodegenOptions(capture_exceptions: bool = True, ir_multithreading: bool = True, custom_builtins: ~asc.codegen.function_visitor.CustomBuiltins = <factory>)

Code generation and AST traversal options

capture_exceptions: bool = True: Capture all exceptions raised by functions called by code generator internally and show the corresponding location in the user function code being traversed that caused an exception instead. Usually, it must always be enabled, but may be disabled for the language debugging purposes.

ir_multithreading: bool = True: Enable parallel processing for the current IR context if possible. Usually, it must always be enabled, but may be disabled for the IR debugging purposes.

class asc.experimental.asctile.CompileOptions(debug: bool = False, strip_loc: bool = False, verify_sync: bool = False, print_ir_before_all: bool = False, run_passes: bool = True, kernel_type: KernelType | None = None, auto_sync: bool | None = True, auto_sync_log: str | None = '', matmul_cube_only: bool = False, insert_sync: bool = True, always_compile: bool = False, bisheng_options: Tuple[str, ...] | None = None, opt_level: int = 3, vf_vec_len: int | None = None, cv_ratio: Literal[1, 2] = 1, reuse_alloc: Literal[0, 1, 2] = 0, static_alloc: bool | None = None, vf_fusion: bool = False)

Binary compilation and IR transformation options (for asctile kernels)

cv_ratio: Literal[1, 2] = 1: The ratio between Vector and Cube units. For example, Ascend950PR_9599 cores have two vector sub-cores per each cube sub-core, hence the supported ratio values are 1 and 2.

debug: bool = False

Enable debug mode for the kernel.

When True, device-side debug operations (device_print, device_assert) are active during kernel execution. When False (the default), these operations are removed from the compiled kernel.

Use this option to enable debugging output and assertions on the device. Note that debug operations may impact performance and should be disabled in production builds.

insert_sync: bool = True: Insert synchronization instructions automatically. This feature is enabled by default, which is usually a must, but may be disabled for the debugging purposes.

reuse_alloc: Literal[0, 1, 2] = 0

Try to reduce the on-chip memory usage by replacing the tensors with those allocated earlier but became unused. Having this feature enabled may help to avoid memory overflow but may introduce performance regressions.

Value	Effect
`0`	Disable the feature (default)
`1`	Enable the feature, use a legacy implementation
`2`	Enable the feature, use a modern implementation (recommended)

static_alloc: bool | None = None

Perform static allocation for tiles instead of relying on Ascend C TPipe backend. The static allocation feature may help to reduce an overhead caused by scalar code.

This feature is enabled by default on supported platforms (such as Ascend950PR_9599).

vf_fusion: bool = False: Fuse groups of consecutive vector operations into VF blocks using Ascend C register API. This feature may help to eliminate unnecessary memory transfers and improve data locality.

always_compile: bool = False: Always run full compilation pipeline instead of fetching a cached objects from the prevoius runs. This option may be useful for testing and parallel execution.

bisheng_options: Tuple[str, ...] | None = None

Append extra arguments to the bisheng command line used to produce an object file for the kernel. Please, run bisheng --help to get the list of options supported by the compiler.

Note: To pass a single argument, use ("-O3",) (with a trailing comma). Without the comma, ("-O3") is just a string, not a tuple.

opt_level: int = 3: Optimization level for the Bisheng compiler. Supported values are 1, 2, 3. Typically, this parameter affects the -O argument of the command line for the compiler.

vf_vec_len: int | None = None: Vector register length for the C310 architecture. This option is only available on supported platforms (such as Ascend950PR_9599). The default value is 256.

class asc.LaunchOptions(core_num: int | None = None, stream: c_void_p | None = None)

Kernel launch and device runtime options.

These options can also be used as positional arguments in [ brackets ] when launch JIT function:

@asc.jit
def kernel(x_ptr, y_ptr):
    ...

def launch(x, y, core_num = 16):
    kernel[core_num](x, y)

core_num: int | None = None: Number of active execution blocks (AI cores) used to launch the kernel. By default, all cores available on the current platform will be used.

Platform configuration

`asc.experimental.asctile.Backend`	Execution backend for kernel compilation and execution.
`asc.experimental.asctile.Platform`	Ascend NPU platform types.

asc.experimental.asctile.set_platform(backend: Backend | str, soc_version: Platform | str | None = None, device_id: int | None = None, check=True) → None

Configure the execution platform and backend for kernel execution.

Sets up the runtime environment by specifying the backend (Model simulator or NPU hardware) and optionally the SoC version and device ID. For the Model backend, a default platform is used if none is specified. For the NPU backend, the actual hardware platform must match the specified soc_version.

Parameters:

backend – Execution backend type, either Backend.Model for simulator or Backend.NPU for hardware execution. Can be specified as a Backend enum or string (“Model” or “NPU”).
soc_version – Target SoC platform version. Can be specified as a Platform enum or string. Required for Model backend; must match actual hardware for NPU backend. Defaults to Ascend910B1 for Model backend if not specified.
device_id – Device ID to use for execution. If specified, sets the device for subsequent kernel executions. Uses 0-th device otherwise.
check – Whether to verify runtime library availability. If True, raises an error if the library is not available.

Raises:

ValueError – If the backend type is unknown, or if the specified soc_version does not match the actual hardware platform when using NPU backend.
RuntimeError – If check is True and the runtime library is not available.

Example

>>> set_platform(Backend.Model, Platform.Ascend910B1)
>>> set_platform("NPU", device_id=0)
>>> set_platform(Backend.Model, "Ascend910B3", check=False)