Docs 2.0.0 · 15 min read

Native code: @native, parallel for and @gpu

Apsi 2.0 compiles functions to machine code. A function marked @native is specialised on the types and units of its arguments, translated to LLVM IR, optimised at -O3 for the exact CPU it runs on (AVX2, AVX-512, FMA, NEON), and called directly — no interpreter in between. parallel for spreads a loop over all CPU cores, and @gpu runs it on NVIDIA GPUs.

use physics.constants

@native
fn orbit_speed(m: Mass, r: Length) -> Velocity = sqrt(G * m / r)

@native
fn energy(x, v, k: Float) -> Float:
    var e = 0.0
    parallel for i in 0..len(x):              # all CPU cores
        e += 0.5 * v[i] * v[i] + 0.5 * k * x[i] * x[i]
    return e

print(orbit_speed(5.972e24kg, 384400[km]) as [km/s])     # 1.01828904573 km/s

Nothing else changes: native functions take and return ordinary Apsi values, call and are called by any other function, and work inside evolve. Results are identical to the bytecode engine.

• How it works
• Performance
• What compiles
• Semantics and safety
• Tensors
• Units
• parallel for
• @gpu
• Tools and settings
• Limits

How it works

 .apsi source ──▶ parser ──▶ checker (names, units, types) ──▶ checked syntax tree
                                                                    │
         first call with new argument types (Int, Float, f32, tensors of a dtype/rank/layout, units)
                                                                    ▼
   specialiser: infers every variable's type by abstract execution, checks units statically and erases them,
   proves index ranges, fuses tensor expressions, outlines parallel loops
                                                                    │
                                                                    ▼
   LLVM IR ──▶ LLVM -O3 (inlining, loop + SLP vectorisation, unrolling) for the host CPU ──▶ machine code
                                                                    │        (signed on-disk cache)
                                                                    ▼
   NativeFunction: checks argument types/units at the boundary, calls the machine code, restores units on the result

• Specialisation. f(2.0) and f(tensor) get separate machine code, each as fast as hand-written code for that type. The first call compiles (typically 10–50 ms); later calls go straight to machine code.
• Machine-code cache. Compiled code is stored in ~/.apsi/cache/native, signed with HMAC-SHA256 using your per-user key; an entry whose signature does not verify is never loaded. The cache key covers the IR, the CPU model and features, and the LLVM and Apsi versions, so code is only reused where it is valid.
• Fast calls. A cheap check of the argument types finds the machine code directly; numbers are passed in registers and tensors as the NumPy object itself (the machine code reads its data pointer, shape and strides from NumPy's struct, whose layout is measured and verified at start-up). A call costs ~0.9 µs, ~1.1 µs with a tensor (it was ~8–10 µs in the first 2.0 build).
• Never breaks a program. If a function uses something the native compiler does not handle, Apsi prints one note with the reason and line, and that function runs on the bytecode engine. Arguments native code cannot take (lists, records, structs, values being differentiated by grad) also use the bytecode engine, so grad(f) of a native f works.
• Translated once, then direct. A processor only executes its own machine instructions, so every language is translated at some point: C, C++, Rust and Go before you ship, Apsi the first time a function is called with new argument types. LLVM IR is the in-memory form the optimiser works on, not an extra layer at run time. After that first call — and in every later run, through the signed cache — the call goes straight into machine code with no translation, interpretation or Python in between. Measured on a 2-vCPU machine: the first compilation of a small loop takes about 90 ms; later runs load it from the cache in about 30 ms, most of which is loading the LLVM library once per process; each call then costs under 1 µs.
• One program, two engines. Native and bytecode functions call each other freely. Calls between native functions are direct machine-code calls, usually inlined.

Performance

Same algorithms, same inputs, best of 5 runs, milliseconds (lower is better). Every language produces the same checksums. Measured on a 2-vCPU cloud machine (Intel Xeon, Emerald Rapids, AVX-512); GCC 13.3, Clang 18.1, Rust 1.97, Go 1.24, Apsi 2.0 with LLVM 22.

The Apsi columns before the last one are single-core. The last column uses both cores of this machine: through parallel for, or automatically for large fused tensor expressions and reductions (the sum(sqrt(…)) row runs on both cores without any change to the code; @native without fast takes 4.95 ms there). C, C++, Rust and Go were measured as written, single-threaded.

How to read it:

• Against the 1.x bytecode engine: 100× to 3,500× faster on loop-heavy code.
• Against C, C++, Rust and Go: @native("fast") is the fastest or tied single-threaded result on four of the seven benchmarks and within 6–20 % of the best on n-body and matmul. Recursive fib remains about 2× behind GCC, which applies recursion-specific transformations, and is faster than Go.
• Default @native keeps checks those languages' release builds do not have (64-bit integer overflow, every index, integer division by zero) and still lands in the same range as Clang, Rust and Go on most benchmarks.
• Why no compiler is 100× faster than C on the same algorithm: C, C++ and Rust already compile to machine code that runs close to what the processor can do; Apsi uses the same LLVM optimiser. Beyond that point, more speed comes from more hardware: parallel for scales with the number of cores (this machine has 2; a 16-core desktop gives up to about 16× over single-threaded C), and @gpu uses the thousands of cores of a GPU. Apsi does this for you: large tensor kernels split across cores automatically, while in C, Rust or Go you would add threads by hand.

Reproduce: sh benchmarks/run_all.sh (needs gcc, g++, rustc, go; clang optional).

Heavy workloads against C, C++, Rust, Go and Mojo

A second suite (benchmarks/heavy/) uses larger, harder programs: dense all-pairs gravity, a PDE stencil, a 1024² matrix product, LU factorisation with pivoting, Ising Monte Carlo, a 100-million-cell dynamic program, a 20-million-step Runge–Kutta integration and k-means clustering. Each is written the same way in Apsi (@native, strict IEEE), C, C++, Rust, Go and Mojo 1.2 (with raw pointers, as performance-oriented Mojo is written); the runner refuses to report times unless all six produce the same checksum. Single core, best of 3, milliseconds:

Run it on your own machine with python3 benchmarks/heavy/run.py (languages whose compiler is missing are skipped).

What compiles

Not compiled yet (the function runs on the bytecode engine and the note says which): strings other than in print/assert, lists, dicts, records, structs, lambdas and closures as values, comprehensions, evolve inside a native function (an evolve that calls native functions is fine), @ on tensors with more than 2 axes, reductions with axis=, reading a global declared with var (pass it as an argument instead), and calls to Python libraries.

Semantics and safety

Native code means the same thing as the bytecode engine. Default @native keeps every check:

@native("fast") is for code that is known to be correct and must be as fast as possible: integer overflow and index checks are removed, and floating-point reassociation and FMA are allowed, so sums may differ in the last digits (like C with -O3 -ffast-math). Use it after the program works with @native.

@native("strict") turns "cannot compile" into an error instead of a note — useful in tests and CI.

Tensors

• Element loops compile to direct memory access: a[i, j] on a contiguous tensor is one address computation. Tensors passed in are used in place (no copy); writes are visible to the caller, as in Apsi.
• Views: m[i] (a row) and a[lo:hi] are views of the same memory, without copying.
• Fused expressions: sqrt(a * a + b * b) + 0.5 * a is computed in one pass with no temporary tensors. Shapes broadcast like NumPy (m * v with m: (6, 5) and v: (5,)); incompatible shapes are an error.
• Reductions fuse too: sum(a * b), mean, min, max, argmin, argmax, prod, dot, norm read their input once without building it.
• All cores automatically: fused expressions and sum/mean/prod/min/max over 131,072 elements or more are split into the same fixed 256 chunks as parallel for and run on every core, without any change to the code (sum(sqrt(a*a + b*b)) over 4M elements: 11.3 ms on one core, 5.2 ms on two). Sums are combined chunk by chunk in a fixed order, so the result is identical for every thread count; it can differ in the last bits from a strictly sequential sum, as NumPy's pairwise sums do.
• New tensors: zeros(n, m), ones, full, zeros_like, results of tensor expressions. Tensors created inside a loop iteration are freed at the end of the iteration when they do not outlive it; a tensor that is returned becomes a normal Apsi tensor with no copy.

Units

Units are checked when a function is compiled and then erased: the machine code works on plain SI numbers, so physical units cost nothing at run time, and the result gets its unit back.

@native
fn kinetic(m: Mass, v: Velocity) -> Energy = 0.5 * m * v ** 2
print(kinetic(2kg, 3[m/s]))          # 9.0 J

@native
fn wrong(x: Length, t: Time) = x + t # UnitError: cannot add Length and Time (at compile time)

Arguments are checked against the parameter annotations on every call, so kinetic(2kg, 3s) is still a UnitError. Calls from unit-erased code (inside evolve) use a separate specialisation without units.

parallel for

@native
fn mean_and_max(x):
    var total = 0.0
    var biggest = -inf
    parallel for i in 0..len(x):
        total += x[i]
        biggest = max(biggest, x[i])
    return total / len(x), biggest

• Iterations run on a pool of threads, one per core (APSI_THREADS=n, native_threads(n) or apsi run --threads n to change). The dispatcher and the worker loop are machine code: workers stay awake for a few milliseconds after a loop, spinning on a shared counter with the CPU's pause instruction, so back-to-back parallel loops start in about 5 µs with no operating-system call (Python's thread pool needed ~70 µs). Idle workers then sleep and use no CPU (APSI_SPIN sets the spin length). Threads claim several consecutive chunks per atomic operation, so cores do not fight over one cache line, and every field threads share has its own cache line.
• Reductions (+=, -=, *=, x = min(x, …), x = max(x, …)) use private copies that are combined in a fixed order. The work is always split into the same 256 chunks, so the result is bit-identical for any number of threads and on every machine.
• Outer variables that are not reductions are read-only inside the loop; a loop that assigns them, or contains return, break or print, runs sequentially, and the note says why.
• A parallel for inside another one runs sequentially in the thread that reaches it.
• Outside native code parallel for is an ordinary for, so programs run unchanged on every engine.

@gpu

@gpu
fn saxpy(a: Float, x, y):
    parallel for i in 0..len(x):
        y[i] = a * x[i] + y[i]

@gpu
fn sum_of_squares(x) -> Float:
    var s = 0.0
    parallel for i in 0..len(x):
        s += x[i] * x[i]
    return s

A @gpu function runs its parallel for with one GPU thread per iteration.

• Requirements: an NVIDIA GPU with compute capability 7.5 or newer (GTX 1660 / RTX 20xx and every newer generation) and the normal NVIDIA display driver, on Windows 10/11 or Linux. No CUDA toolkit: Apsi generates PTX itself (LLVM NVPTX backend, sm_75), and the driver compiles it for the installed GPU and caches it.
• Several GPUs: the iterations are split evenly across all GPUs (choose them with APSI_GPUS=0,1). When every tensor write is to row i of the loop (out[i] = …, out[i, j] = …), each GPU copies back only its own rows; otherwise one GPU is used. Reductions from all GPUs are added together.
• Shape of a GPU function: optional let/var lines, one parallel for over a range, then optionally return of a += reduction variable. The loop body may use arithmetic, comparisons, if/while/for, tensor element reads and writes, sqrt abs floor ceil min max where clip round int float and functions that follow the same rules. exp, log, trigonometric functions, ** with exponents other than 1 and 2 (any integer up to 16 with @gpu("fast")), Float // and %, print and creating tensors are not available inside GPU kernels yet.
• Always runs: without a usable GPU, or for a function outside these rules, the same function runs as native code on all CPU cores, with the same results; one note says why.
• Data movement: tensor arguments are copied to GPU memory for each call and written tensors are copied back. This suits work that is large compared with the data (many operations per element). Consumer GPUs compute f32 far faster than f64: pass f32 tensors for speed.
• Precision: IEEE f64/f32 arithmetic; sqrt is correctly rounded, as on the CPU.
• Status: PTX generation, the kernel rules and the CPU path are covered by the test suite. The launch path uses the CUDA driver API (nvcuda.dll / libcuda.so.1) and needs a GPU to run, so it is not exercised by continuous integration on GPU-less machines. apsi native info and gpu_info() show the GPUs Apsi can use; apsi doctor reports them too.

For training neural networks on the GPU, the PyTorch backend (fit(…, backend="torch"), see BACKENDS.md) remains the right tool: it brings cuDNN, tensor cores and mixed precision. @gpu is for your own loops and kernels.

Tools and settings

The native compiler needs llvmlite (installed with Apsi; it contains LLVM, nothing else to install). Without it, @native functions run on the bytecode engine and apsi native info says why.

Limits

• Native Int is 64-bit; bigger integers stay on the bytecode engine.
• Functions with shape parameters (fn f[N](...)) and parameters typed with structs run on the bytecode engine.
• Several Python threads calling native functions at the same moment share one error slot; run parallel work with parallel for instead of Python threads.
• The playground (WebAssembly) has no native compiler; @native functions run on the bytecode engine there.

Found a mistake? Report it · Markdown source