Docs 2.4.4 · 11 min read

From Python to Apsi (apsi convert)

apsi convert translates Python files — or whole projects — into Apsi with Python's exact semantics: the converted program prints the same text, raises the same errors, trains to the same losses and writes the same files as the Python program, byte for byte. Large real programs convert in seconds and run immediately.

apsi convert script.py                 # writes script.apsi (and .apsi files for the modules it imports), checks them
apsi convert script.py --run           # convert, check and run
apsi convert src/ -o apsi_src/         # a whole project, keeping the folder layout
apsi convert script.py --stdout        # print the Apsi code
apsi convert script.py --idiomatic     # map numpy/math/torch onto Apsi's own built-ins (small numeric scripts)

In VS Code, Cursor and the other VS Code-family editors: right-click a .py file (or a folder) → Apsi: Convert Python File to Apsi.

Typical output:

✓ build_graph_cache.py → build_graph_cache.apsi  (0 notes)
✓ GLA.py → GLA.apsi  (0 notes)
    run with --allow-python=all (it uses subprocess)
✓ gla_brain_dual.py → gla_brain_dual.apsi  (0 notes)

converted 5 files, 0 notes

Modules that sit next to the program (import GLA, from gla_brain_dual import X, packages with __init__.py, from . import x) are converted with it and imported as Apsi files; everything else is loaded from Python.

How exact is exact?

The converted file starts with use pycompat. That line gives the file Python's object model and built-ins while the text stays Apsi (fn, let/var, none/true/false, import "x.apsi"). Concretely:

Converted programs run at the speed of the Python program — attribute access, calls and loops compile to the same Python bytecode — and keep every Apsi feature: use rt, use hp, @native functions, units, the checker. Measured on 2 cores (benchmarks/python/cmp.sh, best of five, interleaved):

Programs that create many objects are faster because Apsi freezes everything loaded before the program starts (the cycle collector never scans it again) and collects the youngest generation every 50,000 allocations instead of 2,000; reference counting is untouched, so objects are still freed the moment they are no longer used (APSI_GC=default restores Python's settings). Pure arithmetic in Python loops runs on the same bytecode interpreter as CPython, so it runs at the same speed; to go far beyond it, put @native on the hot function (machine code through LLVM, see native code).

Conversion itself: the 23,769-line program and three of its modules (29,022 lines) convert and check in 4.5 s.

Permissions — what a converted program may touch

Apsi's sandbox stays on. Without any flag a converted program can use:

• every pure standard module (collections, itertools, functools, re, json, dataclasses, enum, typing, math, random, decimal, time, argparse, threading, queue, datetime, hashlib …)
• sys, os, os.path, pathlib and numpy as permission-checked stand-ins: the same names and behaviour, but file access asks --allow-read/--allow-write, environment variables ask --allow-env, and the parts that would escape the sandbox (os.system, subprocess, sys.modules, numpy.ctypeslib, np.load(allow_pickle=True), array.tofile) are not there
• open() checked against --allow-read / --allow-write

Other libraries need --allow-python (for example torch, scipy, pandas), and libraries that reach the operating system or the network (subprocess, socket, http, ctypes, multiprocessing, shutil …) need --allow-python=all; with =all every module is the real one and nothing is filtered. apsi convert prints the flag a file needs, and apsi run says so when one is missing:

apsi run train.apsi --allow-python                     # torch, numpy, scipy …
apsi run GLA.apsi --allow-python=all -- train --epochs 3   # arguments after -- go to the program (sys.argv)

Introspection that leads to interpreter internals (__globals__, __subclasses__, __builtins__, frame objects, object.__getattribute__, __dict__ of library classes, operator.attrgetter("__…"), eval/exec/__import__) is refused unless the program was started with --allow-python=all.

Checked with real programs

• The Python standard library itself. tests/test_stdlib_convert.py copies 34 pure-Python standard modules next to a driver program — _pydecimal (6,357 lines of arbitrary-precision decimal arithmetic), _pydatetime, argparse, configparser, tomllib, json (a package with relative imports), pickle, dataclasses, enum, random, statistics, fractions, difflib, ipaddress, textwrap, pprint, calendar, shlex, graphlib, heapq, bisect, html.parser, urllib.parse, base64, fnmatch, string, copy, contextlib, operator, reprlib, numbers, _py_abc, colorsys, keyword — converts all of them with the program, runs it, and requires byte-identical output to CPython running the original files. About 38,000 lines of metaclasses, descriptors, generators, __slots__, name mangling, __class_getitem__, exception chaining and 50-digit arithmetic.
• tests/pycompat/ — programs with classes, inheritance from library classes, dataclasses, enums, abstract base classes, context managers, generators with send, closures, threads, collections, heapq, functools, re, json, numpy and every operator. tests/test_python_compat.py converts each one and requires byte-identical output.
• A 23,769-line AI training program with its four companion modules (graph building for 30 languages, web search, safety filter, a dual-brain model) converts in about 3 seconds and checks cleanly; its command-line tools, its multi-process graph-cache builder (results pickled across worker processes, resumable output) and its safety filter produce byte-identical files and output to Python.
• A PyTorch-style training program (an nn.Module hierarchy that registers parameters through __setattr__, autograd, Adam with weight decay, cosine schedule, early stopping, checkpoint) trains to bit-identical losses.

Coverage

Every statement and expression of Python 3.13 converts: there is no "kept as a comment" list any more. Two things differ by design and are not part of a program's behaviour:

• Tracebacks of an uncaught error name the .apsi file and line (train.apsi:212:1: TypeError: …) instead of the .py file; the error type and message are Python's. Programs that catch errors see the real exception objects.
• The sandbox. Without --allow-python=all, introspection that leads to interpreter internals is refused (see above). With it, eval, exec, globals(), frames and every module behave exactly as in Python.

A converted module is reused while it is newer than its .py file and was written by the same Apsi version; a new Apsi version converts it again.

--idiomatic: PyTorch programs → native Apsi (no Python, no PyTorch)

A PyTorch training script becomes an Apsi program that runs without Python and without PyTorch. The converter targets use torch, Apsi's own implementation of the PyTorch API: the same layer names, constructor arguments, parameter names (fc1.weight stored as [out, in]), buffers, losses, optimizers and learning-rate schedulers, implemented natively on Apsi's automatic differentiation. Nothing is approximated and nothing is left for you to finish by hand.

import torch
import torch.nn as nn
import torch.nn.functional as F

class MLP(nn.Module):
    def __init__(self, d_in, hidden, d_out):
        super().__init__()
        self.fc1 = nn.Linear(d_in, hidden)
        self.bn = nn.BatchNorm1d(hidden)
        self.out = nn.Linear(hidden, d_out)
        self.drop = nn.Dropout(0.1)

    def forward(self, x):
        return self.out(self.drop(F.relu(self.bn(self.fc1(x)))))

X = torch.randn(256, 4)
y = (X[:, 0] > 0).long()
model = MLP(4, 32, 2)
opt = torch.optim.AdamW(model.parameters(), lr=1e-2)
sched = torch.optim.lr_scheduler.CosineAnnealingLR(opt, T_max=200)
loss_fn = nn.CrossEntropyLoss(label_smoothing=0.1)
for epoch in range(200):
    opt.zero_grad()
    logits = model(X)
    loss = loss_fn(logits, y)
    loss.backward()
    torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
    opt.step()
    sched.step()
model.eval()
print(loss.item())
torch.save(model.state_dict(), "mlp.safetensors")

apsi convert mlp.py --idiomatic writes:

# Converted from mlp.py by `apsi convert`.
use models as models
use torch as torch

var _aux3 = none
var _grads2 = none
var _loss_of1 = none
var loss = none

fn MLP(d_in, hidden, d_out):
    let fc1 = torch.Linear(d_in, hidden)
    let bn = torch.BatchNorm1d(hidden)
    let out = torch.Linear(hidden, d_out)
    let drop = torch.Dropout(0.1)
    fn init_(seed = 0) = record(fc1=fc1.init(seed=seed * 1009 + 1), bn=bn.init(seed=seed * 1009 + 2), out=out.init(seed=seed * 1009 + 3), drop=drop.init(seed=seed * 1009 + 4))
    fn forward_(p, x, train = false) = out(p.out, drop(p.drop, torch.relu(bn(p.bn, fc1(p.fc1, x, train=train), train=train)), train=train), train=train)
    return torch.Module(init_, forward_, "MLP", record(fc1=fc1, bn=bn, out=out, drop=drop))
let X = randn(256, 4)
let y = tensor(X[:, 0] > 0, dtype=i64)
let model = MLP(4, 32, 2)
var model_params = model.init(seed=0)
let opt = torch.AdamW(lr=0.01)
let sched = torch.CosineAnnealingLR(opt, T_max=200)
let loss_fn = torch.CrossEntropyLoss(label_smoothing=0.1)
for epoch in 0..200:
    fn _loss_of1(_p):
        let logits = model(_p, X)
        let loss = loss_fn(logits, y)
        return loss
    loss, _aux3, _grads2 = torch.value_and_grad_aux(_loss_of1)(model_params)
    _grads2 = torch.clip_grad_norm(_grads2, 1.0)
    model_params = opt.step(model_params, _grads2)
    sched.step()
model.eval()
print(loss)
models.save_safetensors("mlp.safetensors", torch.state_dict(model, model_params))

The saved file is a standard safetensors file with PyTorch's names (bn.running_mean, fc1.weight, …): model.load_state_dict(load_file("mlp.safetensors"), strict=True) in PyTorch accepts it unchanged, and the other direction works the same way.

What is converted exactly

Verified exactness. The test suite (tests/test_convert_torch.py, tests/test_torch_compat.py) runs every layer, loss, optimizer and scheduler above next to real PyTorch with identical weights and compares outputs, gradients and learning-rate sequences. A complete training run — convolution, BatchNorm, LSTM, multi-head attention, AdamW, cosine annealing, gradient clipping and label smoothing — ends with the same weights as PyTorch to within 1e-9.

Random initialisation uses Apsi's generator, so a model trained from scratch starts from different random weights than in PyTorch. To reproduce a PyTorch run exactly, load its initial state_dict.

Memory

• torch.checkpoint_layer(layer, params, x) or torch.checkpoint(fn, x) stores only the block inputs and recomputes the rest during the backward pass: about a third of the peak memory for about twice the compute.
• Create parameters with dtype=f32 (or fit(…, precision="fp32")); gradients and Adam state then stay float32.
• DataLoader builds one batch at a time, so datasets larger than memory can be streamed from a Dataset class.

After converting with --idiomatic, run apsi check file.apsi (done automatically) — and consider adding units to the physics parts: let g = 9.81 → let g = 9.81[m/s^2] turns every later formula into a checked one.

Found a mistake? Report it · Markdown source