Fault tolerance and runtime drivers

What survives a crash on a node, and how sandboxes stay portable across runtimes.

A running sandbox is supervised independently of the node-level orchestrator process that launched it, so that process can crash and restart without taking your sandbox down with it. This is measured, not assumed: killing the orchestrator process outright and killing the sandbox's own runtime process out from under it are both exercised as recovery tests. In both cases the same sandbox — same identity, same durable state — comes back, rather than the incident surfacing as a stuck or lost sandbox.

This protects against a crashed process on the node your sandbox is running on. It is not the same as surviving that entire node going offline — cross-node failover for a dead host is not part of the current guarantee, so avoid depending on a specific sandbox migrating itself to another machine if its node disappears.

Placement also routes new sandboxes around nodes that are genuinely unhealthy, without cordoning a node just because it's mid-way through an ordinary stop, pause, or suspend on one of its sandboxes — routine lifecycle transitions aren't treated as node failures.

Firecracker is the default runtime driver and the one every sandbox can rely on today. Cloud Hypervisor is a second, independently sandboxed runtime driver behind the same lifecycle API and jailer-style isolation model (its own mount/PID/network namespaces, cgroup limits, and seccomp), rolling out opt-in on a per-node basis. It's real, not experimental scaffolding — Linux boot, pause, resume, snapshot, and clone all work on it — but persistent volumes and Windows guests aren't available on it yet, and it isn't the default anywhere. Treat it as available only where a node explicitly advertises it.