How Crossfyre scans survive a crashed node

A reclaimed instance, a closed laptop lid, a target that starts tarpitting. Most tooling loses the whole run to any of those. Here is what we do instead, and why a dead machine costs you nothing.

Long recon runs fail in boring ways. A spot instance gets reclaimed. A laptop lid closes. A target starts rate-limiting and a tool hangs until the SSH session dies. With most scanning setups, that is the whole run gone. You rerun from zero, pay for the compute twice, and hope it finishes this time.

We treat a dropped machine as a Tuesday, not an incident. Whatever it was working on goes to a machine that is still alive and the run keeps moving. Here is how that works.

The problem with "just run the tool"

When you SSH into a box and run a scanner, the run lives and dies with that process. There is no durable record of "this much is done, this much is left." If the process dies at 80 percent, the 80 percent dies with it. Tools that checkpoint to a local file help, but only if the box itself survives, which is exactly what is not guaranteed when you are renting cheap compute or scanning from disposable infrastructure.

The fix is to stop treating the run as one long-lived process and start treating it as a stream of small units of work with durable state behind them.

Workflows become operations on a durable queue

When you launch a workflow, Crossfyre decomposes it into discrete operations. Each operation is published onto a durable, acknowledged work queue: a persistent message log. A node pulls an operation, runs it, streams results back, and only then acknowledges it.

• If a node pulls an operation and then dies before acknowledging, the operation is never marked done.
• The platform redelivers that unacknowledged operation to another available node.
• Results stream back as operations complete, so partial progress is real progress, not a number on a progress bar.

What happens when a node drops

Say you have five nodes chewing through a large content-discovery run and one of them gets reclaimed mid-operation. Here is the sequence:

1. The node stops sending heartbeats and its in-flight operations stop being acknowledged.
2. The platform waits out the acknowledgement window, then re-queues those operations.
3. The four surviving nodes pick them up and keep going.
4. Completed results that already streamed back are untouched. Nothing is re-scanned that was already finished.

You do not babysit any of this. From the dashboard the run just keeps moving, slightly slower for a moment while one node falls out of the rotation.

Resuming a run that was paused or interrupted

Because operation state is durable, a workflow can be halted and resumed deliberately too, not just survive accidents. Pending operations sit on the stream until a node is available to take them. If your whole fleet goes offline overnight and comes back in the morning, the run picks up where it left off rather than starting over.

What a dead node costs you

Nothing, which makes this a short section. Your plan sets your limits and scanning inside them is not metered, so there is no per-scan charge to hand back when a box dies. Crash safety here is an engineering promise, not a billing one.

Why we built it this way

Offensive work happens in unstable conditions by definition. Targets fight back, cheap infrastructure vanishes without warning, and the run that matters most is always the longest one. A platform built for the happy path fails exactly when a rerun is most expensive.

So this is not a feature bolted onto the side. It is the reason spreading work across machines is usable at all, rather than a way to lose five runs instead of one.

Run a long scan and pull the plug on a node. Watch it keep going. Go break a node