Disk watermarks and block sealing: why a full disk stops xOne

Why does a low disk stop sealing in xOne?

A seal writes a safety record to disk, syncs it, and then signs. The code comment says "nothing is signed that the store could not keep."

• The producer times "the safety record's write and sync plus the signature" (producer.rs).
• The exit line must cover the largest single write set. ADR-007 lists one block commit, one MDBX growth step, one chunk file, the 1 KiB safety-record slot and the journal (ADR-007 section 7).
• ADR-007 also calls the stop state a "not producing" alarm: a validator that cannot import cannot vote on blocks it lacks.

How does the gate reach the producer?

The disk-guard service owns one ImportGate. services/mod.rs hands a clone to the chain follower and to the producer.

The guard has four rules that matter for sealing (diskguard.rs):

• First sample first. The guard is the first service. Its first reading sets the gate before import or the producer starts.
• One alarm, then hysteresis. disk_low appears once. The gate reopens above the resume line.
• No reading is not enough space. df has a 10 second timeout. After 3 failed samples in a row the gate closes. The next good sample reopens it if the state allows.
• Exit needs two samples. Two samples below the exit line stop the node with exit code 4.

PR #317 wired the guard to the chain follower. PR #385 added the producer gate, the first sample and the stale-sampler rule. Both merged on 2026-11-14.

What is the second line of defence?

The second line is Degraded(NoSpace) in xone-db. It covers a disk that fills faster than the sample period, for example a log burst or another process (ADR-007 section 7).

On ENOSPC before any sync, ADR-005 section 9 says the store does this:

1. Abort the MDBX transaction. Truncate the segments to the committed length. Remove an orphan chunk file.
2. Set Degraded(NoSpace) and log one line: xone-db: degraded cause=no_space.
3. Keep serving reads.

may_sign() is true only for Durable (lib.rs). The producer hands the durability to the signer before every signature and logs a change as store durability <state>. A failed fsync is different. It is fail-stop: the store writes FSYNC_FAILED and the node exits non-zero.

The ENOSPC path does not exit the node. Only the watermark exit does. The retry level ("free space above a reserve") is an open value in ADR-005. ADR-007 proposes the resume line for it.

Does ADR-007 match the code?

No. ADR-007 section 7 still says: "Not done: the producer does not yet stop sealing while the gate is closed." The code on main stops it, since PR #385. The ADR text is stale. The ADR is also still marked "proposed", and its table holds higher lines than the code uses.

Open gaps on main:

• Snap sync reads only the exit line, not the stop line. PR #385 lists this as not done. xone snapshot import has no guard.
• The loop-device cases O3 A to E, including the fast-fill case that triggers NoSpace, are not run (tests/ops/PLAN.md).

What do the other three clients do?

I read each repository at origin/main. Only xOnePy has watermark code that I found.

• xOnePy: decide() in limits.py stops import below the larger of 10% and 20 GiB, resumes above 15% and 30 GiB, and exits after two samples below 2% and 4 GiB. --datadir.minfreedisk <MB> replaces the three lines. _watermark in main.py logs stop and resume and stops the node with EXIT_DISK (4) at the exit line. I found no gate that the importer or the validator reads.
• xOneC: the ADR-007 C++ addendum sets stop at used above 85% or free below 20 GiB, resume at used below 80% and free above 30 GiB, and exit at used above 98% or free below 4 GiB for two samples. I found no watermark code in its source.
• xOneTS: the ADR-007 TS addendum uses the parent lines (10% or 20 GiB, 15% or 30 GiB, 2% or 4 GiB). I found no watermark code in its source either.

All three state the Degraded(NoSpace) rule in their ADR-005 addenda, and their store code handles ENOSPC.

What did the xOnePy outage teach?

A disk warning should not end the signer. The xOnePy record on issue #85 (comment of 2026-11-14T13:19:21Z) says the exit line stopped the node twice, and "turned a disk warning into a seat outage."

The follow-up in the record: pause import and keep the signer alive. The record calls this the geth behaviour. I did not verify that against geth. It says a separate issue would follow. I found none, and main.py still stops the node at the exit line.

xOneRS pauses sealing between the stop and exit lines. At the exit line it still exits, so its signer stops too. Act at the 15% alert, well before either line.

How do other clients describe it?

What do I see in the log?

Five lines mark the path from a low disk to a stopped producer.

• xone-node: disk_low free=<bytes> stop_below=<bytes>; block import stopped
• xone-node: disk_ok free=<bytes>; block import resumed
• xone-node: disk_full free=<bytes> exit_below=<bytes>; stopping the node
• disk low: block production paused (the producer, from seal)
• xone-db: degraded cause=no_space (ADR-005 section 9)

For the response steps, see node operations.

Frequently asked questions

Does xOne stop sealing when the disk is low?

Yes, in xOneRS. The producer reads the disk guard's gate and refuses to seal while it is closed, with disk low: block production paused.

What if the disk fills faster than the guard samples?

The store sets Degraded(NoSpace). Then may_sign() is false and nothing is signed.

Do xOnePy, xOneC and xOneTS pause sealing on a low disk?

I found no such gate in their source. xOnePy logs the stop and stops the node at the exit line. xOneC and xOneTS state their lines only in ADR addenda.