A frozen model manifest, a generator sealed with drand, and an Arweave record anyone can use to rebuild the exam and check its scores.

I knew the question types but not the actual questions

On 2026-10-17, a bundle carrying my encrypted generator reached an Arweave block. The generator was sealed to a drand round that had not been published yet. I wrote that program, so I knew it would produce integer word problems, short Python programs, and shortest-path questions. I did not know which ones. The questions depended on a beacon the drand network would publish later, at round 33170212, on 2026-10-18T09:00:00Z.

When the round arrived, the same beacon value unlocked the program and seeded it. Four open-weight models answered 360 completions each: 240 on newly generated questions and 120 on a frozen set of public GSM8K items.

But the first two batches I ran should not be cited, and it is worth saying why, because the reason is the most useful thing in this article.

The first batch, b1, was valid under its own release record. Its manifest bound the ciphertext with a SHA-256 hash, but it never bound the ciphertext's transaction ID. Calling that check a pass would be a mistake, and it is WEAK. The v2 protocol exists partly to close that gap.

The second batch, b2, fixed the binding and then failed on the grader. The rule I sealed said an integer answer counts only when the last line is #### followed by nothing but the number. llama.cpp writes its end-of-text marker onto that same line, as #### 392 [end of text]. The registered grader threw the whole line away. Most of b2's arithmetic and GSM8K scores measure that rejection, not the rate at which models produced correct integers.

The ciphertext was already sealed by then, and the grader could not be changed in place. That constraint is the entire point of the protocol, and it applies to me too. So I made b3: a new commitment, sealed before its own round, with the normalization written into the scorer the manifest hashes.

So if you want a number from this project, take one from b3, and take the registered primary accuracy, which counts an answer as wrong when it cannot be parsed and wrong when the output was truncated. Its index transaction is vlVlUfyBn4IUmbXjkSX8079QuySP-Qqr8rN8TbivzTg. That index was published after the run. It is a convenient entry point, not evidence: the evidence that the generator existed before the round is the earlier ciphertext and manifest.

What a reader needs in order to catch me

Public benchmarks leak. Once the questions are online, a high score may only mean the model has seen them.

A private test set looks like the fix, but it hands the same power to one person: whoever writes, stores, selects and reports the questions. That person includes the evaluator, which is me. If I can see the scores and then quietly change the seed, the prompt, the model file, or simply decide which batch to tell you about, I can aim the headline wherever I want.

SealedBench tries to pin all of that down in advance. The generator, the model hashes, the prompts, the runtime settings and the scoring code are bound to one future beacon round. Unless drand's operators reach a threshold and collude to compute that round early, I cannot read the beacon before it is published. If the network stops, the batch never opens; that is the other half of the bargain. After the round, anyone can rebuild the questions from the Arweave record and re-score the answers I published.

Here is every move I could make, and what I gave up in advance to make it visible.

What I could change after seeing scores	What was committed in advance	What you still have to check
The generator or the recipe	Ciphertext and manifest on Arweave before the round	Check L1 block time, not the upload receipt
The seed	The beacon round fixed in the manifest	That the generator really uses that seed
Questions, answers or the scorer, later	Content hashes	A hash cannot show which process wrote a file
Reporting only the flattering batch	Batches declared up front	Encryption cannot force me to publish answers

Freeze, seal, then wait for a block

The manifest freezes the drand chain hash and public key, the release round, the seed rule, a SHA-256 for every GGUF, the llama.cpp commit and build flags, the prompts, the decoding settings, the grader and the reporting policy. Every batch uses quicknet chain 52db9ba70e0cc0f6eaf7803dd07447a1f5477735fd3f661792ba94600c84e971, genesis at unix 1692803367, period 3 seconds.

The generator is sealed to that round with drand time-lock encryption (tlock). The ciphertext and the manifest go up through the Turbo bundler. A receipt is not a commitment. The commitment is the L1 block.

Under the v2 rule a batch is valid only if the block holding its bundle is timestamped no later than unlock_unix − 600, the bundle ID appears in that block, and the data item's bytes hash to the committed SHA-256. Height, timestamp and block hash must agree on two independent gateways, and my own gateway ario.zigza.xyz does not count as either one. b1 used an earlier, simpler rule: manifest and ciphertext had to be in an L1 block before the release round, and a miss made the batch invalid, which I would still have had to report.

After the round, the beacon signature is checked against quicknet's fixed public key. The same beacon both decrypts the generator and enters the seed. One value doing two jobs is not two independent draws, and I do not claim otherwise.

b1's seed is a SHA-256 over a domain string, the chain hash, the round in decimal and the beacon randomness, with hex fields lowercased first. The exact sentence is in b1's manifest. v2 batches add a 32-byte secret stored inside the ciphertext, and hash length-prefixed fields: domain, chain hash, round, signature bytes and that secret. The JSON randomness field is not an input; it must still equal the SHA-256 of the signature. I chose the secret. It does not let me list the questions before the beacon exists. A successful tle -d only shows the ciphertext opens under the signature it was sealed to. The BLS check against the fixed public key is a separate step.

There is one specific gap in v1 worth naming. The November capsule cap and b1 share the same plaintext tarball, SHA-256 e054f1ab7824fde33e6ac399252098cf25c29815ff4ffacabbd311da05eac9c2. Once b1 opened, that program was public. Until 2026-11-15 the unknown is the beacon, not the program. v2 still hides its questions, because the secret sits inside the ciphertext.

Four small models, one CPU, three question families

Every family has a program for an oracle, so grading is code and not a judge model. arith_word produces two-to-five-step integer word problems; the oracle is a function of the sampling parameters, and the prompt contains no answer line. code_output produces short Python programs with no imports and no input, run in an interpreter with a step budget; the oracle is their stdout. graph_p

A frozen model manifest, a generator sealed with drand, and an Arweave record anyone can use to rebuild the exam and check its scores.

I knew the question types but not the actual questions

On 2026-10-17, a bundle carrying my encrypted generator reached an Arweave block. The generator was sealed to a drand round that had not been published yet. I wrote that program, so I knew it would produce integer word problems, short Python programs, and shortest-path questions. I did not know which ones. The questions depended on a beacon the drand network would publish later, at round 33170212, on 2026-10-18T09:00:00Z.

When the round arrived, the same beacon value unlocked the program and seeded it. Four open-weight models answered 360 completions each: 240 on newly generated questions and 120 on a frozen set of public GSM8K items.

But the first two batches I ran should not be cited, and it is worth saying why, because the reason is the most useful thing in this article.

The first batch, b1, was valid under its own release record. Its manifest bound the ciphertext with a SHA-256 hash, but it never bound the ciphertext's transaction ID. Calling that check a pass would be a mistake, and it is WEAK. The v2 protocol exists partly to close that gap.

The second batch, b2, fixed the binding and then failed on the grader. The rule I sealed said an integer answer counts only when the last line is #### followed by nothing but the number. llama.cpp writes its end-of-text marker onto that same line, as #### 392 [end of text]. The registered grader threw the whole line away. Most of b2's arithmetic and GSM8K scores measure that rejection, not the rate at which models produced correct integers.

The ciphertext was already sealed by then, and the grader could not be changed in place. That constraint is the entire point of the protocol, and it applies to me too. So I made b3: a new commitment, sealed before its own round, with the normalization written into the scorer the manifest hashes.

So if you want a number from this project, take one from b3, and take the registered primary accuracy, which counts an answer as wrong when it cannot be parsed and wrong when the output was truncated. Its index transaction is vlVlUfyBn4IUmbXjkSX8079QuySP-Qqr8rN8TbivzTg. That index was published after the run. It is a convenient entry point, not evidence: the evidence that the generator existed before the round is the earlier ciphertext and manifest.

What a reader needs in order to catch me

Public benchmarks leak. Once the questions are online, a high score may only mean the model has seen them.

A private test set looks like the fix, but it hands the same power to one person: whoever writes, stores, selects and reports the questions. That person includes the evaluator, which is me. If I can see the scores and then quietly change the seed, the prompt, the model file, or simply decide which batch to tell you about, I can aim the headline wherever I want.

SealedBench tries to pin all of that down in advance. The generator, the model hashes, the prompts, the runtime settings and the scoring code are bound to one future beacon round. Unless drand's operators reach a threshold and collude to compute that round early, I cannot read the beacon before it is published. If the network stops, the batch never opens; that is the other half of the bargain. After the round, anyone can rebuild the questions from the Arweave record and re-score the answers I published.

Here is every move I could make, and what I gave up in advance to make it visible.

What I could change after seeing scores	What was committed in advance	What you still have to check
The generator or the recipe	Ciphertext and manifest on Arweave before the round	Check L1 block time, not the upload receipt
The seed	The beacon round fixed in the manifest	That the generator really uses that seed
Questions, answers or the scorer, later	Content hashes	A hash cannot show which process wrote a file
Reporting only the flattering batch	Batches declared up front	Encryption cannot force me to publish answers

Freeze, seal, then wait for a block

The manifest freezes the drand chain hash and public key, the release round, the seed rule, a SHA-256 for every GGUF, the llama.cpp commit and build flags, the prompts, the decoding settings, the grader and the reporting policy. Every batch uses quicknet chain 52db9ba70e0cc0f6eaf7803dd07447a1f5477735fd3f661792ba94600c84e971, genesis at unix 1692803367, period 3 seconds.

The generator is sealed to that round with drand time-lock encryption (tlock). The ciphertext and the manifest go up through the Turbo bundler. A receipt is not a commitment. The commitment is the L1 block.

Under the v2 rule a batch is valid only if the block holding its bundle is timestamped no later than unlock_unix − 600, the bundle ID appears in that block, and the data item's bytes hash to the committed SHA-256. Height, timestamp and block hash must agree on two independent gateways, and my own gateway ario.zigza.xyz does not count as either one. b1 used an earlier, simpler rule: manifest and ciphertext had to be in an L1 block before the release round, and a miss made the batch invalid, which I would still have had to report.

After the round, the beacon signature is checked against quicknet's fixed public key. The same beacon both decrypts the generator and enters the seed. One value doing two jobs is not two independent draws, and I do not claim otherwise.

b1's seed is a SHA-256 over a domain string, the chain hash, the round in decimal and the beacon randomness, with hex fields lowercased first. The exact sentence is in b1's manifest. v2 batches add a 32-byte secret stored inside the ciphertext, and hash length-prefixed fields: domain, chain hash, round, signature bytes and that secret. The JSON randomness field is not an input; it must still equal the SHA-256 of the signature. I chose the secret. It does not let me list the questions before the beacon exists. A successful tle -d only shows the ciphertext opens under the signature it was sealed to. The BLS check against the fixed public key is a separate step.

There is one specific gap in v1 worth naming. The November capsule cap and b1 share the same plaintext tarball, SHA-256 e054f1ab7824fde33e6ac399252098cf25c29815ff4ffacabbd311da05eac9c2. Once b1 opened, that program was public. Until 2026-11-15 the unknown is the beacon, not the program. v2 still hides its questions, because the secret sits inside the ciphertext.

Four small models, one CPU, three question families

Every family has a program for an oracle, so grading is code and not a judge model. arith_word produces two-to-five-step integer word problems; the oracle is a function of the sampling parameters, and the prompt contains no answer line. code_output produces short Python programs with no imports and no input, run in an interpreter with a step budget; the oracle is their stdout. graph_p