Compact Merkle Proof Encoding
The compact binary and base64url wire encoding of a Merkle inclusion proof - a depth byte, a packed left/right direction bitfield, and raw 32-byte sibling hashes - and how it maps to and from the JSON "direction:hash" proof form.
Overview
A Merkle inclusion proof is an ordered list of sibling hashes, each tagged with the side it sits on. The plain JSON form of that proof is a list of "direction:hash" strings, where direction is l or r and hash is a 64-character lowercase hexadecimal SHA-256 digest. That form is readable and self-describing, but it is verbose: every sibling hash costs 64 characters of hex plus a two-character prefix, more than double the 32 raw bytes it represents.
The compact encoding is a byte-for-byte-minimal binary form of the same proof, meant for the wire and for storage. It drops the hex expansion and the "l:" / "r:" string prefixes, storing each sibling hash as its raw 32 bytes and packing all the left/right direction flags into a single small bitfield. A base64url text form wraps that binary so it can travel safely inside JSON, a URL, or a header. The two forms are exactly equivalent: any proof can be encoded to the compact form and decoded back to the identical JSON list, with no information lost.
This reference specifies the binary layout, the base64url wrapper, and how each maps to the JSON "direction:hash" form. For what an inclusion proof is and how it is checked against a root, see Merkle inclusion proofs; for how the tree and its hashes are built, see the RFC 6962 Merkle tree.
The JSON proof form
The canonical, human-readable proof is a JSON array of strings, ordered from the bottom of the tree (the leaf’s immediate sibling) up toward the root. Each string is one audit-path entry:
directionisl(the sibling is on the left of the running hash) orr(the sibling is on the right).hashis the sibling’s SHA-256 digest as 64 lowercase hex characters.- The two are joined by a single colon:
"r:5e5c...9ad1".
A one-step proof (a two-leaf tree) is a single-element array:
["r:5e5caeafc27155c368b6f201107d6f8b270747ce636ac5174a56c6e12ef89ad1"]
Two boundary cases carry over from inclusion proofs: a single-leaf tree has an empty proof [], and the number of entries equals the tree depth. The compact encoding round-trips all of these, including the empty proof.
Binary layout
The compact binary form has three parts, in order:
- Depth byte. A single unsigned byte (uint8) holding the number of proof steps, which equals the tree depth. This bounds the whole structure and is the first thing a decoder reads.
- Direction bitfield. A run of
ceil(depth / 8)bytes holding one bit per step. Bit N (counting from the least significant bit of the first byte, little-endian) is the direction of step N:0means the sibling is on the left (l),1means it is on the right (r). Eight steps pack into each byte, so most real proofs need only one or a few bitfield bytes. - Sibling hashes. Exactly
depth * 32bytes: the raw 32-byte SHA-256 digest of each sibling, in the same bottom-to-top order as the steps, one after another with no separators.
So the total size of a non-empty proof is 1 + ceil(depth / 8) + depth * 32 bytes. A one-step proof is 1 + 1 + 32 = 34 bytes, versus 66 characters (a two-character r: prefix plus 64 hex characters) for the equivalent JSON string. As the proof grows, the per-step cost settles toward the 32 raw hash bytes plus one eighth of a byte for the packed direction bit, which is close to the theoretical minimum.
The empty proof is a special single byte: 0x00 (a depth of zero, with no bitfield and no hashes).
The maximum depth accepted on decode is 64 steps, the same proof-depth ceiling that inclusion-proof verification enforces. A depth byte above 64 is rejected, and a binary whose remaining length does not exactly match ceil(depth / 8) + depth * 32 is rejected as malformed.
Worked example: one step
Take the one-step JSON proof above. Its sibling hash 5e5c...9ad1 decodes to 32 raw bytes. The direction is r, so the single direction bit is 1. The 34-byte binary is:
byte 0 : 0x01 depth = 1
byte 1 : 0x01 bitfield: bit 0 = 1 (right)
bytes 2..33 : 5e 5c ae af c2 71 55 ... the 32 raw sibling hash bytes
Decoding reverses this: read depth 1, read one bitfield byte, read 1 * 32 hash bytes, then for each step recover the direction from its bit and re-expand the raw hash to lowercase hex, rebuilding the exact "r:5e5c...9ad1" string.
The base64url text form
Because the compact form is raw binary, it is wrapped in base64url (URL- and filename-safe base64, with no = padding) whenever it needs to live inside text: a JSON string, a query parameter, or an HTTP header. Encoding runs the binary through base64url without padding; decoding reverses that and then parses the binary.
The empty proof [] encodes to the two-character string "AA" (base64url of the single 0x00 byte). An invalid base64url string, or a well-formed one whose decoded bytes are not a valid proof, decodes to an error rather than a wrong proof.
Encoding and decoding functions
Four functions cover the round trips between the JSON list and the two compact forms.
- Encode to binary. Takes a proof list and returns the compact binary. An empty list returns the single
0x00byte. - Decode from binary. Takes the compact binary and returns either the reconstructed proof list on success or an error for a malformed or over-deep binary. It never returns a partial or guessed proof.
- Encode to base64url. Encodes to binary, then wraps in base64url without padding, returning a text string.
- Decode from base64url. Unwraps the base64url text, then decodes the binary, returning the proof list or an error.
Every round trip is lossless and total: encoding then decoding yields the byte-identical original JSON list, and any input that is not a valid proof produces an error rather than a silently wrong result. Both compact forms carry exactly the same information as the JSON list, so a verifier can accept whichever form is convenient, decode it, and then run the ordinary inclusion-proof check described in Merkle inclusion proofs.
Limitations
- The compact form encodes only the audit path (the ordered sibling hashes and their sides). It does not carry the leaf’s own hash, the expected root, or the key. Those come from elsewhere in a proof bundle and are supplied to verification separately.
- The depth byte is a single unsigned byte and decode rejects any depth above 64 steps, matching the proof-depth ceiling that inclusion-proof verification enforces. This is far beyond any practical tree, but it is a hard limit of the format.
- The encoding assumes each sibling hash is exactly 32 bytes (a SHA-256 digest). It is not a general container for arbitrary-length hashes.