JSSC

Can compressed JSON still answer a query without being unpacked? JSSC is an archive format for JSON that keeps every value readable where it lies: a JSONPath query reads the bytes on its path and decompresses only the strings it reaches. Version 8 of the format is an open draft with a reference implementation in Go.

Read the specification Source on GitHub

Two redundancies, removed separately.

JSON repeats itself twice over: keys and values recur from record to record, and the braces, quotes and commas carry nothing once the document is parsed. A general-purpose compressor removes both at once, as one stream, so reading the last record means decompressing everything before it. JSSC removes them separately, so each can be undone on its own.

Every string, once

Keys and string values go into two pools, each distinct string stored a single time and compressed on its own, so one can be read without its neighbours.

A structure you can step over

The document’s shape becomes one-byte tags and references into the pools. Every value says how long it is, so a reader skips what it does not want instead of parsing it.

Read where it lies

A query walks the archive as it is stored and decompresses only the strings it reaches. The rest of the document stays compressed.

How a query reads one value.

1Open the archive
Opening reads section lengths and nothing else. No string is decompressed and no value is parsed, however large the document.
2Turn the field name into a number
A directory of every key, uncompressed, maps a field name to its place in the key pool. A name that is not in it is in no object anywhere, and the query can stop there without reading on.
3Walk the path
Object members are stored in key order, so finding a field is a search over integers. Large arrays and objects carry a table of where each child starts, so the fiftieth element is one step away, not fifty.
4Decompress what it reached
The structure is cut into blocks of about 64 KB, so a read decompresses the block it lands in. Then the one string it needs, and that is the work done.

What a JSS8 archive holds.

Two string pools: One for keys, one for string values, each sorted and stored once per distinct string. Every entry is compressed by itself with FSST, a codec made for short strings; one that would not get smaller is stored as it is. Short values are written straight into the structure instead.
The structure: Tags, lengths and references, with numbers kept as the text they were written in. It is compressed in independent blocks — LZ4 by default, DEFLATE or none on request — so reaching one document never means decompressing the ones before it.
A key directory: Every key’s hash beside its place in the key pool, so a field lookup needs no decompression at all. Keys whose hashes collide are all listed, and the reader tells them apart by comparing the strings.
A key-path filter: On larger documents, a compact set of every key path the document contains. A query for a path that is not there is answered before the walk begins. It can only be wrong in one direction: it never rejects a path the document holds.
Many documents in one archive: Newline-delimited JSON (JSONL, NDJSON) becomes one archive over one shared pool, and is queried as a list of its documents. Each document opens with one read, wherever it sits.

What comes back.

Exactly: Strings, booleans and null; the order of array elements; and every number as written — 1e3 comes back as 1e3, not 1000.
Reordered: Object members come back in key order, which JSON does not give a meaning to. A key repeated inside one object keeps one value.
Not kept: Whitespace and indentation. Decompressing gives compact JSON.

Where it fits, and where it does not.

JSSC trades compression ratio for random access. That trade decides where it belongs.

Large documents, read a few fields at a time: Held locally or in a database and queried far more often than they are written: reading one value touches a small part of the archive, where compressed JSON has to be decompressed and parsed in full first.
Not for shrinking JSON in transit: Compressing each string on its own, so that it can be read on its own, costs size. On most documents gzip, zstd or brotli over the JSON text is smaller, and faster to produce. If the bytes are only ever stored or sent whole, use one of those.
Not for small documents: An archive carries fixed costs — its header, the codec’s table, the directories — that a document of a few kilobytes cannot spread out. Selective reading pays off when there is something to skip.
Not for queries that visit everything: A recursive search over the whole document has nothing to skip, so it does the work of a full read, decompression included.

Try it.

The command line compresses JSON or newline-delimited JSON, queries an archive with JSONPath, and restores it. The same operations are a Go library; the implementation’s README has the API.

# install the command line (Go 1.26 or later)
go install github.com/wizhut/jssc-go-oss/go/cmd/jssc@latest

jssc compress users.json                 # → users.json.jssc
jssc query '$[*].name' users.json.jssc   # decode only the names
jssc query --stats '$[0].email' users.json.jssc
jssc compress --format ndjson events.ndjson
jssc decompress users.json.jssc          # → users.json

Where it stands.

An open draft: The specification is written as an RFC-style draft, revision 5.6.0-draft, and published with the implementation under the Apache 2.0 licence. Until a 1.0 release the wire format may still change, so do not use it yet for archives that must outlive the version that wrote them.
One version: JSS8 is the only version of the format. The development versions before it are described in the specification’s appendix for reference; no implementation reads or writes them, and opening one is refused rather than misread.
A reference implementation in Go: Version 0.2.0: the format, a library, and a command line to compress, decompress, inspect and query. Pure Go, no cgo. Its decoder is fuzzed, so arbitrary bytes fail cleanly, and it takes limits for archives that did not come from you.
Measurements next: No benchmark is published here yet. When one is, it will be tied to the version it measured, and it will say where the format loses as well as where it wins.

An open draft of a JSON format you can query.