Glyph Collections — Tricking developers into guiding MinHash

More text is not more signal. Collections steer fingerprints toward the fields that actually matter — without making you learn aggregation.

Listen

Voice created with generative AI.

Writing is original.

A document’s body is usually the longest thing you have. It is also often the worst thing to fingerprint alone.

Title, summary, tags — those are where the concept lives. The body is where the concept gets buried under asides, code samples, and “as we discussed above.” MinHash does not know which paragraph you meant. It just sees a bag of tokens. Dump the whole file in, and the sketch leans toward whatever is wordiest.

Collections exist because of that failure mode.

The psychology

Collections is based on the idea that more text isn’t necessarily more useful signal. A document’s body can overwhelm a fingerprint even when its title or summary contains the concept you’re searching for. By aggregating multiple fingerprints, Collections can steer the resulting representation toward those useful signals — sacrificing some precision to preserve semantic direction without multiplying comparison complexity.

That last clause is the trick. You could compare a query against title, summary, tags, and body separately. You would get more knobs and more work. Collections fold those sketches into one searchable glyph. The developer still “describes the document.” The library does the unglamorous part: weighting, aggregating, rebuilding the combined fingerprint when a field changes.

Glossier (our proprietary zig implementation of the same ideas) makes the same idea an abstraction. You hand it meaningful fields. It constructs the representation most useful for search. You do not have to understand why title should punch above its token count or how to optimize the text fed into your index. You just stop putting everything in a single string and hoping MinHash reads your mind.

Guiding without a dashboard

The honest version: we wanted developers to guide MinHash without making them feel like they were tuning a search engine.

Ask someone to set field weights and they will either ignore you or overfit. Ask them to pass title, summary, and tags as separate examples in a collection, and they will do it because that is how they already think about content, the metadata almost always exists. The aggregation is the hidden curriculum. Softmax (or whatever aggregator you pick) is allowed to be slightly wrong if it keeps the sketch pointed at the right neighborhood.

Comparing glyphs as a group applies the same idea from the opposite direction. Short queries can score poorly against a single, dense document fingerprint, while multiple labeled fingerprints give the comparison additional signals beyond the body. Collections take this further by trading a small amount of precision for a massive performance gain: they collapse the group into a single aggregate fingerprint, turning an O(N × G) comparison problem into O(N), where N is the index size and G is the query group size.

This portfolio’s query file is that pattern in production: each post is fingerprinted as title, summary, tags, document, and raw, then loaded as a collection at search time. Not because five hashes are fancy. Because one hash of the markdown file kept losing to its own footnotes.

Next: once the representation is cheap, stop building search to scan the world.