Search Architecture¶
This document describes the architectural principles, domain model, and technical design of Badger Commerce's search and discovery platform. It is intended for engineers building on the platform, operators who want to understand what the system can express, and anyone evaluating future direction.
For the customer-facing feature surface, see Search and Discovery. For tenant isolation fundamentals that underpin search, see Multi-Tenancy Architecture.
Why a Search Platform¶
Most ecommerce platforms treat search as a bolt-on: a free-text input box that runs alongside Mongo-driven category pages, manual collections, and faceted filters wired into bespoke admin screens. The result is fragmentation — multiple authoring concepts, multiple data paths, and inevitable drift between how a category page works and how the search results page works.
Badger Commerce takes the opposite stance. Search is the platform's discovery substrate. Category pages, search results, virtual categories, megamenu queries, and ad-hoc landing pages all resolve through the same search layer over the same index. There is one model to learn, one place where ranking and faceting are configured, and one performance profile for everything customer-facing.
Core Concepts¶
Taxonomy Collection
(what the customer is (manually curated
selling — classification) product groupings)
│ │
│ ↑ existing primitives │
▼ ▼
┌────────────────────────────────────────────────┐
│ MerchandisingTree │
│ (how the customer sells — projection) │
│ │
│ MerchandisingNode MerchandisingNode │
│ │ │ │
│ NodeBacking NodeBacking │
│ (one of five) (one of five) │
└────────────────────────────────────────────────┘
│
▼
┌─────────────────────┐
│ Typesense index │
│ (per-tenant) │
└─────────────────────┘
Taxonomy¶
The taxonomy describes what the customer is selling: classes of product, the attributes those classes carry, and how attributes are validated. It is structural metadata that lives once per tenant and is the source of truth for product classification. See Taxonomy, TaxonomyLevel, and TaxonomyAttribute in commerce-core.
Collection¶
A Collection is a manually curated grouping of products, identified by a seoName and optionally arranged into a parent/child hierarchy via parentCollectionIDs. Products declare collection membership via Product.collectionIDs (a list of collection seoNames — the field name is a historical misnomer). Collections existed before the merchandising tree and remain in active use.
MerchandisingTree¶
A named, hierarchical projection of the catalogue that customers actually browse. A site has one or more trees; one is marked primary and drives canonical URLs and SEO. Other trees can power gift guides, seasonal navigation, or megamenus.
A tree is just an ordered set of root MerchandisingNodes. Each node has children. The tree decides nothing about products on its own — that's the backing's job.
MerchandisingNode¶
A node is the unit of merchandising. It carries display data (name, description, image, SEO metadata, facet configuration) and a NodeBacking that specifies how products resolve into it.
NodeBacking¶
The five backing types are the heart of the model. Every node is exactly one:
| Backing | Resolution |
|---|---|
ManualCollectionBacking |
Products in the named Collection (via Product.collectionIDs). |
TaxonomyProjectionBacking |
Products assigned to a taxonomy level (or any descendant), optionally narrowed by attribute filters. |
VirtualQueryBacking |
Products matching a VirtualQuery predicate — e.g. "orange clothes under £100". |
ManualListBacking |
A specific list of products in a specific order. Used for landing pages, gift guides, hand-curated promotions. |
GroupBacking |
No products of its own — a structural container that aggregates children for navigation and SEO. |
Mixing backings inside one tree is encouraged. A typical fashion retailer's primary tree might be a taxonomy projection at the top, with virtual queries pinned to season nodes, manual collections at the leaves, and a manual list for "Featured Today" on the homepage.
VirtualQuery¶
VirtualQuery is a small JSON-serialisable predicate DSL — not a raw query string. It supports clauses like AttributeEquals, AttributeIn, AttributeRange, PriceRange, InStock, InCollection, InTaxonomyLevel, TextMatch, and Nested(VirtualQuery), composed with AND / OR. The DSL is internal, validated, renderable in admin UI, and translated into Typesense filter_by expressions by VirtualQueryTranslator.
The Five Principles¶
These are the load-bearing rules. If you find yourself violating one, stop and check with the maintainers — there's almost certainly a better path.
1. Search is the source of truth for discovery; Mongo is the source of truth for state.¶
Anything customer-facing that browses, filters, faceted-navigates, or full-text-searches goes through Typesense. That includes classic category/collection pages — they are not a special case. Anything that mutates (write a product, update inventory, change a price) goes through Mongo first; the index follows.
This means the rendering layer has one query model and one performance profile. It also means we never debug "why does the category page disagree with the search results page" — they're the same thing.
2. One model, many backings.¶
Manual collections, taxonomy projections, virtual queries, manual lists, and groups are all the same node-shaped thing. They differ only in how products resolve into them. Admin UX, SEO, facet configuration, breadcrumbs, and indexing logic all operate on the abstract node — not on five parallel concepts.
This is what makes the model both expressive enough for large fashion retailers and approachable enough for a small charity shop. Start with one manual collection. Add a virtual query when the use case appears. The platform doesn't make you commit to a paradigm up front.
3. Multi-tenant by isolation, not by filter.¶
Each tenant gets its own Typesense collection. products__{catalogueId} is a stable alias; the data lives in a versioned collection behind it (products__{catalogueId}__v{n}, see Collection lifecycle). Cross-tenant queries are impossible by construction and tenant deletion is a few API calls. Today every tenant shares one product schema (see Schema & Facets); the versioned layout is what lets that schema change safely.
We rejected the "single collection with siteId filter" approach despite its simplicity: it makes noisy-neighbour problems harder to isolate, forces the schema to a lowest-common-denominator, and weakens compliance posture. One Typesense cluster, many tenant collections, is the right balance for both hosted and SaaS deployments.
4. Events drive freshness; reconciliation guarantees correctness.¶
Mongo lifecycle events flow through ApplicationEventPublisher → RabbitMQ → worker → Typesense. Updates land in seconds. But events can be lost (network blips, broker restarts, deployments mid-batch), so a nightly reconciliation job re-asserts the world: it streams everything modified in the last 25 hours and upserts, then diffs Typesense IDs against Mongo IDs and deletes orphans.
Neither mechanism is sufficient alone. Events without reconciliation drift silently. Reconciliation without events makes the catalogue feel stale. Together they give us best-effort freshness with eventual correctness — the right trade-off for ecommerce.
5. Variants roll up to products in the index.¶
Each product is one document. Variant attributes (colours, sizes, SKU codes) are collapsed into multi-valued facet fields on the parent. A search for "red shirts" returns the shirt once, not once per red variant. SKU-level lookups are still possible via dedicated fields — but listings, facets, and dedup logic all flow naturally because the index reflects how shoppers think about products.
The exception we're not making: B2B catalogues sometimes need variant-level documents. We'll cross that bridge when a customer needs it; the schema is not painted into a corner.
Search on or off (useSearch)¶
Search is off by default and switched on per site. A Typesense cluster configured for the
environment (the TYPESENSE_* variables) is not enough: a site only has search when its Use
Search setting (useSearch, search-config, default false) is on as well. SiteSearchSettings
(commerce-core) is the one place that decides it, read by the storefront, the admin and the worker:
| Method | Meaning |
|---|---|
isEnabled(site) |
useSearch on and Typesense configured. Free-text search exists for the site. |
categoryPagesUseIndex(site) |
Typesense configured and the catalogue has a merchandising tree. Its category pages are listed from the product index. |
productIndexNeeded(site) |
Either of the two: the catalogue keeps a product index. |
With search off for a site:
- Storefront.
/searchanswers through the ordinary not-found path (ResourceNotFoundException, so the branded 404 renders);/search/suggestreturns an empty result; there is no content search; theSearchActionis left out of the structured data; and the header search box is hidden on every theme.StorefrontChromeInterceptorputssiteSearchEnabledon the model of every page (a failed lookup counts as off), every theme's header (thesearchMenuslot and the theme's own fallback form) reads it. On category pages the "Search in {category}" box, the header's category scope and any?q=are ignored. - Worker. Nothing is written for the catalogue, and a reindex (the one the settings save requests, an admin rebuild, the nightly reconcile) drops its indexes: the alias, every versioned collection behind it and any pre-alias legacy collection (
SearchSchemaProvisioner.drop). The ten-minute schema check never rebuilds a switched-off catalogue. - Admin. The Search overview says search is switched off and why the rebuild buttons are disabled;
SearchReindexRunService.requestrefuses with the same reason.
Category pages are the exception. /c/... pages (the /c root, every category, facet landing pages), product grids bound to a category (MerchandisingNodeProductsDataProvider), the REST category browse and the sitemap's category counts are all answered from the product index. A catalogue with a merchandising tree therefore keeps its product index with search off (categoryPagesUseIndex), whatever the site's category-navigation-source: the tree's pages render either way. Only free-text search is switched off there. The catalogue keeps exactly the products a category can show (see below), so a product in no category drops off the /c root page too.
Switching search on requests a full reindex (SearchSettingsChangeListener → FullReindexRequestedEvent). The worker finds the product collection missing and builds it in bulk, then indexes content if content search is on. Switching it off requests the same reindex, which drops the indexes. contentSearchEnabled is a sub-switch that only applies while search is on.
There is no migration: after the deploy that introduced the switch, every site without useSearch=true lost its search (and its index, at the next reconcile) until an admin turns it on. The legacy typesenseEnabled key, which never did anything, was removed.
Which products are indexed (search scope)¶
Search finds only the products in the site's search scope, a site-level setting rather than a flag on each product. SearchProductScope (worker) decides it, and SearchIndexer asks it on every write path (single upsert, stock reindex, sweep, bulk rebuild), deleting a product that falls outside, so nothing excluded is resurrected; popularity pushes are partial updates that never create a document.
search-product-scope (search-config, enum):
| Value | A product is in scope when it is |
|---|---|
IN_COLLECTION (default) |
in at least one enabled collection (Product.collectionIDs holds collection seoNames). A product sold only through a direct link (a "200 club" product, say) stays out. |
MERCH_TREE |
in at least one visible category of the primary merchandising tree (the category and its ancestors not hidden), by its static backing (merchandisingNodeIds). A category backed by a saved search (VirtualQuery) is only evaluated when its page is shown, so while the primary tree has a visible one, this mode can't narrow anything and every enabled product is in scope. |
ALL |
enabled (the behaviour before scopes existed). |
search-root-collections (search-config, optional, edited with the collection picker): comma-separated collection seoNames. When set, a product must also be in one of them or in a collection below one of them (Collection.parentCollectionIDs, followed down, cycle-safe).
On a catalogue with category pages the index is shared, so a product a category can show (it resolves to a category of any tree, or the catalogue has a virtual-query category) stays in the index even outside the search scope, and search can find it: it is browsable anyway. The scope fully decides for catalogues without a merchandising tree.
The settings, the enabled collections (one projected query, CollectionService.findHierarchy) and the trees are read once into a snapshot per catalogue, refreshed at the start of every full reindex and on every collection or tree change, and reused for 30 seconds by single-product upserts. Changing either setting reindexes the catalogue (SearchSettingsChangeListener). Saving, disabling, enabling or deleting a collection reindexes its products against a fresh snapshot (CollectionIndexEvent); with root collections set, the whole catalogue is reindexed, since a collection's parents decide what is below a root.
Indexing Pipeline¶
Mongo write ──► ApplicationEventPublisher (in-process)
│
▼
SearchIndexQueueProducer
│
▼
RabbitMQ
(search.index.events)
│
▼
SearchIndexEventListener (worker)
│
▼
SearchIndexer ◄── MerchandisingResolver
│
▼
Typesense
Events¶
| Event | Triggered by | Effect |
|---|---|---|
ProductIndexEvent |
ProductService.save / delete |
Upsert or remove a single product doc. |
CollectionIndexEvent |
CollectionService.save / delete |
Reindex products in that collection (breadcrumb / collectionSeoNames refresh, and in or out of the search scope when the collection is disabled or enabled); the whole catalogue when the scope has root collections. |
MerchandisingNodeIndexEvent |
merch tree mutation | Recompute merchandisingNodeIds on affected products. |
TaxonomyIndexEvent |
active taxonomy change | Logged only. The schema is fixed, so there is nothing to migrate; a reconcile refreshes the denormalised taxonomyLevelIds. |
PageIndexEvent |
page save (Mongo mapping event) / PageService.delete |
Upsert or remove the page in the tenant's content index — only for sites with Content Search. |
ContentReindexRequestedEvent |
stereotype save (in-place sweep); the admin Search overview's Content only (rebuild=true, with a run id) |
Re-sweep the tenant's content index, or rebuild it into a fresh collection. |
FullReindexRequestedEvent |
seeders, tenant import, merchandising-tree changes, saving useSearch, search-product-scope or search-root-collections (SearchSettingsChangeListener); the admin Search overview's rebuild buttons (rebuild=true, with a run id) |
Re-sweep the catalogue in place (building a missing index in bulk, dropping the indexes of a site with search off), or, for the buttons, rebuild into fresh collections. |
Collection lifecycle¶
SearchSchemaProvisioner (worker) owns the collections. For each catalogue and kind (products, content):
- The stable name (
products__{catalogueId},pages__{catalogueId}) is a Typesense alias. Queries (MerchandisingQueryService,ContentQueryService) and single-document writes use it, so they follow whichever physical collection it points at. - The data lives in a versioned physical collection,
products__{catalogueId}__v{epoch millis}. Its metadata holdsschemaFingerprint: a digest of every field's name, type and options (facet, optional, index, stem and so on), the default sort, and a hand-bumpedPRODUCT_INDEX_REVISIONfor document-shape changes the field list can't express. - First use creates a collection and points the alias at it.
- Every physical collection is created in one method, which publishes
SearchCollectionCreatedEvent(catalogueId, collectionName)straight after creation and before the alias can point at it. Listeners run synchronously, so per-collection settings such as synonyms are in place before the collection serves traffic. If a listener throws, the new collection is dropped and nothing goes live.
Rebuild (schema change, zero downtime)¶
A rebuild creates a new versioned collection with the current schema, bulk-loads every enabled product into it (500 per Typesense import, ordered by id), then repoints the alias in one atomic call and drops the collection it replaced. Searches hit the old collection until the swap. If anything fails, including a document Typesense rejects, the new collection is dropped and the old one stays live; the failure is logged and the message goes back to the broker.
The previous collection is dropped straight after the swap rather than kept for rollback. A full load has already proved the new one, keeping it doubles storage, and rolling back is just another rebuild. Versioned collections that aren't live and are over two hours old (a crashed build) are swept on every commit.
What triggers a rebuild
| Trigger | Behaviour |
|---|---|
Worker startup (SearchSchemaMigrationRunner, on its own thread; disable with badger.search.schema-migration.enabled=false) |
Rebuilds each catalogue whose live fingerprint differs from the code's, or whose collection is still a legacy real collection. A schema change therefore ships with the deploy. A content collection that is missing altogether, for a site with content search on, is built too. |
The same check every ten minutes (badger.search.schema-check.interval-ms, default 600000; off with badger.search.schema-check.enabled=false) |
Catches schema changes made by a site setting rather than a deploy: switching semantic search on or off. Runs on its own thread, one pass at a time, so a rebuild never holds up the shared scheduler. |
| Any full reindex (nightly reconcile, seeder, tree change) | Checks the fingerprint first and rebuilds if stale; otherwise sweeps in place as before. |
Admin Search > Overview rebuild buttons (the Merchandising > Trees Search index button links there, and its old POST /admin/merchandising/rest/reindex goes through the same tracked service) |
Always rebuilds, even when the schema is current: everything, products only, or content only. It is the operator's "make the index right" button, so it replaces the live collection rather than patching it, which also clears anything corrupt. Other full-reindex publishers stay cheap in-place sweeps. |
Automatic triggers back off when another worker's rebuild looks to be in flight (a non-live versioned collection under two hours old). The button doesn't, so a crashed build can never block an operator. If two workers do race, both build, the last swap wins, and each drops the collection the alias pointed at when it swapped, so the end state is one full, current collection. The cost is duplicated work, never a bad index.
A collection dropped behind the worker's back (deleted in the Typesense dashboard, or a Typesense reset) is recreated without a restart. Each worker remembers which collections it has already ensured, but a write that fails with "not found" makes it forget, recreate the collection behind its alias and retry the write once, and any state check that finds the collection missing does the same for the next write. For content, the startup and ten-minute checks rebuild a missing collection outright, and the storefront asks for a content reindex (at most once per catalogue per ten minutes per node) when a search finds the collection missing for a site with content search on.
Edits during a rebuild: the in-flight collection is itself the marker. Any versioned collection that the alias doesn't point at and that is under two hours old counts as a build in flight, which every worker can see. Incremental product and page upserts and deletes are applied to the live collection and to every in-flight one, so an edit isn't lost. The marker clears on its own: at the swap the new collection becomes the alias target, and an abandoned build is dropped. Workers cache the lookup for one second, so an edit in the first second of a build, or one made while a page is already loaded into memory by the bulk load, can still be missed; the next nightly reconcile corrects it. A crashed build keeps receiving writes until it ages out after two hours, which is harmless.
Reindex runs (progress for the admin button)¶
An operator-requested rebuild is tracked as a SearchReindexRun (Mongo collection searchReindexRuns, TTL 30 days), owned by SearchReindexRunService in commerce-core:
| Field | Meaning |
|---|---|
catalogueId, scope (ALL, PRODUCTS, CONTENT), requestedBy, requestedAt |
Who asked for what, for which tenant. |
status |
REQUESTED (queued), then RUNNING (worker started), then SUCCEEDED or FAILED. |
startedAt, finishedAt, documentCount, error |
Filled in by the worker. documentCount is read back from the engine after a successful rebuild and may be empty if it couldn't be counted. |
request(siteContext, scope, requestedBy) saves the run, then publishes FullReindexRequestedEvent (ALL, or PRODUCTS with productsOnly) or ContentReindexRequestedEvent (CONTENT) with runId set. SearchIndexQueueProducer copies the run id into SearchIndexMessage.runId and picks the message shape from the scope: secondaryId is rebuild (everything) or rebuild-products (products only), and for CONTENT_REINDEX rebuild means rebuild rather than sweep. Those markers live in secondaryId so a worker that predates them still deserialises the message and does an ordinary reindex.
SearchIndexEventListener (worker) wraps the rebuild: markRunning(runId), do the work, then markSucceeded or markFailed (the failure is still rethrown to the broker, as before). Bookkeeping errors are logged and never fail the reindex. Messages without a runId (nightly reconcile, seeders, stereotype saves) are handled exactly as before and create no run.
Before running, the listener asks SearchIndexEligibility whether there is anything to do. A reindex is skipped when no storefront site uses the catalogue, when search isn't set up for its site, when search is switched off and no category pages read the index, or (content only) when content search or search is off. A skip is logged at INFO with the catalogue and site ids, and a tracked run is marked FAILED with an error starting Skipped: and the reason, which the Search overview shows. It is never reported as SUCCEEDED with 0 documents. (FAILED rather than a new status, so a web node that predates this still reads the run during a rolling deploy.) A skipped reindex is still handed to the indexers, as that is what drops the indexes of a site whose search is now off; for a catalogue with no site or no search engine it does nothing.
Which site's settings govern a catalogue¶
Indexes are per catalogue, but settings (useSearch, the search scope, contentSearchEnabled, semantic search, Typesense config, popularity window) are per site, and several sites can carry one catalogue id. In particular the admin (admin) and superadmin (system) interface sites are seeded with the default storefront's catalogue id. SearchSiteResolver (worker) is the one place that maps a catalogue to its site:
- interface sites (
SiteContext.isInterfaceSite(): modeadmin/system, or those seeded siteIds) are never considered; - among the storefront sites of a catalogue, the primary one wins deterministically: the site whose siteId is the catalogue id, then live over archived, then production over
-stg, then by siteId; - a message carrying a siteId that isn't a catalogue id (stereotype saves) resolves to that site's catalogue's primary site.
The scheduled jobs (SearchReconciliationJob, SearchSchemaMigrationRunner, ProductPopularityJob) walk primarySites() / storefrontSitesByCatalogue(), so each catalogue is processed once, as its primary site. Before this, the resolver took the first site in Mongo's natural order with a matching catalogue id; on an install where that was admin or system, the default storefront's content search read as off, its pages__default collection was never created, and an admin content reindex reported success with 0 documents.
Double submits: if the site already has a run REQUESTED or RUNNING that was requested in the last 30 minutes, request returns that run (alreadyInProgress) instead of starting another. Older unfinished runs are assumed lost and don't block. request refuses (IllegalStateException, shown as HTTP 409) when search isn't configured for the site, when search is switched off and no category pages read the index, or, for CONTENT, when content search (or search) is off.
Index status: SearchIndexStatusService.status(siteContext) reports whether the engine is configured, whether the site's search switch is on, whether its category pages keep the product index, the search scope, whether content search is enabled, whether the engine answered, the product and content document counts, when the live product collection was built (from its __v<epoch> name), a short schema fingerprint, and the latest run. It never throws: with search unconfigured or the engine down it returns that state and the Search overview still renders. The nightly reconcile's last-run time isn't recorded anywhere, so it isn't shown.
The admin pages and JSON endpoints (/admin/search/rest/status, POST /admin/search/rest/reindex, GET /admin/search/rest/reindex/{runId}) are described in the admin guide.
Migrating an existing install¶
Before this existed, products__{catalogueId} was a real collection, and an alias can't share a name with a collection. The first rebuild of such a catalogue (at startup, or on the nightly reconcile) therefore runs: build and fully load products__{catalogueId}__v{n} while the legacy collection keeps serving, drop the legacy collection, then immediately create the alias. The only unavoidable gap is those two calls, milliseconds long. Searches in it return empty results (the query layer already treats a Typesense failure that way) and a write in it is logged and picked up by the next reconcile. Because the check is "fingerprint differs or no alias", it is idempotent, and if a second worker has already migrated, the commit sees an alias, does an ordinary swap, and removes the first worker's collection.
Stemming¶
name, description and searchTerms are indexed with stem: true (Typesense 29's Snowball stemmer, English by default), so "boot" finds "Safety Boots" and "walking" finds "walk". Stemming is a schema option, so enabling it was exactly the fingerprint change the rebuild exists for.
codes (the product's SKU, its barcodes, and its enabled variants' SKUs and barcodes) is not stemmed, and every product search matches it with no typos: see Searching by SKU or barcode.
Reconciliation¶
A nightly @Scheduled job per worker node (SearchReconciliationJob, 03:00, badger.search.reconcile.cron) runs once for every catalogue, as its primary storefront site (see Which site's settings govern a catalogue):
reindexAllProducts: drop the index if the site has search off and no category pages; build it if it is missing; rebuild if the schema is stale; otherwise page through every enabled product (500 at a time) and upsert the ones in scope (deleting the rest), refreshing denormalised fields such asmerchandisingNodeIds.- Prune orphans: export the ids in Typesense and delete any that is not a live, enabled, in-scope Mongo product.
- Do the same for the content index of sites with content search, and drop it for sites without.
It is a full sweep rather than a recent-changes window, so drift of any age is repaired. Per-site failures are logged and don't stop the others. Drift-count metrics are not yet emitted.
Popularity¶
ProductPopularityJob (worker, 02:30, badger.search.popularity.cron; also once at startup after the schema check, badger.search.popularity.on-startup) keeps each product's popularity at the units it sold over the site's search-popularity-days (default 90). Per catalogue:
ProductPopularityService.unitsSoldByProductper site selling from it:OrderService.unitsSoldBySkuruns one Mongo aggregation (orders inSUBMITTED,PACKING,DISPATCHEDorDELIVEREDsubmitted in the window, uncancelled lines, summed per product SKU), then the SKUs are matched to products 500 at a time, a variant's sales counting for its product. Memory is one entry per product sold, whatever the order volume.- Compare with the values stored in
productPopularity(one document per product that sold; none means 0) and keep only the differences, including products that stopped selling. SearchIndexer.pushPopularity: partial updates (import,action=update, 500 a batch) to the live collection and any collection being rebuilt. A 404 (the product isn't indexed) counts as done.- Store the values Typesense accepted. A refused batch isn't stored, so the next run retries it.
Full builds and single-product upserts read the stored value (one productPopularity query per page of 500), so they agree with what the job pushed. See Relevance for how the value is used.
Learned ranking scores¶
LearnedRankingJob (worker, 03:00, after popularity; also at startup after it) keeps rankUnits, rankConversion, rankRevenue and rankMargin at each product's learned score for that goal, for catalogues that qualify: the same compare, push changes, store accepted values pattern, over productRankScores. See Ranking by What Sells.
Schema & Facets¶
Base fields (always present)¶
id, siteId, name, description, miniDescription, seoName, canonicalPath, imageUrls, price, comparePrice, onSale, inStock, manufacturer, rating, featured, popularity (units sold, see Popularity), rankUnits / rankConversion / rankRevenue / rankMargin (learned scores, see Learned ranking scores), publishedDate, collectionSeoNames, taxonomyLevelIds, taxonomyAttributePairs, taxonomyAttributeIds, merchandisingNodeIds, categories (the names of those categories, their parents and the taxonomy level chain, see Searching by category name), searchTerms, a num_<attribute> float array per attribute with numeric values (the num_.* pattern field), and variantCombos / variantComboAttrs (which value pairs one buyable variant carries, for Black + Size 9; variantCombos is faceted so the counts follow the combinations). name, description, searchTerms and categories are stemmed. inStock comes from the inventory service (any enabled variant in stock); see Stock and the In stock filter for how stock changes are coalesced into index writes.
Semantic sites (site config search-semantic-enabled) also have embedding, a float[] that Typesense computes itself from name, description, searchTerms and categories with its built-in ts/all-MiniLM-L12-v2 model (SemanticSearchSettings, commerce-core, holds the field, source fields and model for both sides). Documents don't change: the indexer never sends an embedding. The fingerprint covers the field and its model, so the schema, and the fingerprint, is per site: SearchSchemaProvisioner builds each site's from its setting, switching it makes just that site's collection stale, and every other site's schema is exactly what it was. Embedding is slow (about 19 ms a product on an M1, more on ARM nodes), so a semantic site's rebuild imports 25 products per request on a client with a 120 s read timeout (TypesenseClientFactory.bulkClientFor) instead of 500 on the 5 s one, so an import never times out and gets retried (embedding the batch again). A config read that fails stops the check or rebuild rather than guessing the setting. Queries are covered in Semantic search.
taxonomyLevelIds is denormalized: a product assigned to "Womens > Outerwear > Coats" has all three level IDs in the array, so a query against any ancestor matches without recursive lookups.
merchandisingNodeIds is the magic that makes node-driven listings cheap. At index time, MerchandisingResolver walks every merch tree and asks "does this product resolve into this node?" The answer set is stored on the doc. Browsing a node is then a single equality filter — no per-request resolution.
Per-site attributes¶
The schema is the same for every tenant. Taxonomy attributes aren't separate fields: each is flattened into attributeId:value strings in the facetable taxonomyAttributePairs array (one pair per value of a multi-valued attribute), so a tenant's facets come from its own data without a per-tenant schema. taxonomyAttributeIds lists the attributes a product has a value for; faceting on it gives each attribute's coverage of a result set, which decides which filters the rail shows (see Search and Discovery). For sizes and colours that come from the variants, a product's pairs list only the values an enabled, in-stock variant carries (see Sizes and colours follow stock); taxonomyAttributeIds still counts them, so coverage doesn't drop as sizes sell out. Attribute values that read as numbers are also indexed per attribute as num_<attribute> (one schema entry, the num_.* pattern), so a range filter (length-mm=40-60) is a numeric filter rather than a list of strings (see Colour swatches and number ranges).
Facet configuration cascade¶
A node renders facets from a three-level fallback: node.facetConfig → tree.defaultFacetConfig → taxonomy-derived default (every facetable attribute, ordered by taxonomy display order). Most sites will set the tree default and rarely touch individual nodes.
REST API¶
The search layer is exposed to headless/storefront-API consumers through two read
endpoints (tag Search in the API reference). Both delegate to the same
MerchandisingQueryService.search(...) as the storefront pages, so their products, facet
counts, and pagination match the web /search and /c/** pages exactly.
GET /v1/search/products— catalogue-wide faceted search. Mirrors the storefront/searchpage. Parameters:q(free text),sort,filter,pageNumber,pageSize.GET /v1/merchandising/products?path=/c/...— products within a merchandising-tree node, resolved by canonical path. Mirrors a/c/**category page.404if no node exists at that path or it's hidden. Samesort/filter/paging parameters.
Facet selection uses a repeated filter parameter carrying facetId:value pairs — e.g.
?filter=colour:Black&filter=colour:Blue&filter=manufacturer:Nike. Values for the same
facet are ORed, different facets are ANDed. The facetId is the id of a FacetGroup
returned in the response, so a client round-trips a facet straight back as a filter. Which
facets are aggregated is resolved server-side via the facet configuration
cascade — there's no client control over the facet set.
Both responses are a ProductSearchResults: a page of products (data +
paginationParams), the facets aggregated over the matched set, and a query echo.
pageNumber is 1-based.
Authoring Model¶
Operators don't think about indexes, schemas, or events. They think about:
- "What's my catalogue shape?" → Define a taxonomy, classify products.
- "How do I want customers to browse?" → Build a primary merchandising tree by mixing collections, taxonomy projections, virtual queries, and curated lists.
- "What other entry points do I need?" → Add additional named trees for gift guides, megamenus, seasonal landing pages.
The platform handles index lifecycle, schema migrations, breadcrumb computation, alias swaps, drift reconciliation, and node resolution. None of these are exposed as authoring surface.
What This Is Not (Yet)¶
Round 1 of the search platform delivers the foundation: domain model, persistence, admin scaffolding, and the indexing pipeline. The following are deliberately deferred:
- Rendering off search —
CollectionControllerstill hits Mongo today. Round 2 rewires it. - URL & SEO strategy — canonical paths, redirects from old
/collection/{seoName}to new/c/{path}, sitemap generation, hreflang. - Personalisation and ranking signals — popularity, recency, user-history boosts. Foundation supports it; we haven't built it.
- A/B testing of merchandising trees — the multi-tree model exists partly to enable this later.
- SaaS Sett authoring tooling — collaborative editing, drafts, scheduled publishing.
- Migration tooling — automatic conversion of existing Collection hierarchies into a default merchandising tree. Existing collections continue to work via
ManualCollectionBacking; explicit migration is a future, opt-in tool.
Glossary¶
- Tree —
MerchandisingTree. A named hierarchical projection of the catalogue. A site has one or more. - Primary tree — the tree marked
primaryfor a site. Drives canonical URLs and SEO. - Node —
MerchandisingNode. A position in a tree with display metadata and a backing. - Backing — the strategy by which a node resolves products. One of five types (manual collection, taxonomy projection, virtual query, manual list, group).
- Virtual category — a node backed by a
VirtualQuery. Authored as a saved filter, behaves to the customer as just another category. - Projection — a node backed by a taxonomy level + optional attribute filters. The merchandising tree's view of a slice of the taxonomy.
- Reconciliation — the nightly job that re-asserts Typesense state against Mongo. Catches drift from missed events.
- Alias swap — zero-downtime rebuild: build a new versioned Typesense collection, then atomically repoint the stable alias. See Collection lifecycle.
- Roll-up — variant attributes collapsed onto the parent product doc, so listings dedupe and facets count by product.
Content index¶
Sites that have search on and opt into Content Search get a second per-tenant collection,
pages__{catalogueId}, holding one document per CMS page / blog post (title, headings, body text,
tags). It shares the client, queue and reconciliation machinery above but none of the product schema,
so product search is unaffected whether it's on or off. It uses the same alias, versioned-collection and schema-fingerprint machinery as products, so a content schema change is rebuilt the same way.
Related Documentation¶
-
Content Search — optional search over pages and blog posts.
-
Search and Discovery — customer-facing search features.
- Multi-Tenancy Architecture — tenant isolation principles.
- Product Catalog Management — products, variants, collections.
- Extension System — how rendering extensions consume the search layer.