Listing Traffic Guard¶
Category pages (/c/**), /search and /collection/** are the most expensive pages a shop serves:
each runs a Typesense search and a full render. Their filter links also make the URL space
endless, since every facet value can be added to every other. Well-behaved crawlers respect rel="nofollow"
and noindex, follow; others don't. In October 2026 a crawler walking Halden's facet combinations filled
all 200 of the web pod's Jetty threads, queued about 9,700 more requests behind them, and the pod
failed its health check and was restarted over and over.
Three layers stop that from taking a site down.
1. A finite URL space¶
MerchandisingQueryService.applicableSelections decides which selections a listing applies, and both
MerchandisingPageController and SearchController call it (through CategoryCountService) before
building the query:
- a param is a facet only when the shop has one by that id: a platform facet (brand, price, rating, in stock, on sale, featured) or an attribute of the active taxonomy. Anything else is dropped;
- at most
search-max-selected-facetsfacets (site setting, 5 to 10, default 8) apply, in request order.
FacetParams.onlyApplied then removes the dropped params from the request params the page builds its
links from, so pagination, sort and filter links never carry them on. FacetRailModel.withFacetLimit
removes the links (and min/max inputs) from groups that would add one more facet once the limit is
reached, and every theme's rail shows the fragment/facet-limit :: notice message
(search.filters.limitReached, or search.filters.limitDropped when a URL named more).
2. Load shedding and a per-client throttle (per pod)¶
ListingTrafficGuardFilter (web-mvc, render/filters) runs ahead of tenant and user resolution, so a
refused request costs no database, Redis or visitor record. It answers with short plain text, not an
error page.
Setting (badger.listing-guard.*) |
Default | Meaning |
|---|---|---|
enabled |
true |
Off lets every listing through |
max-concurrent |
40 | Listing requests running at once on a pod (Jetty has 200 threads) |
filtered-max-concurrent |
16 | Of those, filtered ones: a facet, a sort, or any unknown param |
queue-wait-ms |
100 | How long a request waits for a place before a 503 with Retry-After: 5 |
filtered-per-minute |
120 | Filtered listing requests a client may make a minute on a pod, then 429; 0 turns it off |
Each setting also has an environment variable, for example BADGER_LISTING_GUARD_MAX_CONCURRENT.
Plain category browsing keeps at least 24 places however many filtered requests are queued, and
product, basket and checkout pages never wait behind listings. The client is Cloudflare's
CF-Connecting-IP (else the remote address). It is used only as an in-memory key and never logged.
Metrics: badger_listing_guard_rejected_total{reason="busy"|"throttled"} and
badger_listing_guard_in_flight.
3. The edge (Cloudflare)¶
The throttle in the pod is a backstop. The main per-client rate limit is a Cloudflare rate-limiting rule on the listing paths with a query string, which drops the traffic before it reaches the cluster and counts across pods.
Health checks¶
Point the pods' probes at /actuator/health on the management port (MANAGEMENT_PORT, 9090 on the
cluster), which has its own threads. A probe on the application port waits in the same queue as the
listings, so a flood fails the liveness check and the pod is restarted, again and again, instead of
just slowing down.