Version 5.3.0
Server-side performance for large catalogues on the htmx transport, from the load test in
#5. Measured on a catalogue of that
shape — 37,272 products, 134 facets, descriptions — decoding the artifact on each request went from
2.1 s and 967 MB to 0.5 s and 217 MB, or to nothing with the new OPcache setting, and an unrefined
listing from 155 ms to 17 ms. tests/bench/heavy-catalogue.php reproduces the numbers.
Added
- Keep artifacts in OPcache (
cacheArtifacts), off by default. The decoded artifact is written once per publish as a PHP file that OPcache holds in shared memory, so a request reads it without reading or decoding anything. It needs OPcache memory for the whole artifact — about 180 MB for the catalogue above — and Caffeine checks that OPcache actually holds the file, logging a warning and decoding as before when it does not. See "Large catalogues" indocs/TWIG.md. - With
shardPayloadon, a server render no longer reads the payload shard. Publishing writes a small server-only lookup file beside the shards (object ids and a hash of each payload), and the server loads the payloads for the page of hits it renders from the records table in one query. A record rebuilt or removed since the publish fails its hash check and is served from the payload shard, so the server still shows exactly what was published. For the catalogue above that is 179 MB per request instead of 217 MB, and the payloads stay out of OPcache when Keep artifacts in OPcache is on. See §5.1 ofdocs/ARTIFACT.md. Builder::EVENT_DEFINE_SOURCE_QUERY, fired with the query a build loads elements with, so an index can eager-load the variants and relations its mapping reads. Seedocs/EXTENDING.md.caffeine/index/build --all --workers=Nsplits a full build across N processes by element id. Records are pruned once, after every worker finishes, and not at all if one fails.- A "Large catalogues" section in
docs/TWIG.md, including when to give each product type its own index plus one index for site-wide search.
Changed
- The server no longer holds each facet's record → values map. Decoded, it was one PHP array per
record per facet, most of the memory on a catalogue with many facets. Token postings are held as
flat lists for the same reason.
Artifact::$tokenPostingsis replaced by$tokenIds,$tokenWeightsand$tokenOffsets, withtokenPostings(int $token)andrecords(string $facet)to rebuild the old shapes.toArray()still returns the old shape. - Facets are counted from their postings rather than by walking every record in the base set once per facet, so a facet that only one product type carries costs a query in proportion to the products carrying it. A facet with no refinement of its own now reuses the result as its base set instead of recomputing it. Both engines, with a new conformance fixture covering each route a count can take. Facet stats are now summed in value order in both engines (QUERY_SPEC §3.3).
- The artifact format is unchanged, and a browser still running the previous runtime keeps
working. A sharded index gets its lookup file on its next publish (its checksum now includes
the lookup, so that publish is not skipped as unchanged); until then the server reads the
payload shard as before. Run
caffeine/artifact/publishto get it straight away. caffeine/index/build --allshows a real ETA instead of a nonsense one.
Version 5.2.0
Added
Indexes::EVENT_DEFINE_INDEX, so a plugin or module can add attributes and sortings to an index as it loads. Facets can then come from content editors manage, instead of from project config. What a handler adds is never saved to project config. (#4)Indexes::invalidate(), which marks an index's records dirty and schedules an update, for a module whose attribute set just changed.Indexes::getStoredByHandle()/getStoredByUid()/storedByUid(), which return a definition exactly as project config stores it. The index edit screen now uses these.
Changed
- Each publish records a fingerprint of the definition it was built from. A dirty build compares
it with the current definition first and rebuilds every record when they differ, so a change in
what a module adds is caught without
--all. Artifacts published before this release have no fingerprint and are left alone until their next publish. Requires a database migration.
Version 5.1.0
The same code as 5.0.1, under the version it should have had. 5.0.1 added features, which calls for a minor release, not a patch. Nothing changed between them, so upgrading from 5.0.1 is a no-op.
Version 5.0.1
Added
- A
filtersoption on{% caffeine %}for refinements that always apply and stay out of the URL, so a category or brand page can share the shop's listing template. Fixed buckets come back withfixed: true. (#2) caffeine/index/prune-orphans, which runs the same cleanup as Craft's garbage collection on demand. (#3)
Changed
- Garbage collection now also deletes content-addressed files in an index's publish directory that no ledger row lists and the live pointer does not name, such as files left behind by a database restore, plus temporary files from interrupted writes that are over an hour old.
- Publishing, pruning and the sweep take a per-index lock, so two publishes of the same index no longer race each other.
Fixed
- Clicking a facet link in Chrome or Edge, which focus links on click, no longer leaves its region unchanged. Only a region with a text field being typed into is skipped, and focus goes back to the same control after the swap. Back and Forward are fixed by the same change. (#1)
- A page passing a refinement through
paramsthat wasn't in its URL no longer re-fetches its results on every load; usefiltersfor this instead. (#2) - Deleting an index through project config now removes its records, ledger rows and published files, and changing one marks its records dirty. Both handlers read the index uid from the wrong key and returned without doing anything. (#3)
Version 5.0.0
Initial release.
Indexes
- Model an index in the control panel: sources, attributes, facets, sortings, transport and paging. Definitions live in project config, so an index deploys with the code the way sections and fields do. Everything derived from one — records, artifacts, dependency edges — is environment-local.
- Sources for entries, categories, tags, assets, users and Commerce products, plus a registry so a plugin can add its own element type.
- Value extractors read field types that would otherwise index as an opaque object: links (Hyper, FreeLink, Craft's Link field), Google Maps addresses, Money, colours and dropdowns. They match on shape rather than class name, so a link field works whatever produced it, and a plugin can register its own.
- Facets can be strings, hierarchies, numbers, booleans, dates or coordinates. Geo facets are filtered by distance from a point.
- Stopwords and synonyms per index.
- Attribute values can be reached by dotted path, including into related elements and into an
extractor's named parts —
venue.city,link.text.
Querying
- Two query engines, PHP and JavaScript, implementing one written specification
(
docs/QUERY_SPEC.md) and checked against one fixture suite. The server renders the first paint and the browser takes over refinement. - Facet counting follows the disjunctive/conjunctive asymmetry that makes counts behave the way a visitor expects: a disjunctive facet is counted with its own refinements excluded, so "Globex (12)" keeps showing a useful number after "Acme" is ticked.
- Full-text search over an inverted index built in PHP at index time and shipped inside the artifact, so the browser needs no search library — just a binary search over a sorted token array.
- Numeric and date ranges, hierarchical refinement, precomputed sortings, and distance sorting.
The front end
{% caffeine %}and{% caffeineresults %}Twig tags, and eight widgets — search box, refinement list, hierarchical menu, range input, current refinements, sort, stats, pagination.- Everything works without JavaScript. Every facet is a real
<a href>and every widget a real form; the runtime intercepts controls that already work. - Three transports per index:
htmx(fragment swap, the default),client(the browser answers locally, no round trips) andalgolia-json(the Algolia response shape, for InstantSearch and friends). - Widget markup is overridable by dropping a template of the same name under
_caffeine/. - Designed for a full-page cache: cached HTML embeds a stable pointer, never a versioned URL, and the runtime refines a cached canonical page before paint.
Publishing and updates
- Artifacts are content-addressed and immutable, published behind a small mutable pointer, so payloads can be cached forever and the pointer can be cached not at all.
- Publishes to the local web root or to any Craft filesystem (S3, a CDN volume).
.gzand.brsidecars are written alongside. - An identical rebuild writes nothing, spends no version and leaves the pointer's timestamp alone.
- Element saves mark the affected records stale and a debounced queue job rebuilds and
republishes. A
resaveacross thousands of entries collapses to one job. - Denormalised values stay honest:
caffeine_depsrecords every element read to build a record, so renaming a category marks every record that copied its title. - A failed build leaves the previously published artifact live and untouched.
The control panel
- Index list with health — record count, what is stale, live version, artifact size, last build.
- A definition builder, a record preview that shows what the mapper produces for one real element, and a query playground that runs a state against the published artifact.
- Manual rebuild and republish, and a settings screen.
Console
caffeine/index/status, build, preview, touch; caffeine/artifact/publish, status,
versions, verify, prune.
Editions
- Lite (free): one index, entries only, string and boolean facets, the fragment transport, relevance plus one named sorting.
- Pro: unlimited indexes, every element type, every facet type including geo, all three transports, unlimited sortings, stopwords and synonyms, and the control panel's preview and query playground.
A site whose Pro licence lapses is downgraded rather than broken: the stored definition is left intact and the index keeps working with the Lite feature set, so renewing restores exactly what was there.
Requirements
Craft 5.3+ and PHP 8.2+. No runtime dependencies beyond Craft itself; the browser runtime ships as ES modules with no build step.