Patterns and testing
Cache-aside as one call, keeping the copy from going stale, and proving both in a test that needs no redis.
Cache-aside with Remember
A cache that is down, disabled or holding an older shape is not an error — the loader runs and the result is served. Also stampedes (RememberOnce), TTL jitter, and negative caching.
go
package main
import (
"errors"
"math/rand/v2"
"time"
core "gitlab.finema.co/finema/idin-core/v2"
)
// --- Example 4: cache-aside with Remember -----------------------------------
//
// Look in the cache; on a miss compute the value and store it. Remember is that
// in one call, and its guarantee is what makes it safe to sprinkle around: a
// cache that is down, disabled, or holding a value written by an older version
// of the struct is not an error — the loader runs and the result is served.
// Only the loader failing fails the call.
//
// Adding a cache therefore cannot introduce a new way for a request to fail.
// The worst case is the speed you had before it.
//
// with redis: one GET
// without redis: one loadUser, every time
// with a stale-shaped value: one loadUser, and the key is overwritten
// errNotFound stands in for the repository's not-found error.
var errNotFound = errors.New("not found")
func runRemember(ctx core.IContext) {
// Jitter the TTL: a thousand keys written in the same second expire in the
// same second, and the cliff arrives at the database as a spike. This turns
// a periodic cliff into a flat line for the price of one addition.
u, err := core.Remember(ctx.Cache(), "user:v1:42", jitter(5*time.Minute, 30*time.Second),
func() (user, error) { return loadUser(ctx, "42") })
if err != nil {
ctx.Log().Error("could not load the user at all", "err", err)
return
}
// RememberOnce puts a lock around the loader: one caller computes, the rest
// wait for it to publish, and fall back to loading it themselves rather
// than failing. Use it when the loader is expensive *and* many callers want
// the same key at once — Remember is cheaper and right for everything else.
//
// The wait should be a little longer than the loader's p99. Too short and
// everybody falls through and computes it anyway; too long and a slow
// loader holds a thousand requests instead of failing them.
report, err := core.RememberOnce(ctx.Cache(), "report:tenant-1",
time.Hour, // how long the result is cached
3*time.Second, // how long to wait for whoever is building it
func() (string, error) { return buildReport(ctx, "tenant-1") })
found, _ := lookupUser(ctx, "does-not-exist")
ctx.Log().Info("remember", "user", u.Name, "report", report, "found", found != nil, "err", err)
}
// jitter spreads expiries. math/rand is the right tool here: this is
// load-shaping, not a secret.
func jitter(base, spread time.Duration) time.Duration {
return base + time.Duration(rand.Int64N(int64(spread)))
}
// userLookup is how a cache stores "nothing". A nil value is indistinguishable
// from a miss once it comes back out, so the answer has to carry a flag.
type userLookup struct {
User *user `json:"user"`
Found bool `json:"found"`
}
// lookupUser caches the absence too. A key that is looked up constantly and
// does not exist hits the database every single time otherwise — an easy way
// for one bad client to become a load problem.
//
// The negative TTL is deliberately much shorter than the positive one: somebody
// who has just signed up should not be told they do not exist for the next five
// minutes.
func lookupUser(ctx core.IContext, id string) (*user, core.IError) {
res, err := core.Remember(ctx.Cache(), "user:v1:lookup:"+id, 30*time.Second,
func() (userLookup, error) {
u, err := loadUser(ctx, id)
if errors.Is(err, errNotFound) {
return userLookup{Found: false}, nil
}
if err != nil {
return userLookup{}, err
}
return userLookup{User: &u, Found: true}, nil
})
if err != nil {
return nil, err
}
if !res.Found {
return nil, core.New(404, "USER_NOT_FOUND", "user not found")
}
return res.User, nil
}
func loadUser(_ core.IContext, id string) (user, error) {
if id == "does-not-exist" {
return user{}, errNotFound
}
return user{ID: id, Name: "ann"}, nil
}
func buildReport(_ core.IContext, tenantID string) (string, error) {
return "report for " + tenantID, nil
}
// Choosing what to cache is most of the work:
//
// worth caching not worth caching
// expensive reads, far more read than written written more often than read
// the result of an external API call data that must be exactly current
// computed aggregates and reports large blobs — that is storage
// session and token lookups the only copy of anything
//
// The measurement worth doing first: how long does the uncached path actually
// take, and how often is it called? A cache in front of a 2ms query called
// twice a minute adds a class of bug and saves nothing.
//
// And the boundary that matters: if the uncached path cannot serve traffic at
// all, the cache is not a cache any more. It is a dependency — and this one is
// designed to be lost.Invalidation
Write first, then invalidate — the other order lets a concurrent read put the old value back. Plus DelByPrefix for derived keys, versioned keys for when there are too many to list, and pub/sub for the copies held in other processes.
go
package main
import (
"fmt"
"time"
core "gitlab.finema.co/finema/idin-core/v2"
)
// --- Example 5: invalidation ------------------------------------------------
//
// Two strategies, and mixing them badly is where stale data comes from.
//
// expire and forget a short TTL; stale for at most that long. Simplest, and
// correct wherever "up to a minute old" is fine.
// write-through delete the key when the underlying data changes.
//
// Order matters — write first, then invalidate. Invalidating before the write
// lets a concurrent read repopulate the cache with the old value, and the cache
// is then wrong until the TTL rescues it.
func runInvalidation(app *core.App, ctx core.IContext) {
sub := watchConfigChanges(app)
if err := sub.Start(); err != nil {
// Subscribe is the one cache operation that fails loudly with no cache,
// rather than degrading. A read that misses recomputes and is still
// correct; a subscriber that silently receives nothing forever is a
// service that looks healthy while doing none of its work.
ctx.Log().Warn("no pub/sub: in-process copies will not be invalidated", "err", err)
return
}
defer func() { _ = sub.Stop(ctx) }()
_ = updateUserName(ctx, "42", "ann-the-second")
publishConfigChange(ctx, 7)
// Pub/sub has no acknowledgement, so this is a demo affordance, not a
// pattern: give the handler a moment before the process moves on.
time.Sleep(50 * time.Millisecond)
}
// updateUserName is the write-through shape, in the order that is safe.
func updateUserName(ctx core.IContext, id, name string) core.IError {
if err := saveUser(ctx, id, name); err != nil { // 1. the durable write
return core.Wrap(err, "save user")
}
// 2. then the cache. Forget deletes and ignores the error, because failing
// a request over a failed cache *invalidation* trades a stale read for an
// outage. Use Del where a stale value is not survivable.
core.Forget(ctx.Cache(), "user:v1:"+id, "user:v1:lookup:"+id)
// Derived views live under one prefix so a single call clears all of them.
// The thing that actually breaks cache invalidation is a value derived from
// another value, cached under a key nobody remembers to delete.
if _, err := ctx.Cache().DelByPrefix("user:v1:" + id + ":"); err != nil {
ctx.Log().Warn("could not clear derived keys", "id", id, "err", err)
}
return nil
}
// Versioning is the other answer, and the better one for an entity with many
// derived views: there is no list of keys to keep in sync, and no window where
// half of them are invalidated and half are not.
func permissionsKey(ctx core.IContext, id string) string {
version, _ := ctx.Cache().Incr("ver:user:"+id, 0, core.NoExpiry) // delta 0 reads
return fmt.Sprintf("user:v1:%s:v%d:permissions", id, version)
}
// bumpUserVersion makes every derived key of this user unreachable at once. The
// old keys are not deleted — they simply stop being addressed and expire on
// their own, which is why every derived key still needs a TTL.
func bumpUserVersion(ctx core.IContext, id string) {
_, _ = ctx.Cache().Incr("ver:user:"+id, 1, core.NoExpiry)
}
// Deleting a key in redis invalidates it for everyone, because there is one
// redis. Invalidating something held *in a process* — a config struct, a
// feature-flag map, a compiled template — needs a message, and pub/sub runs on
// the same cache connection, so there is nothing extra to configure.
func watchConfigChanges(app *core.App) core.ISubscriber {
sub := app.NewSubscriber()
sub.On("config.changed", func(ctx core.IContext, msg *core.PubSubMessage) error {
version, err := core.BindMessage[int](msg)
if err != nil {
return err
}
// Reload rather than trusting the payload: the message says *that*
// something changed, and the shared store says what it changed to. A
// handler that applies the payload directly is one dropped message away
// from a replica that disagrees with the others forever.
ctx.Log().Info("configuration changed, reloading", "version", version)
return nil
})
return sub
}
func publishConfigChange(ctx core.IContext, version int) {
core.Forget(ctx.Cache(), "config") // the shared copy
_ = ctx.PubSub().Publish("config.changed", version) // the in-process copies
}
func saveUser(_ core.IContext, _, _ string) error { return nil }
// Two caveats worth carrying:
//
// - The memory cache's pub/sub is in-process, so an invalidation message
// reaches this binary and nobody else. It makes the code testable; it does
// not make it distributed.
//
// - permissionsKey and bumpUserVersion read and write a counter that has no
// TTL. That is deliberate — a version that expires silently reuses old key
// names — but it is also the one key here that leaks if the entity is
// deleted, so delete it with the entity.Testing, and surviving no cache
NewMemoryCache() is a real cache in-process, so the caching path is exercised rather than stubbed. Running the same code against NewNoopCache() is what pins down that it still works with nothing at all. See Testing.
go
package main
import (
"time"
core "gitlab.finema.co/finema/idin-core/v2"
)
// --- Example 6: testing, and code that survives no cache --------------------
//
// Tests need no redis. core.NewMemoryCache() is a real cache in the same
// process — same encoding, real expiry, atomic counters, working locks and
// pub/sub — so the caching path is exercised rather than stubbed out.
//
// What it cannot do is span processes, so anything whose *point* is
// coordination between processes is not being tested by it: a distributed lock
// has one holder to begin with, a rate limit counts one process out of N,
// pub/sub fans out only to this binary, and Redis() returns nil. Those need a
// real redis under the integration tag.
//
// The test this file is really about is the one below. Running the same code
// against the memory cache and against the disabled cache — and expecting 1
// call and then 2 — pins down both halves at once: that the caching is real,
// and that the code still works without it.
//
// func TestCachesTheUser(t *testing.T) {
// c := core.NewMemoryCache()
// defer c.Close()
// app, err := core.NewApp(env, core.WithCache("default", c))
// require.NoError(t, err)
// ctx := app.NewContext(context.Background(), core.ModeTest)
//
// var calls int32
// load := func() (user, error) {
// atomic.AddInt32(&calls, 1)
// return user{ID: "1", Name: "ann"}, nil
// }
// _, _ = core.Remember(ctx.Cache(), "user:v1:1", time.Minute, load)
// _, _ = core.Remember(ctx.Cache(), "user:v1:1", time.Minute, load)
//
// // the assertion that matters needs no cache internals at all
// require.Equal(t, int32(1), atomic.LoadInt32(&calls))
//
// // and the memory cache is inspectable when the internals do matter
// ttl, err := c.TTL("user:v1:1")
// require.NoError(t, err)
// require.InDelta(t, time.Minute.Seconds(), ttl.Seconds(), 2)
// }
func runTesting(ctx core.IContext) {
// The same assertion as running code: 1 against a real cache, 2 against
// none. Anything else means the caching is not doing what it claims.
ctx.Log().Info("loader calls",
"memory_cache", countLoads(core.NewMemoryCache()),
"no_cache", countLoads(core.NewNoopCache()))
}
func countLoads(c core.ICache) int {
defer func() { _ = c.Close() }()
calls := 0
load := func() (user, error) {
calls++
return user{ID: "1", Name: "ann"}, nil
}
_, _ = core.Remember(c, "user:v1:1", time.Minute, load)
_, _ = core.Remember(c, "user:v1:1", time.Minute, load)
return calls
}
// requireRealCache is for the few paths that genuinely cannot work without a
// cache. Everything else should not ask: the whole point of the disabled cache
// is that cache-aside code runs unchanged in an environment that has none.
//
// The three answers a disabled cache gives that change *behaviour* rather than
// just speed, and are therefore worth a check:
//
// Incr returns 0 a limit built on it never trips
// SetNX returns true every caller believes it is the first
// Lock is granted every replica holds every lock
//
// On a laptop all three are the right answers. On a multi-replica deployment
// that lost its CACHE_* configuration by accident, all three are silent
// correctness bugs.
func requireRealCache(ctx core.IContext) core.IError {
if !ctx.Cache().Enabled() {
return core.New(503, "CACHE_REQUIRED", "this endpoint needs a cache")
}
return nil
}
// logCacheMode belongs at boot, where somebody reads it once, rather than in a
// handler where it would be noise. "Which cache did this process actually get"
// is the first question of every stale-data and every double-charge
// investigation, and it costs one line to have the answer already written down.
func logCacheMode(app *core.App) {
c := app.Cache()
app.Log().Info("cache",
"enabled", c.Enabled(), // false => the disabled backend
"distributed", c.Redis() != nil, // false => memory or disabled: this process only
"prefix", c.Prefix())
}
// Which backend for which test:
//
// NewMemoryCache() almost every test — fast, hermetic, no
// container, and a real cache
// NewNoopCache() asserting the code works with no cache
// real redis (make test-integration) the cluster and sentinel paths, and
// anything that has to be believed
//
// Expiry is real in the memory cache, so a 50ms TTL is testable without
// waiting. Keep those tests rare and the durations small: a suite that sleeps
// its way through TTL assertions gets slow, and sleeping tests are the first
// ones to go flaky on a loaded CI runner.