Home / Glossary
Short, accurate definitions of the terms that come up in my work, each linked to a deeper write-up where there is one.
Last updated
A delivery guarantee where every message is processed one or more times, so nothing is lost but duplicates are possible. It is the default for most Kafka consumers, because a consumer that fails before committing its offset will receive the same messages again.
Read more →Keeping data consistent between two systems where changes can start on either side. It needs a clear owner for each field, version-aware updates so a system ignores echoes of its own writes, and a periodic reconcile job to catch drift.
Removing or refreshing cached data when the event that changes it happens, rather than waiting for a timer to expire. It keeps caches fast without serving stale values for long.
The number of messages a consumer group still has to process in a partition: the gap between the latest offset written and the group's committed offset. Rising lag means consumers are falling behind producers.
The process in which a Kafka consumer group reassigns partitions among its members, triggered when a consumer joins, leaves or stops responding. Messages processed but not yet committed before a rebalance are delivered again to the new owner.
Read more →Kafka's guarantee that a read-process-write loop within Kafka takes effect once, by committing consumed offsets and produced records in a single transaction. It does not extend to side effects in external databases or APIs.
Read more →A test that passes and fails on the same code without any change. Most flakiness comes from shared test data, timing assumptions, environment dependencies or real race conditions in the product.
Read more →A consumer whose processing has the same effect whether a message arrives once or several times. It is usually built by storing a unique message key in the same transaction as the side effect and skipping keys it has already seen.
Read more →Checking a language model's response before it is used, typically against a schema and business rules, and retrying or falling back when it fails. It stops one malformed response from breaking every step after it.
Read more →A sandboxed software project, usually a repository in a container, where an AI agent attempts a task and a verifier scores the result. The score becomes the reward signal for reinforcement learning or the result of an evaluation.
Read more →A Kafka consumer setting, group.instance.id, that gives each consumer a stable identity. A consumer that restarts within the session timeout rejoins with its old partitions and does not trigger a rebalance.
Read more →A pattern where a service writes an outgoing message to an outbox table in the same database transaction as its state change, and a separate process delivers it. It prevents changes that are saved but never announced, or announced but never saved.
Read more →The automated check that decides whether an AI agent completed a task, such as hidden tests or invariant checks run after the agent finishes. A good verifier checks behaviour, cannot be edited by the agent, and fails plausible but wrong fixes.
Read more →