Data performance starts with the contract

[ 01 ]

Start with the question, not the database

A slow screen is often described as a database problem before anyone has established which query it needs. A report that requests every related record, resolves them one at a time and filters in application memory has a contract problem as well as a performance problem.

My work with GraphQL services, data models and caching has made the boundary between the client’s question and the server’s data particularly important. Start with the decision the screen supports, its authorization scope, acceptable freshness and expected volume. Those constraints tell us what to fetch and how to measure success.

[ 02 ]

GraphQL and REST solve different problems

GraphQL can let clients request precisely the fields they need and compose data from multiple domains. It also makes query cost less obvious: nested fields can create N+1 database calls or traverse relationships with unbounded cardinality. REST endpoints make operations and caching boundaries explicit, but a poor endpoint design can cause over-fetching or a chain of client requests.

With GraphQL Yoga and Node.js, enforce pagination and cost limits, use request-scoped batching for repeated lookups, and ensure each resolver applies the same authorization rules as an equivalent REST operation. A request-scoped loader must not become a global cache containing one tenant’s objects for another tenant.

Use cursor pagination with a stable ordering and a tie-breaker when datasets change during traversal. Offset pagination can be appropriate for small administrative lists; for large changing datasets it can become expensive and produce duplicates or gaps.

[ 03 ]

Model for access and consistency

MongoDB embedding can make a bounded aggregate cheap to read and update together. References suit independently changing or unbounded relationships, but require additional queries or aggregation. Avoid an ever-growing document for events or transactions; model retention and read patterns before selecting the shape.

For relational data, foreign keys and transactions can protect relationships that must remain consistent. Composite indexes should reflect equality filters, ordering and actual query plans. Include tenant and branch predicates where the query requires them, and measure write amplification and index size alongside read latency.

Aggregation pipelines should filter early when that can use an index, project only needed fields and be inspected with execution statistics. A pipeline that performs a large lookup before narrowing the organization can consume resources and risks leaking data if scope is wrong.

[ 04 ]

A cache needs an invalidation story

Redis is useful for data that is expensive to compute and can tolerate a known freshness window. A key should include every dimension affecting the answer: tenant, resource, filters and, when needed, permission context. Cache only the minimum safe representation and set an explicit lifetime.

For a cache-aside read, check the cache, query the authoritative store on a miss, then populate it. Writes must invalidate or update relevant entries after a successful commit. A time-to-live limits staleness but does not guarantee immediate correctness; use a different strategy for balances, permissions or decisions that require current state.

type CacheScope = { organizationId: string; catalogVersion: string };

function catalogKey(scope: CacheScope, category: string): string {
  return ['catalog', scope.organizationId, scope.catalogVersion, category]
    .map(encodeURIComponent)
    .join(':');
}

[ 05 ]

Bound relationship fan-out

A GraphQL selection that looks small at the top level can multiply work at each nested relationship. A page of organizations containing branches, each with a list of transactions and each transaction with line items, can become a very large request even when each resolver is individually simple. Limit list sizes and nesting according to a defensible cost model, not just query depth.

Batching reduces repeated lookups when keys can be fetched together. It does not remove the need for an efficient query or authorization: a batch must still apply tenant scope, and its result mapping must not expose records that the caller cannot read. Keep loaders request-scoped so permissions and cached values do not leak across users.

The client also has responsibility for choosing useful fields. Separate summary views from detailed inspection, paginate collections independently and avoid polling a large graph just to update one status. Persisted or reviewed operations can make the expected workload easier to observe, although they do not eliminate validation and access checks.

[ 06 ]

Choose a document boundary deliberately

Suppose an order has a small set of line items that are normally read and updated with it. Embedding those lines can make that aggregate convenient to retrieve and can support atomic updates within the document. An unlimited history of status events is different: its growth and retention requirements may justify a separate collection keyed by order and time.

A reference is not free. A report that reads a page of orders and then fetches each customer independently can create a network round trip per row. Batch the lookup, use an appropriate aggregation, or maintain a small snapshot when the business meaning calls for historical values. A customer’s current name and the name printed on an old invoice may be different pieces of information.

Use explain output and representative data to check whether the intended index is selected and how many documents are examined. Index order should follow the actual filters and sort, including tenant scope, rather than a generic recipe. Every additional index consumes storage and adds work to writes, so remove redundant indexes only after checking all dependent queries.

[ 07 ]

Invalidation can race with a read

A simple cache-aside implementation has a subtle race: a reader misses the cache and reads an old database value; a writer commits a new value and invalidates the key; then the reader fills the cache with the old value. A short TTL bounds the duration but does not prevent the stale refill. Whether that risk is acceptable depends on the decision being made from the result.

Versioned keys or a revision checked when populating a cache can make old results inaccessible after a change, provided the revision itself is updated and read consistently. Event-driven invalidation can reduce delay but requires durable delivery and recovery after missed events. For security-sensitive permissions or financial state, bypassing a shared cache may be the simpler correct design.

A popular expired key can also trigger a stampede of identical database reads. Request coalescing, a bounded lock or carefully chosen stale-while-revalidate behavior can reduce load. Each has failure modes: locks need expiry and ownership, and stale responses must never be used where freshness is a correctness requirement.

[ 08 ]

Make data changes observable and reversible

Large data changes should be treated as operational workflows. Migrate in bounded batches, track a checkpoint, make each batch safe to retry and monitor database pressure. A migration that assumes no concurrent writes can lose updates even when it succeeds in a staging environment.

For expensive reports, distinguish time spent waiting on the database from serialization, network transfer and frontend rendering. Trace a representative request across those boundaries and examine tail latency rather than only the average. Test with skewed tenant sizes: one organization with much more data can expose an access pattern hidden by a small development dataset.

Backups and restore tests are part of data architecture. Define recovery objectives with the people who depend on the system, verify that a backup can actually be restored, and protect the restored data with the same access controls. A fast query is not a successful data platform if its authoritative records cannot be recovered.

When a migration changes how a value is calculated, compare the old and new results on a representative sample before switching readers. Investigate discrepancies rather than assuming that the newer calculation is correct. Keep a documented reconciliation procedure for records changed during the transition and a clear point at which the old representation can be retired.

Cache and database monitoring should include error rates and resource saturation as well as throughput. A cache outage should not unexpectedly drive every request to an already overloaded primary database. Rate limits and a defined degraded response can protect the source while the team restores the cache or reduces demand.

[ 09 ]

Observe the path end to end

Measure request latency, resolver or endpoint cost, database execution plans and cache hit behavior together. A high cache-hit rate is not useful if invalidation is wrong; a low average latency can conceal a slow tail or a query that only fails for large tenants.

Load tests should include realistic tenant sizes, relationship fan-out and concurrent writes. Protect downstream services with timeouts and bounded retries. Use tracing or correlation identifiers to connect an API request to a query without placing sensitive payloads in logs.

Before adding another cache or index, remove unnecessary work. A smaller response, bounded relationship and properly scoped query can eliminate complexity that would otherwise need monitoring and invalidation forever.

The fastest answer is not useful if it belongs to another tenant or describes a state that no longer exists.