Databases: query paths and storage engines
Match access patterns to database families, compare scans with indexed lookups, and reason about B-tree and LSM write costs.
20 min7 steps
By the end you can
- Choose a database family from access patterns and constraints
- Compare full-scan and indexed query work including returned rows
- Explain B-tree page updates and LSM flush and compaction
- Calculate write amplification and explain its workload dependence
Start with the access pattern
Relational SQL
- Relations, joins and constraints
- Flexible indexed queries
Key-value / document
- Key lookups / aggregate documents
- Indexes depend on engine
Wide-column / graph
- Partition-key and range access
- Or relationship traversal
NoSQL covers multiple families, not one feature set. Key-value stores suit keyed access; documents group nested data; wide-column stores organize partitioned rows; graph databases emphasize relationships. SQL systems can also store JSON. Choose from query shape, invariants and operational needs.
Avoid a label-based guarantee
Session lookup
- Read by opaque session id
- TTL expiration useful
Relationship traversal
- Find paths across many hops
- Edge traversal is central
A key-value store is a natural fit for simple keyed sessions. A graph model may fit multi-hop relationships. Neither label guarantees transactions, scale or speed. Measure the actual workload and verify durability and consistency requirements separately.
Compare examined rows with returned rows
- Full scan: examined rows1,000,000 rows
- Indexed: matching rows fetched10 rows
A full scan examines the table. A suitable B-tree index navigates to a key range, then reads matches. A rough comparison is O(N) versus O(log N + K), where K is returned entries. Index traversal and heap fetches still cost work; this picture is not a latency benchmark.
A usable index can still lose
Selective predicate
- Few matches
- Index path often useful
Most rows match
- Many scattered row fetches
- Sequential scan may win
Indexes consume storage and add write work. They help only when their ordering and predicates support the query. If most rows match, sequential I/O can beat many random fetches. EXPLAIN shows the chosen plan; EXPLAIN ANALYZE executes it and measures actual work.
Update ordered pages
Read path
- Navigate internal pages
- Read ordered leaf entries
Write path
- Modify affected pages
- Split pages when required
B-tree indexes maintain ordered pages, supporting equality and range scans. Updates modify affected pages and may cause splits; databases typically add logging and buffering. Every additional secondary index adds maintenance. Ordered lookup does not make writes free or define the transaction isolation level.
Buffer, flush and merge
Write
Log for durability; insert into memory table
Flush
Write a sorted immutable file
Compact
Merge sorted files; discard obsolete versions when safe
An LSM engine buffers writes and flushes sorted runs. Compaction merges those runs, rewriting data to control overlap and space. Reads may consult memory and multiple files; Bloom filters and indexes can reduce probes. LSMs trade read, write and space amplification through policy choices.
Measure physical work per logical byte
- Logical writes100 MB
- Engine A: 3x300 MB physical
- Engine B: 8x800 MB physical
Write amplification = physical bytes written / logical bytes written. At 8?, 100 MB becomes 800 MB. Compare the same measurement boundary: device writes, engine writes and logged bytes differ. Neither B-tree nor LSM has one universal factor; workload, compaction, page size and compression matter.
Loading question…
Mastery check
5 questions on fresh variants. One attempt each, no time limit. Score 80% or more to master the lesson.
Answer this step to continue.