Senior Scalability & Performance Interview Questions (50 MCQs)

Question 1

In a load test, going from 8 to 16 nodes reduced total throughput. Per the Universal Scalability Law, what is the most likely explanation?

Accepted Answer

Coherency cost (pairwise coordination between nodes) grows quadratically and outweighs the parallelism gain

Answer

Amdahl's law predicts throughput always drops after 8 nodes

Answer

The load balancer can only spread traffic across at most 8 nodes

Answer

The new nodes stay permanently slower due to cold caches

Question 2

How are hedged requests correctly implemented to reduce tail latency?

Accepted Answer

If no response arrives by about p95, send a second request to another replica and use the first reply

Answer

Every request is sent to two replicas upfront and the slower reply is used

Answer

Slow requests are cancelled and retried to the same replica with exponential backoff

Answer

All timeouts are lowered to p50 so slow requests cannot exist

Question 3

A page makes 200 parallel backend calls to render. Why should the team set backend SLOs on p99.9 rather than p99?

Accepted Answer

With p99, ~87% of pages hit at least one slow call; with p99.9 it is ~18%

Answer

Because p99.9 is always lower than p99 and causes fewer alerts

Answer

Because computing p99 in distributed systems is impossible

Answer

Because p99.9 only counts successful requests

Question 4

Your load tool is closed-loop: each worker sends the next request only after the previous reply. During a 5-second server stall, what error appears in the percentiles?

Accepted Answer

Coordinated omission: requests that should have been sent during the stall are never recorded, so the tail looks far better than reality

Answer

All requests in that window are double-counted, so p99 looks too large

Answer

The tool overstates throughput but percentiles stay accurate

Answer

No error occurs because closed-loop is the most accurate method

Question 5

An architecture has three layers (gateway → service → DB client), and each layer makes up to 3 total attempts on error. When the DB fails completely, how many DB attempts can one user request cause?

Accepted Answer

27 attempts

Answer

9 attempts

Answer

3 attempts

Answer

81 attempts

Question 6

What problem does a retry budget (e.g. retries may be at most 10% of requests) solve that exponential backoff alone does not?

Accepted Answer

It caps total retry load during widespread failure, since backoff only spreads each client's retry timing

Answer

It guarantees every failed request eventually succeeds

Answer

It halves the latency of successful requests

Answer

It removes the need for server-side idempotency

Question 7

A brief traffic spike caused overload; the spike ended but the system stayed overloaded for hours. What is this pattern and its usual root cause?

Accepted Answer

Metastable failure; a feedback loop like retries, timeouts, or cache misses keeps load above capacity

Answer

Split brain; two leaders process requests at the same time

Answer

A memory leak; memory slowly fills, unrelated to the spike

Answer

Clock skew; node clocks have drifted apart

Question 8

Consider a single-worker service with 10 ms mean service time, assuming M/M/1. At 90% utilization, what is the approximate mean total response time?

Accepted Answer

About 100 ms

Answer

About 11 ms

Answer

About 19 ms

Answer

About 900 ms

Question 9

Two services have the same utilization and mean service time, but one has highly variable service times (some requests 100x slower). Which statement is correct?

Accepted Answer

The variable service has much more queueing and latency, since waiting grows with service-time variance

Answer

Both have the same latency because utilization and mean are equal

Answer

The variable service has lower latency because most requests are fast

Answer

Variance only affects throughput, not latency

Question 10

After an incident, a queue has an 18 million message backlog. Production is 4,000 msg/s and consumption capacity is 10,000 msg/s. How long until the backlog drains?

Accepted Answer

About 50 minutes

Answer

About 30 minutes

Answer

About 75 minutes

Answer

About 5 minutes

Question 11

Amdahl's law predicts a 20x cap for a job with 5% serial work, yet big-data systems benefit from thousands of nodes. How does Gustafson's law explain this?

Accepted Answer

As nodes are added the problem size grows too, so the serial part's share of total work shrinks

Answer

Gustafson proves the serial part becomes parallel automatically with more nodes

Answer

Gustafson only applies to GPUs, not distributed systems

Answer

Amdahl is wrong and speedup is always linear in node count

Question 12

At peak, API latency and app CPU rose (from waiting and serialization). The autoscaler went from 20 to 60 pods and the system collapsed entirely. What is the most likely reason?

Accepted Answer

The real bottleneck was the database, and more pods sent it more connections and concurrent queries

Answer

Kubernetes automatically throttles the network above 50 pods

Answer

The new pods ran an older version of the code

Answer

The autoscaler should have measured memory because CPU is never valid

Question 13

80 pods each have `maxPoolSize=25`, and Postgres has `max_connections=500`. What happens during a rolling deploy with `maxSurge=25%`?

Accepted Answer

Potential demand reaches 2,500 connections and pods get connection errors at peak; a pooler or smaller pools are needed

Answer

Postgres automatically raises `max_connections` based on demand

Answer

No problem occurs because pools never open connections until needed

Answer

Only old pods get errors and users see no impact

Question 14

A JVM service has ~5 ms p50 but ~400 ms p99.9, and spikes correlate with major GC times. Which remedy is most directly effective?

Accepted Answer

Reduce allocation rate and use a low-pause collector like ZGC or G1 with a pause target

Answer

Double the pod count without changing JVM settings

Answer

Raise client timeouts to 1 second

Answer

Enable compression on every response

Question 15

Why do some systems (e.g. at Facebook) switch their request queue from FIFO to LIFO under overload?

Accepted Answer

Old requests' clients have likely timed out; serving the newest preserves goodput

Answer

LIFO always has lower mean latency than FIFO, even at low load

Answer

LIFO guarantees per-user processing order

Answer

LIFO completely removes the need for server-side timeouts

Question 16

Libraries like Netflix concurrency-limits use adaptive concurrency limits instead of fixed rate limits. What is the core idea?

Accepted Answer

They dynamically adjust the in-flight request cap from latency signals (like TCP Vegas)

Answer

They take a fixed RPS number from a load test and never change it

Answer

They reject all requests until CPU drops below 50%

Answer

They increase pod count based on latency

Question 17

A timeline uses fan-out on write (denormalizing each post into every follower's inbox). A user with 50 million followers posts. What is the best approach?

Accepted Answer

Hybrid: merge posts from mega-accounts at read time, keep fan-out on write for everyone else

Answer

Keep fan-out on write and add workers to write 50 million rows

Answer

Convert the whole system to fan-out on read

Answer

Limit every user to 1 million followers

Question 18

A GPU inference service batches requests. The SLO is p99 < 150 ms and running a batch takes ~60 ms. What is a sensible maximum batching wait window?

Accepted Answer

About 30–40 ms, so with execution, network, and queueing margin it stays under 150 ms

Answer

About 150 ms, since bigger batches are always better

Answer

Zero; batching always violates a latency SLO

Answer

About 1 second for maximum GPU efficiency

Question 19

Billions of small JSON messages (~300 bytes) are replicated across regions and egress cost is high. gzip per message barely helps. What is the best option?

Accepted Answer

zstd with a dictionary trained on sample messages, or compressing batches of messages

Answer

The highest gzip level on each message individually

Answer

Base64-encoding messages before sending

Answer

Sending every message twice to ensure delivery

Question 20

The SLO is 99.9% over 30 days and the current error rate is 1.44%. What is the burn rate, and how long would the month's budget last if it continues?

Accepted Answer

Burn rate 14.4; the budget lasts about 2 days

Answer

Burn rate 1.44; the budget lasts about 20 days

Answer

Burn rate 144; the budget lasts about 5 hours

Answer

Burn rate 0.1; the budget never runs out

Question 21

Why do burn-rate SLO alerts usually use two windows at once (e.g. 1 hour and 5 minutes)?

Accepted Answer

The long window confirms significant budget burn and the short one confirms the problem is still ongoing

Answer

Because monitoring tools cannot compute windows longer than 5 minutes

Answer

So the alert is sent twice and more likely to be noticed

Answer

Because SLOs must be defined separately for latency and availability

Question 22

Your availability SLI comes from app server logs and reads 99.98%, but users complain about outages. What is the most likely measurement flaw?

Accepted Answer

Requests that never reached the server (DNS, load balancer, crashes) never show up in server logs

Answer

Server logs always record errors twice

Answer

The SLI should only be computed from server CPU

Answer

99.98% is statistically incompatible with user complaints

Question 23

A service is spread evenly across 3 AZs and must survive losing a whole AZ without degradation. What is the maximum target utilization in normal operation?

Accepted Answer

About 66% of each AZ's acceptable capacity

Answer

About 33% of each AZ's acceptable capacity

Answer

About 90% of each AZ's acceptable capacity

Answer

About 50% of each AZ's acceptable capacity

Question 24

2 billion log events of ~500 bytes are produced daily; compression gives 5:1, replication is 3x, retention is 90 days. Roughly what is the steady-state storage?

Accepted Answer

About 54 TB

Answer

About 90 TB

Answer

About 270 TB

Answer

About 18 TB

Question 25

One key in a Redis Cluster (a product's stock in a flash sale) gets 40% of all traffic and its shard is saturated. Which combination is most effective?

Accepted Answer

A short-lived local cache for reads and splitting the write counter into several sub-keys (sharded counter)

Answer

Adding more shards to the cluster

Answer

Increasing the TTL of that same key in Redis

Answer

Moving the key to a bigger Redis instance with the same access pattern

Question 26

A video view counter does `UPDATE views = views + 1` per view, and viral videos' rows suffer heavy lock contention. What is the best redesign?

Accepted Answer

Aggregate increments in memory or a stream and periodically flush one `UPDATE` with the sum

Answer

Use a `SERIALIZABLE` transaction for each increment

Answer

Read the value, increment in the app, and write it back without locks

Answer

Add an index on the `views` column

Question 27

A service's latency rises 5x under load, yet CPU is only 30% and a normal (on-CPU) flame graph shows nothing unusual. What is the next analysis step?

Accepted Answer

Off-CPU profiling to see time blocked on locks, I/O, or pool waits

Answer

Upgrade the CPU because a faster CPU always reduces latency

Answer

Increase the sampling rate of the same on-CPU profiler

Answer

Add a cache in front of every endpoint without further analysis

Question 28

An endpoint's SLO is p99 < 300 ms and it must call auth (p99 40 ms), profile (p99 90 ms), recommendations (p99 180 ms), and pricing (p99 70 ms). Which design meets the SLO?

Accepted Answer

Auth first, then the other three in parallel, with a timeout and fallback for recommendations

Answer

All four calls sequentially with the same 300 ms timeout each

Answer

All four sequentially, since summed p99s exactly give the overall p99

Answer

Skip auth to save 40 ms

Question 29

In a 5-service chain, each service has an independent 2 s timeout. Under overload, downstream services keep working on requests the user abandoned long ago. What is the principled fix?

Accepted Answer

Deadline propagation: pass the request's absolute deadline at each hop and abandon work once it passes

Answer

Raise all services' timeouts to 10 seconds

Answer

Remove all timeouts and rely on TCP keepalive

Answer

Add retries at each hop to compensate for timeouts

Question 30

As incoming load exceeded capacity, the server's raw throughput stayed flat but successful on-time responses dropped sharply. What is this?

Accepted Answer

Congestion collapse; goodput falls because the server works on requests whose clients have timed out

Answer

A natural latency increase with no effect on goodput

Answer

A sign capacity is sufficient, since throughput stayed flat

Answer

A monitoring bug; throughput and goodput are always equal

Question 31

A popular page's cache key expires and 5,000 concurrent requests all fire the same heavy query at the database. Which app-level technique directly solves this?

Accepted Answer

Request coalescing (singleflight): one request runs the query and the rest wait for its result

Answer

Increase the connection pool size to 5,000

Answer

Lower the key's TTL so it refills sooner

Answer

Immediately retry the requests that timed out

Question 32

A database on SSD with ~1 ms fsync, doing one fsync per commit, seemingly caps at ~1,000 TPS. How do databases reach tens of thousands of commits per second without weakening durability?

Accepted Answer

Group commit: many concurrent transactions are made durable with one shared WAL fsync

Answer

Disabling fsync and relying on the OS cache

Answer

Writing each transaction to a separate disk

Answer

Acknowledging commit before writing to the WAL

Question 33

A Cassandra cluster under steady writes shows worsening read p99 and saturated disk I/O over time, while QPS is constant. What is the most likely cause?

Accepted Answer

Compaction has fallen behind and SSTables piled up, raising read and write amplification

Answer

The memtable is too large and directly blocks reads

Answer

The replication factor increased automatically

Answer

Clients repeat requests due to keep-alive

Question 34

Every time the autoscaler scales in, a small spike of 502 errors appears. What are the root cause and correct fix?

Accepted Answer

Instances are killed before leaving the load balancer and finishing in-flight requests; use connection draining and graceful shutdown

Answer

The autoscaler should never scale in

Answer

The load balancer should switch to a random algorithm

Answer

Remaining instances need more memory

Question 35

A serverless API has ~2 s p99 under bursty traffic, while warm execution is 50 ms. Which remedy targets the root cause?

Accepted Answer

Provisioned concurrency or minimum warm instances, plus reducing init time (smaller bundle, lazy connections)

Answer

Increase database memory for faster responses

Answer

Raise the function timeout to 30 seconds

Answer

Reduce requests with a strict rate limit

Question 36

A team says: 'We put a queue in front of the service, so we can handle any traffic.' The sustained arrival rate is 120% of consumption capacity. What happens?

Accepted Answer

Backlog and latency grow without bound; a queue only absorbs temporary bursts, not a sustained capacity deficit

Answer

The queue automatically compresses 20% of messages to compensate

Answer

Latency stays constant because the queue preserves order

Answer

No problem occurs because messages are stored on disk

Question 37

A benchmark of a new service showed 20,000 RPS at 8 ms p99, but production saturated at 5,000 RPS. Which benchmark flaw is most likely?

Accepted Answer

Unrealistic data and access patterns; e.g. a few repeated keys all served from cache

Answer

The benchmark ran on faster hardware, which always gives a 4x gap

Answer

Production uses HTTP/2, which is inherently 4x slower

Answer

Benchmark p99 is always larger than production's

Question 38

An analytics job takes 20 minutes on a 50-node Spark cluster. The data is 200 GB. Why might a senior engineer suggest running it on one big machine?

Accepted Answer

The data fits in one machine's RAM, and removing network, shuffle, and coordination overhead may make it faster and cheaper

Answer

Because Spark always fails on more than 10 nodes

Answer

Because vertical scaling is always faster than horizontal

Answer

Because a big machine removes any need for redundancy

Question 39

After moving from a 16-core to a 2-socket 96-core machine, a service got only 2x faster, not 6x. Which hardware factor is likely?

Accepted Answer

NUMA effects: threads accessing the other socket's memory are slower and cache coherency adds traffic

Answer

More cores always halve each core's speed

Answer

Operating systems do not support more than 32 cores

Answer

RAM is stored compressed on bigger machines

Question 40

In a multi-tenant SaaS, whenever one big customer runs a heavy export, everyone else's p99 degrades. What is the most suitable architectural remedy?

Accepted Answer

Resource isolation: separate queues and worker pools for heavy jobs plus per-tenant quotas/fair queuing

Answer

Increase total instances without any separation

Answer

Lower timeouts on all requests for every customer

Answer

Disable the export feature for everyone

Question 41

A service has 20,000 writes/s at 2 KB average, and every write is replicated to 2 other regions. Roughly what is the monthly cross-region traffic?

Accepted Answer

About 200 TB

Answer

About 100 TB

Answer

About 20 TB

Answer

About 2 PB

Question 42

Before replacing a critical service with a rewrite, what is the most reliable way to validate its capacity and correctness under real load?

Accepted Answer

Shadow traffic: mirror production traffic to the new service without using its responses, and compare results

Answer

Running unit tests with 100% coverage

Answer

Switching all traffic at once during off-peak hours

Answer

A synthetic load test with one endpoint and one repeated user

Question 43

A service gets 5,000 RPS at 100 ms latency. A dependency starts failing 20% and clients retry each failure once, while latency rises to 300 ms. Roughly what concurrency is now needed?

Accepted Answer

About 1,800, up from 500

Answer

About 600, up from 500

Answer

About 1,500, up from 500

Answer

Still 500, since user RPS has not changed

Question 44

An inventory service is 95% reads; product-page display tolerates seconds of staleness but checkout must see exact values. Which design is best?

Accepted Answer

Cache or replicas for product pages, and direct transactional reads/reservations on the primary at checkout

Answer

All reads from a cache with a 1-hour TTL

Answer

All reads from the primary so there is no staleness

Answer

All reads from replicas, with checkout retrying until the value is correct

Question 45

A dashboard shows p99 in 1-minute windows, and the team averages those p99s for a monthly report. Why is this misleading?

Accepted Answer

Low- and high-traffic minutes get equal weight, and the true monthly p99 is not derived from the full request distribution

Answer

A 1-minute window always shows p99 as zero

Answer

The average of p99s always equals the monthly p50

Answer

This is correct because averaging removes sampling error

Question 46

During events traffic grows up to 10% per minute, and it takes 6 minutes from metric detection to a new instance being ready. Roughly how much headroom is needed for reactive autoscaling to suffice?

Accepted Answer

About 70–80% extra capacity over current load

Answer

About 10% extra capacity over current load

Answer

About 6% extra capacity over current load

Answer

No headroom, because the autoscaler is automatic

Question 47

A cache has a 99% hit ratio and the database is sized for only 1% of traffic (plus little margin). If the cache cluster fully restarts, what happens and what is the design lesson?

Accepted Answer

DB load jumps up to ~100x; the cache is load-bearing and needs gradual warm-up or load limiting

Answer

DB load rises only 1% because the hit ratio stays the same

Answer

Users just see slightly staler data with no DB impact

Answer

The DB automatically takes over the cache role with no performance loss

Question 48

A gRPC service sits behind an L4 load balancer. The autoscaler adds pods, but old pods stay hot while new pods are nearly idle. What is the cause?

Accepted Answer

Long-lived HTTP/2 connections stay pinned to old pods; request-level (L7 or client-side) balancing is needed

Answer

New pods fail health checks and never get traffic

Answer

gRPC can only run on one pod per node

Answer

The autoscaler should use memory instead of CPU

Question 49

A system precomputes recommendation feeds nightly for all 200 million registered users, but only 15 million are daily active. Which change is most efficient?

Accepted Answer

Precompute only for recently active users and compute on demand with caching for returning users

Answer

Grow the cluster so the precomputation finishes faster

Answer

Drop precomputation entirely and compute every feed per request

Answer

Precompute hourly instead of nightly for all users

Question 50

A service shares one 200-thread pool for calls to 5 dependencies. One dependency slows (20 ms → 2 s) and the whole service fails. What is the quantitative explanation and fix?

Accepted Answer

By Little's law even 100 RPS to that dependency occupies 200 threads; a separate pool or concurrency cap per dependency is needed

Answer

Java thread pools do not support more than 100 threads

Answer

The slow dependency consumes the service's CPU, so upgrade the CPU

Answer

Increase the pool to 10,000 threads so it never fills