Middle Reliability & Fault Tolerance Interview Questions (50 MCQs)

Question 1

A request passes through a load balancer (`99.99%`), an app (`99.9%`), and a database (`99.95%`) in series. What is the overall availability, roughly?

Accepted Answer

About `99.84%`

Answer

About `99.9%`

Answer

About `99.95%`

Answer

About `99.61%`

Question 2

In a two-node active-active cluster, what is the maximum normal load per node if the service must not overload when one node fails?

Accepted Answer

About 50% of capacity

Answer

About 75% of capacity

Answer

About 90% of capacity

Answer

About 25% of capacity

Question 3

Three service layers are chained and each makes up to 3 attempts on failure. If the deepest dependency is down, how many requests can one user request send to it?

Accepted Answer

27

Answer

9

Answer

3

Answer

6

Question 4

To avoid exponential retry amplification in a deep call chain, which approach is recommended?

Accepted Answer

Retry at only one layer, usually closest to the failure or the edge

Answer

Let every layer retry but without any backoff

Answer

Increase the number of retries at every layer to 10

Answer

Remove the timeouts from every layer entirely

Question 5

How does a 'retry budget' (e.g., a 10% cap) prevent retry storms?

Accepted Answer

It caps retries to a fraction of normal requests, so extra load during outages is bounded

Answer

It lets each request retry up to 10 times and then stop

Answer

It allows retries only during the 10% quietest hours of the day

Answer

It divides the financial cost of retries between teams

Question 6

With 'full jitter', how is the wait before retry number `n` computed?

Accepted Answer

`random(0, min(cap, base × 2^n))`

Answer

`min(cap, base × 2^n) + 1s`

Answer

`base × n` with no randomness

Answer

`random(0, base)` for all retries

Question 7

What is a good basis for setting the timeout on a call to a dependency?

Accepted Answer

A bit above the dependency's high-percentile latency (e.g., p99.9)

Answer

Exactly equal to the dependency's average latency

Answer

A large value like 60 seconds for every dependency

Answer

The lowest latency ever observed from that dependency

Question 8

What problem does 'deadline propagation' (like gRPC deadlines) solve?

Accepted Answer

Downstream services stop doing work for requests the caller no longer awaits

Answer

All services use the same fixed timeout value

Answer

Requests are retried indefinitely until they succeed

Answer

Server clocks are synced with a central NTP server

Question 9

When the server receives a repeated idempotency key whose original request completed, what should it do?

Accepted Answer

Return the stored response of the first request without re-executing

Answer

Execute the operation again and return the new response

Answer

Always return a `500` error so the client notices

Answer

Delete the key and silently ignore the request

Question 10

If two requests with the same idempotency key arrive concurrently while the first is still running, what should the server do?

Accepted Answer

Atomically lock the key and reject the second with `409` or hold it until the first ends

Answer

Run both in parallel because the key is the same

Answer

Cancel the first request and execute only the second

Answer

Reject both and block the key permanently

Question 11

When the circuit breaker is Open, what should the client's retry logic do?

Accepted Answer

Not retry, and return the fail-fast error directly or use a fallback

Answer

Retry faster so the breaker closes sooner

Answer

Bypass the breaker and call the dependency directly

Answer

Hold the request in an unbounded queue until it closes

Question 12

Why does the Half-Open state let only a limited number of trial requests through?

Accepted Answer

So a freshly recovered dependency isn't knocked over by the full traffic flood

Answer

So logging costs are reduced during recovery

Answer

Because the breaker lacks memory in this state

Answer

So the trial requests can clear the cache first

Question 13

Why do circuit breakers usually have a 'minimum request volume' in the window besides an error-rate threshold?

Accepted Answer

So a few random errors under low traffic don't trip it needlessly

Answer

So the breaker is only active during peak hours

Answer

So the number of allowed retries per client is capped

Answer

So the breaker's memory usage always stays constant

Question 14

A service with one shared 200-thread pool calls three dependencies. One became very slow and the whole service stalled. Which change directly fixes this?

Accepted Answer

Giving each dependency its own pool or semaphore with a fixed cap

Answer

Growing the shared pool from 200 to 2,000 threads

Answer

Removing timeouts so slow requests can complete

Answer

Adding more retries for the slow dependency

Question 15

The product pricing service is temporarily down. Which fallback is usually most reasonable for the product page?

Accepted Answer

Show the last cached (stale) price, disabling checkout temporarily if needed

Answer

Show a price of zero until the service comes back

Answer

Retry forever until a price arrives, even if the page never loads

Answer

Return a `500` error for the whole product page

Question 16

During overload of an online store, which requests does smart load shedding usually drop first?

Accepted Answer

Low-priority ones like prefetches, analytics, and batch jobs

Answer

Users' checkout and payment requests

Answer

Login requests from existing users

Answer

Requests that arrived earlier than the others

Question 17

A service queues requests and clients time out after 2 seconds. What is an important optimization under overload?

Accepted Answer

Discard requests that have waited over 2 seconds before processing them

Answer

Fully process every queued request in FIFO order

Answer

Make the queue unbounded so no request is rejected

Answer

Raise the client timeout to 2 minutes

Question 18

Why is an unbounded queue between two components dangerous?

Accepted Answer

It hides overload until latency and memory grow without limit

Answer

Because messages leave an unbounded queue in random order

Answer

Because an unbounded queue only works with one consumer

Answer

Because an unbounded queue cannot persist messages

Question 19

A token bucket refills at 10 tokens/second with capacity 50. What can a client that has been idle for a while do?

Accepted Answer

Burst up to 50 requests instantly, then continue at 10 per second

Answer

Only 10 requests per second, with no bursting at all

Answer

Up to 60 requests per second, sustained indefinitely

Answer

50 requests per second, sustained indefinitely

Question 20

What is the main difference between a leaky bucket (as a queue) and a token bucket?

Accepted Answer

A leaky bucket smooths output to a constant rate; a token bucket allows bursts

Answer

A leaky bucket allows bursts; a token bucket makes output perfectly constant

Answer

A leaky bucket can only be implemented on the client side

Answer

Both behave identically and differ only in name

Question 21

What is the main drawback of the sliding window log algorithm?

Accepted Answer

Storing a timestamp per request, which uses lots of memory at high volume

Answer

Allowing double the limit at window boundaries

Answer

Being impossible to implement with Redis

Answer

Lower accuracy than a fixed window counter

Question 22

The limit is 100/minute. The previous minute had 80 requests, the current one has 30 so far, and 25% of it has elapsed. What does the sliding window counter estimate?

Accepted Answer

`30 + 80 × 0.75 = 90`, so the request is allowed

Answer

`30 + 80 = 110`, so the request is rejected

Answer

`30 + 80 × 0.25 = 50`, so the request is allowed

Answer

Just `30`, since the previous minute is ignored

Question 23

In a distributed Redis rate limiter, why is an atomic `INCR` or a Lua script typically used?

Accepted Answer

So the read-then-write pattern across instances avoids race conditions

Answer

So the data is automatically encrypted on disk

Answer

Because Redis doesn't allow writes without Lua

Answer

So counters are synchronized across regions

Question 24

What is the main trade-off of local (in-memory per instance) vs global (central store) rate limiting?

Accepted Answer

Local is fast and dependency-free but approximate; global is precise but adds latency and a dependency

Answer

Local is more precise; global is faster and has no dependency

Answer

Local works only for HTTP; global works only for gRPC

Answer

There's no difference if the instance count is fixed

Question 25

Why is rate limiting only by IP a weak choice for an authenticated public API?

Accepted Answer

Many users share one IP behind NAT, and attackers can rotate IPs

Answer

Because the IP address can't be read under HTTPS

Answer

Because IPv6 doesn't support rate limiting

Answer

Because hashing an IP address is very slow

Question 26

Why is a 'deep' health check that also checks the shared database risky for load balancer decisions?

Accepted Answer

When the DB slows, every instance turns unhealthy at once and leaves rotation

Answer

Because a deep health check modifies database data

Answer

Because the load balancer can't parse JSON responses

Answer

Because deep health checks only work on port 443

Question 27

In automatic database failover, what is 'split-brain' and what prevents it?

Accepted Answer

Two nodes both think they're primary; quorum and fencing prevent it

Answer

The DB splits into two unequal shards; rebalancing prevents it

Answer

Two queries read the same row at once; an index prevents it

Answer

A replica lags the primary; adding RAM prevents it

Question 28

Why can DNS-based failover be slower than expected?

Accepted Answer

Resolvers and clients cache the old record until the TTL expires (or longer)

Answer

Because DNS only updates records once per day

Answer

Because changing a DNS record requires restarting all servers

Answer

Because DNS doesn't support IPv4 records

Question 29

Which order correctly ranks DR strategies from cheapest (highest RTO) to most expensive (lowest RTO)?

Accepted Answer

Backup/restore, pilot light, warm standby, multi-site active-active

Answer

Pilot light, backup/restore, multi-site, warm standby

Answer

Warm standby, pilot light, backup/restore, multi-site

Answer

Multi-site, warm standby, pilot light, backup/restore

Question 30

In the 'pilot light' DR strategy, what is running in the backup region?

Accepted Answer

Only core pieces like a DB replica; app servers are started during a disaster

Answer

A full copy at full capacity serving half the traffic

Answer

Nothing; only backups are kept in storage

Answer

A full but scaled-down copy that serves no real traffic

Question 31

What is the advantage of warm standby over pilot light?

Accepted Answer

The full stack runs at low capacity, so only scale-up is needed and RTO is lower

Answer

It costs less because no servers run in the second region

Answer

Its RPO is always zero because it uses daily backups

Answer

It needs no data replication between regions

Question 32

A database replicates asynchronously to a second region with a typical lag of 5 seconds. If the primary region vanishes suddenly, what is the RPO?

Accepted Answer

At least the replication lag at that moment (about 5 seconds or more)

Answer

Zero, because replication was running

Answer

Equal to the time needed to promote the replica

Answer

24 hours, since replication isn't a backup

Question 33

What is the main cost of synchronous replication between two distant regions to achieve zero RPO?

Accepted Answer

Every write waits for a cross-region round-trip, raising write latency

Answer

The amount of data in each region is halved

Answer

Reads can no longer be served from the second region

Answer

The need for backups disappears entirely

Question 34

A developer accidentally dropped a table at 14:05. Which capability lets you restore the database to 14:04?

Accepted Answer

Point-in-time recovery using a base snapshot plus WAL/binlog replay

Answer

Promoting a synchronous read replica

Answer

Increasing the number of database replicas

Answer

Restarting the primary and clearing the cache

Question 35

To automatically decide whether to promote or roll back a canary, its metrics are compared against what?

Accepted Answer

A concurrent baseline of the current version with similar traffic and size

Answer

The same service's metrics from last week

Answer

Staging environment metrics from test time

Answer

A fixed value set at design time

Question 36

To rename a database column without breaking rollback in a blue-green deployment, which approach is safe?

Accepted Answer

Expand/contract: add the new column, dual-write, migrate, drop the old one in a later release

Answer

Rename the column directly at the same time as the deploy

Answer

Drop the old column before deploying to force use of the new one

Answer

Take the service fully offline for every schema change

Question 37

In a gradual feature flag rollout (e.g., 5% of users), why are users usually selected by hashing the user ID rather than randomly per request?

Accepted Answer

So each user has a consistent experience and doesn't flip between versions

Answer

Because hashing the user ID strengthens password security

Answer

Because random selection is impossible on distributed servers

Answer

So the user percentage grows automatically over time

Question 38

Under a typical error budget policy, what happens when a service exhausts its error budget for the period?

Accepted Answer

Non-essential feature releases freeze and focus shifts to reliability work

Answer

The SLO is automatically lowered for that period

Answer

The team is allowed to double its deploy pace

Answer

All alerts are silenced until the period ends

Question 39

A service has a `99.9%` success SLO and receives 10 million requests per month. How many failed requests is its monthly error budget?

Accepted Answer

10,000

Answer

1,000

Answer

100,000

Answer

100

Question 40

What does a 'burn rate' of 1 mean for a 30-day SLO?

Accepted Answer

The error budget is consumed at a pace that exhausts it exactly at 30 days

Answer

The whole error budget is used up within one hour

Answer

Exactly one error occurs every day

Answer

The service had no errors in this period

Question 41

What is the difference between the RED and USE monitoring methods?

Accepted Answer

RED (rate, errors, duration) for services; USE (utilization, saturation, errors) for resources

Answer

RED for hardware resources; USE for request-driven services

Answer

RED is only for logs; USE is only for traces

Answer

RED is for the frontend; USE is only for databases

Question 42

For distributed tracing to follow a request across several microservices, what is required?

Accepted Answer

Propagating trace context (like the `traceparent` header) on every call

Answer

All services using one shared database

Answer

Running all services on one physical server

Answer

Writing all services' logs to one shared file

Question 43

Why is it recommended to page on-call based on symptoms rather than causes?

Accepted Answer

Because alerting on user impact (like error rate) is precise while cause alerts are noisy

Answer

Because cause metrics like CPU can't be collected

Answer

Because cause alerts always arrive later than symptom alerts

Answer

Because on-call may only view user-facing metrics

Question 44

What is usually the first step in designing a chaos engineering experiment?

Accepted Answer

Defining a measurable 'steady state' and a hypothesis it holds during failure

Answer

Randomly killing production servers without telling the team

Answer

Turning off all alerts to avoid noise

Answer

Running several failure types across the whole system at once

Question 45

Why are quorum-based systems (like etcd or Raft) usually deployed across 3 AZs rather than 2?

Accepted Answer

With 3 AZs losing one still keeps a majority; with 2 AZs the majority may be lost

Answer

Because cloud providers mandate at least 3 AZs

Answer

Because data transfer across 3 AZs costs less than 2

Answer

Because Raft only compiles with an odd number of nodes

Question 46

During a deploy, what should an instance do on receiving `SIGTERM` to avoid user errors?

Accepted Answer

Fail readiness, stop accepting new requests, and finish in-flight ones

Answer

Immediately exit the process with `exit(0)`

Answer

Abort all in-flight requests with a `500` error

Answer

Ignore the signal until the orchestrator kills it

Question 47

Why do health checks usually have 'healthy' and 'unhealthy' thresholds (e.g., 3 consecutive checks)?

Accepted Answer

So a single transient failure or success doesn't cause flapping in and out

Answer

So health check costs in the cloud drop by a factor of three

Answer

Because HTTP needs at least three requests to connect

Answer

So new instances always get traffic before warming up

Question 48

Why keep some spare capacity (headroom) normally even with autoscaling in place?

Accepted Answer

Because new instances take minutes to launch and can't absorb a sudden spike

Answer

Because autoscaling only works during business hours

Answer

Because autoscaling never reduces the instance count

Answer

Because headroom removes the need for a load balancer

Question 49

Why is sending headers like `RateLimit-Remaining` and `RateLimit-Reset` to clients useful?

Accepted Answer

Clients can pace themselves before ever hitting a `429`

Answer

Because browsers won't send requests without them

Answer

So the server can enforce limits entirely client-side

Answer

Because these headers automatically cache the response

Question 50

What is the benefit of structured logging (e.g., JSON) with a request ID on every log line?

Accepted Answer

Precisely filtering and finding all logs for one request across services

Answer

Shrinking log volume to under a tenth of its size

Answer

Removing the need for metrics and traces

Answer

Automatically encrypting sensitive data in logs