Vertical vs Horizontal Scaling

Key Scaling Strategies

graph TD
    SCALE["Scaling Strategies"]
    SCALE --> LB["Load Balancing<br/>(distribute traffic)"]
    SCALE --> CACHE["Caching<br/>(reduce DB load)"]
    SCALE --> SHARD["Database Sharding<br/>(split data across DBs)"]
    SCALE --> ASYNC["Async Processing<br/>(message queues)"]
    SCALE --> CDN["CDN<br/>(serve static content at edge)"]
    SCALE --> MS["Microservices<br/>(scale services independently)"]

    style SCALE fill:#56cc9d,stroke:#333,color:#fff
    style LB fill:#ffce67,stroke:#333
    style CACHE fill:#6cc3d5,stroke:#333,color:#fff

Stateless vs Stateful Services

Stateless (preferred for horizontal scaling):
  - Server holds NO user session data
  - Any server can handle any request
  - Session stored in external store (Redis, DB)
  - Easy to add/remove servers

Stateful (harder to scale):
  - Server holds user session in memory
  - Requests must be routed to same server (sticky sessions)
  - Server failure loses user state
  - Scaling requires state migration

Reliability Patterns

Availability Levels

Failure Handling Flow

Request comes in:
  1. Load balancer routes to healthy server
     - If server unreachable → try next server
  2. Server processes request
     - If downstream service fails → circuit breaker
       - Circuit CLOSED: forward request normally
       - Circuit OPEN: return cached/fallback response immediately
       - Circuit HALF-OPEN: try one request, if success → close
  3. Database write
     - Write to primary → replicate to replicas
     - If primary fails → promote replica (automatic failover)
  4. Return response
     - If timeout → client retries with exponential backoff
     - If persistent failure → graceful degradation (partial response)

Performance Optimization by Layer

Caching Strategy Deep Dive

graph TD
    REQ["Request"]
    REQ --> APP["Application"]
    APP --> CHECK{"Cache<br/>hit?"}
    CHECK -->|"Hit"| RETURN["Return cached data<br/>(< 1ms)"]
    CHECK -->|"Miss"| DB["Query Database<br/>(10-100ms)"]
    DB --> UPDATE["Update Cache"]
    UPDATE --> RETURN2["Return data"]

    style CHECK fill:#ffce67,stroke:#333
    style RETURN fill:#56cc9d,stroke:#333,color:#fff

Cache Invalidation Strategies

CAP Theorem

Every distributed data store can provide at most two of three guarantees simultaneously:

Since network partitions WILL happen, you must choose:

  CP (Consistency + Partition Tolerance):
    → During partition: reject requests rather than return stale data
    → Examples: MongoDB, HBase, Redis Cluster, ZooKeeper
    → Use for: Banking, inventory, leader election

  AP (Availability + Partition Tolerance):
    → During partition: serve requests even if data might be stale
    → Examples: Cassandra, DynamoDB, CouchDB
    → Use for: Social media feeds, shopping carts, analytics

Consensus Algorithms

Consistency Models

Infrastructure Components Reference

Monolith vs Microservices

When to Move from Monolith to Microservices

Start with a monolith. Split when:
  1. Team size > 10-15 engineers (coordination overhead)
  2. Different components have vastly different scaling needs
  3. Deployment of one feature blocks another team
  4. Different services need different tech stacks
  5. You need independent failure isolation

Do NOT split prematurely — microservices add operational complexity:
  - Service discovery
  - Distributed transactions (Saga pattern)
  - Network latency between services
  - Distributed debugging and tracing
  - Data consistency across service boundaries

API Style Comparison

REST API Design Principles

Good REST API design:

  Resources (nouns, not verbs):
    ✅ GET    /api/v1/users              → List users
    ✅ GET    /api/v1/users/123          → Get user 123
    ✅ POST   /api/v1/users              → Create user
    ✅ PUT    /api/v1/users/123          → Update user 123
    ✅ DELETE /api/v1/users/123          → Delete user 123
    ❌ GET    /api/v1/getUser?id=123     → Verb in URL (bad)

  Nested resources:
    GET  /api/v1/users/123/orders       → Orders for user 123
    GET  /api/v1/users/123/orders/456   → Specific order

  Pagination:
    GET /api/v1/users?page=2&limit=20
    GET /api/v1/users?cursor=abc123&limit=20  (cursor-based, preferred)

  Filtering and sorting:
    GET /api/v1/users?role=admin&sort=-created_at

  Versioning:
    /api/v1/users  → URL path versioning (most common)
    Accept: application/vnd.api.v1+json  → Header versioning

HTTP Status Codes

Idempotency

Pagination: Offset vs Cursor

Database Selection Guide

SQL vs NoSQL Trade-offs

Database Scaling Strategies

graph TD
    DBSCALE["Database Scaling"]
    DBSCALE --> RR["Read Replicas<br/>(scale reads)"]
    DBSCALE --> SHARD["Sharding<br/>(scale writes + storage)"]
    DBSCALE --> PART["Partitioning<br/>(split tables)"]
    DBSCALE --> POOL["Connection Pooling<br/>(scale connections)"]

    RR --> RR_D["Primary handles writes<br/>Replicas handle reads<br/>Async replication"]
    SHARD --> SHARD_D["Split data by key<br/>(user_id % N shards)<br/>Each shard is a full DB"]

    style DBSCALE fill:#56cc9d,stroke:#333,color:#fff

Sharding Strategies

How a Web Request Works (End-to-End)

graph LR
    BROWSER["Browser"]
    BROWSER -->|"1. DNS Lookup"| DNS["DNS Server<br/>→ IP address"]
    DNS -->|"2. TCP Handshake"| LB["Load Balancer"]
    LB -->|"3. TLS Handshake<br/>(HTTPS)"| APP["App Server"]
    APP -->|"4. Process Request"| DB["Database"]
    DB -->|"5. Response"| APP
    APP -->|"6. HTTP Response"| BROWSER

    style BROWSER fill:#6cc3d5,stroke:#333,color:#fff
    style LB fill:#56cc9d,stroke:#333,color:#fff
    style APP fill:#ffce67,stroke:#333

Step-by-step breakdown:
  1. DNS resolution: browser.com → 93.184.216.34  (~50ms first time, cached after)
  2. TCP handshake: SYN → SYN-ACK → ACK           (~1 RTT = 0.5ms same DC, 150ms cross-region)
  3. TLS handshake: Certificate exchange, key setup (~1-2 RTT additional for HTTPS)
  4. HTTP request: GET /api/users                   (headers + body)
  5. Server processes, queries DB, builds response
  6. HTTP response: 200 OK + JSON payload
  7. Browser renders response

Communication Protocols

Real-Time Communication Patterns

DNS and Load Balancing at Network Level

Authentication vs Authorization

Token-Based Authentication Flow (OAuth 2.0 + JWT)

1. User logs in → Auth Server validates credentials
2. Auth Server issues:
   - Access token (JWT, short-lived: 15-60 min)
   - Refresh token (opaque, long-lived: 7-30 days)
3. Client sends Access token in header: Authorization: Bearer <token>
4. API Gateway / Service validates JWT:
   - Verify signature (no DB call needed)
   - Check expiration
   - Extract user ID, roles from claims
5. Token expired → client uses Refresh token to get new Access token
6. Refresh token expired → user must log in again

JWT Structure

Header.Payload.Signature

Header:  {"alg": "RS256", "typ": "JWT"}
Payload: {"sub": "user123", "role": "admin", "exp": 1716300000, "iat": 1716296400}
Signature: HMACSHA256(base64(header) + "." + base64(payload), secret)

Key design decisions:
  - Use RS256 (asymmetric) for microservices (public key verification, no shared secret)
  - Keep payload small (don't put entire user profile)
  - Set short expiration (15 min) + use refresh tokens
  - Never store sensitive data in JWT (it's base64, not encrypted)

Common Security Threats and Mitigations

Security Checklist for System Design

✅ HTTPS/TLS for all communication (internal and external)
✅ Authentication at the API gateway layer
✅ Authorization checks at the service level
✅ Input validation and sanitization at system boundaries
✅ Rate limiting per client/IP/API key
✅ Encryption at rest for sensitive data (AES-256)
✅ Password hashing with bcrypt or argon2 (never plain text or MD5)
✅ Secrets in vault (HashiCorp Vault, AWS Secrets Manager) — not in code
✅ Audit logging for security-relevant events
✅ Principle of least privilege for service accounts
✅ Network segmentation (private subnets for DBs, no public access)

Power of 2 Reference

Common Data Sizes

QPS (Queries Per Second) Estimation

Formula: QPS = DAU × queries_per_user / seconds_per_day

Example: Twitter
  - 500M DAU
  - Each user views feed 5 times/day, each feed = 10 API calls
  - Total queries/day = 500M × 50 = 25B
  - QPS = 25B / 86,400 ≈ 290,000 QPS
  - Peak QPS ≈ 2 × average ≈ 580,000 QPS

Quick shortcut:
  - Seconds in a day ≈ 100,000 (actual: 86,400)
  - 1M requests/day ≈ 10 QPS
  - 100M requests/day ≈ 1,000 QPS
  - 1B requests/day ≈ 10,000 QPS

Storage Estimation

Formula: Storage = records_per_day × record_size × retention_period

Example: Chat application
  - 500M DAU, 100 messages/user/day
  - Message size: ~100 bytes (text) + ~100 bytes (metadata) = 200 bytes
  - Daily: 500M × 100 × 200 bytes = 10TB/day
  - Yearly: 10TB × 365 = 3.65 PB/year
  - 5 years with replication (3x): ~55 PB total

Bandwidth Estimation

Formula: Bandwidth = QPS × avg_response_size

Example: Image serving
  - 100K QPS, average image = 200KB
  - Bandwidth = 100,000 × 200KB = 20GB/s = 160 Gbps
  - With CDN absorbing 90%: origin bandwidth ≈ 16 Gbps

Server Estimation

Rule of thumb:
  - 1 web server handles ~1,000-10,000 QPS (depends on complexity)
  - 1 DB server handles ~1,000-5,000 QPS (depends on query complexity)
  - 1 cache server (Redis): ~100,000-500,000 QPS

Example: 500K QPS API
  - App servers: 500K / 5,000 = 100 servers (with headroom: 150)
  - DB (with read replicas): 1 primary + 10 read replicas
  - Cache: 500K / 200K = 3 Redis nodes (with replication: 6)

Layer 4 vs Layer 7 Load Balancing

Load Balancing Algorithms Deep Dive

graph TD
    subgraph RR["Round Robin"]
        RR1["Request 1 → Server A"]
        RR2["Request 2 → Server B"]
        RR3["Request 3 → Server C"]
        RR4["Request 4 → Server A"]
    end

    subgraph LC["Least Connections"]
        LC1["Server A: 5 active"]
        LC2["Server B: 2 active ← next request"]
        LC3["Server C: 8 active"]
    end

    subgraph WRR["Weighted Round Robin"]
        WRR1["Server A (weight 5): gets 5 of every 8"]
        WRR2["Server B (weight 2): gets 2 of every 8"]
        WRR3["Server C (weight 1): gets 1 of every 8"]
    end

    style RR fill:#56cc9d,stroke:#333,color:#fff
    style LC fill:#ffce67,stroke:#333
    style WRR fill:#6cc3d5,stroke:#333,color:#fff

Session Persistence (Sticky Sessions)

Problem: User state (shopping cart, login session) lives on one server.
         If next request goes to different server → state lost.

Solutions (from worst to best):
  1. Sticky sessions (cookie/IP-based routing to same server)
     - Simple but defeats load balancing purpose
     - Server failure = lost sessions

  2. Session replication (broadcast sessions to all servers)
     - Network overhead grows O(n²)
     - Memory wasted on every server

  3. Centralized session store (Redis/Memcached) ← RECOMMENDED
     - Any server can handle any request
     - Session stored in Redis with TTL
     - Server failure has zero impact on sessions
     - Scales independently

Health Check Design

Health check state machine:
  HEALTHY → 3 consecutive failures → UNHEALTHY (remove from pool)
  UNHEALTHY → 2 consecutive successes → HEALTHY (add back to pool)
  
  Drain mode: stop sending new requests, wait for active to complete

Caching Patterns (Implementation Detail)

graph LR
    subgraph CacheAside["Cache-Aside (Lazy Loading)"]
        A1["App checks cache"]
        A1 -->|"miss"| A2["App queries DB"]
        A2 --> A3["App writes to cache"]
    end

    subgraph WriteThrough["Write-Through"]
        B1["App writes to cache"]
        B1 --> B2["Cache writes to DB"]
        B2 --> B3["Confirm to app"]
    end

    subgraph WriteBehind["Write-Behind (Write-Back)"]
        C1["App writes to cache"]
        C1 --> C2["Return immediately"]
        C2 -.->|"async"| C3["Cache writes to DB later"]
    end

    style CacheAside fill:#56cc9d,stroke:#333,color:#fff
    style WriteThrough fill:#ffce67,stroke:#333
    style WriteBehind fill:#6cc3d5,stroke:#333,color:#fff

Cache Eviction Policies

Redis vs Memcached

Cache Stampede Prevention

Problem: Cache key expires → hundreds of requests simultaneously hit DB → DB overload

Solutions:
  1. Lock/mutex: Only one request fetches from DB, others wait
     cache_key = "user:123"
     lock_key = f"lock:{cache_key}"
     if not redis.get(cache_key):
         if redis.set(lock_key, "1", nx=True, ex=5):  # acquire lock
             data = db.query(...)
             redis.set(cache_key, data, ex=300)
             redis.delete(lock_key)
         else:
             wait_for_cache()  # spin until cache populated

  2. Probabilistic early recomputation:
     - Each read checks: should I refresh? (probability increases near TTL)
     - Spreads refresh across time window

  3. Background refresh (refresh-ahead):
     - Background job refreshes popular keys before expiry
     - No stampede possible

When to Use a Message Queue

Kafka vs RabbitMQ vs SQS

Kafka Architecture Deep Dive

graph TD
    subgraph Producers
        P1["Producer 1"]
        P2["Producer 2"]
    end

    subgraph Kafka["Kafka Cluster"]
        subgraph Topic["Topic: orders (3 partitions)"]
            PART0["Partition 0<br/>[msg1, msg4, msg7...]"]
            PART1["Partition 1<br/>[msg2, msg5, msg8...]"]
            PART2["Partition 2<br/>[msg3, msg6, msg9...]"]
        end
    end

    subgraph ConsumerGroup["Consumer Group: order-processors"]
        C1["Consumer 1<br/>← Partition 0"]
        C2["Consumer 2<br/>← Partition 1"]
        C3["Consumer 3<br/>← Partition 2"]
    end

    P1 --> PART0
    P2 --> PART1
    PART0 --> C1
    PART1 --> C2
    PART2 --> C3

    style Kafka fill:#56cc9d,stroke:#333,color:#fff
    style ConsumerGroup fill:#ffce67,stroke:#333

Delivery Guarantees

Exactly-once in practice:
  - Kafka: Idempotent producer + transactions + consumer offset commit
  - Application-level: Idempotency key in each message
    → Consumer checks: "Have I processed message with ID X?"
    → If yes → skip (dedup)
    → If no → process + record ID in DB (same transaction)

Dead Letter Queue (DLQ)

Message processing flow:
  1. Consumer picks up message
  2. Processing fails → retry (exponential backoff: 1s, 5s, 30s, 5min)
  3. After max retries (e.g., 5 attempts) → move to Dead Letter Queue
  4. DLQ messages are inspected manually or by automated systems
  5. Fix the bug → replay DLQ messages back to original queue

Why DLQ matters:
  - Prevents poison messages from blocking the queue
  - Preserves failed messages for debugging
  - Allows retry after fix is deployed

Microservices Design Principles

Service Communication Patterns

Saga Pattern for Distributed Transactions

graph LR
    subgraph Happy["Happy Path"]
        O1["Create Order<br/>(PENDING)"] --> P1["Reserve Payment"]
        P1 --> I1["Reserve Inventory"]
        I1 --> O2["Confirm Order<br/>(CONFIRMED)"]
    end

    subgraph Compensate["Compensation (on failure)"]
        I_FAIL["Inventory fails"] --> P_COMP["Refund Payment"]
        P_COMP --> O_COMP["Cancel Order"]
    end

    style Happy fill:#56cc9d,stroke:#333,color:#fff
    style Compensate fill:#ff7851,stroke:#333,color:#fff

Service Discovery

Problem: Services scale dynamically (pods come and go).
         How does Service A find Service B's current address?

Solution: Service Registry
  1. Service starts → registers itself (IP:port) with registry
  2. Service wants to call another → queries registry for addresses
  3. Registry health-checks registered services, removes dead ones
  4. Client-side load balancing across returned addresses

Tools:
  - Consul (HashiCorp) — service mesh + KV store + health checks
  - Eureka (Netflix) — Java-focused, Spring Cloud native
  - Kubernetes DNS — built-in (service-name.namespace.svc.cluster.local)
  - etcd — distributed KV store (used by Kubernetes internally)

Replication Strategies

Replication Topologies

Replication Lag Problems

Scenario: User updates profile (write to primary), immediately reads (from replica)
Problem: Replica hasn't received the update yet → shows stale data

Solutions:
  1. Read-your-writes consistency:
     → After write, read from primary for N seconds
     → Or track last-write timestamp, read from replica only if up-to-date

  2. Monotonic reads:
     → Always route same user to same replica (sticky reads)
     → Prevents seeing data go "backward"

  3. Causal consistency:
     → Track dependencies between writes
     → Replica only serves reads after all causal dependencies are applied

Sharding (Partitioning) Deep Dive

Cross-Shard Operations

Challenge: Query that spans multiple shards (e.g., "all orders > $100")

Approaches:
  1. Scatter-gather: Query all shards, merge results (expensive)
  2. Denormalize: Copy needed data into each shard (storage trade-off)
  3. Global index: Secondary index service spans all shards
  4. Avoid: Design schema so most queries hit single shard
     → Shard by user_id, and most queries are user-scoped

Core Kubernetes Objects

Deployment Strategies in Kubernetes

graph LR
    subgraph Rolling["Rolling Update (default)"]
        R1["v1 v1 v1"] --> R2["v2 v1 v1"] --> R3["v2 v2 v1"] --> R4["v2 v2 v2"]
    end

    subgraph BlueGreen["Blue-Green"]
        BG1["Blue (v1) ← traffic"] --> BG2["Green (v2) ← traffic"]
    end

    subgraph Canary["Canary"]
        CAN1["v1: 90% traffic<br/>v2: 10% traffic"] --> CAN2["v1: 0%<br/>v2: 100%"]
    end

    style Rolling fill:#56cc9d,stroke:#333,color:#fff
    style BlueGreen fill:#ffce67,stroke:#333
    style Canary fill:#6cc3d5,stroke:#333,color:#fff

Resource Management

# Pod resource specification
resources:
  requests:           # Guaranteed minimum
    cpu: "250m"       # 0.25 CPU cores
    memory: "256Mi"   # 256 MB RAM
  limits:             # Maximum allowed
    cpu: "1000m"      # 1 CPU core
    memory: "512Mi"   # 512 MB RAM

# HPA (Horizontal Pod Autoscaler)
# Scale between 3-10 pods when CPU > 70%
minReplicas: 3
maxReplicas: 10
targetCPUUtilizationPercentage: 70

Kubernetes Networking

CI/CD Pipeline Stages

CI/CD Best Practices

Pipeline design principles:
  1. Fast feedback: fail early (lint → unit tests → integration)
  2. Immutable artifacts: build once, deploy to all environments
  3. Environment parity: staging mirrors production
  4. Infrastructure as Code: Terraform/Pulumi for infra changes
  5. GitOps: desired state in Git, reconciler applies it (ArgoCD)
  6. Feature flags: decouple deployment from release
  7. Rollback plan: every deployment has automated rollback trigger

Branch strategy:
  - Trunk-based development (preferred for fast teams):
    → Short-lived feature branches (< 1 day)
    → Merge to main frequently
    → Feature flags hide incomplete work
    → Main is always deployable

  - GitFlow (for teams needing release management):
    → develop → feature branches → release branches → main
    → More overhead, longer release cycles

GitOps with ArgoCD

GitOps workflow:
  1. Developer merges PR → main branch
  2. CI pipeline builds image → pushes to registry (e.g., v1.2.3)
  3. CI updates manifest repo (Helm values / kustomize with new image tag)
  4. ArgoCD detects drift between Git manifest and cluster state
  5. ArgoCD applies changes to Kubernetes cluster
  6. If deployment fails health checks → ArgoCD auto-rollback

Benefits:
  - Git is single source of truth
  - Full audit trail (who changed what, when)
  - Easy rollback (git revert)
  - Declarative (describe desired state, not imperative steps)

Three Pillars of Observability

Key Metrics (The Four Golden Signals)

Structured Logging

{
  "timestamp": "2026-05-21T10:30:45.123Z",
  "level": "ERROR",
  "service": "order-service",
  "trace_id": "abc-123-def-456",
  "span_id": "span-789",
  "user_id": "user_42",
  "method": "POST",
  "path": "/api/v1/orders",
  "status_code": 500,
  "duration_ms": 2345,
  "error": "ConnectionRefusedError: payment-service:8080",
  "message": "Failed to process payment for order"
}

Distributed Tracing

Request: User places order
  
  ┌─ API Gateway (12ms) ─────────────────────────────────────┐
  │  ┌─ Order Service (45ms) ──────────────────────────────┐  │
  │  │  ┌─ User Service (8ms) ────┐                        │  │
  │  │  └─────────────────────────┘                        │  │
  │  │  ┌─ Payment Service (320ms) ← BOTTLENECK ─────────┐│  │
  │  │  │  ┌─ Stripe API (280ms) ───────────────────────┐││  │
  │  │  │  └────────────────────────────────────────────┘││  │
  │  │  └────────────────────────────────────────────────┘│  │
  │  │  ┌─ Inventory Service (15ms)──┐                     │  │
  │  │  └────────────────────────────┘                     │  │
  │  └─────────────────────────────────────────────────────┘  │
  └────────────────────────────────────────────────────────────┘
  Total: 392ms (Payment → Stripe is 71% of total time)

SLOs, SLIs, and SLAs

Alerting Strategy

Alert design principles:
  1. Alert on symptoms, not causes
     ✅ "Error rate > 5% for 3 min"  (symptom)
     ❌ "CPU > 90%"  (may not impact users)

  2. Severity levels:
     - P1 (Critical): Revenue impacting, page immediately
     - P2 (High): Degraded service, page during business hours
     - P3 (Medium): Non-urgent, ticket in queue
     - P4 (Low): Informational, dashboard only

  3. Reduce noise:
     - Group related alerts
     - Require duration threshold (not single spike)
     - Suppress during maintenance windows
     - Escalation: Slack → PagerDuty → phone call

Event Types

Event Sourcing

graph LR
    CMD["Command:<br/>PlaceOrder"]
    CMD --> ES["Event Store<br/>(append-only log)"]
    ES --> E1["OrderCreated"]
    ES --> E2["ItemAdded (x3)"]
    ES --> E3["PaymentReceived"]
    ES --> E4["OrderShipped"]

    ES -->|"Replay events"| STATE["Current State:<br/>Order #123<br/>Status: Shipped<br/>Items: 3<br/>Total: $59.99"]

    style ES fill:#56cc9d,stroke:#333,color:#fff
    style STATE fill:#6cc3d5,stroke:#333,color:#fff

CQRS (Command Query Responsibility Segregation)

graph TD
    CLIENT["Client"]
    CLIENT -->|"Write (Command)"| WRITE["Write Model<br/>(normalized DB)"]
    CLIENT -->|"Read (Query)"| READ["Read Model<br/>(denormalized views)"]

    WRITE -->|"Events"| PROJ["Projection Service"]
    PROJ --> READ

    style WRITE fill:#56cc9d,stroke:#333,color:#fff
    style READ fill:#6cc3d5,stroke:#333,color:#fff
    style PROJ fill:#ffce67,stroke:#333

Idempotent Event Processing

Problem: Network failures → events may be delivered multiple times.
         Consumer must handle duplicates safely.

Solutions:
  1. Idempotency key in every event:
     Event: { "id": "evt_abc123", "type": "PaymentReceived", "data": {...} }
     Consumer: 
       - Before processing, check: "Have I seen evt_abc123?"
       - If yes → skip
       - If no → process + record evt_abc123 in processed_events table

  2. Idempotent operations (naturally safe):
     - SET operations (overwrite): last write wins
     - Upsert with same data: same result regardless of count

  3. Transactional outbox pattern:
     - Write business data + event to same DB (single transaction)
     - Background process reads outbox table → publishes to Kafka
     - Guarantees: if data saved, event will eventually publish

What a Service Mesh Provides

Service Mesh Comparison

Traffic Management Patterns

When to Use (and NOT Use) a Service Mesh

Use a service mesh when:
  ✅ Running 10+ microservices in production
  ✅ Need mTLS between all services (zero trust)
  ✅ Want consistent observability without code changes
  ✅ Complex traffic routing (canary, A/B, fault injection)
  ✅ Need policy-based access control

Do NOT use when:
  ❌ Fewer than 5 services (overhead not worth it)
  ❌ Team doesn't have Kubernetes expertise
  ❌ Simple request-response with no special routing
  ❌ Latency-critical paths where sidecar overhead matters (~1-3ms)
  ❌ Monolith or early-stage product

Requirements

Key Design Decisions

Short Code Generation:
  Option A: Hash (MD5/SHA256) → take first 7 chars → collision check
  Option B: Pre-generated key service (counter-based, Base62 encoded)
  Option C: Snowflake ID → Base62 encode

  Recommended: Counter-based with Base62 encoding
    - 7 chars of Base62 = 62^7 = ~3.5 trillion unique codes
    - No collision checking needed
    - Monotonically increasing → good for DB indexing

Database Schema

-- URLs table
CREATE TABLE urls (
    short_code  VARCHAR(7) PRIMARY KEY,  -- Base62 encoded
    original_url TEXT NOT NULL,
    user_id     BIGINT,
    created_at  TIMESTAMP DEFAULT NOW(),
    expires_at  TIMESTAMP,
    click_count BIGINT DEFAULT 0
);

-- Analytics (separate table for write performance)
CREATE TABLE clicks (
    id          BIGSERIAL PRIMARY KEY,
    short_code  VARCHAR(7),
    clicked_at  TIMESTAMP DEFAULT NOW(),
    user_agent  TEXT,
    ip_address  INET,
    referrer    TEXT
);

Trade-offs

Requirements

Algorithm Comparison

Token Bucket Design (Recommended)

# Redis-based Token Bucket (distributed)
# Key: rate_limit:{client_id}
# Fields: tokens (float), last_refill (timestamp)

async def is_allowed(redis, client_id: str, max_tokens: int, refill_rate: float) -> bool:
    """
    max_tokens: bucket capacity (e.g., 100)
    refill_rate: tokens per second (e.g., 10)
    """
    key = f"rate_limit:{client_id}"
    now = time.time()

    # Lua script for atomicity (no race conditions)
    lua_script = """
    local tokens = tonumber(redis.call('hget', KEYS[1], 'tokens') or ARGV[1])
    local last_refill = tonumber(redis.call('hget', KEYS[1], 'last_refill') or ARGV[3])
    local now = tonumber(ARGV[3])
    local max_tokens = tonumber(ARGV[1])
    local refill_rate = tonumber(ARGV[2])

    -- Refill tokens based on elapsed time
    local elapsed = now - last_refill
    tokens = math.min(max_tokens, tokens + elapsed * refill_rate)

    if tokens >= 1 then
        tokens = tokens - 1
        redis.call('hset', KEYS[1], 'tokens', tokens, 'last_refill', now)
        redis.call('expire', KEYS[1], 3600)
        return 1  -- Allowed
    else
        redis.call('hset', KEYS[1], 'tokens', tokens, 'last_refill', now)
        return 0  -- Rejected
    end
    """
    result = await redis.eval(lua_script, 1, key, max_tokens, refill_rate, now)
    return result == 1

Where to Place the Rate Limiter

Requirements

Key Components

Message Delivery Flow

1. Alice sends message via WebSocket → Chat Gateway
2. Gateway publishes to Kafka topic (partitioned by chat_id)
3. Message Router consumes from Kafka:
   a. Persist message to Cassandra (status: "sent")
   b. Look up Bob's connection in Session Store (Redis)
   c. If online → push via WebSocket → update status: "delivered"
   d. If offline → send push notification
4. Bob opens app → fetch undelivered messages → update: "delivered"
5. Bob reads message → client sends ACK → update: "read"

Group Messaging Fan-out

Requirements

Fan-out Strategy: The Core Decision

graph LR
    subgraph FanOutWrite["Fan-out on Write (Push)"]
        POST1["New Post"] --> COPY["Copy to all<br/>followers' feeds"]
        COPY --> F1["Alice's Feed Cache"]
        COPY --> F2["Bob's Feed Cache"]
        COPY --> F3["Carol's Feed Cache"]
    end

    subgraph FanOutRead["Fan-out on Read (Pull)"]
        OPEN["Open Feed"] --> FETCH["Fetch posts from<br/>all followed users"]
        FETCH --> M1["User A's posts"]
        FETCH --> M2["User B's posts"]
        FETCH --> M3["User C's posts"]
    end

    style FanOutWrite fill:#56cc9d,stroke:#333,color:#fff
    style FanOutRead fill:#6cc3d5,stroke:#333,color:#fff

Hybrid Approach (Twitter/Instagram’s Actual Design)

Normal users (<10K followers):
  → Fan-out on write: pre-compute feed for all followers
  → Feed is ready when they open the app

Celebrity users (>10K followers):
  → Fan-out on read: don't pre-compute
  → When user opens feed, merge:
      - Pre-computed feed (from normal users they follow)
      - On-demand fetch (from celebrities they follow)
  → Rank the merged result

Feed Ranking

Requirements

Chunked Upload Design

Why chunking?
  - Resume interrupted uploads (mobile networks)
  - Deduplicate at chunk level (save storage)
  - Parallel upload of chunks (faster)
  - Delta sync: only upload changed chunks

Chunk size: 4MB (balance between overhead and resume granularity)

Upload flow:
  1. Client splits file into 4MB chunks
  2. Client computes SHA-256 hash per chunk
  3. Client asks server: "Do you have chunk with hash X?"
     - Yes → skip (deduplication)
     - No → upload chunk
  4. After all chunks uploaded → server assembles file
  5. Server updates metadata DB with file record
  6. Notification service alerts other devices to sync

Data Model

-- Files metadata
CREATE TABLE files (
    file_id     UUID PRIMARY KEY,
    user_id     BIGINT NOT NULL,
    filename    VARCHAR(255),
    path        TEXT,
    size_bytes  BIGINT,
    checksum    VARCHAR(64),
    version     INT DEFAULT 1,
    created_at  TIMESTAMP,
    updated_at  TIMESTAMP
);

-- File chunks (for dedup and resume)
CREATE TABLE chunks (
    chunk_hash  VARCHAR(64) PRIMARY KEY,  -- SHA-256
    size_bytes  INT,
    s3_location TEXT,
    ref_count   INT DEFAULT 1  -- for garbage collection
);

-- File-to-chunk mapping
CREATE TABLE file_chunks (
    file_id     UUID,
    chunk_index INT,
    chunk_hash  VARCHAR(64),
    PRIMARY KEY (file_id, chunk_index)
);

Sync and Conflict Resolution

Requirements

Video Processing Pipeline

Upload → Original Storage → Transcoding → CDN Distribution

Transcoding outputs (per video):
  ┌────────────────────────────────────────┐
  │ Resolution   Bitrate    File Size      │
  │ 360p         800 kbps   ~50MB          │
  │ 480p         1.5 Mbps   ~100MB         │
  │ 720p         3 Mbps     ~200MB         │
  │ 1080p        6 Mbps     ~400MB         │
  │ 4K           20 Mbps    ~1.5GB         │
  └────────────────────────────────────────┘
  + Audio tracks (multiple languages)
  + Subtitles (multiple languages)
  + Thumbnail generation (every 10s for preview)

Adaptive Bitrate Streaming (ABR)

graph LR
    CLIENT["Video Player"]
    CLIENT -->|"Measures bandwidth"| ABR["ABR Algorithm<br/>(DASH / HLS)"]
    ABR -->|"Good network"| HD["1080p chunks"]
    ABR -->|"Poor network"| SD["480p chunks"]
    ABR -->|"Very poor"| LOW["360p chunks"]

    style ABR fill:#56cc9d,stroke:#333,color:#fff

CDN Strategy

Requirements

Architecture Deep Dive

Event flow:
  1. Service emits event: {"type": "order_shipped", "user_id": 123, "data": {...}}
  2. Notification Service:
     a. Check user preferences (opted-in channels, quiet hours)
     b. Check rate limits (max 5 push/hour per user)
     c. Render template with user data
     d. Enqueue to channel-specific queues with priority
  3. Channel workers:
     a. Dequeue message
     b. Call provider API (APNs, Twilio, SES)
     c. Handle retries with exponential backoff
     d. Log delivery status
  4. Analytics:
     - Track: sent, delivered, opened, clicked, unsubscribed

Handling Scale and Reliability

Requirements

Trie Data Structure

Trie for ["system", "systems", "syslog", "syntax"]:

         root
          |
          s
          |
          y
         / \
        s    n
        |    |
        t    t
        |    |
        e    a
        |    |
        m    x
       /
      s

Each node stores:
  - Character
  - Top-K queries passing through this prefix
  - Frequency / score for ranking

Two-Phase Architecture

Ranking Signals

Requirements

CAP Theorem Trade-offs

graph TD
    CAP["CAP Theorem:<br/>Pick 2 of 3"]
    CAP --> C["Consistency<br/>(every read gets latest write)"]
    CAP --> A["Availability<br/>(every request gets a response)"]
    CAP --> P["Partition Tolerance<br/>(works despite network failures)"]

    C --- CP["CP Systems:<br/>MongoDB, HBase, Redis Cluster"]
    A --- AP["AP Systems:<br/>Cassandra, DynamoDB, CouchDB"]

    style CAP fill:#56cc9d,stroke:#333,color:#fff
    style CP fill:#6cc3d5,stroke:#333,color:#fff
    style AP fill:#ffce67,stroke:#333

Key Design Components

Consistency Levels (DynamoDB/Cassandra Style)

API Gateway Responsibilities

Load Balancing Algorithms

Health Checks

Active health checks:
  - Gateway pings /health on each backend every 5-10s
  - Unhealthy after 3 consecutive failures
  - Healthy after 2 consecutive successes
  - Remove unhealthy servers from rotation

Passive health checks:
  - Monitor response codes and latency
  - If >50% of requests to a server fail → mark unhealthy
  - Automatic recovery when success rate improves

High Availability Design