Design Amazon/E-commerce
Amazon কেন Complex?
Amazon-এ প্রতি second হাজারো orders হয়। Black Friday-তে traffic ১০০x spike করে। এখানে সবচেয়ে কঠিন problem হলো inventory management + order processing + payment — এই তিনটা atomic হতে হবে। একই সময়ে 1000 user একটা শেষ item কিনতে চাইলে কী হবে?
📌 Core Challenge
Overselling Problem: Stock = 1, কিন্তু 100 user একই সময়ে "Buy Now" click করল। Race condition prevent করতে না পারলেন 100 জনের order accept হবে কিন্তু deliver করা যাবেন না।
Features কী কী?
✅ Functional Requirements
- →Product catalog (search, browse)
- →Shopping cart
- →Order placement + payment
- →Inventory management
- →Order tracking
- →Product reviews and ratings
- →Seller dashboard
- →Recommendations
⚡ Non-Functional Requirements
- →300M+ active users
- →Product search < 200ms
- →Payment ACID guarantees
- →No overselling inventory
- →99.99% checkout availability
- →Black Friday 100x traffic spike
Amazon Scale
🔢 Black Friday Calculation
Normal: 140 orders/sec। Black Friday 10x = 1400 orders/sec। Prime Day আরো বেশি। System-কে এই spike handle করতে auto-scaling দরকার।
Microservices Architecture
Amazon early 2000s-এ monolith থেকে microservices-এ migrate করেছেনিল। এখন thousands of microservices।
Amazon Microservices Architecture
Product Catalog এবং Search
Amazon-এ 350M+ products আছে। User যখন "wireless headphones" search করে তখন milliseconds-এ relevant results দেখাতে হবে। এজন্য দরকার সঠিক database strategy।
🔍 Elasticsearch — Full-Text Search
Elasticsearch inverted index দিয়ে full-text search করে। 350M products-এ keyword search milliseconds-এ। Faceted search (filter by brand, price, rating), fuzzy matching, relevance scoring — সব Elasticsearch করে। DynamoDB product catalog store করে, Elasticsearch search index হিসেবে কাজ করে।
| Data | Database | Why? |
|---|---|---|
| Products catalog | DynamoDB + Elasticsearch | Flexible schema, search |
| Orders (critical) | Aurora/PostgreSQL | ACID, financial data |
| Inventory | MySQL (with row locking) | Atomic updates, locking |
| Shopping cart | Redis | Ephemeral, fast, TTL |
| User sessions | Redis | Fast auth check |
| Product images | S3 + CloudFront | Object storage + CDN |
| Reviews/ratings | Cassandra | High write volume |
| Analytics/clickstream | Redshift/Snowflake | OLAP, data warehouse |
💡 Orders DB-তে কেন ACID?
Order = financial transaction। Double charge, lost orders = real money loss। ACID গ্যারান্টি দেয় partial writes হবে না। PostgreSQL/Aurora প্রতিটা financial transaction-এর জন্য industry standard।
Inventory Management এবং Flash Sales
⚠️ Inventory Race Condition
Stock = 1, user A ও B একই সময়ে checkout করছে। Step 2-তে যদি proper locking না থাকে দুজনই "stock available" দেখবেন। Solution: Optimistic Locking বা Database row lock with version check।
def reserve_inventory(product_id, quantity, db):
while True:
# Current stock এবং version পড়ো
product = db.get(f"SELECT id, stock, version FROM products WHERE id={product_id}")
if product.stock < quantity:
raise InsufficientStockError("Out of stock")
# Version check দিয়ে atomic update
rows_updated = db.execute("""
UPDATE products
SET stock = stock - %s, version = version + 1
WHERE id = %s AND version = %s AND stock >= %s
""", (quantity, product_id, product.version, quantity))
if rows_updated == 1:
# Success! কেউ আগে update করেনি
return "Reserved"
else:
# Conflict! কেউ আগে stock নিয়ে গেছে, retry করুন
continue # বা raise errorFlash Sale Strategy
Flash sale: 100 units, 10,000 requests একসাথে। Database-এ directly পাঠালে DB crash করবেন। Solution: Redis atomic counter ব্যবহার করুন।
💡 Redis DECR — Flash Sale Solution
Redis DECR atomic operation। Counter = 1000। DECR → 999, 998... → 0। 0-এর নিচে গেলে reject। DB-তে 1M requests পাঠানো মানে DB crash। Redis in-memory তাই 1M req/sec handle করতে পারে।
Order Placement — সবচেয়ে Critical Flow
🛒 Checkout Flow — Step by Step
| Step | Action | Service | Critical? |
|---|---|---|---|
| 1. Cart Review | Items এবং prices confirm | Cart Service | Medium |
| 2. Lock Inventory | Stock reserve করুন (soft lock) | Inventory Service | ⚠️ Very Critical |
| 3. Payment | Card charge করুন | Payment Service | ⚠️ Most Critical |
| 4. Confirm Order | Order create করুন DB-তে | Order Service | Critical |
| 5. Reduce Inventory | Actual stock deduct করুন | Inventory Service | Critical |
| 6. Notify | Email/SMS পাঠাও | Notification Service | Low (async) |
🎯 SAGA Pattern
Distributed systems-এ traditional ACID transaction নেই। SAGA Pattern use করুন: প্রতিটা step এর জন্য compensating transaction define করুন। Payment fail হলে → inventory unlock করুন → cart restore করুন। Chain of events with rollback।
Step 1 — Reserve Inventory
Inventory Service-এ stock soft-lock করুন। Compensation: inventory.release(cart.items)
Step 2 — Process Payment
Payment Service-এ card charge করুন। Compensation: payment.refund(payment_id)
Step 3 — Create Order
Order Service DB-তে order create করুন। Compensation: order.cancel(order_id)
Step 4 — Async Notifications (Kafka)
Event publish করুন — Notification, Fulfillment service async consume করবেন। Non-blocking ✓
class PlaceOrderSaga:
def execute(self, cart, payment_info):
steps_completed = []
try:
# Step 1: Reserve inventory
self.inventory.reserve(cart.items)
steps_completed.append('inventory')
# Step 2: Process payment
payment_id = self.payment.charge(payment_info, cart.total)
steps_completed.append('payment')
# Step 3: Create order
order = self.order.create(cart, payment_id)
steps_completed.append('order')
# Step 4: Async notifications (Kafka)
self.events.publish("order.placed", order)
return order
except Exception as e:
# Compensating transactions (rollback)
if 'payment' in steps_completed:
self.payment.refund(payment_id)
if 'inventory' in steps_completed:
self.inventory.release(cart.items)
raise eBlack Friday Scaling এবং Tech Stack
Scaling Decisions
Auto-scaling: AWS Auto Scaling Groups। Traffic বাড়লে automatically EC2 instances add হয়। Black Friday-তে 10x scale up, afterward scale down।
Product Cache: Popular products Redis-এ cache করুন। 80% product views cached। DB load drastically কমে।
Queue-based Order Processing: Order requests Kafka queue-তে নাও। Backend async process করে। User immediately "Order Received" দেখে।
Flash Sale Problem: 100 units, 10,000 requests একসাথে। Redis SETNX দিয়ে atomic reservation। DB overload prevent করে কিন্তু Redis সত্য থেকে diverge হতে পারে।
Read Replicas: Product reads অনেক বেশি। Read replicas দিয়ে scale করুন কিন্তু replication lag এ newly added products কিছুক্ষণ দেখা না-ও যেতে পারে।
Amazon-এর Tech Stack
Backend Services
Databases
Infrastructure
Payment এবং Recommendations
💳 Payment System
Payment = most critical path। Stripe বা Internal payment processor। Idempotency key দিয়ে double charge prevent করুন। PCI DSS compliance mandatory। Synchronous flow — user wait করে।
🤖 Recommendations
Collaborative filtering ML model + offline batch computation। "User A এবং User B একই products কিনেছে" — pattern থেকে suggest করুন। Offline job (Spark) daily compute। Real-time serving Redis থেকে।
🎯 Interview Tips — Amazon System Design
1) সবসময় overselling problem mention করুন এবং Optimistic Locking solution দিন।
2) Order processing-এ SAGA pattern explain করুন — distributed transaction problem জানাও।
3) Database choices justify করুন: Orders → PostgreSQL (ACID), Cart → Redis (ephemeral), Search → Elasticsearch।
4) Black Friday spike-এর জন্য Auto-scaling + Queue-based processing mention করুন।
5) Notification async (Kafka) — checkout speed বাড়ানোর জন্য।
SUMMARY — আজকে যা শিখলাম
| Challenge | Solution | Technology |
|---|---|---|
| Overselling | Optimistic locking | MySQL version check |
| Distributed transactions | SAGA pattern | Kafka events |
| Product search | Inverted index | Elasticsearch |
| Traffic spikes | Auto-scaling | AWS Auto Scaling |
| Async processing | Event-driven | Kafka / SQS |
| Flash sale | Atomic counter | Redis DECR |
| Shopping cart | In-memory storage | Redis (TTL) |
| Financial orders | ACID transactions | PostgreSQL/Aurora |
| Product images | CDN + Object storage | S3 + CloudFront |
| Recommendations | Collaborative filtering | ML + Redis serving |