Ebay Auction (used)
Topic: Ebay Auction (used)
Interviewer: im_J
Interviewee: 朱杰
Level: L5 (Senior)
Event notes for historical events:
https://designclub.commitway.com
Topic
Mock System Design Interview Summary
Interview Overview
Date: 4/10/2022
Target level: L5 senior
Duration: 45 minutes
Topic covered: ebay auction
Drawing tool used: excalidraw
Requirements
Functional requirements
Auction website
Auction success depends on payment success
Functional Requirements/Workflow
Seller enlist a product, (product, inventory)
seller create an auction (auction)
Bidder is able to search the product
Auction starts, bidder can place bid
Bidder can see the realtime bidding price
Bidder limit bid in a min(v2)
Waiting list (v2)
Auction, winner -> payment -> auction finish
Non functional requirements
100M DAU, 1M product, 100 queries/product/day
Query qps: 10^6 * 100 / 10^5 = 1000 QPS
Bidding write QPS: 100M * 20 / 10^5 = 200QPS
50%, 50M connections (for realtime prices display)
Minute 15:
Timeline:
Check table for existing product
Add invetory
Product table
Product (key), desc,
System Design
External APIs
System design
Seller side
Please focus on bidding
Flink:
Supports “exact once”
====
Q: what does bid agent do?
Inform the bidder the latest price
Additional thoughts: can keep top-N prices
HashTable: userA 90, UserB 75
LinkedList:
Database schema
==
Add Elastic search
Ensure kafka processes exactly once
Q: how do you update the database?
Q: nosql database because write throughput is high?
Will the throughput still be high? Given there are message queues
Cache write mechanism
Use memory to write to DB
Every DB have records
Q: How do you handle large size of the record?
A:
Option 1: HDFS, append log,
Option 2: shard by option ID, or time range
Interviewer and Audience Feedback
Interviewer feedback:
Strong foundation
Exactly once kafka
Tradeoff: nosql/sql
Kafka exactly once
Flink
Quant analysis: strong, quantified
Requirement:
Keep focus, spread too thin
Interviewer wants to focus on the bidding process
Spent too much time on seller bidding
Bidding:
Stream processing is fine, but it may be too complex
Conflicting:
Nosql vs exactly once
Try to be more consistent
===
Interviewee
What I prepared has different focus: two phase commit in payment, etc
I like the fact the interviewer keep me on track
Time manage is hard
===
Audience
QPS estimation: what value to pick
100M users, seems high
Ebay or amazon may have lots of users
If the number is too high - do we direct the interviewer toward this direction?
Scaling
100M may be too high
Replica and sharding
Bid agent, bid service
Bid agent: websocket call
2 way communication
QPS is fine
Traffic estimation peak. X3 or x10 for QPS
Why are there 2 kafka?
Exact 1-1 processing, 2 phase commit
Wants the upstream and downstream have data consistency
Alex Xu likes 2 queues, but it’s not too clear why.
First one is bidding topic resource
Second one is after stream process
The second one is aggregated
Topic can be for the same auction
The second one has been aggregated
It probably not 2 kafka
But 2 different topics in kafka
The first one is from a log system
It’s has fixed number topic, processed from log system
First kafka sends data to Second kafka based on rules
E.g. Monitor CPU, monitor topic
If we expand 2nd kafka, we can pause the second kafka
Top-K
First one is for handling different events
Second one is to handle different topic
Why does this support exactly kafka
One kafka can already guarantee exactly one
2 kafka
Can allow the second kafka to expand
Flink
2 kafka, job manager for 2PL.
2PC and saga
If the first one fails, then it will be passed to the 2nd person
Ebay: bid: current price and can set maximum price
First one $104, $200
Second one $105 -> $106
Third one: $108
Eventually the person place the higher max price
The hot spot is the last minute (may require in-memory operation)
Hot item. Ebay web OS brought down EBay website
Sharding. Hot item
Product: highest price. Not second price
If the first one fails payment, the seller can decide if the order can be passed to the second highest bidder
1-2 days
Payment succeeds: auction completes
Payment fails: retry, 15 minutes. Notifies the second price bidder
Saga, Inventory, Payment Service
Payment provider
Auction: usually is for a single item
==
Redis: when do we persist it to DB?
Writeback buffer for redis
This may create data loss. Every 2 seconds data is updated
Streaming process
Redis is in memory. When will this write to DB?
After we complete the auction, we will save to the DB
All reports can happen in DB
Redis: in memory, only the last few minutes
DB - insert/update
2 phase transaction, very slow
Q: There are kafka and streaming process. We can buffer the peak service. Why do we need redis?
A: good question
Q: Redis order set?
A: hashmap + linkedList. Yes.
We can directly write to redis
If we use Redis, we can use saga mechanism
The second kafka is for exactly once.
Redis can also be sharded.
Ron: kafka, redis queue. Topic, auction ID. message queue
Can we create 10,000 message queues?
If we need only 20-30 topics
The first kafka, sharding hard to change
Second kafka, can change # of partitions, consumers
Easy to do sharding
We don’t need to
Azure service bus: 32 queue
Max 1000 queues, but need to file ticket to azure
In christmas we need to change sharding from 1000 to 2000
If the first one needs to be expanded, there is a problem.
Can we use a load balancer to rotate from first to second kafka
Second one may be due to multi-data center requirements.
Mirror maker to collect from multiple data center into a single one
If we need to write data to different database
2 kafkas: event service: streaming service for aggregation in the middle. This will reduce the load to the second kafka.
Interviewer:
inventory/database: transaction
How to handle spiky traffic
Read and write: 100k per single node for redis