Design a limited-time offers e-commerce system

Background

Business background

As an architect of an e-commerce start-up company, you are responsible for the design of the 6.18 (June 18th) flash sale system. Your business model is as follows:

1. We will select the best-selling and well-received products on major e-commerce platforms for sale. Each category does not exceed 20 products and currently has 10 categories;

2. This 6.18 limited-time offers event selected 1,000 power banks and 10 iPhone 14 as the seckill products;

3. The average daily active users are about 1 million;

4. The boss requires zero incidences.

Technical background

• Java tech stack team.

• Most users are from mobile Apps (iOS & Android) and the WeChat mini-program. To promote the transformation of users into App users, only those Apps can participate in the fresh sale;

• Single Equipment Room.

Current State

The team is doing a micro-service architecture.

Business Case

User (event) Journey

Event Page

The event page will show the details of the event and the products, e.g. power bank and iPhone 14.

Including:

• Event details, time.

• Product info

• Details

• Price

Checkout Page

• Product

• Price

• Product amount

• Delivery info

• Method

• Address

• Contact info

Payment Page

• Payment details

The whole flow

Capacity Estimation and Constraints

We have 1 million daily active users, let's assume 35% of users will participate in the event. So there will be 350 thousand users.

As the user journal for this event shows, there will be 3 steps:

• Check the event page, and click the buy button.

• Checkout.

• Payment.

Event details page

Most users will keep refreshing the event page at the last minute. And will double or triple click the buy button once it becomes clickable.

Let's assume each user will refresh the page 5 times at the last minute and click the button twice within 10 seconds once the button becomes clickable.

The QPS will be 350000 * 5 / 60s ≈ 30000 QPS

The TPS will be 350000 * 2 / 10s ≈ 70000 TPS

Order Page

Assume users already have their info (address, contact info etc.) and only need to confirm the product info and submit the order. Let's say each user will submit an order within 10s.

The TPS will be 350000 * 1 / 10 = 35000 TPS

Payment page

Only those users who successfully submit the order will reach the payment page, the total number will be 1010. So small that we can ignore it.

Design

Overall architecture design

Storage Design

CDN

We host all the static pages (HTML, CSS), images and other media resources on CDN since those resources most likely would stay the same, especially during the event. This will improve the performance and reduce the load from our server.

Database

As the decision details above show, we are doing a Master-slave architecture.

The table for the event will be

id								bigint				// event idproduct_id				bingint   		// foreign key to product tablestatus						emum 					// event statusprice							bdecimalstock							intstart_time				datetimeend_time					datetimelimit							int						// total amount of product for the event

High Performance

Load Balance Architecture

Similar to what we did in The Availability and Performance analytics of Sina Weibo comment, we are using Multi-level load-balancing architecture here.

The Caching Architecture

High Availability

Rate Limit (Flow control)

We will use the Bucket Algorithm for rate limiting. Since we only have 1010 products, we can set the bucket with a low number, like 10000. Once we reach the limit of 10000, we will return the fail directly to the client (App).

Queuing

We will show a queuing page on the app for those in the rate-limiting queue. The whole flow will be

1. User hits the Buy button, and the App will call the Queuing Server. And the App will show the queuing page.

2. The Queuing Server will push the message to the Queue. If the queue is full, the server will return an error to App, and the App will show the relative page.

3. The Domain Server will fetch messages from the Queue and forward the process.

4. The Domain Server will callback to Queueing Server, and notify which request is being processed.

5. The Queuing Servier will return a response to the App with the token and redirect URL to the App.

6. The App will call the Domain Server with the token return from Queuing Server to continue the process.

The Aggressive Rate Limit

We can have an aggressive rate limit strategy: randomly reject 90% of the request on the client site (App) or the Servier site to handle some unexpected situations.