|$ curl https://forge-ai.dev/api/markdown?path=docs/system-design/url-shortener

$cat docs/url-shortener.md

updated Recently·35 min read·published

URL Shortener

◆System Design◆Intermediate🎯Free Tools

Introduction

A URL shortener converts long URLs into short, shareable aliases. When a user visits the short URL, the service redirects them to the original URL. This classic system design interview question tests your ability to design a scalable read-heavy service with a simple data model.

The core challenge is generating unique short codes efficiently and redirecting users with minimal latency. Reads dominate writes by orders of magnitude, so caching and fast lookups are critical.

This case study designs a service similar in scale to bit.ly. It covers requirements, capacity estimation, API design, hashing strategies, database schema, caching, analytics, and deployment.

Requirements

Begin by clarifying functional and non-functional requirements. For a URL shortener, the functional surface is small, but scale and availability requirements drive the architecture.

Functional Requirements

◆Given a long URL, generate a unique short URL

◆Given a short URL, redirect to the original long URL

◆Optionally allow custom aliases and expiration times

◆Track click analytics: count, referrer, geo, timestamp

Non-Functional Requirements

◆High availability: redirection must work even if creation is degraded

◆Low latency: redirects under 100 ms at p99

◆Scalability: handle 100M new URLs/month and 10B redirects/month

◆Short URLs should be difficult to guess (not sequential)

ℹ

info

Read-heavy workloads like redirection benefit enormously from caching. Optimize the read path first; the write path has much lower throughput requirements.

Capacity Estimation

Estimating scale helps validate design choices. Assume 100 million new short URLs per month and a 100:1 read-to-write ratio.

capacity.txt

TEXT

1	Assumptions:
2	- 100M new short URLs/month
3	- 100:1 read:write ratio → 10B reads/month
4	- Average long URL: 500 bytes
5	- Short code: 7 characters
6	- Metadata per URL: ~200 bytes
7	- Analytics row per redirect: ~100 bytes
8
9	Writes per second: 100M / (30 × 86,400) ≈ 39 writes/sec
10	Reads per second: 10B / (30 × 86,400) ≈ 3,858 reads/sec
11	Peak traffic: 5× average ≈ 19,290 reads/sec
12
13	Storage per month (URL mappings):
14	100M × (500 + 200) bytes ≈ 65 GB
15
16	Storage per month (analytics):
17	10B × 100 bytes ≈ 931 GB
18
19	Five-year URL mappings: 65 GB × 60 ≈ 3.9 TB
20	Five-year analytics: 931 GB × 60 ≈ 55 TB

📝

note

Analytics storage dwarfs URL mapping storage. Consider separating analytics into a columnar store or data warehouse optimized for batch analysis rather than transactional lookups.

API Design

The API has two primary operations: create a short URL and resolve a short URL. Analytics can be exposed as a separate endpoint.

api.txt

TEXT

1	POST /api/v1/shorten
2	Request:
3	{
4	"longUrl": "https://example.com/very/long/path?query=1",
5	"customAlias": "mylink", // optional
6	"expiresAt": "2027-01-01T00:00:00Z" // optional
7	}
8
9	Response (201):
10	{
11	"shortUrl": "https://forgelearn.dev/aBc3xK9",
12	"shortCode": "aBc3xK9",
13	"longUrl": "https://example.com/very/long/path?query=1",
14	"expiresAt": "2027-01-01T00:00:00Z"
15	}
16
17	GET /{shortCode}
18	Response: 302 redirect to long URL
19	404 if not found
20	410 if expired
21
22	GET /api/v1/analytics/{shortCode}
23	Response:
24	{
25	"shortCode": "aBc3xK9",
26	"totalClicks": 15420,
27	"topReferrers": [{"source": "twitter.com", "count": 8900}],
28	"topCountries": [{"country": "US", "count": 7200}]
29	}

✓

best practice

Use 302 redirects for analytics tracking, not 301. A 301 is cached by browsers, causing subsequent visits to bypass your service and preventing click counting.

Short Code Generation

The short code must be unique, short, and non-sequential. There are two common approaches: hash-based encoding and counter-based encoding with obfuscation.

Hash-Based Approach

Hash the long URL using a cryptographic hash such as SHA-256, then encode a portion of the hash in base62. If a collision occurs, append a salt and rehash. This approach is stateless but cannot support custom aliases and may have rare collisions.

hash-approach.txt

TEXT

1	Hash-based short code:
2
3	1. hash = SHA-256(longUrl + salt)
4	2. Take first 7 bytes of hash
5	3. Encode as base62 (a-z, A-Z, 0-9)
6	4. Check database for collision
7	5. If collision, increment salt and retry
8
9	Base62 encoding gives 62^7 ≈ 3.5 trillion codes,
10	enough for a 7-character short URL.
11
12	Downside: same long URL always maps to same code
13	unless salted per user, preventing per-user analytics.

Counter-Based Approach

Use a distributed counter to assign unique IDs, then encode the ID in base62. To prevent sequential, guessable codes, apply a permutation such as the Knuth multiplicative hash. This guarantees uniqueness and supports custom aliases.

counter-approach.txt

TEXT

1	Counter-based short code:
2
3	1. id = counter.incrementAndGet() // globally unique
4	2. obfuscated = knuth_permutation(id)
5	3. shortCode = base62_encode(obfuscated)
6
7	Example:
8	id = 1000001
9	obfuscated = 4829103756
10	base62 = "aBc3xK9"
11
12	Knuth permutation uses a large prime and modulo
13	the keyspace to scatter sequential IDs.

Approach	Pros	Cons
Hash-based	Stateless, no counter needed	Collisions, no custom aliases easily
Counter-based	Unique, supports custom aliases	Requires distributed counter

✓

best practice

Use the counter-based approach for production URL shorteners. It guarantees uniqueness, supports custom aliases, and produces deterministic code lengths. Manage the counter with a database sequence, Redis INCR, or a pre-allocated range per app server.

Database Design

The URL mapping table is small and read-heavy. A relational database with a primary key on shortCode and an index on longUrl for deduplication is sufficient at most scales. For massive scale, shard by shortCode.

schema.sql

TEXT

1	Schema:
2
3	CREATE TABLE url_mappings (
4	short_code VARCHAR(10) PRIMARY KEY,
5	long_url TEXT NOT NULL,
6	created_at TIMESTAMP DEFAULT NOW(),
7	expires_at TIMESTAMP NULL,
8	user_id BIGINT NULL,
9	click_count BIGINT DEFAULT 0
10	);
11
12	CREATE INDEX idx_long_url ON url_mappings(long_url);
13
14	Analytics table:
15
16	CREATE TABLE url_analytics (
17	id BIGINT PRIMARY KEY,
18	short_code VARCHAR(10) NOT NULL,
19	clicked_at TIMESTAMP DEFAULT NOW(),
20	country VARCHAR(2),
21	referrer VARCHAR(255),
22	user_agent VARCHAR(255),
23	ip_hash VARCHAR(64)
24	);
25
26	CREATE INDEX idx_analytics_code ON url_analytics(short_code, clicked_at);

ℹ

info

The read path only needs shortCode → longUrl. Consider caching hot mappings aggressively and storing the full mapping in Redis or a KV store with sub-millisecond latency.

High-Level Design

The architecture follows a standard layered pattern optimized for read-heavy traffic. Writes flow through the API to the database and cache. Reads hit the cache first, then the database, and finally redirect the user.

architecture.txt

TEXT

1	Architecture:
2
3	Client ──▶ CDN / WAF ──▶ Load Balancer
4	│
5	┌─────────┴─────────┐
6	▼ ▼
7	Write Path Read Path
8	POST /shorten GET /{shortCode}
9	│ │
10	▼ ▼
11	API Servers API Servers
12	│ │
13	▼ ▼
14	Short Code Generator Cache (Redis)
15	│ │
16	▼ ▼
17	Database (primary) Database (replica)
18	│
19	▼
20	Message Queue
21	│
22	▼
23	Analytics Pipeline

⚠

warning

Keep the redirect path as short as possible. Every additional hop adds latency. Ideally, serve redirects from cache at the edge or from an in-memory store for hot links.

Caching Strategy

Caching is the key to achieving sub-100 ms redirects. Use Redis as a cache-aside layer for shortCode → longUrl mappings. Set TTLs based on expiration and access patterns. For extremely hot URLs, consider CDN-level redirects if your provider supports them.

cache-flow.txt

TEXT

1	Redirect flow with cache:
2
3	1. Client requests /aBc3xK9
4	2. Check CDN / edge cache for redirect rule
5	3. Check Redis for key short:aBc3xK9
6	4. Cache hit: return 302 immediately
7	5. Cache miss: query database, populate cache, return 302
8	6. If not found: return 404
9
10	TTL strategy:
11	- Popular links: longer TTL (days)
12	- Unpopular links: shorter TTL (hours)
13	- Expired links: do not cache, return 410

✓

best practice

Pre-warm the cache when a new short URL is created. This avoids a cache miss on the first redirect, which is common for URLs shared immediately after creation.

Analytics

Analytics should not block the redirect path. Fire an asynchronous event to a message queue or stream for each click, then process events in batches. This protects redirect latency from analytics write load.

analytics.txt

TEXT

1	Click analytics pipeline:
2
3	Redirect request
4	│
5	├─▶ Return 302 immediately
6	│
7	└─▶ Emit ClickEvent to Kafka
8	│
9	▼
10	Stream processor (Flink / Kafka Streams)
11	│
12	├─▶ Update counter in Redis
13	├─▶ Write aggregated metrics to warehouse
14	└─▶ Store raw events for ad-hoc analysis

ℹ

info

For real-time counters, use Redis HyperLogLog for unique visitors and simple counters for total clicks. For detailed analytics, batch writes into a columnar store like ClickHouse or BigQuery.

Tradeoffs

Every design decision in a URL shortener involves tradeoffs. Be ready to discuss them in an interview.

Decision	Option A	Option B
Code generation	Hash-based, stateless	Counter-based, unique
Redirect status	302, track every click	301, faster but fewer analytics
Analytics	Async pipeline	Synchronous counter update
Storage	Relational DB	Key-value store

🔥

pro tip

The URL shortener is intentionally simple. Interviewers often add constraints to test deeper thinking: custom domains, link previews, malware detection, rate limiting, and GDPR right-to-erasure.

$Blueprint — Engineering Documentation·Section ID: SD-URL-01·Revision: 1.0

X GitHub YouTube Instagram Facebook Discord Slack Telegram Buy me a coffee Ko-fi Sponsor