|$ curl https://forge-ai.dev/api/markdown?path=docs/python/regex

$cat docs/python-—-regular-expressions.md

updated Recently·16 min read·published

Python — Regular Expressions

◆Python◆Intermediate

Introduction

Regular expressions (regex) are a powerful tool for pattern matching and text manipulation. Python's re module provides a full regex engine based on the PCRE (Perl Compatible Regular Expressions) standard. Regex patterns let you search, validate, extract, and replace text with surgical precision.

While regex can seem cryptic at first, mastering a handful of core concepts unlocks enormous productivity gains — from parsing log files and validating user input to extracting structured data from unstructured text.

re Module Functions

The re module offers several top-level functions for working with patterns. Each serves a different use case depending on whether you need to search, match, split, or replace.

Function	Description	Returns
re.search()	Searches anywhere in the string	Match object or None
re.match()	Matches only at the beginning	Match object or None
re.fullmatch()	Matches the entire string	Match object or None
re.findall()	Returns all non-overlapping matches	List of strings or tuples
re.finditer()	Returns an iterator of match objects	Iterator of Match objects
re.sub()	Replaces matches with a replacement	New string
re.subn()	Like sub(), but also returns count	(new_string, count)
re.split()	Splits string by pattern	List of strings
re.compile()	Pre-compiles a pattern for reuse	Compiled pattern object

re_search.py

Python

1	import re
2
3	# re.search — finds first match anywhere in string
4	match = re.search(r"\d+", "There are 42 apples")
5	if match:
6	print(match.group()) # "42"
7	print(match.span()) # (10, 12)
8
9	# re.match — matches only at the beginning
10	match = re.match(r"\d+", "42 apples")
11	print(match.group()) # "42"
12
13	match = re.match(r"\d+", "apples 42")
14	print(match) # None — no match at start
15
16	# re.fullmatch — must match the entire string
17	re.fullmatch(r"\d{3}-\d{4}", "555-1234") # Match
18	re.fullmatch(r"\d{3}-\d{4}", "Call 555-1234") # None
19
20	# re.findall — returns all matches as a list
21	emails = re.findall(r"\S+@\S+", "Contact: a@b.com or c@d.org")
22	print(emails) # ['a@b.com', 'c@d.org']
23
24	# re.finditer — returns iterator of match objects
25	for m in re.finditer(r"\d+", "x1 y22 z333"):
26	print(f"{m.group()} at {m.span()}")

re_sub_split.py

Python

1	import re
2
3	# re.sub — replace matches
4	text = re.sub(r"\bfoo\b", "bar", "foo foobar foo")
5	print(text) # "bar foobar bar"
6
7	# re.sub with a function for dynamic replacement
8	def double_number(m):
9	return str(int(m.group()) * 2)
10
11	result = re.sub(r"\d+", double_number, "a1 b2 c3")
12	print(result) # "a2 b4 c6"
13
14	# re.subn — like sub but also returns the count
15	new_text, count = re.subn(r"\s+", " ", " too many spaces ")
16	print(new_text) # " too many spaces "
17	print(count) # 4
18
19	# re.split — split by pattern
20	parts = re.split(r"[,;:\s]+", "one, two; three four")
21	print(parts) # ['one', 'two', 'three', 'four']
22
23	# Split with capture groups — separators included
24	parts = re.split(r"([,;])", "one, two; three")
25	print(parts) # ['one', ',', ' two', ';', ' three']

ℹ

info

Use re.search() by default — it checks the entire string. Use re.match() only when you specifically need to match at position 0. Use re.fullmatch() when validating complete strings (emails, phone numbers, etc.).

Pattern Syntax

Regex patterns combine literals, metacharacters, and quantifiers to define what to match. Here is the comprehensive reference of every pattern element.

Pattern	Description	Example
abc	Matches literal characters	r"hello" matches "hello"
.	Any character except newline	r"a.c" matches "abc", "a1c"
\\d	Digit [0-9]	r"\\d+" matches "42"
\\D	Non-digit [^0-9]	r"\\D+" matches "abc"
\\w	Word character [a-zA-Z0-9_]	r"\\w+" matches "hello_42"
\\W	Non-word character	r"\\W" matches " ", "@"
\\s	Whitespace [ \\t\\n\\r\\f\\v]	r"\\s+" matches " \\t"
\\S	Non-whitespace	r"\\S+" matches "word"
\\b	Word boundary	r"\\bcat\\b" matches "cat" not "catch"
\\B	Non-word boundary	r"\\Bcat\\B" matches in "concatenate"
\\\\	Escaped special character	r"\\." matches literal "."
[abc]	Character set (any one)	r"[aeiou]" matches vowels
[^abc]	Negated set (not a, b, or c)	r"[^0-9]" matches non-digits
[a-z]	Range of characters	r"[A-Za-z]" matches letters
\|	Alternation (OR)	r"cat\|dog" matches "cat" or "dog"

pattern_syntax.py

Python

1	import re
2
3	# Character classes in action
4	text = "The price is $12.99 and the code is ABC-123"
5
6	# Find all dollar amounts
7	dollars = re.findall(r"\$\d+\.\d{2}", text)
8	print(dollars) # ['$12.99']
9
10	# Find alphanumeric codes
11	codes = re.findall(r"[A-Z]+-\d+", text)
12	print(codes) # ['ABC-123']
13
14	# Alternation — match either pattern
15	html_tags = re.findall(r"<b>\|<i>\|<u>", "This is <b>bold</b> and <i>italic</i>")
16	print(html_tags) # ['<b>', '<i>']

Character Classes

Character classes let you match a set of characters with a single expression. You can use predefined classes or define your own with brackets.

character_classes.py

Python

1	import re
2
3	# Custom character classes with brackets
4	hex_pattern = r"[0-9a-fA-F]+"
5	print(re.findall(hex_pattern, "Color: #FF5733")) # ['FF5733']
6
7	# Negated character classes
8	non_alpha = r"[^a-zA-Z]+"
9	print(re.findall(non_alpha, "Hello, World! 123")) # [', ', '! ']
10
11	# Unicode property classes (Python 3.11+ with regex)
12	# Basic approach with re module
13	ascii_only = r"[\x00-\x7F]+"
14
15	# Character class shortcuts recap
16	print(re.findall(r"\d", "12 34")) # ['1', '2', '3', '4']
17	print(re.findall(r"\D", "12 34")) # [' ']
18	print(re.findall(r"\w", "hello-world")) # ['h','e','l','l','o','w','o','r','l','d']
19	print(re.findall(r"\W", "hello-world")) # ['-']
20	print(re.findall(r"\s", "a b\tc")) # [' ', '\t']
21	print(re.findall(r"\S", "a b\tc")) # ['a', 'b', 'c']
22
23	# Combining classes
24	# Match any word, digit, or punctuation
25	token_pattern = r"[\w\d\p{P}]+" # Not in re module directly
26	# Practical alternative:
27	token_pattern = r"[a-zA-Z0-9_]+"
28	print(re.findall(token_pattern, "hello_world 42!")) # ['hello_world', '42']

⚠

warning

Inside character classes, most metacharacters lose their special meaning. The caret ^ is special only as the first character (negation). The dash - is special only between characters (range). The backslash \\ still works for escaping.

Quantifiers

Quantifiers specify how many times a character, group, or character class must be present. By default, quantifiers are greedy — they match as much as possible. Append ? to make them lazy.

Quantifier	Meaning	Example
*	Zero or more	r"ab*c" matches "ac", "abc", "abbc"
+	One or more	r"ab+c" matches "abc", "abbc"
?	Zero or one (optional)	r"colou?r" matches "color", "colour"
{n}	Exactly n times	r"\\d4" matches "2026"
{n,}	n or more times	r"\d{3,}" matches "100", "99999"
{n,m}	Between n and m times	r"\d{1,3}" matches "1", "42", "999"
*?	Zero or more (lazy)	r"<.*?>" matches shortest tags
+?	One or more (lazy)	Matches minimal characters

quantifiers.py

Python

1	import re
2
3	# Greedy vs lazy quantifiers
4	html = "<b>bold</b> and <i>italic</i>"
5
6	# Greedy — matches everything between first < and last >
7	greedy = re.findall(r"<.*>", html)
8	print(greedy) # ['<b>bold</b> and <i>italic</i>']
9
10	# Lazy — matches each tag individually
11	lazy = re.findall(r"<.*?>", html)
12	print(lazy) # ['<b>', '</b>', '<i>', '</i>']
13
14	# Quantifier {n} — exact count
15	phone = re.findall(r"\d{3}-\d{3}-\d{4}", "Call 555-123-4567 or 800-555-0199")
16	print(phone) # ['555-123-4567', '800-555-0199']
17
18	# Quantifier {n,m} — range
19	ip_octet = r"\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}"
20	ip = re.findall(ip_octet, "Server 192.168.1.1 and 10.0.0.255")
21	print(ip) # ['192.168.1.1', '10.0.0.255']
22
23	# Optional parts
24	date = r"\d{4}[-/]\d{2}[-/]\d{2}"
25	print(re.findall(date, "2026-01-15 and 2026/07/09")) # ['2026-01-15', '2026/07/09']

Groups

Parentheses create capture groups that extract substrings from matches. Named groups provide readable access, and non-capturing groups enable logical grouping without extra captures.

Syntax	Description	Example
(abc)	Capture group	match.group(1)
(?P<name>abc)	Named capture group	match.group("name")
(?:abc)	Non-capturing group	Groups without capturing
(?P=name)	Backreference to named group	Matches same text again
(?:...\|...)	Non-capturing alternation	Group without extra match

groups.py

Python

1	import re
2
3	# Named groups — the clearest approach
4	date_pattern = r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})"
5	m = re.match(date_pattern, "2026-07-09")
6	if m:
7	print(m.group("year")) # "2026"
8	print(m.group("month")) # "07"
9	print(m.group("day")) # "09"
10	print(m.groupdict()) # {'year': '2026', 'month': '07', 'day': '09'}
11
12	# Numeric groups
13	m = re.search(r"(\d{3})-(\d{3})-(\d{4})", "555-123-4567")
14	print(m.group(0)) # "555-123-4567" (full match)
15	print(m.group(1)) # "555"
16	print(m.group(2)) # "123"
17	print(m.group(3)) # "4567"
18	print(m.groups()) # ('555', '123', '4567')
19
20	# Non-capturing group — groups without capturing
21	pattern = r"(?:https?\|ftp)://\S+"
22	urls = re.findall(pattern, "Visit https://a.com or ftp://b.org")
23	print(urls) # ['https://a.com', 'ftp://b.org']
24
25	# Backreference — match repeated words
26	pattern = r"(?P<word>\w+)\s+(?P=word)"
27	doubles = re.findall(pattern, "the the quick brown fox fox")
28	print(doubles) # ['the', 'fox']

Anchors

Anchors match positions, not characters. They ensure a pattern appears at the start, end, or a specific boundary of the string or line.

Anchor	Position	With re.MULTILINE
^	Start of string	Start of each line
$	End of string	End of each line
\\A	Start of string (always)	Start of string only
\\Z	End of string (always)	End of string only

anchors.py

Python

1	import re
2
3	# ^ matches start of string
4	print(re.match(r"^Hello", "Hello World")) # Match
5	print(re.match(r"^World", "Hello World")) # None
6
7	# $ matches end of string
8	print(re.search(r"World$", "Hello World")) # Match
9	print(re.search(r"Hello$", "Hello World")) # None
10
11	# Without MULTILINE, ^ and $ match entire string
12	print(re.findall(r"^\w+", "line1\nline2\nline3")) # ['line1']
13
14	# With MULTILINE, ^ and $ match each line
15	print(re.findall(r"^\w+", "line1\nline2\nline3", re.MULTILINE))
16	# ['line1', 'line2', 'line3']
17
18	# \A and \Z are not affected by MULTILINE
19	print(re.match(r"\A\w+", "line1\nline2")) # Match — 'line1'
20	print(re.search(r"\w+\Z", "Hello\nWorld")) # Match — 'World'
21
22	# Practical: validate a full string with anchors
23	is_valid_email = bool(re.fullmatch(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}", "user@example.com"))
24	print(is_valid_email) # True

✓

best practice

Always use \\A and \\Z anchors when validating entire strings, unless you need line-level matching with re.MULTILINE. They are immune to flag changes and make intent explicit.

Flags

Flags modify regex behavior. They can be passed as the third argument to re functions or embedded in the pattern with (?flags).

Flag	Constant	Description
i	re.IGNORECASE	Case-insensitive matching
m	re.MULTILINE	^ and $ match start/end of each line
s	re.DOTALL	. matches any character including newline
x	re.VERBOSE	Ignore whitespace and allow comments in patterns
a	re.ASCII	Make \\w, \\d, \\s match only ASCII characters
u	re.UNICODE	Make \\w, \\d, \\s match Unicode (default in Python 3)

flags.py

Python

1	import re
2
3	# re.IGNORECASE — case-insensitive matching
4	print(re.findall(r"python", "Python PYTHON python", re.IGNORECASE))
5	# ['Python', 'PYTHON', 'python']
6
7	# re.DOTALL — dot matches newline
8	html = "<div>\nHello\n</div>"
9	print(re.findall(r"<div>(.+)</div>", html)) # [] — dot stops at newline
10	print(re.findall(r"<div>(.+)</div>", html, re.DOTALL))
11	# ['\nHello\n']
12
13	# re.MULTILINE — anchors match per-line
14	log = "ERROR: disk full\nINFO: ok\nERROR: timeout"
15	errors = re.findall(r"^ERROR.*", log, re.MULTILINE)
16	print(errors) # ['ERROR: disk full', 'ERROR: timeout']
17
18	# re.VERBOSE — readable patterns with comments
19	email_pattern = re.compile(r"""
20	^ # Start of string
21	[a-zA-Z0-9._%+-]+ # Username
22	@ # @ symbol
23	[a-zA-Z0-9.-]+ # Domain name
24	\. # Dot
25	[a-zA-Z]{2,} # TLD
26	$ # End of string
27	""", re.VERBOSE)
28
29	print(bool(email_pattern.match("user@example.com"))) # True
30
31	# Combining flags
32	pattern = re.compile(r"hello\s+world", re.IGNORECASE \| re.DOTALL)
33	print(bool(pattern.match("Hello\nWorld"))) # True

🔥

pro tip

re.VERBOSE is the secret weapon for complex patterns. Use it to break long regex into labeled segments with comments. Your future self will thank you when debugging a 50-character pattern.

Common Patterns

These battle-tested patterns handle the most common real-world text processing tasks. Each is production-ready and thoroughly commented.

common_patterns.py

Python

1	import re
2
3	# --- Email validation ---
4	email_re = re.compile(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")
5	print(bool(email_re.fullmatch("user@example.com"))) # True
6
7	# --- US phone numbers (multiple formats) ---
8	phone_re = re.compile(r"(?:\+?1[-.\s]?)?$?(\d{3})$?[-.\s]?(\d{3})[-.\s]?(\d{4})")
9	phones = phone_re.findall("Call 555-123-4567, (555) 234-5678, or +1.555.345.6789")
10	print(phones) # [('555','123','4567'), ('555','234','5678'), ('555','345','6789')]
11
12	# --- URL extraction ---
13	url_re = re.compile(r"https?://[^\s<>"']+")
14	urls = url_re.findall("Visit https://example.com or http://test.org/path?q=1")
15	print(urls) # ['https://example.com', 'http://test.org/path?q=1']
16
17	# --- Date formats (YYYY-MM-DD, MM/DD/YYYY, DD.MM.YYYY) ---
18	date_re = re.compile(r"\d{4}[-/]\d{2}[-/]\d{2}\|\d{2}[/.]\d{2}[/.]\d{4}")
19	dates = date_re.findall("Born 12/25/1990, started 2026-01-15, end 31.12.2026")
20	print(dates) # ['12/25/1990', '2026-01-15', '31.12.2026']
21
22	# --- IPv4 address ---
23	ip_re = re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b")
24	ips = ip_re.findall("Servers: 192.168.1.1, 10.0.0.1, 256.1.1.1")
25	print(ips) # ['192.168.1.1', '10.0.0.1', '256.1.1.1']
26
27	# --- HTML tags ---
28	tag_re = re.compile(r"<(/?)(\w+)([^>]*?)>")
29	tags = tag_re.findall('<div class="x"><br/><img src="a.png"/></div>')
30	print(tags) # [('', 'div', ' class="x"'), ('/', 'br', '/'), ...]
31
32	# --- Hex color codes ---
33	hex_re = re.compile(r"#[0-9a-fA-F]{6}\b\|#[0-9a-fA-F]{3}\b")
34	colors = hex_re.findall("Colors: #FF5733, #00f, #abc123")
35	print(colors) # ['#FF5733', '#00f', '#abc123']

Use Case	Pattern	Match Examples
Email	[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}	user@domain.com
Phone (US)	\$?\\d3\$?[-.\\s]?\\d3[-.\\s]?\\d4	(555) 123-4567
URL	https?://[^\\s]+	https://example.com/path
Date (ISO)	\\d4[-/]\\d2[-/]\\d2	2026-07-09
IPv4	\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b	192.168.1.1
Hex Color	#[0-9a-fA-F]{3,6}\b	#FF5733, #00f
HTML Tag	<\/?\w+[^>]*?>	<div class="x">
Integer	-?\\d+	42, -7, 0
Float	-?\d+\.\d+	3.14, -0.5
Whitespace	\\s+	spaces, tabs, newlines

Performance

Compiled patterns avoid re-parsing the regex on every call. For any pattern used more than once, always compile it first. The performance difference is significant at scale.

performance.py

Python

1	import re
2	import time
3
4	# BAD: recompiles the pattern on every iteration
5	def find_emails_slow(texts):
6	results = []
7	for text in texts:
8	results.extend(re.findall(r"[\w.]+@[\w.]+\.\w+", text))
9	return results
10
11	# GOOD: compile once, reuse everywhere
12	EMAIL_RE = re.compile(r"[\w.]+@[\w.]+\.\w+")
13
14	def find_emails_fast(texts):
15	results = []
16	for text in texts:
17	results.extend(EMAIL_RE.findall(text))
18	return results
19
20	# Benchmark
21	texts = ["Contact: user@example.com"] * 100_000
22
23	start = time.perf_counter()
24	find_emails_slow(texts)
25	slow_time = time.perf_counter() - start
26
27	start = time.perf_counter()
28	find_emails_fast(texts)
29	fast_time = time.perf_counter() - start
30
31	print(f"Without compile: {slow_time:.3f}s")
32	print(f"With compile: {fast_time:.3f}s")
33	print(f"Speedup: {slow_time / fast_time:.1f}x")

⚠

warning

Beware of catastrophic backtracking. Patterns like r"(a+)+b"on a string of only "a"s cause exponential time. Avoid nested quantifiers and keep patterns specific. If a pattern takes more than a second on a short string, suspect backtracking.

Performance tips:

◆Compile patterns used more than once

◆Anchor patterns with ^ and $ to reduce search space

◆Use specific character classes instead of . when possible

◆Prefer re.search() over re.match() when you need middle-of-string matches

◆Use non-capturing groups (?:...) when you don't need the captured value

◆For simple string operations, use str methods instead of regex

◆Profile with timeit before optimizing regex performance

Alternatives to Regex

Regex is powerful but not always the right tool. Python built-in string methods are faster and more readable for simple operations. Know when to reach for string methods instead.

Task	Regex	String Method (Better)
Check if starts with	re.match(r"^prefix", s)	s.startswith("prefix")
Check if ends with	re.search(r"suffix$", s)	s.endswith("suffix")
Find substring	re.search(r"needle", s)	"needle" in s
Simple replace	re.sub(r"a", "b", s)	s.replace("a", "b")
Split by delimiter	re.split(r",", s)	s.split(",")
Check if all digits	re.fullmatch(r"\\d+", s)	s.isdigit()
Strip whitespace	re.sub(r"^\\s+\|\\s+$", "", s)	s.strip()
Contains word	re.search(r"\\bword\\b", s)	"word" in s.split()

alternatives.py

Python

1	# String methods vs regex — when to use what
2
3	text = " Hello, World! "
4
5	# Stripping whitespace
6	print(text.strip()) # "Hello, World!" — use this
7	print(re.sub(r"^\s+\|\s+$", "", text)) # "Hello, World!" — overkill
8
9	# Case-insensitive check
10	data = "Hello World"
11	print(data.lower().startswith("hello")) # True — use this
12	print(bool(re.match(r"hello", data, re.IGNORECASE))) # True — overkill
13
14	# Simple split
15	csv_line = "one,two,three"
16	print(csv_line.split(",")) # ['one', 'two', 'three'] — use this
17	print(re.split(r",", csv_line)) # Same result, slower
18
19	# When regex IS necessary
20	import re
21
22	# Complex pattern: validate email
23	email = "user@example.com"
24	print(bool(re.fullmatch(r"[\w.+-]+@[\w.-]+\.[a-zA-Z]{2,}", email)))
25
26	# Replace with capture groups
27	full_name = "John Smith"
28	formatted = re.sub(r"(\w+) (\w+)", r"\2, \1", full_name)
29	print(formatted) # "Smith, John"
30
31	# Extract all numbers from mixed text
32	mixed = "Order #42: 3 items at $9.99 each"
33	numbers = re.findall(r"\d+\.?\d*", mixed)
34	print(numbers) # ['42', '3', '9.99']

✓

best practice

Rule of thumb: if str.replace(), str.split(), str.startswith(), or in can do the job, use them. Reserve regex for patterns that involve character classes, quantifiers, groups, or multiple possible forms. Regex should feel like a scalpel, not a hammer.

$Blueprint — Engineering Documentation·Section ID: PYTHON-REGEX·Revision: 1.0

X GitHub YouTube Instagram Facebook Buy me a coffee Sponsor