|$ curl https://forge-ai.dev/api/markdown?path=docs/python/environments

$cat docs/python-—-virtual-environments.md

updated Recently·12 min read·published

Python — Virtual Environments

◆Python◆Intermediate

Introduction

Python projects depend on third-party libraries, and different projects often need different versions of the same library. Virtual environments solve this by creating isolated Python installations per project, preventing dependency conflicts and keeping your global site-packages clean.

This guide covers the full ecosystem: venv for lightweight isolation, pyenv for Python version management, pip-tools and uv for fast dependency resolution, poetry for all-in-one project management, conda for data science workflows, and python-dotenv for environment variables.

Why Virtual Environments?

Without virtual environments, all Python packages install globally under site-packages. This leads to version conflicts — Project A needs Django 3.2, Project B needs Django 4.2. They can't both be installed globally. Virtual environments give each project its own site-packages directory, so dependencies are isolated per project.

ℹ

info

Even for small scripts, always use a virtual environment. It prevents accidental upgrades that break other projects and makes your project reproducible for other developers and deployment.

terminal

Bash

1	# Without a venv — packages land in global site-packages
2	pip install requests # global install — all projects affected
3
4	# The global site-packages location:
5	python -c "import site; print(site.getsitepackages())"
6	# → /usr/local/lib/python3.12/site-packages

venv — Built-in Virtual Environments

Python ships with the venv module in the standard library. It creates a lightweight environment with its own Python binary and package directories. No external tools required.

terminal

Bash

1	# Create a virtual environment (common convention: .venv or venv)
2	python -m venv .venv
3
4	# Activate on macOS / Linux:
5	source .venv/bin/activate
6
7	# Activate on Windows (PowerShell):
8	.venv\Scripts\Activate.ps1
9
10	# Activate on Windows (cmd.exe):
11	.venv\Scripts\activate.bat
12
13	# Once activated, the prompt changes:
14	(.venv) $ pip install requests
15
16	# Deactivate when done:
17	(.venv) $ deactivate

ℹ

info

Use .venv (with leading dot) instead of venv as the directory name. The dot keeps it hidden in file listings and is the community convention. Add .venv/ to your .gitignore.

terminal

Bash

1	# Freeze installed packages to share the environment
2	pip freeze > requirements.txt
3
4	# Recreate the environment on another machine
5	python -m venv .venv
6	source .venv/bin/activate
7	pip install -r requirements.txt
8
9	# Check which Python you're using inside the venv
10	which python # → .venv/bin/python
11	python --version # Python version is inherited from parent

The python -m venv approach is zero-dependency and works everywhere Python is installed. It is the most portable option and is the recommended starting point for most projects.

check_env.py

Python

1	# Check if you're inside a virtual environment (from Python)
2	import sys
3	print(sys.prefix != sys.base_prefix) # True if inside venv
4
5	# sys.prefix points to the venv directory when activated
6	print(sys.prefix) # → /home/user/project/.venv

pyenv — Python Version Management

venv isolates packages but uses the same Python version as the system. When you need Python 3.10 for one project and Python 3.12 for another, you need pyenv. It installs multiple Python versions side-by-side and switches between them per directory.

terminal

Bash

1	# Install pyenv (macOS via Homebrew)
2	brew install pyenv
3
4	# Install specific Python versions
5	pyenv install 3.10.15
6	pyenv install 3.12.3
7	pyenv install 3.13.0
8
9	# List installed versions
10	pyenv versions
11
12	# Set global default (system wide)
13	pyenv global 3.12.3
14
15	# Set local version (project specific — writes .python-version)
16	cd my-project
17	pyenv local 3.10.15
18
19	# Python version automatically switches when you cd into the dir
20	cat .python-version # → 3.10.15
21	python --version # → Python 3.10.15

⚠

warning

After installing pyenv, you must add eval "$(pyenv init -)" to your shell profile (~/.zshrc or ~/.bashrc). Without it, the python command won't switch versions.

terminal

Bash

1	# Combine pyenv with venv for full isolation
2	pyenv local 3.10.15 # set Python version
3	python -m venv .venv # create venv with that version
4	source .venv/bin/activate # activate
5	python --version # → Python 3.10.15
6
7	# pyenv-virtualenv plugin (optional — automates the combo)
8	pyenv virtualenv 3.12.3 my-project
9	pyenv activate my-project
10	pyenv deactivate

pip-tools — Lock Files for pip

pip freeze > requirements.txt captures every installed package including transitive dependencies. This makes it hard to upgrade or understand what you actually depend on. pip-tools introduces a two-file pattern: requirements.in for direct dependencies and requirements.txt as a fully-pinned lock file.

terminal

Bash

1	# Install pip-tools
2	pip install pip-tools
3
4	# requirements.in — list only direct dependencies:
5	# requests
6	# flask>=2.0,<3.0
7
8	# Compile into a locked requirements.txt
9	pip-compile requirements.in # → generates requirements.txt
10
11	# The output pins every transitive dependency with hashes:
12	# flask==2.3.3
13	# via -r requirements.in
14	# jinja2==3.1.4
15	# via flask
16	# markupsafe==2.1.5
17	# via jinja2
18	# ...
19
20	# Sync your environment to match exactly
21	pip-sync requirements.txt # removes unrelated packages too
22
23	# Upgrade a specific dependency
24	pip-compile --upgrade-package requests requirements.in

ℹ

info

The requirements.in / requirements.txt split is the most maintainable way to use pip in production. Committing the lock file ensures reproducible deployments.

terminal

Bash

1	# Common requirements.in patterns:
2
3	# --- requirements-dev.in ---
4	# Add dev-only tooling (not shipped to production)
5	# -r requirements.txt # include prod deps
6	# pytest
7	# pytest-cov
8	# black
9	# ruff
10
11	# Compile both:
12	pip-compile requirements-dev.in # → requirements-dev.txt
13
14	# Install dev environment:
15	pip-sync requirements-dev.txt

uv — Fast Rust-based Package Manager

uv is a drop-in replacement for pip, pip-tools, pipenv, poetry, and virtualenv written in Rust. It is 10-100x faster than pip and handles Python version management, virtual environments, and dependency resolution in a single binary.

terminal

Bash

1	# Install uv
2	curl -LsSf https://astral.sh/uv/install.sh \| sh
3
4	# Or via Homebrew
5	brew install uv
6
7	# Create a virtual environment (replaces both pyenv + venv)
8	uv venv # creates .venv
9	uv venv --python 3.10 # create with specific Python version
10	source .venv/bin/activate
11
12	# Install packages (replaces pip install)
13	uv pip install requests
14
15	# Install from requirements.txt (replaces pip install -r)
16	uv pip sync requirements.txt
17
18	# Compile requirements (replaces pip-compile)
19	uv pip compile requirements.in -o requirements.txt
20
21	# Add a dependency
22	uv add "requests>=2.28"

uv resolves dependencies globally, producing a deterministic lock file even without explicit version pins. Its speed comes from Rust's parallelism and a global package cache that avoids re-downloading packages across projects.

terminal

Bash

1	# uv tool — run tools without installing them
2	uvx ruff check . # ruff runs in ephemeral env
3	uvx black --check . # black runs, no install needed
4
5	# uv pip tree — inspect the dependency tree
6	uv pip tree
7
8	# uv cache management
9	uv cache clean
10	uv cache dir # show cache location

ℹ

info

uv is the future of Python packaging. It implements pip, pip-tools, and venv APIs exactly so existing workflows work without changes. Adopt it incrementally — start with uv pip install in your existing projects.

Poetry — All-in-One Project Management

poetry combines dependency management, virtual environment creation, build, and publish into one tool. It uses pyproject.toml (PEP 621) for project metadata and poetry.lock for deterministic installs.

terminal

Bash

1	# Install poetry
2	curl -sSL https://install.python-poetry.org \| python3 -
3
4	# Start a new project
5	poetry new my-project
6	cd my-project
7
8	# Or add poetry to an existing project
9	poetry init
10
11	# Add dependencies
12	poetry add requests
13	poetry add --group dev pytest black
14
15	# Install from existing lock file
16	poetry install
17
18	# Activate the virtual environment
19	poetry shell
20
21	# Or run a command inside the env without activating
22	poetry run python my_script.py
23
24	# Update a dependency
25	poetry update requests
26
27	# Export to requirements.txt (for Docker)
28	poetry export --format requirements.txt --output requirements.txt

pyproject.toml

TOML

1	# pyproject.toml — poetry project example
2	[tool.poetry]
3	name = "my-project"
4	version = "0.1.0"
5	description = ""
6	authors = ["Alice <alice@example.com>"]
7
8	[tool.poetry.dependencies]
9	python = "^3.11"
10	requests = "^2.31.0"
11	click = "^8.1"
12
13	[tool.poetry.group.dev.dependencies]
14	pytest = "^7.4"
15	ruff = "^0.1"
16
17	[build-system]
18	requires = ["poetry-core"]
19	build-backend = "poetry.core.masonry.api"

⚠

warning

Poetry manages its own virtual environments (in ~/.cache/pypoetry/) by default. Run poetry config virtualenvs.in-project true to create .venv inside the project directory instead.

Conda — Environments for Data Science

conda is a cross-platform package and environment manager popular in data science. Unlike pip, it manages non-Python dependencies (CUDA, BLAS, compilers) and resolves dependencies across languages. It ships with Anaconda (full distribution) or Miniconda (minimal bootstrap).

terminal

Bash

1	# Create an environment with a specific Python version
2	conda create -n my-env python=3.11
3
4	# Activate and deactivate
5	conda activate my-env
6	conda deactivate
7
8	# Install packages (from conda-forge channel)
9	conda install numpy pandas scikit-learn
10
11	# Install with specific channel
12	conda install -c conda-forge pytorch
13
14	# List environments
15	conda env list
16
17	# Export environment (cross-platform)
18	conda env export > environment.yml
19
20	# Recreate from environment file
21	conda env create -f environment.yml
22
23	# Remove an environment
24	conda remove -n my-env --all

environment.yml

YAML

1	# environment.yml — conda environment specification
2	name: ml-project
3	channels:
4	- conda-forge
5	- defaults
6	dependencies:
7	- python=3.11
8	- numpy=1.26
9	- pandas=2.1
10	- scikit-learn=1.3
11	- matplotlib=3.8
12	- pytorch=2.1
13	- pip # pip inside conda for PyPI-only packages
14	- pip:
15	- transformers # installed via pip inside the conda env

ℹ

info

Install Miniconda (not Anaconda) — it's faster and you install only what you need. Pin channels and versions in environment.yml for reproducible data science projects. Prefer conda-forge as the default channel.

.env Files & python-dotenv

Environment variables keep secrets (API keys, database URLs) out of your source code. The python-dotenv library reads key-value pairs from a .env file and loads them into os.environ.

terminal

Bash

1	# Install python-dotenv
2	pip install python-dotenv

.env

Bash

1	# .env — NEVER commit this file!
2	DATABASE_URL=postgresql://user:pass@localhost:5432/db
3	SECRET_KEY=super-secret-key-123
4	API_KEY=sk-proj-abc123
5	DEBUG=false
6	REDIS_URL=redis://localhost:6379/0
7
8	# Comments start with #
9	# Use quotes for values with spaces or special chars:
10	GREETING="Hello, World!"

load_env.py

Python

1	# Load .env in your application
2	from dotenv import load_dotenv
3	import os
4
5	load_dotenv() # loads .env from current directory
6
7	# Now access via os.environ (or os.getenv)
8	db_url = os.getenv("DATABASE_URL")
9	secret = os.getenv("SECRET_KEY")
10	debug = os.getenv("DEBUG", "false").lower() == "true"
11
12	# Specify a custom path
13	load_dotenv(".env.local")
14
15	# Override existing env vars (off by default)
16	load_dotenv(override=True)

⚠

warning

Never commit .env files. Add .env to your .gitignore. Instead, commit a .env.example with placeholder values so other developers know what variables are required.

.env.example

Bash

1	# .env.example — committed to git, serves as documentation
2	DATABASE_URL=postgresql://user:pass@localhost:5432/db
3	SECRET_KEY=change-me-in-production
4	API_KEY=
5	DEBUG=false
6	REDIS_URL=redis://localhost:6379/0
7
8	# Patterns for different environments:
9	# .env — loaded by default
10	# .env.local — local overrides (gitignored)
11	# .env.production — CI/CD injects these
12	# .env.example — template (committed)

Best Practices

Choosing an environment management strategy depends on your team size, deployment target, and project complexity. These recommendations cover the most common scenarios.

ℹ

info

For most projects: Use venv + requirements.in / pip-tools. It is simple, standard-library-based, and works everywhere Python runs. If you want more speed, replace pip-tools with uv.

terminal

Bash

1	# Recommended project layout:
2	my-project/
3	├── .venv/ # virtual environment (gitignored)
4	├── .env # local secrets (gitignored)
5	├── .env.example # template for required vars (committed)
6	├── .gitignore # includes .venv, .env, __pycache__, etc.
7	├── .python-version # pyenv local version (committed)
8	├── requirements.in # direct dependencies (committed)
9	├── requirements.txt # pinned lock file (committed)
10	├── requirements-dev.in # dev dependencies
11	├── requirements-dev.txt # dev lock file
12	├── pyproject.toml # project metadata + tool config
13	└── src/ # application code

Guidelines by scenario:

Quick scripts / simple tools — venv + requirements.txt
Team project / production app — venv + pip-tools or uv with lock files
Multiple Python versions — pyenv to manage versions, venv per project
Data science / ML — conda for CUDA/non-Python deps
Library publishing — poetry for seamless build + publish
Large monorepo — uv workspace or poetry with dependency groups

.gitignore

Bash

1	# .gitignore essentials:
2	.venv/
3	.env
4	__pycache__/
5	*.pyc
6	*.pyo
7	.pytest_cache/
8	.mypy_cache/
9	.ruff_cache/
10	dist/
11	*.egg-info/
12
13	# Keep committed:
14	# requirements.in (or pyproject.toml)
15	# requirements.txt (lock file)
16	# .python-version
17	# .env.example

ℹ

info

Whichever tool you choose, commit the lock file. It makes builds deterministic, enables dependency auditing (e.g., pip-audit or safety), and lets you roll back to a known good state. The lock file is the source of truth for what runs in production.

$Blueprint — Engineering Documentation·Section ID: PYTHON-ENV·Revision: 1.0

X GitHub YouTube Instagram Facebook Buy me a coffee Sponsor