|$ curl https://forge-ai.dev/api/markdown?path=docs/ai/llm-fundamentals

$cat docs/llm-fundamentals.md

updated Recently·40 min read·published

LLM Fundamentals

◆AI Engineering◆LLMs◆Beginner to Intermediate

Introduction

Large Language Models (LLMs) are deep neural networks trained on enormous text datasets to understand and generate human-like text. They are the foundation of modern AI applications, powering everything from chatbots to code assistants to document analysis systems.

This section explains what LLMs are, how they work at a high level, how they are trained, what they can and cannot do, and how to choose the right model for your application.

📝

note

LLMs are not "intelligent" in the human sense. They are next-token prediction engines that have learned statistical patterns from text. Understanding this distinction is crucial for building reliable applications.

What Are LLMs?

An LLM is a type of neural network trained to predict the next token (word, subword, or character) given a sequence of previous tokens. What makes them "large" is the scale: billions of parameters trained on trillions of tokens from the public internet, books, and other sources.

At inference time, the model generates text autoregressively — it predicts one token at a time, feeding each new token back into the input until it reaches a stopping condition (like reaching a maximum length or generating an end-of-sequence token).

autoregressive-generation.py

Python

1	# The core of LLM inference is autoregressive generation
2	def generate(model, prompt, max_tokens=100):
3	tokens = tokenize(prompt)
4	for _ in range(max_tokens):
5	# Model predicts next token probability distribution
6	logits = model.forward(tokens)
7	next_token = sample_from_distribution(logits[-1])
8
9	# Append and check for stopping
10	tokens.append(next_token)
11	if next_token == EOS_TOKEN:
12	break
13
14	return detokenize(tokens)
15
16	# In practice, you use library APIs:
17	from openai import OpenAI
18	client = OpenAI()
19
20	response = client.chat.completions.create(
21	model="gpt-4o",
22	messages=[{"role": "user", "content": "Hello!"}],
23	max_tokens=50,
24	temperature=0.7
25	)
26	print(response.choices[0].message.content)

🔥

pro tip

The temperature parameter controls randomness in generation. Low values (0.0-0.3) produce deterministic, focused outputs ideal for factual tasks. High values (0.7-1.0) produce creative, varied outputs. Never use temperature above 1.0 — it produces incoherent text.

Transformer Architecture (High-Level)

Almost all modern LLMs are based on the Transformer architecture, introduced in the landmark 2017 paper "Attention Is All You Need" by Vaswani et al. The core innovation is the attention mechanism, which allows the model to weigh the importance of different tokens in the input when producing each output token.

The Attention Mechanism

Attention computes a weighted sum of values based on the similarity between queries and keys. For each token, the model can "attend to" other tokens in the sequence, learning long-range dependencies that were difficult for previous architectures like RNNs and LSTMs.

attention.py

Python

1	# Simplified scaled dot-product attention
2	import numpy as np
3
4	def attention(query, key, value):
5	"""
6	query: (batch, seq_len, d_k)
7	key: (batch, seq_len, d_k)
8	value: (batch, seq_len, d_v)
9	"""
10	d_k = query.shape[-1]
11
12	# Compute attention scores
13	scores = np.matmul(query, key.transpose(0, 2, 1))
14	scores = scores / np.sqrt(d_k) # Scale
15
16	# Apply softmax to get attention weights
17	weights = np.exp(scores) / np.sum(np.exp(scores), axis=-1, keepdims=True)
18
19	# Weighted sum of values
20	output = np.matmul(weights, value)
21	return output, weights

preview

Key Components

Component	Purpose	Detail
Self-Attention	Contextual token relationships	Each token attends to all others in the sequence
Feed-Forward	Non-linear transformation	MLP that processes each token independently
Layer Norm	Training stability	Normalizes activations across the feature dimension
Positional Encoding	Token order information	Sine/cosine or learned embeddings for position
Residual Connections	Gradient flow	Skip connections that prevent vanishing gradients
Multi-Head	Parallel attention patterns	Multiple attention heads capture different relationships

ℹ

info

You do not need to implement transformers from scratch to use LLMs effectively. But understanding the architecture helps you debug unexpected behavior, optimize prompts, and make informed model choices. The key takeaway: attention is what gives LLMs their contextual understanding.

Training Process

Training a modern LLM is an enormous engineering undertaking that proceeds through three main stages. Understanding these stages helps you choose between base models and chat models, and informs decisions about fine-tuning.

Stage 1: Pre-Training

The model is trained on a massive, diverse corpus of internet text (trillions of tokens) using self-supervised learning. The objective is simple: predict the next token. This stage gives the model broad knowledge of language, facts, and reasoning patterns. Pre-training costs tens of millions of dollars and requires thousands of GPUs running for weeks or months.

pretraining-data.txt

TEXT

1	Pre-training dataset composition (typical):
2
3	- CommonCrawl (filtered) — 60% — Web pages
4	- Books (fiction + non) — 15% — Long-form text, narrative
5	- Wikipedia — 10% — Factual, structured content
6	- Academic papers — 10% — Technical, scientific text
7	- Code (GitHub) — 5% — Programming languages
8
9	Total: ~10-15 trillion tokens
10	Model size: 7B to 405B parameters
11	Compute: Thousands of GPU-months

⚠

warning

Base (pre-trained) models are not instruction-following. They excel at continuation and completion but will not answer questions or follow commands. You need a chat or instruct model for interactive applications. This is why you should never deploy a raw pre-trained model as a chatbot.

Stage 2: Fine-Tuning (Supervised)

The pre-trained model is fine-tuned on curated datasets of instruction-input-output triples. This teaches the model to follow instructions, answer questions, and maintain a conversation. Models like Llama-3-Instruct, GPT-4o, and Claude have gone through this stage.

fine-tuning.py

Python

1	# Conceptual fine-tuning data format
2	training_data = [
3	{
4	"instruction": "Explain what a transformer is",
5	"input": "",
6	"output": "A transformer is a neural network architecture that uses self-attention to process sequential data..."
7	},
8	{
9	"instruction": "Write a Python function",
10	"input": "Calculate the Fibonacci sequence up to n terms",
11	"output": "def fibonacci(n):\n a, b = 0, 1\n for _ in range(n):\n print(a)\n a, b = b, a + b"
12	},
13	]
14
15	# Fine-tuning with LoRA (efficient method)
16	from peft import LoraConfig, get_peft_model
17	from transformers import AutoModelForCausalLM
18
19	model = AutoModelForCausalLM.from_pretrained("llama-3-8b")
20	lora_config = LoraConfig(
21	r=16,
22	lora_alpha=32,
23	target_modules=["q_proj", "v_proj"],
24	lora_dropout=0.05
25	)
26	model = get_peft_model(model, lora_config)

Stage 3: RLHF / Preference Alignment

Reinforcement Learning from Human Feedback (RLHF) or Direct Preference Optimization (DPO) aligns the model with human preferences. The model learns to prefer helpful, honest, and harmless responses over unhelpful or harmful ones. This is what makes models like ChatGPT and Claude feel safe and cooperative.

dpo-loss.py

Python

1	# DPO loss function (simplified)
2	import torch.nn.functional as F
3
4	def dpo_loss(chosen_logps, rejected_logps, beta=0.1):
5	"""
6	chosen_logps: log probs of preferred response
7	rejected_logps: log probs of dispreferred response
8	beta: temperature parameter for the policy
9	"""
10	log_ratio = chosen_logps - rejected_logps
11	loss = -F.logsigmoid(beta * log_ratio).mean()
12	return loss
13
14	# Training data format: preferred vs rejected
15	preference_data = [
16	{
17	"prompt": "How do I hack a computer?",
18	"chosen": "I cannot provide instructions for hacking...",
19	"rejected": "Here is how to hack a computer..."
20	},
21	]

🔥

pro tip

DPO is simpler and more stable than RLHF for most fine-tuning use cases. It directly optimizes the policy on preference pairs without needing a separate reward model. If you are fine-tuning for preference alignment, start with DPO.

Capabilities & Limitations

Understanding what LLMs can and cannot do is essential for building effective applications. Overestimating model capabilities is the most common cause of AI project failure.

Capabilities

◆Text generation and completion across virtually any domain

◆Summarization of long documents into concise overviews

◆Translation between major languages with high fluency

◆Code generation, debugging, and explanation across programming languages

◆Creative writing: stories, poems, marketing copy, scripts

◆Question answering from provided context (with RAG for grounding)

◆Tool use and function calling for API integration

◆Reasoning and problem solving, especially with chain-of-thought

◆Classification, extraction, and structured output generation

Limitations

◆Hallucinations: models confidently produce false information

◆No true reasoning: they pattern-match, not reason logically

◆Knowledge cutoff: no awareness of events after training date

◆No inherent truthfulness: will agree with incorrect premises

◆Context window limits: cannot process arbitrarily long inputs

◆Biased outputs: reflect biases in training data

◆No causal understanding: correlation vs causation confusion

◆Mathematical unreliability: poor at arithmetic without tools

◆High computational cost and latency compared to traditional software

preview

✓

best practice

Design your AI system assuming the LLM will hallucinate, make mistakes, and contradict itself. Implement grounding (RAG), validation (structured outputs), and human-in-the-loop review for critical decisions. Never give an LLM unmonitored write access to your database.

Popular Models

The LLM landscape evolves rapidly. Here are the major model families as of 2026, their characteristics, and typical use cases.

Model	Developer	Size Range	Context Window	Best For
GPT-4o	OpenAI	Unknown (estimated ~1.8T MoE)	128K tokens	General purpose, vision, multilingual
Claude 3.5 Sonnet	Anthropic	Unknown (~200B estimated)	200K tokens	Long context, coding, nuanced reasoning
Llama 3.1 405B	Meta	8B, 70B, 405B	128K tokens	Self-hosting, customization, research
Gemini 2.0	Google	Flash, Pro, Ultra	1M+ tokens	Multimodal, massive context, Google ecosystem
Mistral Large	Mistral AI	7B, 8x22B, Large	128K tokens	European data sovereignty, efficiency
DeepSeek-V3	DeepSeek	671B MoE (37B active)	128K tokens	Cost-effective inference, strong math/code

Choosing a Model

Model selection depends on your specific requirements. Here is a decision framework:

Cost-Sensitive Applications

Use GPT-4o Mini, Claude Haiku, Mistral Small, or Llama 3.1 8B. These offer strong performance at a fraction of the cost of flagship models. Ideal for classification, extraction, and simple Q&A.

Complex Reasoning & Code

Use GPT-4o, Claude 3.5 Sonnet, or DeepSeek-V3. These excel at multi-step reasoning, complex code generation, and tasks requiring deep understanding. Higher cost but better reliability.

Self-Hosted / Privacy

Use Llama 3.1 405B, Mistral Large, or DeepSeek-V3 with vLLM or TGI for deployment. Complete data control, no API dependencies, but significant infrastructure costs.

Basic Inference with Different Providers

multi-provider-inference.py

Python

1	# OpenAI
2	from openai import OpenAI
3	client = OpenAI()
4	response = client.chat.completions.create(
5	model="gpt-4o",
6	messages=[{"role": "user", "content": "Hello!"}]
7	)
8
9	# Anthropic
10	from anthropic import Anthropic
11	client = Anthropic()
12	response = client.messages.create(
13	model="claude-3-5-sonnet-20241022",
14	max_tokens=1024,
15	messages=[{"role": "user", "content": "Hello!"}]
16	)
17
18	# Google Gemini
19	from google import genai
20	client = genai.Client()
21	response = client.models.generate_content(
22	model="gemini-2.0-flash",
23	contents="Hello!"
24	)
25
26	# Self-hosted (Ollama)
27	import requests
28	response = requests.post(
29	"http://localhost:11434/api/chat",
30	json={
31	"model": "llama3.1:70b",
32	"messages": [{"role": "user", "content": "Hello!"}]
33	}
34	)

ℹ

info

Do not lock yourself into a single provider. Design your application with a model abstraction layer so you can switch providers as pricing, performance, and capabilities evolve. Use libraries like LiteLLM that provide a unified interface across providers.

$Blueprint — Engineering Documentation·Section ID: AI-02·Revision: 1.0

X GitHub YouTube Instagram Facebook Buy me a coffee Sponsor