Series: The AI Engineer's Path. This is Part 1 of 10. Each part builds on the last, ending with a full self-hosted RAG stack you've trained, quantized, and deployed yourself.
Hardware required for this part: Any laptop. No GPU needed.
Python version: 3.11+.
Why Most "Python for AI" Tutorials Miss the Point
Most "Python for AI" tutorials teach Python like it's a scripting language. AI engineering is software engineering with AI components. The parts of Python that matter are the parts that hold up under production load: type hints, async, context managers, dependency management, tests. Without those, your "AI app" is a notebook that breaks the moment a second user shows up.
This post is the foundation the rest of the series stands on. We won't touch an LLM in this part — that's Part 2. Here we build a CLI tool that processes text. Then Parts 2 through 10 extend the same project: an LLM call, a FastAPI service, embeddings, RAG, fine-tuning, quantization, serving, distillation, and finally scaling.
What You'll Build
A CLI tool, text_tool.py, that reads a text file and prints word-frequency statistics. Trivial as a project, but the scaffolding around it — structure, types, errors, tests — is the same scaffolding every part of the series uses.
Project Structure
Every project I ship looks like this:
ai-engineer-path/
├── pyproject.toml
├── src/
│ └── text_tool/
│ ├── __init__.py
│ ├── cli.py
│ └── processing.py
└── tests/
└── test_processing.pyUse uv for dependency management. It's faster than pip and the tooling is sharper. Install once:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv init ai-engineer-path
cd ai-engineer-path
uv add --dev pytest==8.4.0Your pyproject.toml ends up like this:
[project]
name = "ai-engineer-path"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = []
[dependency-groups]
dev = ["pytest==8.4.0"]
[tool.pytest.ini_options]
pythonpath = ["src"]
testpaths = ["tests"]Type Hints — Use Them Everywhere
Type hints are not bureaucracy. They're documentation that the language can check. Modern Python (3.11+) lets you write them naturally:
from __future__ import annotations
from typing import TypedDict
class WordCount(TypedDict):
word: str
count: int
def top_words(text: str, n: int = 10) -> list[WordCount]:
"""Return the top-n most frequent words in text."""
from collections import Counter
words = text.lower().split()
counts = Counter(words).most_common(n)
return [{"word": w, "count": c} for w, c in counts]
def is_empty(text: str | None) -> bool:
return text is None or text.strip() == ""from __future__ import annotations makes all annotations evaluated lazily, which lets you use modern syntax (list[str], str | None) on every supported Python version without quotes. Use it on every file.
Dataclasses vs Pydantic v2 — Rule of Thumb
Pydantic at API boundaries. Dataclasses internally.
Pydantic validates and coerces. That's expensive but necessary anywhere data crosses a trust boundary — HTTP request body, LLM JSON output, config file. Dataclasses are cheap and structural. Use them inside your own code where you control the inputs.
from dataclasses import dataclass
from pydantic import BaseModel, Field
# Internal state — no validation needed, your code controls it
@dataclass
class ProcessedDocument:
word_count: int
top_words: list[tuple[str, int]]
source_path: str
# API boundary — validates input from the outside world
class ProcessRequest(BaseModel):
file_path: str = Field(..., description="Path to the text file")
top_n: int = Field(10, ge=1, le=100)
lowercase: bool = TrueWe'll use ProcessRequest for real in Part 3 when we put a FastAPI in front of this code.
Context Managers — Beyond File Handles
The pattern with open(...) as f: guarantees the file closes even if an exception fires. The same pattern matters for anything with a finite lifetime: HTTP clients, GPU memory, database connections.
from pathlib import Path
from contextlib import contextmanager
import time
def read_text(path: Path) -> str:
"""Read a UTF-8 text file. Closes the file even on error."""
with open(path, encoding="utf-8") as f:
return f.read()
@contextmanager
def timed(label: str):
"""Time a block of code. Use as: `with timed("loading"): load_thing()`."""
start = time.perf_counter()
try:
yield
finally:
elapsed = time.perf_counter() - start
print(f"[{label}] {elapsed:.3f}s")
# Usage
with timed("file read"):
content = read_text(Path("input.txt"))You'll see the timed pattern again in Part 2 around LLM calls and Part 8 around vLLM startup.
Async — Just Enough for Part 2
I'm not going to over-explain async. The thing to internalise: any code that spends time waiting on I/O (network, disk, an LLM API call) should be async. The thread doesn't sit idle — it goes off and does other work until the response arrives. For LLM workloads, this is huge: ten requests in flight cost the same wall-clock as one.
import asyncio
async def fake_llm_call(prompt: str) -> str:
"""Pretend to be an LLM. Sleep for 1s, return a string."""
await asyncio.sleep(1.0)
return f"Echo: {prompt}"
async def main() -> None:
# These run concurrently — total wall-clock is ~1s, not 3s
results = await asyncio.gather(
fake_llm_call("first"),
fake_llm_call("second"),
fake_llm_call("third"),
)
for r in results:
print(r)
if __name__ == "__main__":
asyncio.run(main())Run it. You'll see three lines printed after roughly one second — not three. That's the entire reason async matters for AI work.
Errors — Custom Exceptions, Not String Comparisons
If you find yourself writing if "rate limit" in str(e):, you've already lost. Define exception classes. Catch by class. Log structured.
import logging
logger = logging.getLogger(__name__)
class TextToolError(Exception):
"""Base error for this tool."""
class FileNotReadable(TextToolError):
"""Raised when the input file can't be read."""
def safe_read(path: str) -> str:
try:
with open(path, encoding="utf-8") as f:
return f.read()
except FileNotFoundError as e:
logger.error("file not found", extra={"path": path})
raise FileNotReadable(f"no such file: {path}") from e
except UnicodeDecodeError as e:
logger.error("not utf-8", extra={"path": path})
raise FileNotReadable(f"file is not valid UTF-8: {path}") from eUse logging, not print. Logs you can filter, aggregate, search; print you can only stare at. The extra={...} dict is the seed of structured logging, which Part 10 expands into structlog.
The Complete CLI Tool
Save this as src/text_tool/cli.py:
"""text_tool: a CLI that prints word-frequency stats for a text file."""
from __future__ import annotations
import argparse
import logging
import sys
from pathlib import Path
from collections import Counter
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
)
logger = logging.getLogger(__name__)
class TextToolError(Exception):
"""Base error."""
class FileNotReadable(TextToolError):
"""Input file unreadable."""
def read_text(path: Path) -> str:
try:
with open(path, encoding="utf-8") as f:
return f.read()
except FileNotFoundError as e:
raise FileNotReadable(f"no such file: {path}") from e
except UnicodeDecodeError as e:
raise FileNotReadable(f"not valid UTF-8: {path}") from e
def top_words(text: str, n: int = 10) -> list[tuple[str, int]]:
"""Return the n most frequent lowercased words."""
words = [w.strip(".,!?;:\"'()[]") for w in text.lower().split()]
words = [w for w in words if w]
return Counter(words).most_common(n)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Word-frequency CLI.")
parser.add_argument("path", type=Path, help="UTF-8 text file to analyse")
parser.add_argument("-n", "--top", type=int, default=10, help="how many top words to print")
args = parser.parse_args(argv)
try:
text = read_text(args.path)
except FileNotReadable as e:
logger.error("read failed: %s", e)
return 2
for word, count in top_words(text, args.top):
print(f"{count:6d} {word}")
return 0
if __name__ == "__main__":
sys.exit(main())Try it:
echo "the quick brown fox jumps over the lazy dog the fox" > sample.txt
uv run python -m text_tool.cli sample.txt -n 5Output:
3 the
2 fox
1 quick
1 brown
1 jumpsTests — pytest, Not "I Ran It Once"
Save this as tests/test_processing.py:
from pathlib import Path
import pytest
from text_tool.cli import top_words, read_text, FileNotReadable
def test_top_words_basic():
text = "the the the quick brown fox"
result = top_words(text, n=2)
assert result == [("the", 3), ("quick", 1)]
def test_top_words_strips_punctuation():
text = "Hello, hello! Hello?"
result = top_words(text, n=1)
assert result == [("hello", 3)]
def test_top_words_empty_string():
assert top_words("", n=5) == []
def test_read_text_missing_file_raises(tmp_path: Path):
with pytest.raises(FileNotReadable):
read_text(tmp_path / "does-not-exist.txt")
def test_read_text_returns_content(tmp_path: Path):
p = tmp_path / "input.txt"
p.write_text("hello world", encoding="utf-8")
assert read_text(p) == "hello world"Run them:
uv run pytest -vYou'll see five passing tests. The tmp_path fixture is one of pytest's quiet wins — it gives every test its own throwaway directory, so your tests can write real files without polluting the repo or stepping on each other.
Key Takeaways
- Type hints are documentation the language checks. Use
from __future__ import annotationsand modern syntax (list[str],str | None) on every file - Pydantic at trust boundaries (HTTP, LLM JSON, config). Dataclasses inside your own code
- Context managers for everything with a finite lifetime — not just files
- Async whenever you wait on I/O. LLM calls are I/O. Most of your code in this series will be async
- Custom exception classes, never string-matching on error messages.
logging, neverprint - Tests with pytest, with
tmp_pathfor filesystem work. If a function is worth writing, it's worth one test
Next Up
Part 2 takes this CLI and gives it an LLM. We'll build a provider-agnostic client wrapper with retries, streaming, structured output, and cost tracking — the module the rest of the series imports.