Skip to main content

Capstone Project - Type-Safe Configuration Manager

The Configuration Problem in Production​

Every real application needs configuration. When you deploy to production, you need different database credentials than your local development environment. Your API timeout settings change. Your logging level shifts from DEBUG to INFO. These values cannot be hardcoded in your source code—they belong in configuration files, environment variables, or command-line arguments.

But here's where it gets tricky: configuration is fragile. A typo in an environment variable name silently uses a default value instead of failing. Missing required settings crash the app hours into production rather than at startup. Different environments have different precedence rules, confusing developers about where values come from. Without type safety, you don't discover missing fields until runtime.

This is the capstone project: you'll build a production-quality ConfigManager that:

  • Loads configuration from multiple sources (YAML files, environment variables, CLI arguments)
  • Enforces type safety with Pydantic and Generics
  • Implements clear precedence rules (CLI overrides environment, environment overrides files)
  • Validates configuration on startup, failing fast if anything is wrong
  • Provides helpful error messages so developers know exactly what's misconfigured
  • Includes comprehensive tests proving it works in all scenarios

By the end, you'll have a portfolio-worthy project demonstrating mastery of Pydantic, Generics, and production engineering practices—something you can show in technical interviews or include on GitHub.

Section 1: Requirements and Architecture​

Before writing a single line of code, let's clarify what a production config system needs.

Functional Requirements (What it does)​

Your ConfigManager must:

  1. Load from YAML files — Read config.yaml, dev.yaml, or prod.yaml and parse structured data
  2. Load from environment variables — Allow overrides via APP_DATABASE_HOST, APP_LOG_LEVEL, etc.
  3. Load from CLI arguments — Accept --debug or --log-level=DEBUG to override everything else
  4. Merge with precedence — CLI args win over env vars, which win over file values, which win over defaults
  5. Validate everything — Ensure types, required fields, and constraints are satisfied

Non-Functional Requirements (How it must work)​

  1. Type-safe access — Use Generics so config.get[DatabaseConfig]("database") returns a typed object with full IDE autocomplete
  2. Fail fast — If config is invalid, crash at startup with a clear error, not 3 hours into production
  3. Testable — Unit tests can verify each loading strategy independently
  4. Documented — A user reading the code understands why each piece exists
  5. Secure — Never log passwords; handle secrets safely

Architecture Diagram​

┌─────────────────────────────────────────────────────────────┐
│                    ConfigManager                             │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  1. ConfigLoader                                             │
│     ├─ load_yaml() → dict                                    │
│     ├─ load_env() → dict                                     │
│     └─ load_cli() → dict                                     │
│                                                               │
│  2. Merge with Precedence                                   │
│     ├─ defaults (BaseModel field defaults)                  │
│     ├─ + YAML file values                                   │
│     ├─ + Environment variable values                        │
│     └─ + CLI argument values (highest priority)             │
│                                                               │
│  3. Pydantic Validation                                      │
│     └─ Validate merged dict against AppConfig model         │
│                                                               │
│  4. Generic[T] Wrapper                                       │
│     ├─ Type-safe access: config.get[DatabaseConfig]("db")   │
│     └─ IDE autocomplete on DatabaseConfig fields            │
│                                                               │
│  5. Return Validated AppConfig                               │
│     └─ App uses with confidence: no more type errors        │
│                                                               │
└─────────────────────────────────────────────────────────────┘

💬 AI Colearning Prompt​

"Compare Pydantic BaseSettings vs manually reading environment variables with os.getenv(). What are the tradeoffs of each approach?"

Design Decisions: Why This Architecture?​

Why Pydantic? Type hints alone don't enforce constraints. Pydantic validates at runtime, ensuring your configuration is actually correct before your app tries to use it.

Why BaseSettings? It automates the common pattern of "load from env vars with a prefix." Without it, you'd manually check os.environ.get("APP_DATABASE_HOST") for every single field.

Why Generic[T] wrapper? When you write config.get("database"), Python doesn't know what type you're getting back—is it a dict? A DatabaseConfig object? The Generic wrapper lets you specify the return type: config.get[DatabaseConfig]("database"), and your IDE gives you perfect autocomplete on all DatabaseConfig fields.

🎓 Expert Insight​

In AI-native development, configuration is your specification for deployment. When you use Pydantic for config, you're creating executable documentation: the schema IS the validation IS the type hints. This specification-as-code pattern scales from local development to production without translation layers.

Section 2: Defining Config Models​

Let's build the nested Pydantic models that describe your application's configuration.

Creating the DatabaseConfig Model​

Loading Python environment...

The env_prefix means environment variables like APP_DATABASE_HOST automatically map to the host field. This eliminates manual string matching and reduces typos.

Creating the APIConfig Model​

Loading Python environment...

Creating the Top-Level AppConfig Model​

Loading Python environment...

The env_nested_delimiter is key: it lets you set nested values from environment variables. APP_DATABASE__HOST=prod-db.example.com sets the database's host field without repeating the full path.

🤝 Practice Exercise​

Ask your AI: "Scaffold the three config models (DatabaseConfig, APIConfig, AppConfig) with realistic defaults and validation constraints. Add validation to ensure port is 1-65535, timeout is > 0, and log_level is one of 'DEBUG', 'INFO', 'WARNING', 'ERROR'."

Expected Outcome: You'll see how to structure nested configuration models with comprehensive validation, understanding how Field() constraints and validators work together to enforce business rules at the config layer.

Section 3: Multi-Source Loading​

Now implement the ConfigLoader that reads from all three sources and merges them with proper precedence.

Loading from YAML Files​

Loading Python environment...

Loading from Environment Variables​

Loading Python environment...

Loading from CLI Arguments​

Loading Python environment...

Merging with Precedence​

Loading Python environment...

The Complete load_config() Function​

Loading Python environment...

Section 4: Generic Type-Safe Access​

Now we add the ConfigValue[T] wrapper that provides type-safe configuration access with IDE autocomplete.

Why Type-Safe Access Matters​

Without Generics, when you retrieve a config subsection, Python doesn't know its type:

Loading Python environment...

With Generics, you make the type explicit:

Loading Python environment...

Implementing ConfigValue[T]​

Loading Python environment...

Adding get() Method to AppConfig​

Loading Python environment...

Using Type-Safe Access​

Loading Python environment...

Section 5: Error Handling and Validation​

Production systems must fail gracefully. Configuration errors should crash at startup with clear messages, not 3 hours into production.

Validating Required Fields​

Loading Python environment...

Logging Configuration Sources​

Loading Python environment...

Section 6: Testing​

A production system needs comprehensive tests. You can't deploy configuration code to production without proving it handles all scenarios.

Test Setup with Temporary Files​

Loading Python environment...

Testing YAML Loading​

Loading Python environment...

Testing Environment Variable Overrides​

Loading Python environment...

Testing Precedence Rules​

Loading Python environment...

Testing Validation Errors​

Loading Python environment...

Section 7: Project Deliverables​

Your capstone project should include all of these components:

Project Structure​

config-manager/
├── config_manager/
│   ├── __init__.py
│   ├── models.py           # DatabaseConfig, APIConfig, AppConfig
│   ├── loader.py           # load_yaml, load_env, load_cli, merge_configs
│   ├── manager.py          # ConfigManager class with get[T]() method
│   └── exceptions.py       # Custom exceptions
├── configs/
│   ├── dev.yaml           # Development configuration
│   ├── prod.yaml          # Production configuration
│   └── .env.example       # Example environment variables
├── tests/
│   ├── conftest.py        # Pytest fixtures
│   ├── test_yaml_loading.py
│   ├── test_env_loading.py
│   ├── test_precedence.py
│   ├── test_validation.py
│   └── test_integration.py
├── example_app.py         # Demo application using ConfigManager
├── README.md              # Project documentation
├── requirements.txt       # Dependencies (pydantic, pyyaml)
└── pytest.ini            # Pytest configuration

Example Configuration Files​

configs/dev.yaml:

debug: true
log_level: DEBUG
database:
  host: localhost
  port: 5432
  name: myapp_dev
  user: dev_user
  password: dev_password
api:
  base_url: http://localhost:8000
  timeout: 5
  retry_count: 1

configs/prod.yaml:

debug: false
log_level: INFO
database:
  host: prod-db.example.com
  port: 5432
  name: myapp_prod
  user: prod_user
  password: ${DB_PASSWORD}  # Load from env
api:
  base_url: https://api.example.com
  timeout: 30
  retry_count: 3

Example Application​

Loading Python environment...

Test Coverage​

Run tests with:

pytest tests/ -v --cov=config_manager

Aim for 90%+ test coverage of your ConfigManager code.

Common Mistakes to Avoid​

Mistake 1: Not Validating at Startup​

Loading Python environment...

Loading Python environment...

Mistake 2: Hardcoding Defaults in Code​

Loading Python environment...

Loading Python environment...

Mistake 3: Not Documenting Precedence​

Loading Python environment...

Loading Python environment...

Mistake 4: Overcomplicating the System​

Loading Python environment...

Lesson: Start simple. Add remote configs and secrets management only when you actually need them (that's your extension activity for B2+ students).

Try With AI​

Integrate Pydantic and generics into a complete type-safe configuration system through AI collaboration.

🔍 Explore System Architecture:

"Design config manager using BaseSettings for environment loading, nested Pydantic models for validation, generic ConfigLoader[T: BaseModel] for type safety. List required components and validation strategy."

🎯 Practice Config Validation:

"Build DatabaseConfig, APIConfig, FeatureFlags models with Pydantic. Create AppConfig composing them. Use BaseSettings with env_prefix, env_nested_delimiter. Validate complex constraints with @model_validator."

🧪 Test Generic Loading:

"Create generic ConfigLoader[T] that loads from .env, JSON, or YAML, validates with Pydantic model T, handles errors gracefully. Show type safety preserving T throughout."

🚀 Apply Production System:

"Build complete config management system: environment variable loading, file fallbacks, validation with clear errors, type-safe access, hot reload support. Reflect on Pydantic+generics enabling this architecture."