One Model is a Toy: Designing a Database-Driven ML Platform for N Models

The moment v1 stops being enough is the exact moment you realize your inference script is hardcoded to one model's features. To add a second model, you have to fork the container. That is not a platform; that is a script. Deleting the hardcoded factory pattern and moving to a database-driven inference engine is what enables real operational scale.

In Part 1, I built a reliable, single-model infrastructure using AWS CDK, Feast, and MLflow. It worked perfectly for one customer churn model. But when it came time to support multi-model batch scoring, the cracks in the single-model paradigm became obvious.

This post documents the architectural pivot from a hardcoded codebase to a dynamic, multi-model orchestration engine.

The Problem with Hardcoded Abstractions

When engineers move from one model to two, the first instinct is to build a class hierarchy. You create a BaseMLModel factory, then subclass it into ChurnModel and FraudModel. You hardcode the Feast feature dependencies, the expected label column, and the S3 data paths directly into the Python class definition.

I initially built exactly that factory pattern. And then I deleted it completely.

In a mature platform, model logic (like scikit-learn Pipelines) should not be hardcoded into the platform's repository as Python classes. The platform should simply act as an execution engine. It should retrieve the serialized pipeline from MLflow, read the required feature definitions from a database, and execute the scoring. By keeping the model logic inside the codebase, any change to a model's feature requirements forces a pull request, a code review, and a full infrastructure redeployment.

That is an unacceptable operational bottleneck. The solution is to remove the model definition from the codebase and put it into a database.

System Architecture: The Model Catalog

To decouple the execution engine from the model definitions, I introduced a Model Catalog. This is a PostgreSQL database managed via SQLModel and FastAPI, serving as the definitive source of truth for orchestration rules.

Loading architecture diagram...

The catalog does not replace MLflow. MLflow remains the authority on artifact storage and version lineage. Instead, the catalog manages the operational metadata that MLflow ignores. The most critical piece of that metadata is the feature_refs column.

By storing a comma-separated string of required Feast features directly in the database (customer_features:age, customer_features:account_balance), the platform knows exactly what data the model requires before it ever loads the artifact.

Database-Driven Inference

With the catalog in place, I rewrote the batch inference entrypoint (predict.py) to be fully dynamic. The ECS Fargate task receives a single environment variable: MODEL_NAME. Everything else is resolved at runtime.

Here is the exact code that fetches the metadata and retrieves the features:

python

# predict.py: dynamically fetching features based on catalog metadata
def _fetch_model_metadata(config: InferenceConfig) -> Model:
    """Fetch the model configuration (including feature_refs) from the Catalog DB."""
    engine = create_engine(config.database_url, echo=False)
    with Session(engine) as session:
        statement = select(Model).where(Model.model_name == config.model_name)
        model = session.exec(statement).first()
        if not model:
            raise ValueError(f"Model '{config.model_name}' not found in the Catalog database.")
        return model

def _retrieve_features(
    entity_df: pd.DataFrame, store: FeatureStore, model: Model
) -> pd.DataFrame:
    # feature_refs is a comma-separated string from the DB
    feature_list = [f.strip() for f in model.feature_refs.split(",") if f.strip()]
    
    job = store.get_historical_features(
        entity_df=entity_df,
        features=feature_list,
    )
    return job.to_df()

Once the features are retrieved, the script leverages MLflow's pyfunc interface to load the production model using the @champion alias.

python

# predict.py: framework-agnostic execution
def _run_inference(
    feature_df: pd.DataFrame, config: InferenceConfig, model: Model,
) -> list[PredictionRecord]:
    model_uri = f"models:/{config.model_name}@champion"
    loaded_model = mlflow.pyfunc.load_model(model_uri)

    drop_cols = ["entity_id", "event_timestamp"]
    X = feature_df.drop(columns=[c for c in drop_cols if c in feature_df.columns])
    
    raw_scores = loaded_model.predict(X)
    # ... wraps predictions in Pydantic records and returns them

The inference service does not need to know whether the champion model is a random forest, a gradient booster, or a neural network. It simply queries the database for the required features, asks Feast for the data, asks MLflow for the executable pipeline, and runs the prediction.

Key Takeaways: From Script to Platform

This architectural pivot yields three major operational unlocks:

Action	Previous Single-Model Paradigm	New Database-Driven Paradigm
Adding a new model	Requires a PR to add a new class, followed by a CDK redeployment.	Requires a single `POST /v1/models` API call. Zero infrastructure changes.
Updating feature dependencies	Requires modifying the `feature_refs` constant in Python and redeploying the container.	Requires a `PUT` request to update the database row. Takes effect on the next scheduled run.
Framework agnostic execution	Hardcoded logic binds the platform to a specific training framework.	`mlflow.pyfunc` provides a uniform execution abstraction across all models.

The Ugly Part: Local Development Constraints

Building cloud-native infrastructure is excellent for production, but replicating it locally can destroy a standard developer laptop. During this phase, I attempted to run the PostgreSQL database, the MLflow tracking server, and the FastAPI catalog locally using docker-compose.

The result was an immediate system crash. My machine hit its 16GB RAM limit, causing Docker to hang indefinitely.

To unblock development, I abandoned the Docker approach and built scripts/run_local.py, a zero-Docker Python subprocess manager. Instead of spinning up heavy containers, this script boots the FastAPI server and the MLflow Tracking Server natively, utilizing local SQLite database files (sqlite:///local_catalog.db and sqlite:///mlruns.db). This ultra-lightweight environment consumes fewer than 100MB of RAM, proving that you do not need a heavy containerized stack to validate orchestration logic locally.

Conclusion: Where We Go Next

We have successfully transitioned from a single-model script to a multi-model batch scoring platform. The platform now holds the metadata necessary to batch score, trace, and monitor several models concurrently.

The next evolution is real-time serving. In Part 3, we will integrate BentoML to provide dynamic process isolation for online serving, retrieving features in real-time from Feast's DynamoDB online store.

The Repository & Decision Log

The complete source code and infrastructure for this multi-model architecture are open-source.

Source Code: github.com/danivpv/ml-platform
Decision Log: docs/ml-platform-prd.md

If you are building a second model into a single-model codebase, I highly recommend reviewing the coupling table in the repository's road-to-prod.md document before you begin.