Production MLOps for a Dual-Head Vision Model (Detection + Segmentation)
Training a dual-head vision model is the easy part. We built the production MLOps ecosystem — on a massive Azure Databricks cluster — to tune it across competing objectives, scale distributed trials, visually debug it, and keep the training data legally clean for commercial use. The same system also powers real-time detection and anonymization of people in the video feed.
The challenge
For a heavy-industry client we had to deploy a custom multi-objective vision model — simultaneous object detection and semantic segmentation — at scale, not just in a notebook. Building the model is only about 20% of the job; the other 80% is wrestling cloud infrastructure, balancing competing mathematical objectives, and building the evaluation tools to actually prove it works.
Our approach
Multi-objective tuning without a weakest link
Standard hyperparameter tuning breaks with two heads — blindly optimize bounding-box mAP and segmentation F1 tanks. We built a distributed Ray Tune pipeline on Databricks with custom composite scoring (geometric mean + harmonic F1) that forced Optuna to penalize the weakest link and balance both tasks.
Dynamic config at scale
Scaling distributed trials means generating thousands of unique configurations, and standard CLI overrides break on parameters nested deep in list structures (like augmentation probabilities). We engineered a custom syntax parser to inject those nested overrides into training configs at scale.
Visual debugging for stakeholders (Voxel51)
Aggregate metrics are liars — a high global mAP won’t tell you why the model fails on specific frames. We integrated Voxel51 into the pipeline and computed per-image mIoU directly into the metadata, so even non-technical stakeholders could filter for false positives and visually debug edge-case failures.
Commercially-clean training data
You can’t just scrape the internet for training data. We navigated a minefield of restrictive, non-commercial dataset licensing — leaning on transfer learning and heavily filtered, permissively-licensed hybrid datasets — to keep the whole pipeline legally compliant for commercial enterprise use without sacrificing accuracy.
The outcome
The result is a production-grade MLOps ecosystem — Ray, MLflow, Optuna, and Voxel51 on Azure Databricks — that tunes and balances a multi-objective model, scales distributed trials reliably, and gives stakeholders visual proof it works, all on a training-data pipeline that is legally clean for commercial use.
Technology
Everyone loves to talk about the latest vision models, but building the model is only 20% of the job. We don’t just train custom models — we engineer the distributed systems that make them production-ready.