Safety has always been fundamental in safety-critical industries, especially automotive. For autonomous systems to gain public trust, they must demonstrate safety, reliability, and transparency. This requires certifiable, traceable algorithms validated with high-quality human feedback.
Technical teams face significant challenges throughout the machine learning lifecycle when developing safe perception systems. While these obstacles provide valuable learning opportunities, the right approach and tools can make the journey more efficient. At Kognic, we believe machines learn faster with human feedback—and that integrating cost-efficient, scalable human feedback into your data flywheel is key to success.
Developing ML models for perception is inherently iterative, with several interconnected phases. You'll move between requirements definition, data management, model development, testing and verification, and deployment—often multiple times before reaching production readiness.
Below, we outline common pitfalls in each phase and how teams can minimize unwanted outcomes while maximizing the value of human feedback in their workflows.
Success begins with solid foundations: defining your operational design domain (ODD) and establishing clear metrics (KPIs or SPIs) to evaluate model performance.
Most critical are annotation guidelines. Since machine learning is "programming by example," the quality of your human feedback directly determines what your model learns. Guidelines define whether your system distinguishes between fire trucks and tow trucks, or recognizes pedestrian intent at crossings.
Poor foundations can lead to:
The solution: Invest time in thoughtful requirements and comprehensive guidelines. Learn more about effective guideline development.
This phase transforms specifications into usable datasets through data collection, selection, and annotation. Whether using real-world data, simulation, or both, ensuring your data represents the ODD and follows guidelines is essential. Advanced techniques like active learning help focus human feedback on the most informative examples.
Common challenges include:
Monitor closely: Track data coverage and annotation quality throughout the process. At Kognic, we help teams get the most annotated autonomy data for their budget through our unique combination of Platform, People, and Process.
This is where machine learning happens. Set aside validation data, start simple, iterate systematically, and compare against baselines. Standard ML best practices apply here. 🚀
Does your model meet expectations? Testing reveals performance on fresh data and helps identify overfitting. Even if you've achieved the best possible KPIs, ask: are they good enough for my application? Consider broader aspects beyond your initial metrics, such as common sense tests.
Successful testing produces a verified model ready for deployment. However, you may discover needs for additional data, better ODD coverage, improved annotations, or guideline adjustments—requiring iteration back to earlier phases.
Take this phase seriously to avoid unpleasant surprises later.
With verification complete, it's time for MLOps: integration, deployment, and monitoring. This phase also presents opportunities to collect new data and retrain models with expanded human feedback.
Potential deployment challenges:
Mitigation strategies: Deploy on appropriate platforms, implement ODD-exit detection mechanisms, and always evaluate performance after retraining before deployment.
Developing ML models for safe autonomy requires time, resources, skilled teams, and the right tooling.
The real challenge isn't just building an ML model—it's building an integrated system that operates reliably in production. The iterative nature of perception development means continuous refinement across all phases.
These efforts are essential for building systems that society can trust. We have a responsibility to test, identify issues, iterate, and ultimately demonstrate safety before launching autonomous products.
At Kognic, we're focused on the multi-modal, real-world data that trains and validates autonomous systems. Our platform is built on the principle that human feedback accelerates machine learning—particularly the expert judgment required for complex 3D sensorfusion data. By making human feedback as productive as possible, we help teams develop safer, more reliable autonomous systems faster and more cost-effectively.
For those interested in learning more about ML lifecycle management, here are valuable resources:
https://developers.google.com/machine-learning/guides/rules-of-ml
https://arxiv.org/abs/2108.02497