I share this perspective having been in this space since 2015 with my first AI company - Mighty AI (acquired by Uber), and firmly believe that while the barrier to enter the market for data annotation is relatively low, not all players in the space are created equal. It spans the gamut from a few people in an office, to thousands of humans in offices dedicated to a specific task for an enterprise. But in ADAS/AV, it’s imperative that your partner has thought through the requirements to enter this challenging space, and that they truly understand the nuanced details needed to support arguably one of the most challenging tasks in computer vision.
What we’ve seen over time is a constant battle internally of build vs. buy, and in my opinion it’s not really that hard of a choice. I understand why companies want to build, but what I believe gets overlooked is that dataset management or annotation aren’t necessarily core to what these companies are trying to build. It’s absolutely a component of it, but at some point in my lifetime, my assumption is that we will get to a level of automation where humans aren’t needed to process the raw data being generated, in which do companies want to take their eyes and resources off of the end goal to build out a data management / annotation system to solve this challenge. I say, find the right partner (obviously 🙂).
If you’re going to partner, choosing the right one is crucial and what key features to look for:
Beyond annotations: While many platforms excel at basic image annotation, sensor fusion demands more. Non-negotiables to think through on how handle and manage are:
- Multimodal Data: LiDAR point clouds, radar returns, camera images, and more – the platform should seamlessly support diverse data types, at scale.
- 3D Annotations: Annotating objects and their spatial relationships in 3D space is crucial for accurate perception. Choose a platform with 3D annotation tools like cuboids, bounding boxes, and semantic segmentation for various sensor outputs.
- Temporal Tracking: Objects move, and the platform should be able to track them across different sensor data streams and timeframes.
- Technological integration: Tremendous strides have been made over the years on using technology to drive efficiency in data annotation work. While the holy grail is to automate the entire pipeline, that’s not possible, so human-in-the-loop is still needed to meet the quality demands of ADAS/AV use cases. However, there are various models that can be integrated into a tool set that can exponentially increase the efficiency of the humans generating the ground-truth dataset.
Accuracy is Paramount: Errors in annotations have disastrous consequences. Ensure that whatever platform you chose offers:
- Active Learning & Quality Control: Leverage AI-powered suggestions and human-in-the-loop validation to minimize errors and ensure consistent quality.
- Domain Expertise: Look for a platform with annotators experienced in ADAS and AV datasets, familiar with industry-specific objects and scenarios.
Collaboration is Key: Annotation is often a team effort. The platform should enable:
- Granular Access Control: Assign different roles and permissions to team members based on their needs.
- Seamless Communication: Facilitate efficient communication and collaboration between annotators and project managers.
- Integrations: Integrate with your existing development and data management tools for a streamlined workflow.
Security and Scalability Matter:
- Data Security: Ensure the platform complies with industry standards and regulations to protect sensitive data.
- Scalability: As your datasets and annotation needs grow, choose a platform that can scale efficiently to accommodate your expanding requirements.
The right data annotation platform is an investment, not just an expense. By considering these key features and prioritizing quality, you ensure your sensor fusion datasets become the foundation for safe, reliable, and performant ADAS and AV systems, paving the way for a future of safe, intelligent mobility.
