Quality Through Communication: How Human Feedback Accelerates Machine Learning

Ensuring Quality Through Communication: How Human Feedback Accelerates Machine Learning

Training safe autonomous systems requires high-quality annotated data. At Kognic, we believe machines learn faster with human feedback—and we've built our entire process around making that feedback as productive, scalable, and cost-efficient as possible.

As the leader in autonomy data annotation, we help customers integrate human feedback into their data pipelines through our unique combination of Platform, Process, and People. This is what enables us to deliver the most annotated autonomy data for your budget. But there's another crucial factor behind our success: continuous communication across every step of the quality assurance process.

In this article, we explore how our teams collaborate to ensure you receive reliable training data for safe perception systems—and why clear communication is essential to making this work smoothly at scale.

The Guideline Agreement Process: Aligning on Quality Expectations

The journey begins with understanding your use case. After meeting with our Sales team and establishing an account team, your requirements are handed to our Data Delivery team, who create a project execution plan. To enable successful deliveries, this team selects a Lead Quality Manager (LQM) for quality assurance and the most skilled BPO partner for annotation.

Meera Ranganathan, Data Delivery Manager, explains the importance of this phase: "Regular and clear communication makes a huge difference when working with teams across different parts of the world, especially for ensuring they understand our quality expectations." Despite the complexity, she describes the collaboration "as an absolute delight."

Next comes the Guideline Agreement Process. Our Perception Experts work closely with you to understand your use case and its implications. We guide you on which annotation approach will deliver the most value for your investment, then collaborate on formulating clear, unambiguous labeling guidelines. This reduces misinterpretation and identifies potential issues before annotation even begins.

Together with LQMs, our Perception Experts remove ambiguity in annotation guidelines as early as possible, setting the foundation for productive collaboration. By identifying which errors your use case is most sensitive to, we establish the right quality and cost balance. During this stage, communication between Perception Experts, LQMs, and customers is continuous—through Slack conversations or meetings. This ensures both parties can evaluate project quality efficiently in later phases.

Onboarding: Ensuring Consistent Human Feedback at Scale

After stakeholders agree on guidelines, the next challenge is ensuring knowledge transfers effectively to annotators. Human interpretation naturally varies, which can lead to inconsistency. To make perception systems reliable and autonomous vehicles trustworthy, our workforce must be aligned when providing human feedback on data.

The question: how do we ensure thousands of annotators provide consistent feedback?

We provide comprehensive training to annotation teams, and our LQMs and Quality Managers (QMs) support annotators with uncertainties, edge cases, and questions. This ensures the entire workforce understands the guidelines before production begins. The quality monitoring process identifies who needs additional training and who performs well—only qualified annotators move to production.

Anna Khmura, an experienced LQM, explains: "Before every project starts, we hold brief meetings with each BPO to explain how to work on tasks, expectations, and goals. Each mistake must be fixed according to guidelines, and quality-assured tasks should always be golden. Until I see that an annotator works without errors, I can't approve them for production tasks." We leave nothing to chance.

How do we maintain clear communication during onboarding? Beyond training sessions where experienced LQMs and QMs explain expectations in detail, these managers have continuous conversations with annotators who need help. At Kognic, transparency is in our DNA. This approach enables us to support all annotators during training and preparation. LQMs and QMs also gather common questions in a shared feedback document, so everyone benefits from collective learning.

As one QM describes it, everything works through effective communication. Providing regular, positive, and constructive feedback—along with one-on-one coaching sessions—helps us identify where people struggle and provide appropriate support to avoid systematic errors. That's why we encourage our workforce to ask questions while working.

Production Phase: Maintaining Quality at Scale

During production, Perception Experts review dataset quality to understand which error types exist and how severe they are for your use case. With integrated quality reports and data quality deep dives, Perception Experts can spot and eliminate systematic errors in the dataset. Maintaining close contact with LQMs to understand their challenges is essential. As Laura Wörns puts it, "We need to ensure annotation guidelines and quality expectations are understood by our LQMs, since they are correcting and teaching our workforce."

How do we ensure our communication workflow operates as it should during this stage? Apart from the information sharing and training sessions, where experienced LQMs and QMs explain in detail what is expected from annotators, these managers have continuous conversations with annotators who need help. At Kognic, being transparent is in our DNA. This way of working enables us to help the rest of the annotators who participate in the training and prepare for the annotation examination tasks. Another example of this is the fact that LQMs and QMs gather all general doubts in a shared feedback document, so that everyone can benefit from that.

As one of our QMs narrates, everything works thanks to good communication. Giving regular, positive and constructive feedback, and setting meetings with the annotators to give one-on-one coaching is key. This helps us determine where the person struggles, why it is so and provide the appropriate support to avoid structure errors. That is why at Kognic we always encourage our workforce to ask questions when doing their tasks.

Ensuring a continuous dialogue also during the production phase

During the production phase, Perception experts review quality in datasets, being the goal of it to get an understanding of which error types are hidden and how severe these are for the customer use case. With the help of the integrated quality reports and data quality deep dives, Perception Experts can spot and get rid of structured errors in the dataset. Keeping close contact with LQMs to understand their challenges and doubts and trying to solve them play a key part. As Laura Wörns puts it, “We need to ensure the annotation guidelines and quality expectations are understood from our LQMs, since they are correcting and teaching up our workforce”.

Our LQMs and QMs work closely with annotators to identify and fix errors—whether repetitive mistakes, edge cases, new client clarifications, or similar issues—ensuring everyone stays aligned. With guidelines as the foundation, they ensure work remains consistent. LQMs pay special attention to false negatives, false positives, bounding box dimensions (both 2D and 3D), assigned properties, assigned classes, and understanding of guideline rules.

We've Got Your Back

We aim to deliver reliable training data that ensures safe perception. To achieve this goal, clear and continuous communication is paramount. As Meera Ranganathan expressed, it's about consistency and safety—but also about adding a human touch to everything our platform enables, and about learning and improving together. "We receive amazing ideas, feedback, and positive comments which motivates us to provide better support and create better workflows."

Communication, collaboration, action, iteration. It comes down to setting the right expectations from the start, maintaining clear and constructive communication, paying attention to details, and iterating when needed to deliver on our promises. No matter how many Kognic teams are involved, our communication processes ensure a positive customer experience while our platform maximizes productivity. This is how we deliver on our promise: the most annotated autonomy data for your budget.