The past decade has witnessed explosive growth in Machine Learning (ML) in automotive and other industries. In the future, AI will automate jobs and get complicated tasks done without human involvement. However, this is a misconception based on the assumption that if we do not see the human assistance behind AI, then there isn’t any.
As we know at Kognic, today Artificial Intelligence depends on labeled data sets used to ‘learn’ and identify patterns or predict the future. We offer leading automation when producing labels, but that quality depends on reliable humans. The challenge is to minimize the amount of human judgement required while maximizing quality.
Recently, some news have raised concerns about annotator working conditions. And, as a company that collaborates with annotators, we feel strongly about this. We cannot keep silent about a situation that affects the whole sector, not only the companies that let this happen.
This post will explain what the data labeling sector looks like and then describe our efforts to improve this situation. The hope is that with this knowledge about how to set up fair labeling working conditions and adequate tools we can inspire other companies to change their methods. We are aware that we are not perfect, though, and we are always willing to be part of the conversation to make a difference.
What does the data labeling ecosystem look like?
In the last few years, there has been a boom in private annotation firms dedicated to providing data labeling and annotation services. These private firms, also referred to as offshore business process outsourcing (BPO) companies, employ and provide staff for data annotations. Data labeling can be demanding and sometimes there can be elevated stress levels, depending on the situation.
Why is this so? First of all, annotators are required to label large volumes of data in a single shift. They have to follow strict quality requirements and deadlines, and are subjected to quality assurance processes, such as on the spot review and monitoring of tasks. In this regard, we see common sources of stress for annotators such as atypical work schedules with long and irregular hours including night shifts (to cater for an international clientele), limited vacation time, unpaid parental leave, specific shift times and even timed bathroom breaks. In addition, annotators are usually paid the minimum wage and the job has demanding performance requirements. You can now see a clear picture of annotators' working conditions.
Although this can happen anywhere in the world, there are data annotation companies which utilize their employees in large cities of emerging countries to do the data annotation work. They state they have fair working conditions for their employees in terms of gender equality, maternity leaves, minimum wage, etc. However, it is noteworthy to mention that those statements are provided as self-declarations, rather than a third-party ethical compliance certification.
Our take on this issue
At Kognic we acknowledge annotators' big contribution to the advances in AI and therefore we don't think the current approach is the right way forward. We believe that all industry players need to ensure that annotators have economic security and fair working conditions, and that they are not exploited in the name of cheap labor. Together, we must strive not just to conform to existing regulations, but contribute to the dialogue to improve and develop them. This is not just our moral obligation, but a substantial requirement to provide a sustainable industry where all employees can thrive.
At the same time, we are aware of the fact that it will be difficult to globally regulate the borderless market of data labeling services, but none of these challenges are insurmountable. Overcoming these requires regulatory efforts, as well as cooperation between industry protagonists. But until that moment comes, we need to make sure that we as companies are doing our bit to make a difference.
So, how is Kognic making efforts to create fair working standards for annotators?
We believe that a careful evaluation of the fit between the company’s ethical compliance requirements and the solution provider is what can ensure that employers create fair conditions for data annotation work. In order to be more specific, in the following paragraphs we present concrete advice as well as the way we approach every step of the process.
BPO selection
The first area where we can make a difference to our workforce is in the selection process of new Business Process Outsourcing (BPO) partners. In our case, we have a rigorous selection procedure to weed out companies that put profit over people. Our advice is to be selective, gather as much information as you possibly can and to ask a lot of questions to make sure you can choose a partner whose annotators work in fair conditions.
Providing decent working conditions
There are many steps that we as companies can take to provide good working conditions. These include but are not limited to:
- Limiting the number of working hours.
- Allocating time for breaks.
- Providing the right technical and ergonomic equipment to do the work (stable internet connection, reliable computers, displays and office space).
- Providing a safe working environment (including building safety, fire safety, working proactively to avoid accidents in the workplace, having first aid kits available, adequate cleaning, heating, lighting, access to recreational spaces, etc.).
- Ensuring the partners you work with offer long-term contracts to their employees.
Visiting the BPO location and meeting the annotation team
In order to get an accurate picture of the culture and working conditions of your BPOs, you should make site visits. Only when meeting your BPO partner and their annotators in person can you gain firsthand experience and understand their daily operations.
In addition, onsite visits help to build relationships and trust. This is because they allow face to face communication with the BPO leaders that will enhance dialogue and collaboration. Also, by meeting the annotators you can understand issues affecting them personally and find potential solutions.
Getting feedback from annotators
As just mentioned, another crucial factor to consider is getting feedback from the annotators themselves. In our experience, apart from the information you can get from them onsite, anonymous surveys sent out on a regular basis can provide good insight into what works well and what needs improvement.
Be mindful when formulating questions, and include visual resources where possible. Finally, it is important that you show you are taking the feedback into account by updating your workforce on plans and improvements. Otherwise you won’t be helping your annotators in reality.
Fair salary levels
To ensure all collaboration partners pay fair wages, you can investigate local minimum salary levels for the BPO location and compare these to the wages paid to the workforce. Wages should match or exceed the regulated minimum wage to be apropriate. We suggest paying an hourly wage instead of paying per unit, as annotators then know how much they get.
Kognic’s minimum requirement is that employers shall pay at least the statutory minimum wage, the prevailing industry wage, or the wage negotiated in a collective agreement, whichever is higher. Pay is based on experience, qualifications and performance, and employees have the right to a written specification of how the wage has been calculated. Other essential factors that we ensure with BPOs are that wages are paid regularly and on time; and that contracts are clear and written in the local language.
Reasonable working hours
In our agreements we dictate that all annotators must have the freedom to take breaks (and we encourage them to take these), and decide their schedule, as long as they meet working expectations (active annotation time is expected to be about 30 hours per week). No BPO companies working with Kognic can exceed 45 hours of scheduled working hours. In addition, weekends are free for annotators. It shouldn’t be any other way, everyone deserves time off to rest and to have a life after work!
Therefore, as an employer, you should plan for work to take place during the weekdays, so that the annotators get the time off they deserve. We also encourage you to clearly communicate these expectations to your collaborators. And on the rare occasion where overtime or weekend work could be required, remember that this should be justly compensated.
Providing the right tools & processes
As we’ve said, part of facilitating good work conditions is about providing the right equipment and in our case also the right tools. At Kognic, we continuously work to improve the annotation platform used by our annotators, to guarantee that it is user-friendly and minimizes the manual annotation effort. Additionally, we have developed processes to reduce unnecessary stress. For example, we have a structured onboarding process which allows all employees to learn how to use our platform and how to annotate according to the project guidelines in a stress-free environment.
Legally binding agreements
Last but not least, It is important to have legally binding agreements that will help to provide your annotators fair conditions and a good work-life balance. At Kognic, we have a comprehensive Code of Conduct which we share with partners and include in our legal contracts. In this document, we detail our expectations on many matters.
We know it’s impossible to include all eventualities, but setting the expectations and getting confirmation that your partners share your values and prioritize good working conditions is definitely one step in the right direction.
Let’s be part of the solution
Today’s world is facing many challenges, one of them being inequality. We believe that the place where you live or where you’ve been born shouldn’t determine having better or worse working conditions, liveable wage or social protection etc. At the end of the day, every person in the world has the right to have a decent life, and any company which supports human rights should care about this issue as much as we do.
At Kognic, we can only encourage all companies in the AI industry to not see the lack of standards as an excuse to mistreat employees, but rather as an opportunity to create new standards which we can all be proud of. A good start is to ask yourself: How are you making sure that your annotators have the right working conditions?
We strive to become better every day and be observant about our processes and the environment we provide for our workers. But we know there’s always room for improvement. Do you have any additional ideas to ensure these conditions are met? We’d love to know.
Let’s be part of the solution together. 🚀
