Challenges With Crowdsourcing Platforms

Using crowdsourcing platforms for labeling data to aid the development of machine learning models

Table of contents

A crowdsourcing platform is a place where we can find workers who can perform elementary tasks. Often these tasks are trivial and will require a lot of time and patience to be completed by a human. While some parts of the task can be automated, computers are yet to achieve human-level accuracy when it comes to solving certain problems. These platforms can be leveraged to solve such easy-to-solve human problems. These are online-only and often asynchronous. A few examples of such tasks include, but are not limited to image data labeling, translation of text from one language to another, data entry, etc.,

Gig Economy

We have seen the rise of the “gig economy” where people are working remotely and independently. A crowdsourcing platform can be a fantastic way of getting started to earn a little bit of money while completing easy tasks. Also, these remote workers are often from developing countries, often paid too little despite considering the economic conditions of the region where they work from. It might seem like an employment opportunity for these workers, but it is often exploitation of these minimum wage workers. Tasks such as image labeling are often monotonous and require careful attention from the labelers. Sometimes, they are responsible for labeling adult or explicit content. This could cause long-term mental health effects on these workers with no entity or employer taking accountability. A background study on the workers identified that the workers were previously minimum-wage workers. Global pandemics like COVID-19 led to their unemployment and thus, they were forced to take up any available jobs. While workers such as these helps aid projects with small budgets, there should be a systematic reform to avoid the exploitation of these online workers.

The Future

Governments of the respective countries of the gig workers have set up labor laws and minimum wages but the tasks that they perform are often asynchronous and a traditional payroll might not be feasible for most employers. These workers are not one-time participants. These platforms follow an incentive and rating mechanism that will ensure that the workers have some credibility as well as the employers can instill trust in the people they are planning to hire. While this metric of evaluating the performance of a worker is controversial, it can help build a system around employers hiring the same workers again because of their high credibility and past work experience doing similar jobs. Some studies have shown that the workers themselves don’t feel that they are being exploited as they are willing to participate as they are being paid fairly. Crowdsourcing can be a fantastic way of lifting people out of poverty in third-world countries by providing them with opportunities. These kinds of jobs can be adequate for their income, despite the pricing being low. On the other hand, the building of highly accurate machine learning models needs accurately labeled data. Many important models have been created for state-of-the-art applications like image recognition, and natural language processing. These platforms can help create them, which in the future can be useful for the betterment of humanity.

Did you find this article valuable?

Support Sudhanva Narayana by becoming a sponsor. Any amount is appreciated!