An Offer One Can’t Refuse
The game is a group-based exercise around the data sourcing and annotation experience. Emulating the limited resources available in the real world, players must think of creative ways to complete annotation tasks done in different systems.The Backdrop
Datasets used to train AI are often labeled/annotated. This work is done by data annotators. It has been reported that data annotators are often underpaid and exploited, see here, here. There have been several discussions and attempts to understand how data workers must be compensated, and their work valued (see here and here). This game is an attempt to think through and explore how we may arrive at answers to these critical questions by placing players in the shoes of data vendors who must ensure completion of tasks with limited resources. It flips the present dynamic between a vendor and annotators to one where it's the data vendors who must convince annotators to come onboard under the terms they have set out. Players are therefore pushed to compute and create a compensation model under real-world constraints in a ‘fair’ manner, and/or propose creative alternatives to supplant the model. This game was first tried out at the Mapping Data Work conference in March 2025, in partnership with the University of Amsterdam and Aapti Institute.Why is Tattle doing this?
In 2023, with support from Mozilla through the Data Futures Lab, we began thinking through our annotation process in building Uli, and arrived at a question: “How can we recognise the contributions of the annotators who work with us on Uli beyond monetary compensation?” We have been speaking with stakeholders and data workers to address this question of value and recognition for data annotators; you can read more about it here. This activity frames this question, and opens up the room for people to participate in developing models of compensation for a data annotation pipeline.Game Play
Goal
Your job is to make a pitch to data annotators describing their wage and other benefits.Personas: Data Vendor, Data Annotator and Data Client (aka looming figure in the background waiting for the annotations to be delivered)Set Up
- Each player is sorted into a group of 4-5 people (the group size can vary).
- Groups represent organizations/entities (Data Vendors) seeking qualified data annotators for annotation tasks.
- The goal is to train AI-assisted service offerings using the annotated data.
- Each group must make a pitch to the audience (who represent the data annotators) as to why they should work with each organization/entity.
- The groups are constrained by the scenarios they get, i.e., money for each task, and the time period to complete the project.
The Rules of the Game
- Each group gets a scenario, budget, and dataset size, some with additional conditions.
- The information from the scenario allotted to each group is immutable. It forms the baseline for the activity.
- The group must then prepare a pitch to up to 40 annotators (can be imaginary/ think of the other people in the room as the annotators) to convince them to work for them within these constraints.
- Each group has 15-20 minutes to prepare.
- With the given information, the groups make a 1-minute pitch.
- The calculator is an assistive device. To win the game you need to make an offer beyond just the monetory compensation, so get creative!
Is there anything I need to set up?
- If you are playing offline, and have one disinterested friend who is better off as a timekeeper/referee, have them generate the scenarios on their phone and allot it to each of you.
- If you are a game nerd, you can also keep some colour markers and chart paper handy so each group can get creative with their pitches.
- If you are playing online, set up a meeting on a platform that allows you to breakout into groups, and have a timer handy.
The Calculator
If you’d like to use the calculator as a standalone functionality outside of the game, please feel free to do so. Don’t click the generate scenario button, and you should be good to go.Data Annotator Wage Calculator
