Crowdsourcing

The LARA Portal contains initial support for crowdsourcing of projects. By this, we mean a process where a project is cut up into several pieces, and the work of annotating the pieces (e.g. adding audio, translations, etc) is split between a number of people, typically one person for each piece. At the end, the annotated pieces are stuck together again into a complete project. The portal helps with the mechanics of making this work.

In this section, we will describe the LARA crowdsourcing capabilities, using an extended example. First, a little terminology. We will refer to the person who creates the top-level project and drives the crowdsourcing effort as the crowd-requester. The people who accept and carry out the crowdsourced subprojects are the crowd-workers. All top-level crowdsourcing actions are performed though the portal’s crowdsourcing dashboard. This is divided into three subtabs:

The functionality of each subtab is what you would expect from the names. The Available tasks subtab shows the crowdsourced tasks that can currently be picked up by crowd-workers. The My crowd-requester subtab shows the crowdsourced tasks (if any) which you have created as a crowd-requester. The My crowd-worker subtab shows the tasks (if any) that you have picked up and are currently working on as a crowd-worker.

Creating a crowdsourced task

We’ll now get into the details of creating and performing crowdsourced tasks. As our toy example, we’ll use the traditional nursery rhyme “Mary Had a Little Lamb”. Here is the initial text:

Mary Had a Little Lamb

Traditional

Verse 1

Mary had a little lamb
Its fleece was white as snow.
And everwhere that Mary went
That lamb was sure to go.

Verse 2

He followed her to school one day
Which was against the rule;
It made the children laugh and play,
To see a lamb at school.

As the crowd-requester, I’m happy to do the tagging myself, but I want to delegate the tasks of adding audio and translations to two other people. I start by creating a project where the tagged version of the text is as follows:

<h1>Mary Had#have# a Little Lamb||</h1>
<b>Traditional||</b>

<h2>Verse 1||</h2>
Mary had#have# a little lamb||
Its#it# fleece was#be# white as snow.||
And everwhere that Mary went#go#||
That lamb was#be# sure to go.||
<cut>
<h2>Verse 2||</h2>
He followed#follow# her#she# to school one day||
Which was#be# against the rule;||
It made#make# the children#child# laugh and play,||
To see a lamb at school.||

All of this should look familiar, except for one new element: the <cut> tag in the middle. This says how to divide up the text into two parts for crowdsourcing. The first part is the text before the <cut>, and the second part is the text after it. After creating my project and doing Create resources, I go to the My LARA texts tab:

Crowdsourcing the project couldn’t be easier. All I have to do is click on the icon under Distribute project:

The portal confirms. When we go to the My crowd-requester tasks subtab, we can see the new tasks waiting to be picked up:

Picking up and performing a task

Now I’m going to log out and then log in as larauser, who’s going to be our first crowdworker. larauser looks at the Available tasks subtab. There are a lot of tasks here, and he knows what he’s looking for, so he searches for “Mary”. The display looks like this:

To pick up the first task, I click on the icon under Get it.

Now I can see it under My crowd-worker tasks:

I can click on “Edit project” to open it like any other project. I start by assigning the segment recording voice to larauser and doing Create resources.

When I got to the second tab, I see a Segment translation task:

I can click on it and fill in the French translations I found on the web:

and I can also record sentence audio in the usual way:

Note: you have to do the Create pages step too, otherwise the audio won’t be downloaded to the portal.

Notifying the task-requester

Having got this far, I, in my capacity as crowd-worker larauser, think I should notify crowd-requester testlara about what I’ve done. I go back to My crowd-worker tasks and use the Send for revision control:

I get a pop-up box which lets me send a message:

If I, as testlara, want to find out what’s going on, I can do that by going to My crowd-requester tasks. I see there’s a pencil icon under Comment for worker telling me I have a message to respond to. I find it by clicking on the Content notes control:

CALLecting a project

To find out what the current state of the full project is, I, as testlara, go back to My LARA texts and use the CALLect icon:

This sticks together the currently valid pieces of the cut-up project. Now I can edit the full project as usual and do Create resources followed by Create pages:

I look at the preview:

We can see that there is audio for the first half of the text.

Notifying the crowd-worker

After having reviewed the CALLected project, I respond by clicking on Comment for worker:

The crowdworker, larauser. can read the message by clicking on Content notes under My crowd-worker tasks:

Directly viewing the crowdworker’s progress

The task-requester can access the crowdworker’s work directly without going through the process of CALLecting it and remaking the full project. They can do this using the View progress control under My crowd-requester tasks:

This opens the subproject through the usual three-tab view, but only letting the task-requester look at things rather than change them:

Finalising a subtask

Let’s suppose that the crowd-requester is satisfied and doesn’t need anything else from the crowd-worker. They should now finalise the subtask, which removes it from the crowd-worker’s control:

Taking back and giving back a subtask

Another possible situation is that the crowd-requester is impatient with the worker’s lack of progress and wants to reassign the task. They can do this with the Take back control:

Similarly, a crowd-worker who has lost interest in a task can use their Give back control: