Amazon Mechanical Turk Integration
The module at its current form is a minimum viable product. It is designed to be a framework similar to the Drupal Commerce module, and requires helper modules to provide out-of-the box functionality. It has the following functionality:
- Issue Mturk API calls directly from Drupal, and record the API calls to display in Views. (Note: currently only a few most often used APIs are supported, together with some extended APIs.)
- Create HITs directly from Drupal. New HITs are created automatically when new nodes are created, and the HITs are hosted on the Drupal site as "ExternalQuestion" so you can easily modify how workers view the HITs. (Note: requires customized code.)
- A new content type "Mturk Task" to save Mturk access information and HITs properties.
- Show/hide blocks based on whether a user is an Mturk worker ("accept" or "preview") or the user is not from Mturk.
- A GUI interface to send messages to selected workers who have worked on your HITs.
- A GUI interface to grant grant bonus to selected workers who have worked on your HITs.
- Rules integration. Support 2 Rules events: "mturk worker login" and "mturk worker submit"
- Webform integration. The Mturk HITs are created using Webforms, and the results are redirected to Mturk when workers submit HITs with a Webform.
- Themekey integration. Display different themes based on whether a user is a Mturk worker or not.
- View mode support. Display nodes differently based on whether a user is a Mturk worker or not.
- Load results from Mturk, compute workers agreement and quality, clean the results, and save results back to relevant Drupal fields. (Note: customized code required).
- Display leader board based on Mturk workers finished HITs and overall quality. (Note: customized code required).
To make it fully functional requires the completion of the following issues (not prioritized):
If you want to see this module fully functional, please +1 on.
Introduction: About Amazon Mechanical Turk
Amazon Mechanical Turk is a popular platform for paid, micro-task crowd-sourcing (for a list of different genres of crowd-sourcing, see here), which is often referred to as "human computation". The idea is to use massive, low cost (if not free) human labor to solve hard algorithmic problems. For example, it is hard to develop a computer algorithm to label product images for your eCommerce site, but you can use Mturk to do it with real people at a reasonable cost.
For a brief introduction of human computation, see Artificial Intelligence, With Help From the Humans
Typical usage scenarios and case study
Example 1: A large Drupal ecommerce site has over ten thousand products. The site owner would like the product pages to embed a relevant youtube video to help online shoppers. He would use this module and create a task on Mturk, and thousands of Mturk workers would find relevant youtube video links and post them to the Drupal site.
Example 2: A university library has their book catalog in Drupal. The books metadata are stored as unstructured data in the "body" field, and they want to extract the metadata to be structured RDF format. They can't find any good automated tools to do it, but instead, they use Mturk and have thousands of workers manually clean the metadata.
Example 3: An online community wants to create some initial contents to attract more users. They can use this module to have thousands of Mturk workers contributes contents. As the community grows, they can have Mturk workers filter out bad quality contents (not necessarily spam).
Case study: We use this module to build the https://redbluegray.com site (still under development), where we classify political articles into conservative (red), liberal (blue), and neither (gray). Each day we randomly select a few articles to be manually classified on Mturk, and then we use these articles to train a classifier to automatically classify the rest of the articles. This is to make sure our classifier is up-to-date and uses all the new keywords for better classification. A list of articles manually classified by the Mturk workers is at https://redbluegray.com/ground-truth-list.
Q: Will there be a D6 backport?
A: No. This module is D7 only because it relies heavily on D7 only features.
Q: How to increase Mturk workers accuracy and get them work fast?
A: This is actually a hot research topic, and there's no definite answer. You can do a search on Google Scholar and find a large literature about it. I will write a short tutorial later too.
A: We provide paid customization. Please contact me at email@example.com.
|7.x-3.x-dev||tar.gz (31.69 KB) | zip (36.68 KB)||2013年11月19日||发布说明||简 | 繁 | 更多|