Introducing the Mapper API

Introducing the Mapper API

At Google I/O we announced the Mapper API. Built completely on top of public App Engine APIs today, this API is only the first component of App Engine's MapReduce toolkit, but can be extremely useful on its own.

The Mapper API can already be of use to many developers who would otherwise need to build their own tool for doing large scale data manipulation. In addition to taking care of the distribution of these jobs over task queues, it provides the ability to store state, batch datastore writes via mutation pools, and ships with an easy to use administrative interface for job management, all optimized for the constraints of App Engine's dynamic serving environment. Some examples of the types of operations that work with minimal configuration with this tool:

• Report Generation
• Large scale migration of entity properties
• Datastore cleanup
• Computing statistics and metrics

For an introduction to the Mapper API, watch Mike Aizatsky's video from Google I/O, where he demonstrates building a source code indexer. The slides can be downloaded here, and the video is below: