Collecting and Analysing App Engine Logs Using the Logs Reader API and Fantasm.

Posted on by

Google recently released the Logs Reader API which allows programmatic access to App Engine logs. A quick perusal of the documentation showed its interface to be consistent with the continuation interface provided by Fantasm.

As a proof of concept, we built the logcollector application that combines the Logs Reader API and Fantasm to send email alerts in the event of user specified log level (eg. CRITICAL) messages.

The sample code can easily be extended for your own purposes.

Downloading and Installing

Python code for logcollector can be downloaded from Fantasm’s SVN Google code repository:

> svn export fantasm
> cd fantasm/application/logcollector

To use logcollector, simply deploy the application as a separate version into your App Engine application.

> head -n 2 app.yaml
application: myapp
version: logcollector
> update .


logcollector can be used to monitor the logs of any deployed version by hitting the following URL:

…and filling out the form shown in Figure 1. Version must be a valid deployed version of myapp, Sender must be a valid admin email address, and To can be any email address you want to send the alert email to. Choosing a Minimum Log Level that is too low may result in a very full inbox!

Interface to start logcollector
Figure 1. Interface to start a LogCollector.

Currently the Log Reader API can only query logs from one version at a time. You can monitor several versions simultaneously by submitting the form for each version.