REST API

Treeherder provides a REST API which can be used to query for all the push, job, and performance data it stores internally. To allow inspection of this API, we use Swagger, which provides a friendly browsable interface to Treeherder’s API endpoints. After setting up a local instance of Treeherder, you can access Swagger at http://localhost:8000/docs/. You can also view it on our production instance at https://treeherder.mozilla.org/docs/.

Python Client

We provide a library, called treeherder-client, to simplify interacting with the REST API. It is maintained inside the Treeherder repository, but you can install your own copy from PyPI using pip:

pip install treeherder-client

It will install a module called thclient that you can access, for example:

from thclient import TreeherderClient

By default the production Treeherder API will be used, however this can be overridden by passing a server_url argument to the TreeherderClient constructor:

# Treeherder production
client = TreeherderClient()

# Treeherder stage
client = TreeherderClient(server_url='https://treeherder.allizom.org')

# Local vagrant instance
client = TreeherderClient(server_url='http://localhost:8000')

When using the Python client, don’t forget to set up logging in the caller so that any API error messages are output, like so:

import logging

logging.basicConfig()

For verbose output, pass level=logging.DEBUG to basicConfig().

User Agents

When interacting with Treeherder’s API, you must set an appropriate User Agent header (rather than relying on the defaults of your language/library) so that we can more easily track API feature usage, as well as accidental abuse. Default scripting User Agents will receive an HTTP 403 response (see bug 1230222 for more details).

If you are using the Python Client, an appropriate User Agent is set for you. When using the Python requests library, the User Agent can be set like so:

r = requests.get(url, headers={'User-Agent': ...})

Authentication

A Treeherder client instance should identify itself to the server via the Hawk authentication mechanism. To apply for credentials or create some for local testing, see Managing API credentials below.

Once your credentials are set up, if you are using the Python client pass them via the client_id and secret parameters to TreeherderClient’s constructor:

client = TreeherderClient(client_id='hawk_id', secret='hawk_secret')
client.post_collection('mozilla-central', tac)

Remember to point the Python client at the Treeherder instance to which the credentials belong - see here for more details.

To diagnose problems when authenticating, ensure Python logging has been set up (see Python Client).

Note: The system clock on the machines making requests must be correct (or more specifically, within 60 seconds of the Treeherder server time), otherwise authentication will fail. In this case, the response body will be:

{"detail":"Hawk authentication failed: The token has expired. Is your system clock correct?"}

Managing API credentials

To submit data to Treeherder’s API you need Hawk credentials, even if you’re submitting to your local server. The recommended process is slightly different for a development server versus submitting to Treeherder staging or production, see below for details.

Generating and using credentials on a local testing instance

To generate credentials in the Vagrant instance run the following:

vagrant ~/treeherder$ ./manage.py create_credentials my-client-id

The generated Hawk secret will be output to the console, which should then be passed along with the chosen client_id, and Vagrant instance server_url to the TreeherderClient constructor. For more details see the Submitting Data section.

Generating and using credentials on treeherder stage or production

Users can generate credentials for the deployed Mozilla Treeherder instances (and view/delete existing ones) using the forms here: stage / production. It is recommended that the same client_id string be used for both stage and production. Once you’ve created your set of credentials, you can get access to the Hawk secret by clicking on the link that should appear on the credentials list page.

The credentials must be marked as approved by a Treeherder admin before they can be used for submitting to the API. Request this for stage first, by filing a bug in Treeherder: API. Once any submission issues are resolved on stage, file a new bug requesting approval for production.

Once the credentials are approved, they may be used exactly in exactly the same way as with a local testing instance (see above).

Treeherder administrators can manage credentials here: stage / production. Note: Bugs must be filed to document all approvals & changes, to ease debugging and coordinating with credential owners in case of any later issues.