Running the tests¶
You can run flake8, isort and the py.test suite inside the Vagrant VM, using:
vagrant ~/treeherder$ ./runtests.sh
Or for more control, run each tool individually:
vagrant ~/treeherder$ py.test tests/ vagrant ~/treeherder$ py.test tests/log_parser/test_tasks.py vagrant ~/treeherder$ py.test tests/etl/test_buildapi.py -k test_ingest_builds4h_jobs
To run all tests, including slow tests that are normally skipped, use:
vagrant ~/treeherder$ py.test --runslow tests/
For more options, see
py.test --helpor http://pytest.org/latest/usage.html
vagrant ~/treeherder$ flake8
NB: If running flake8 from outside of the VM, ensure you are using the same version as used on Travis (see
To run interactively:
vagrant ~/treeherder$ isort
Or to apply all changes without confirmation:
vagrant ~/treeherder$ isort --apply
NB: isort must be run from inside the VM, since a populated (and up to date) virtualenv is required so that isort can correctly categorise the imports.
Profiling API endpoint performance¶
On our development (vagrant) instance we have django-debug-toolbar installed, which can give information on exactly what SQL is run to generate individual API endpoints. Just navigate to an endpoint (example: http://localhost:8000/api/repository/) and you should see the toolbar to your right.
Add a new Mercurial repository¶
To add a new repository, the following steps are needed:
Append new repository information to the fixtures file located at treeherder/model/fixtures/repository.json
Load the file you edited with the loaddata command:
vagrant ~/treeherder$ ./manage.py loaddata repository
Restart any running gunicorn/celery processes.
For more information on adding a new GitHub repository see Add GitHub repository.
Building the docs locally¶
vagrant sshinto the VM, or else activate a virtualenv on the host machine.
From the root of the Treeherder repo, run:
> pip install -r requirements/docs.txt > make -C docs html
The built docs can then be found inside
- Always use
yarnto make changes, not
npm, so that
yarn.lockremains in sync.
- Add new packages using
yarn add <PACKAGE> --no-bin-links(
yarn.lockwill be automatically updated).
- After changes to
yarn install --no-bin-linksto install them and automatically update
- For more details see the Yarn documentation.
Note: To work around symlink issues for Windows hosts, use
--no-bin-links with any command that adds/modifies packages. Whilst this is technically unnecessary with non-Windows hosts, it’s still recommended since otherwise your local changes might inadvertently rely on
node_modules/.bin/ symlinks that won’t exist in a newly created Vagrant environment. Unfortunately yarn doesn’t yet support setting this option via the global yarn config, otherwise we could just enable it by default.
Releasing a new version of the Python client¶
Determine whether the patch, minor or major version should be bumped, by inspecting the client Git log.
File a separate bug for the version bump.
Open a PR to update the version listed in client.py.
Use Twine to publish both the sdist and the wheel to PyPI, by running the following from the root of the Treeherder repository:
> pip install -U twine wheel > cd treeherder/client/ > rm -rf dist/* > python setup.py sdist bdist_wheel > twine upload dist/*
Release Engineering::Builddutybug requesting that the sdist and wheel releases (plus any new dependent packages) be added to the internal PyPI mirror. For an example, see bug 1236965.
Hide Jobs with Tiers¶
To hide jobs we use the job’s
tier setting. Jobs with
tier of 3 are
hidden by default. There are two ways to set a job to be hidden in Treeherder:
- TaskCluster - Edit the task definition to include the
tiersetting in the Treeherder section.
- BuildBot - You must get the signature hash of the job from the UI and add that
signature hash to the
buildapi.pyfile in the Treeherder repo. To get the signature, click the job and then click the
siglink in the Job Details Panel. That will place the signature hash in the filter field.