When we convert hg repositories to git, and vice versa, the hg changeset SHA (the 40 character hexadecimal string that you get when you commit a change) is different to the git commit id (the equivalent SHA used by git).
In order to keep track of which hg changeset SHAs relate to which git commit SHAs, we keep a database of the mappings, together with details about the project the SHAs come from, and what time they were inserted into the database.
The vcs sync tool (checked into mozharness) is the tool which performs the conversion between hg repos and git repos, and this is documented separately. It is responsible for performing the conversion - this is outside the scope of mapper.
Mapper is an HTTP API that allows:
- insertion of new mappings and projects (a "project" is essentially the name of the repo - e.g. build-tools) (HTTP POST)
- insertion of git/hg mappings for a given project (HTTP POST)
- retrieval of mappings for a given project (HTTP GET)
Behind the scenes, it is reading/writing from the database (using sqlalchemy).
The vcs sync tool is a client of the mapper: it is vcs sync that inserts into mapper (i.e. uses the HTTP POST methods).
The other clients of mapper are:
- people / developers - wanting to query mappings
b2g_build.py- the build script for b2g - since this needs to lookup SHAs in order to reference frozen commit versions in manifests
Request authentication credentials
Open a bug on bugzila and request new taskcluster client credentials that that you will then use in.
Use the following points to guide you opening the bug:
- Product field should be
- Component field should be
- Summary field should be
Requesting taskcluster client credentials to use with mapper
- Description field should contain:
- who is the responsible person and which is the responsible team
- what is the purpose of usage
- what should be the expiration date of the credentials (suggested is one year)
- which level of access is required:
- Create new project
- Insert mappings
How to generate taskcluster client credentials
Make sure you are logged into taskcluster.
Create New Clientform:
Make sure to include the Bug number by following the template
Who is responsible and which team, also where is this token used.
Requested expiration, by default set it to 1 year.
List of scopes requested based on the requested level of access:
Create new project
accessTokenin a JSON authentication file via https://send.firefox.com. Format of authentication file should be:
Use docker-compose up to run api (and postgresql database). The API will be available at https://localhost:8004/apidocs. When visiting https://localhost:8004 you will be redirected to the location of the frontend.
To trigger the deployment you have to push the code to the branch with the same name as environment you want to deploy to.
This will start Taskcluster graph which will build and push docker image to docker hub (mozilla/releng-mapper) with the same tag as is the environment.
Cloudops team Jenkins is listening for the change and will deploy it to GCP once it confirms that the docker images was build in a trusted environment. It usually takes around 5min for deployment to be done. For more how things are configures you can check cloudops infrastructure.:
You can check that the service was deployed correctly by visiting the
/__version__ endpoint which should include
We have a number of deployed ToolTool environments.
URL: https://mapper.testing.mozilla-releng.net/ Taskcluster Secret: project/releng/mapper/config:dev Taskcluster Client ID: project/releng/mapper/dev
URL: https://mapper.staging.mozilla-releng.net/ Taskcluster Secret: project/releng/mapper/config:staging Taskcluster Client ID: project/releng/mapper/staging
URL: https://mapper.mozilla-releng.net/ Taskcluster Secret: project/releng/mapper/config:production Taskcluster Client ID: project/releng/mapper/production