Skip to content
No description, website, or topics provided.
Python Shell Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docker.d
requirements
src
tests
.gitignore
.taskcluster.yml
Dockerfile
MANIFEST.in
README.rst
docker-compose.yml
settings.py
setup.cfg
setup.py
tox.ini
version.txt

README.rst

Mapper

When we convert hg repositories to git, and vice versa, the hg changeset SHA (the 40 character hexadecimal string that you get when you commit a change) is different to the git commit id (the equivalent SHA used by git).

In order to keep track of which hg changeset SHAs relate to which git commit SHAs, we keep a database of the mappings, together with details about the project the SHAs come from, and what time they were inserted into the database.

The vcs sync tool (checked into mozharness) is the tool which performs the conversion between hg repos and git repos, and this is documented separately. It is responsible for performing the conversion - this is outside the scope of mapper.

Mapper is an HTTP API that allows:

  • insertion of new mappings and projects (a "project" is essentially the name of the repo - e.g. build-tools) (HTTP POST)
  • insertion of git/hg mappings for a given project (HTTP POST)
  • retrieval of mappings for a given project (HTTP GET)

Behind the scenes, it is reading/writing from the database (using sqlalchemy).

Note

The vcs sync tool is a client of the mapper: it is vcs sync that inserts into mapper (i.e. uses the HTTP POST methods).

The other clients of mapper are:

  • people / developers - wanting to query mappings
  • b2g_build.py - the build script for b2g - since this needs to lookup SHAs in order to reference frozen commit versions in manifests

Request authentication credentials

Open a bug on bugzila and request new taskcluster client credentials that that you will then use in.

Use the following points to guide you opening the bug:

  1. Product field should be Release Engineering
  2. Component field should be Applications: Mapper
  3. Summary field should be Requesting taskcluster client credentials to use with mapper
  4. Description field should contain:
    • who is the responsible person and which is the responsible team
    • what is the purpose of usage
    • what should be the expiration date of the credentials (suggested is one year)
    • which level of access is required:
      • Create new project
      • Insert mappings

How to generate taskcluster client credentials

  1. Go to https://tools.taskcluster.net/auth/clients.

  2. Make sure you are logged into taskcluster.

  3. Fill the Create New Client form:

    ClientId:

    Make sure to include the Bug number by following the template

    project/releng/services/mapper/bug<NUMBER>
    
    Description:

    Who is responsible and which team, also where is this token used.

    Expires:

    Requested expiration, by default set it to 1 year.

    Client Scopes:

    List of scopes requested based on the requested level of access:

    • Create new project

      project:releng:services/mapper/api/project/insert
      
    • Insert mappings

      project:releng:services/mapper/api/mapping/insert
      
  4. Send clientId and accessToken in a JSON authentication file via https://send.firefox.com. Format of authentication file should be:

Local Development

Use docker-compose up to run api (and postgresql database). The API will be available at https://localhost:8004/apidocs. When visiting https://localhost:8004 you will be redirected to the location of the frontend.

Deployment process

To trigger the deployment you have to push the code to the branch with the same name as environment you want to deploy to.

This will start Taskcluster graph which will build and push docker image to docker hub (mozilla/releng-mapper) with the same tag as is the environment.

Cloudops team Jenkins is listening for the change and will deploy it to GCP once it confirms that the docker images was build in a trusted environment. It usually takes around 5min for deployment to be done. For more how things are configures you can check cloudops infrastructure.:

You can check that the service was deployed correctly by visiting the /__version__ endpoint which should include

Deployed Environments

We have a number of deployed ToolTool environments.

You can’t perform that action at this time.