Linux Commit Analyzer (CommA)

This tool populates a database with the details of which patches that are upstream (in linux-mainline) are missing from distros in their downstream kernels (e.g. Ubuntu kernels). For our use case, we are looking at Hyper-V patches and checking downstream azure specific kernels, but this is adaptable.

Getting Started

Install Dependencies

Install python3 via apt and then it as default:

This is not technically required but is a recommended configuration, as poetry expects some Python at python. You could also install Python 2.7.

apt install python3 python3-pip python3-venv
update-alternatives --install /usr/bin/python python /usr/bin/python3 1
update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 1

Install poetry:

Caution: this modifies your .bashrc.

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python

Setup Microsoft SQL repos (this is for mssql-tools):

Caution: these commands use sudo and add a package repository. Change the Ubuntu version to your version (cat /etc/os-release for the number).

curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
curl https://packages.microsoft.com/config/ubuntu/18.04/prod.list | sudo tee /etc/apt/sources.list.d/msprod.list

Install the required packages for connecting to the Microsoft SQL database:

apt update
apt install mssql-tools unixodbc-dev

Install Python packages via poetry:

poetry install

For the symbol matcher, install exuberant-ctags as the default ctags will not work:

apt install exuberant-ctags

Running CommA

1. Enter the setup environment with poetry shell
2. Provide the URL to the secrets repo with export COMMA_SECRETS_URL=<URL/to/secrets/repo>.
3. Run ./CommA.py run --upstream --downstream

This will parse the upstream and downstream repos.

Setting Up Secrets

You should have a separate Git repository which contains an XML file with your database secrets, structured like:

</secrets><?xml version="1.0"?>
<secrets>
    <DatabaseServer>your.database.server.url</DatabaseServer>
    <DatabaseName>your_database_name</DatabaseName>
    <DatabaseUser>your_database_user</DatabaseUser>
    <DatabasePassword>your_database_password</DatabasePassword>
</secrets>

The file should be named PatchTrackerSecrets.xml and exist at the root level.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Legal Notices

Microsoft and any contributors grant you a license to the Microsoft documentation and other content in this repository under the Creative Commons Attribution 4.0 International Public License, see the LICENSE-DOCS file, and grant you a license to any code in the repository under the MIT License, see the LICENSE file.

Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.

Privacy information can be found at https://privacy.microsoft.com/en-us/

Microsoft and any contributors reserve all other rights, whether under their respective copyrights, patents, or trademarks, whether by implication, estoppel or otherwise.