Complete "Technical Infrastructure" page

[voisey]

The Technical Infrastructure page in the wiki is missing information and so needs to be completed.

It may be suitable to migrate certain information from https://wiki.magmacore.org/display/HOME/DevOps+@Magma but note that some sections might no longer be relevant. For instance, I believe we no longer use Netlify. We may also want to add certain other sections:

• Vagrant Cloud
• Dependabot: Only mentioned very briefly in Security Overview for Contributors. What does dependabot do? Which admin should be asked? How is it configured?
• Codecov: Mentioned briefly in two places in the wiki (excluding earlier versions of the docs): Magma CI and Contributing Code. What does it do? How does it work? How does one get access? How can one change the configuration?
• CI integration with Slack: Also add a section about how CI results are integrated with Slack, including a mention of the "#ci" channel. Link this section in the Magma CI docs
• CI dashboard: Point to dashboard and repo where it is configured

[nstng]

The following is a change request for https://github.com/magma/magma/wiki/Technical-Infrastructure. For reviewers: feel free to edit obvious issues directly in this comment. For questions and requests please use the comment function here. Write a comment if you are fine with the current state (will be seen as accepted review).

---

This page documents external services and frameworks that are used in the technical and organizational infrastructure of Magma.

Slack

A Magma Slack to engage with the community is hosted at https://slack.magmacore.org/. The Slack is also used for CI notifications, e.g., #ci where various workflows report failures of runs.

Support

If you have issues with using the Magma Slack, feel free to ask for support in public channels. If you have a general problem connecting to the Magma Slack, you can open an issue at https://github.com/magma/magma/issues.

GitHub

Magma uses GitHub at https://github.com/magma/magma to host the Magma source code. This includes, among others, pull request workflows, CI, docker artifacts (for CI), and issue tracking.

Support

Admin access to the account is managed by the TSC (can be contacted via Slack at #governance-tsc).

Artifactory

Release and CI artifacts are hosted at https://linuxfoundation.jfrog.io. The artifactory is maintained by the Linux Foundation. Write access is restricted to credentials which are deposited as GitHub secrets. This is, only workflows that run on the GitHub Magma account have write access.

Support

The workflows that push artifacts are owned by @magma/approvers-infra. Linux Foundation maintainers can be contacted in the Magma Slack, but this should usually be accompanied by opening a ticket in the Linux Foundation service desk: https://jira.linuxfoundation.org/plugins/servlet/desk.

1Password

Magma uses a 1Password account for sharing credentials in a team.

Support

You can discuss access to the account with Kendall Waters Perez, Projects Coordinator at the Linux Foundation (mail: kperez@linuxfoundation.org, Slack: @ Kendall Waters Perez).

CI Dashboard and Firebase

Results of selected workflows that run after a pull request is merged to master are summarized on an external CI dashboard at https://magma-ci.web.app/. The board displays results of GitHub CI workflows, but also results of external CI systems can be published. The data transfer happens via a Firebase (https://firebase.google.com/) account.

The code and documentation for the dashboard can be found at https://github.com/magma/ci-dashboard.

Support

For Magma workflows that publish to the dashboard, find the owner group of respective workflows in the workflow definitions. But in general @magma/approvers-infra is a good group to find support.

AWS

There is a Linux Foundation AWS account that was used for test deployments and various CI functionality. As of 01/2023 all running services were shut down. Further usage of this AWS account is under discussion.

Support

For general question contact TSC or Linux Foundation members in Slack. For technical issues use the Linux Foundation service desk: https://jira.linuxfoundation.org/plugins/servlet/desk.

Vagrant Cloud

Magma has a Vagrant Cloud account that can be found at https://app.vagrantup.com/magmacore. Hosted images are pre-built for CI workflows and local development. See https://github.com/magma/magma/wiki/Creating,-Testing,-and-Publishing-Magma-VM-Base-Images for documentation on how these images are created and updated.

Support

Generally @magma/approvers-infra can be contacted for issues. Getting access is currently managed by existing members. For technical issues use the Linux Foundation service desk: https://jira.linuxfoundation.org/plugins/servlet/desk.

Confluence

A Confluence hosted by the Linux Foundation can be accessed at https://wiki.magmacore.org. The Confluence is currently mostly used to document meetings and related documents. But it can also used for other purposes. The Confluence has read accesses without login, but a self sign-up is possible in order to contribute.

Support

Login via self sign-up. Contact Linux Foundation members in Slack for additional questions.

Dependabot

Magma uses Dependabot (https://docs.github.com/en/code-security/dependabot) for automated pull requests with suggested dependency upgrades.

Support

Dependabot is currently running in its default configuration. For general integration into the Magma repository you can contact people with admin access to the Magma GitHub account (TSC via Slack).

Codecov.io

Results of code coverage analyses are published to Codecov.io. This is also integrated in the Magma CI checks where code coverage metrics of subsequent revisions are compared. The checks fail if thresholds, configured in codecov.yml (root of Magma repository) are exceeded.

Support

In general people in @magma/approvers-infra can be contacted for questions. It is possible to do a self sign-up and link your GitHub account, but in general this should not be necessary. The Codevov.io account is linked by to the Magma GitHub account (GitHub admin access needed).

Sentry

Sentry is integrated into various parts of Magma, but needs to be enabled and configured in order to send data to a Sentry account. The intended use case is to enable Sentry for HWIL environments (currently this is not used). See the official Magma documentation for technical details

Support

Linux Foundation maintainers can be contacted in the Magma Slack, but this should usually be accompanied by opening a ticket in the Linux Foundation service desk: https://jira.linuxfoundation.org/plugins/servlet/desk.

[Neudrino]

Please be aware that @mentions with Github usernames or teams do (sadly) not work in Github wiki, but you would manually need to add the link for to the personal pages of the respective users or groups .

[LKreutzer]

I fixed some typos and added some links to slack channels.

Is it worth mentioning https://github.com/MagmaCIBot or is that too obscure? No idea how the configuration/login is managed.

[LKreutzer]

Follow-up: Are we sure that all of these services are accessible with the credentials from 1password?

[nstng]

Is it worth mentioning https://github.com/MagmaCIBot or is that too obscure? No idea how the configuration/login is managed.

I suggest to add this for now in a backporting documentation - I don't know if the bot is used for more use cases.

[nstng]

Follow-up: Are we sure that all of these services are accessible with the credentials from 1password?

They are not - 1password is currently barely used. Some parts do not need special credentials/access (e.g. slack, confluence), others are managed bei LF/Kendal. Where are you missing a documentation for getting access/credentials?

[LKreutzer]

Follow-up: Are we sure that all of these services are accessible with the credentials from 1password?

They are not - 1password is currently barely used. Some parts do not need special credentials/access (e.g. slack, confluence), others are managed bei LF/Kendal. Where are you missing a documentation for getting access/credentials?

I think reads might ask how they can get access to Firebase or Codecov.

I just remembered: Should we add the Sentry account here?

[nstng]

I think reads might ask how they can get access to Firebase or Codecov.

Codecov is a self sign up - I will add this
Firebase I actually have to research how access is granted here

I just remembered: Should we add the Sentry account here?

Yes, thank you! I will add a section

[voisey]

Looks great, thanks! Made a few very small modifications to the text in the comment.

[nstng]

added changes to https://github.com/magma/magma/wiki/Technical-Infrastructure - will close this issue