[tracker] Update and expand Thoth architecture diagrams

[codificat]

Is your feature request related to a problem? Please describe.

Thoth is relatively complex, and it can be challenging to understand what are all the components and their interactions - both internal and external.

High-level Goals
list of high-level Acceptance Criteria/Goals (that is signed off by the team, and unchangeable)

As an individual who wants to understand how Thoth works, I would like to have a set of diagrams about Thoth components so that I can have a visual understanding of its architecture, integrations, and interaction points.

Describe the solution you'd like

A single diagram would not be enough here, because in order to be comprehensive it would have to be too complex, so multiple diagrams would be involved:

• a simple and high level overview diagram, showing external integration/interaction and only component names
• a diagram per component, i.e. a zoom in at each of the components of the overview

Describe alternatives you've considered

Additional context
Add any other context or screenshots about the feature request here.

There are some existing diagrams that can contribute to the whole story:

• https://github.com/thoth-station/core/blob/master/docs/architecture.png could probably be updated as part of this effort
• aicoe-ci pipelines https://github.com/AICoE/aicoe-ci#architecture
• meteor deep dive diagram on aicoe-pipelines Add deep dive into AICoE Pipelines to Meteor Flowchart AICoE/meteor#62
• the Kebechet diagram at Create a schema diagram for Kebechet AICoE/data-driven-development#24

Acceptance Criteria
checklist of detailed Acceptance Criteria (that might extend over the lifetime of the card)

• high level overview diagram
• component diagrams
  • AICoE pipelines: Create a general documentation for Thoth CI/CD Pipelines AICoE/aicoe-ci#153
• links exist between diagrams (e.g. a single reference page)
• the various diagrams are visually consistent

[pacospace]

See also investigator docs: https://github.com/thoth-station/investigator

[codificat]

Also possibly related / could be part of this: #253

[pacospace]

Also possibly related / could be part of this: #253

that issue might be closed, as we will consider sig-store, maybe it can be repurposed

[pacospace]

Some example of the sequence diagrams, let me know what you think! https://app.diagrams.net/#G1972Ny8BnPgsVUcF85r6qmj_H5AvqnQaa

[pacospace]

[pacospace]

[pacospace]

[goern]

/triage accepted
/priority important-soon
/assign @pacospace
/assign @codificat
/kind demo

[pacospace]

new Thoth diagram https://app.diagrams.net/#G1XN1L8KLNtUNPIYL1QHEFcBv6lvcjN63M

[codificat]

Many thanks for these diagrams @pacospace, they look great!

Some comments about them:

• For the first two (internal and external triggers) I would probably just use the complete version of each. Maybe make the "main flow" black line slightly thicker, like you did with the last of the diagrams, for better clarity... but at that level of detail, we probably don't need a "more simple" version of each
• In the internal trigger diagram, I believe it would be better to have "gitub" be a small box towards the bottom, where the issue/pr arrow goes, similar to the thoth diagram, instead of taking all the left margin.

Beyond that, a more general comment about the approach here, in the context of this issue overall.

These are great diagrams to understand the details. However, we are still missing the higher level / general overview ones. In that sense, I wonder if it would not be better to start with these high-level ones: I believe it would help us achieve more consistency across the whole set.

In practical terms: this being a tracker issue for a "whole story" set of diagrams, should we split individual diagrams to their own separate issue/PR?

As a side note: I can't access the direct links to the diagram sources. Sharing the drawio file should work, but I'm not sure how practical that is.

[pacospace]

Many thanks for these diagrams @pacospace, they look great!

Some comments about them:

• For the first two (internal and external triggers) I would probably just use the complete version of each. Maybe make the "main flow" black line slightly thicker, like you did with the last of the diagrams, for better clarity... but at that level of detail, we probably don't need a "more simple" version of each
• In the internal trigger diagram, I believe it would be better to have "gitub" be a small box towards the bottom, where the issue/pr arrow goes, similar to the thoth diagram, instead of taking all the left margin.

I ll adjust those thanks!

Beyond that, a more general comment about the approach here, in the context of this issue overall.

These are great diagrams to understand the details. However, we are still missing the higher level / general overview ones. In that sense, I wonder if it would not be better to start with these high-level ones: I believe it would help us achieve more consistency across the whole set.

what kind of high level ones are we talking about? is the thoth diagram good for high level of all components we maintain?

In practical terms: this being a tracker issue for a "whole story" set of diagrams, should we split individual diagrams to their own separate issue/PR?

Sounds good to open one for each diagram we want, thanks!

As a side note: I can't access the direct links to the diagram sources. Sharing the drawio file should work, but I'm not sure how practical that is.

I ll check why the links do not work!

[codificat]

/sig docs

[goern]

ping?

[sesheta]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[sesheta]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

[sesheta]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

[sesheta]

@sesheta: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[codificat]

/reopen
/lifecycle frozen

discussing this in today's SIG-UX meeting

[sesheta]

@codificat: Reopened this issue.

In response to this:

/reopen
/lifecycle frozen

discussing this in today's SIG-UX meeting

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.