Solve missing Thoth dependencies

[mayaCostantini]

Is your feature request related to a problem? Please describe.

Some of thoth packages do not seem to be registered in the database, making thamos advise fail on the adviser repository.
Solve the following packages:

• thoth-solver
• fext
• termial-random

Additional context

  https://thoth-station.ninja/j/unresolved                 │ Resolver failed as it was unable to resolve direct        │ thoth-solver   │ ❌ ERROR   
                                                           │ dependency 'thoth-solver'                                 │                │            
  https://thoth-station.ninja/j/unresolved                 │ Resolver failed as it was unable to resolve direct        │ fext           │ ❌ ERROR   
                                                           │ dependency 'fext'                                         │                │            
  https://thoth-station.ninja/j/unresolved                 │ Resolver failed as it was unable to resolve direct        │ termial-random │ ❌ ERROR   
                                                           │ dependency 'termial-random'                               │                │            

[mayaCostantini]

/sig stack-guidance
/good-first-issue
/priority important-soon

[sesheta]

@mayaCostantini:
This request has been marked as suitable for new contributors.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue command.

In response to this:

/sig stack-guidance
/good-first-issue
/priority important-soon

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[goern]

do we have a "runbook"/piece of documentation what specifically needs to be done to add the packages? shall we create a tekton pipeline or a script for this?

[mayaCostantini]

I don't think a pipeline or a script would be necessary for the moment, but we can document how to register a package via management-api as I am not sure if this is already written somewhere.

[goern]

/triage accepted
/assign @shreekarSS

[shreekarSS]

Added missing dependencies using management api: https://management.stage.thoth-station.ninja/api/v1/ui/#/Python%20Solver/post_solve_python

To verify if dependencies are added or not, Tried endpoint https://khemenu.thoth-station.ninja/api/v1/ui/#/PythonPackages/list_python_packages but got no response ( issue thoth-station/user-api#1782 )

[goern]

@shreekarSS any update in this?

[shreekarSS]

@shreekarSS any update in this?

I tried to add missing dependencies via management api, getting 202 code, which is not registered the packages yet. I'm trying to figure it out.

[mayaCostantini]

@shreekarSS Did you try to use user API to verify if the packages are present in the database?

[harshad16]

i m taking a look at this,
there seems to be something broken on the solver pipeline
will post detail report shortly.

[harshad16]

Further investigation shows:

There is total 68k package unsolved already known to the system,
based on this , package * per version of is already triggered to be solved
there are about 900k+ solver runs to be executed, to fulfill these requirements.

Based on this, it would take 38 days to complete the overall queue. (i.e (900k {wf}/1000 {wf/hr})/24).

our systems are already on full force ingestion, without much space to increase the ingestion.

Actions we can do:

• keeping track of the solver ids of the package we triggered manually, setup an action to that would calculate the time it would be available, so user can use them.

[harshad16]

still in process of checking if anything is hindering in solver runs itself.

[harshad16]

Found the bug which was causing hindrance in data update.
solver were able to complete, however graph-sync is failing.

opening the issue in the relevant repo for fix.

[harshad16]

Here is the issue: thoth-station/storages#2703

Step to reach to bug:

1. Log in the cluster where solver is scheduled.
2. Check the thoth-middletier-<stage|prod> namespace where solvers are executed.
3. Check the argo list or oc get wf to check which worklfow are executing
4. Explore the failed ones.

[shreekarSS]

Added missing dependencies using management api: https://management.stage.thoth-station.ninja/api/v1/ui/#/Python%20Solver/post_solve_python

thoth-solver
"analysis_id": [
"solver-rhel-8-py38-220929160717-79968ff899dbab76",
"solver-fedora-34-py39-220929160717-c1ab2ea6e327e1e5",
"solver-fedora-35-py310-220929160717-104979b14c477b11"
],

fext
"analysis_id": [
"solver-rhel-8-py38-220929160806-983fbb5bb4dcd900",
"solver-fedora-34-py39-220929160806-e794f09be22a9dd8",
"solver-fedora-35-py310-220929160806-57e2e909a17e9400"
],

termial-random
"analysis_id": [
"solver-rhel-8-py38-220929160849-da44125b71eb2555",
"solver-fedora-34-py39-220929160849-90a0c3dc51c4684e",
"solver-fedora-35-py310-220929160849-5120239e134a8fc9"
],

[shreekarSS]

Packages have been added, closing the issue.
https://stage.thoth-station.ninja/api/v1/python/package?page=0&per_page=25&os_name=ubi&os_version=8&python_version=3.8&like=thoth-solver

https://stage.thoth-station.ninja/api/v1/python/package?page=0&per_page=25&os_name=ubi&os_version=8&python_version=3.8&like=fext

https://stage.thoth-station.ninja/api/v1/python/package?page=0&per_page=25&os_name=ubi&os_version=8&python_version=3.8&like=termial-random

[mayaCostantini]

Closing as completed