ignore some jobs on highstate page

[erwindon]

Hello,
it's not a bug, but questions or improvements.
I am using the separate SaltGUI (1.28.0) solution in a nginx container (nginx:1.21.6).
Everything works fine but I have 2 problems with the hightstate page:

1. my saltstack master uses a Gitlab as a backend, so to deploy a specific branch with salt, I use this command on the host as an example: salt-call state.apply saltenv=XXX pillarenv=XXX test=true (XXX = git branch)
   The problem is that the result of the command will also be displayed on the saltgui hightstate page, but we should only see the results of the master branch and not the result of my XXX branch, because only the master branch is important to define if the server is in highstate or not.
   Is it possible to hide these parasitic results in order to keep only the results of the master branch? or any other solution (like to show only the state.highstate and hide everything else) ?

2. I have a lot of minions and states, and in the highstate page, the result of the highstate is far too long:
   Is it possible to have only the final result ? and the detail if we click on the green/yellow/red circle ? or another proper solution.
   I already tried this solution: https://github.com/erwindon/SaltGUI#performance, but I didn't see any change, maybe because I use a separate SaltGUI.
   Screenshots

thanks a lot.

NOTE: nr. 2 is now in #459
NOTE: question about salt-syndic (below) is now in #460

[pju51]

Is there also a solution if we use several salt masters (multimaster with salt-syndic) ?
(see #460)

[erwindon]

@pju51
you can ask unlimited questions, but please limit to 1 question per github-issue.
I will ignore the 2nd question (long lines) and 3rd question (salt-syndic) in this issue.
just copy those to new github issues please.
(I cannot do that for you because then I become the owner of the issues)

[erwindon]

I'm not considering to be selective on the commands that were used (state.highstate/state.apply/etc) as that will differ too much per user.

I see the following solutions:

1. adjust the page to group the results per saltenv and/or per pillarenv
2. add a configuration parameter to the /etc/salt/master file that contains a list of saltenv to ignore (or include)

one of the consequences of both solutions is that it might not be sufficient to look for the latest 10 highstate related commands. e.g. the latest 10 jobs might have been for XXX leading to no results being shown.
the only available alternative is to add metadata to the important jobs, so that they can be found more easily using the metadata search facility. unfortunately, function runners.jobs.list_jobs does not have a search parameter for saltenv (or pillarenv). the easiest solution for this additional problem is to change constant MAX_HIGHSTATE_JOBS into a configuration parameter.

From the proposed 2 solutions, I think that nr. 2 is the easiest one as there are no GUI changes involved.
The detailed activities for nr. 2 are:

• add parameter saltgui_hide_envs / saltgui_show_envs
• ignore all jobs on page 'highstate' that have a saltenv named parameter, and which is (not) listed in these variables.
• make MAX_HIGHSTATE_JOBS a new configuration item saltgui_max_highstate_jobs (default=10)
• adjust the hidden page (ctrl-click on title 'SaltGUI') to show the 3 new parameters

the new parameters may be used to limit the jobs shown on the jobs-page, or on the jobs-panel. but there wouldn't be a way to make the jobs visible again. so I'll ignore that possibility for now.

[pju51]

Thanks for your answer, and I will create new tickets for my other questions, sorry.

from my point of view I think the results of state.apply should be in the job page and state.highstate in the highstate page.
the solution to hide is not possible because we will constantly create new git branches, so I cant create a list, maybe a wildcard
(*) is accepted ?

the other thing, if I run this command:
salt-call state.apply STATE_NAME test=true
I will have also the result in the hightstate page when it should be in the job page, because in my command I specified which state I want to deploy not all.
A highstate status should not be linked to a particular state but to all states applied to a server, so the result of my command should not be in the highstate page

[erwindon]

the solution to hide is not possible because we will constantly create new git branches,

that's why I use 2 lists: saltgui_hide_envs / saltgui_show_envs
only one of these is filled in
in your case that would be "saltgui_show_env: master"
and all other env-s are hidden

[pju51]

Ah yes can be a good solution.

I tested; I added saltgui_show_env: master in my /etc/salt/master.d/master.conf file on my saltstack master server
then I restarted the services salt-api and salt-master but I still have the problem.

Did I miss anything?

on the hidden page I can't find this information added.

[erwindon]

Did I miss anything?

:-)
you made the assumption that this was already created.
but no, it is still only the solution proposal for the questions in this github issue

Ah yes can be a good solution.

I will go ahead to create this solution.
It will take a few days to build that.
I'll invite you to review the PR (a PR to myself) that I create for it

[pju51]

Did I miss anything?

:-) you made the assumption that this was already created. but no, it is still only the solution proposal for the questions in this github issue

Ah yes can be a good solution.

I will go ahead to create this solution. It will take a few days to build that. I'll invite you to review the PR (a PR to myself) that I create for it

my bad, yes, I thought it was already implemented and that I had missed something lol :D

ok I will review and test when its ready, no problem.

[erwindon]

note: changes will be on branch highstate-envs.

note: during build of this feature, I realized that there are several occasions where structured data is read back from the session storage. I added 2 utility functions to convert such data back to an object or an array.

note: during build of this feature, I realized that there are several occasions where allowlist+denylist matching was used. I added a utility function to compare an item to an allowlist and a denylist in one function call.

[erwindon]

@pju51: the configuration parameters have a slightly different name now:
saltgui_show_saltenvs
or
saltgui_hide_saltenvs
to prevent any confusion with pillar-envs; and to indicate that the parameter can take a list of names (a string value is considered to be a 1-item list)

[erwindon]

@pju51
Can you please give the new HighState page in this branch a try?

I guess that this solution should be extended to the "Issues" page. That page tries to find failed states (and a handful of other categories). But let's do this one page at a time.

[pju51]

@erwindon
Thanks, I'll test tomorrow and let you know the result.

[pju51]

@erwindon
So I added in my saltmaster config:
saltgui_show_saltenvs: master
then I restarted the services: salt-api + salt-master

After this I used your branch highstate-envs, but seems not work:

I always see the result of other branches in the highstate page.

also on the hidden page, I dont see the variablessaltgui_hide_saltenvs or saltgui_show_saltenvs, but maybe its normal?!

[pju51]

I also tested :

saltgui_show_saltenvs:
    - base

saltgui_show_saltenvs:
    - master

unfortunately same result, not work.

[erwindon]

You are using the correct branch.
Even when the behaviour is broken, you should see the new setting on the (hidden) Options page.

just in case this is an effect of the browser-cache, can you SHIFT-refresh the relevant pages?
In all known browers, shift-refresh ignores the brower-cache and reloads the actual page.
Otherwise, I may not have pushed my changes, but I can only verify that in a few hours time...

[erwindon]

I did push out all changes, so that was not the problem.
So I'm hoping that the SHIFT-refresh (or even the normal refresh) fixes the issue...

[pju51]

So I tested again, and after a redeployment of saltgui and a refresh of the hidden page, I have the new settings, but unfortunately I see no more results, I tested both parameters:

saltgui_show_saltenvs: base

saltgui_show_saltenvs: master

I can't get any more results on the highstate page, whether I use a gitlab branch or not in my salt command but I have the result in jobs page

also another problem, when I use your branch, I lose these parameters when I want to test (or apply) salt on a minion :

when I should have something like this:

[pju51]

Hello @erwindon ,

it was wrong, because I added :
saltgui_show_saltenvs: base (and master after for test)

so normally it should exclude from display all commands that do not use the branch base (or master) but in my case it displays them. I don't use an old code, I pull from your repo. Otherwise I didn't understand what saltgui_show_saltenvs is for.

from what I understood the parameter saltgui_show_saltenvs: allows to display only the commands which use the specified env?

maybe because of the browser cache, so I will try again with your new code + and cache refresh between each tests.

[erwindon]

@pju51
welcome back!

In case you are wondering... I just forgot to mark this one as 'draft' :-)

(and no rebase was needed here)

[pju51]

I pulled the branch again, I tried in private mode on my browser (+ SHIFT-refresh to be sure)

My saltstack master is configured with "saltgui_show_saltenvs: master" and salt services (salt-api, salt-master) has been restarted.

I executed this on saltgui:

and the result :

I added a pillar env but the result still appear on the highstate page when it should not.
what should I see in the browser console ??

[pju51]

humm very weird...
currently it doesn't show me any commands in the hightstate page, as if it was filtered and I haven't changed anything from my last test.
Also, I always see the cmds on the right panel (recents jobs) in the high status page.

I will test with this parameter
saltgui_show_saltenvs: base

EDIT:
I tested with saltgui_show_saltenvs: base and same, I have no result in the hightstate page with or without pillars env in my cmd). Its very weird.

EDIT2:
I confirm its the filter because when I switch to the master branch, I have the result in the hightstate page. So it's a good news, the filter works but the it doesn't seem to recognise my key (base/master)

[erwindon]

@pju51
let's clear one confusion...
the filter works on the parameter saltenv= that was included in the command.
therefore, the only reasonable setting is: saltgui_show_saltenvs: SOP-4122, with exactly the same value as it was used in the command.

with the code in this branch, the browser console should show 2 types of lines:

1. one line with an overview of the parameters saltgui_show_saltenvs and saltgui_hide_saltenvs as they will be used by SaltGUI. both should be array values. when a single string was configured as value, it will be shown as a 1-element array here.
2. one line with the JOB-ID and the SALTENV per job that was found (each job may apply to multiple minions).

Using saltgui_show_saltenvs: base will only find the jobs that really have a parameter saltenv=base. SaltGUI cannot know what the default saltenv is, even though in 99% of the cases it would be base, it can even be different per minion.
Using saltgui_show_saltenvs: master will most likely not be useful since that is a git branch-name, which neither SaltStack, not SaltGUI has any knowledge of.
Using ``: defaultwill catch the jobs that do not have a parametersaltenv=` (or a parameter `saltenv=default`). That was an addition I made just before your holiday.

can you please specify a single combination that is NOT working?

[pju51]

it's simple.

if I add this :
saltgui_show_saltenvs: master
I would like to see only the result of this command in the highstate page :
salt-call state.apply test=true (with or without test=true)

and if I try with a saltenv parameter like :
salt-call state.apply saltenv=XXX pillarenv=XXX test=true (with or without test=true)
it should not be displayed.

[erwindon]

@pju51

if I add this :
saltgui_show_saltenvs: master
I would like to see only the result of this command in the highstate page :
salt-call state.apply test=true (with or without test=true)

and if I try with a saltenv parameter like :
salt-call state.apply saltenv=XXX pillarenv=XXX test=true (with or without test=true)
it should not be displayed.

I cannot literally add the first part to SaltGUI as the word master is the name of a git branch.
But I already implemented the same functionality, using the keyword default.
Except for this detail, it should work like you expect.

[pju51]

Thanks @erwindon I will test soon.

I was a little busy previous weeks

[pju51]

Hello @erwindon

so I tested and....
it works like a charm with the keyword default ! 💯

This MR is done for me, I validated and you can merge ! Thanks a lot

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication