[develop] Adding jids in master & minion logs #48660

garethgreenaway · 2018-07-18T20:41:28Z

What does this PR do?

This change adds the optional ability to include the JIDs in the master & minion logs for all entries related to a particular Salt job.

What issues does this PR fix or reference?

#45282

Commits signed with GPG?

Yes

Please review Salt's Contributing Guide for best practices.

See GitHub's page on GPG signing for more information about signing commits with GPG.

isbm

Unit Tests? 😉

gtmanfred

Nice, well done.

rallytime

I have a very minor change request.

Otherwise, great idea using tornado's StackContext!

rallytime · 2018-07-20T14:22:08Z

doc/ref/configuration/logging/index.rst

This should be log_fmt_jid

garethgreenaway · 2018-07-27T21:49:20Z

Tests will pass under Python3 if logging is not disabled

➜ python ~/code/salt/tests/runtests.py -n integration.logging.test_jid_logging
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 * Python Version: 3.6.5 (default, Jul 27 2018, 13:22:44) [GCC 7.3.0]
 * Transplanting configuration files to '/tmp/salt-tests-tmpdir/config'
 * Current Directory: /home/gareth/code/salt
 * Test suite is running under PID 18747
 * Logging tests on /tmp/salt-runtests.log
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 * Max open files settings is too low (soft: 1024, hard: 4096) for running the tests
 * Trying to raise the limits to soft: 3072, hard: 4096
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 * Setting up Salt daemons to execute tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 * Starting salt-master ... STARTED!                                                                                                                                                                                                          
 * Starting salt-minion ... STARTED!                                                                                                                                                                                                          
 * Starting sub salt-minion ... STARTED!                                                                                                                                                                                                      
 * Starting syndic salt-master ... STARTED!                                                                                                                                                                                                   
 * Starting salt-syndic ... STARTED!                                                                                                                                                                                                          
==============================================================================================================================================================================================================================================
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Starting integration.logging.test_jid_logging Tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.
----------------------------------------------------------------------
Ran 1 test in 0.272s

OK

===========================================================================================================  Overall Tests Report  ===========================================================================================================
***  No Problems Found While Running Tests  **************************************************************************************************************************************************************************************************
==============================================================================================================================================================================================================================================
OK (total=1, skipped=0, passed=1, failures=0, errors=0) 
===========================================================================================================  Overall Tests Report  ===========================================================================================================

isbm

👍

rallytime · 2018-07-31T15:40:53Z

@garethgreenaway One lint failure here. Can you fix that up, please?

https://jenkinsci.saltstack.com/job/pr-lint/job/PR-48660/2/warnings52Result/

…om times.

…mport for logging to import the salt.log.setup to ensure that we're using the customized salt logging.

…he minion logs as well.

…Context so we have control the format of what the JID will look like in the logs.

…ormat for log_fmt_jid.

… if it is included in the log format string.

…ng for PY3.

garethgreenaway · 2018-08-01T03:49:26Z

re-run all

cachedout

This is extremely well-done. Hats off, @garethgreenaway

garethgreenaway requested a review from a team as a code owner July 18, 2018 20:41

ghost self-requested a review July 18, 2018 20:41

isbm suggested changes Jul 19, 2018

View reviewed changes

gtmanfred approved these changes Jul 19, 2018

View reviewed changes

rallytime suggested changes Jul 20, 2018

View reviewed changes

doc/ref/configuration/logging/index.rst Outdated

Copy link

Contributor

rallytime Jul 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be log_fmt_jid

rallytime added the Awesome label Jul 20, 2018

rallytime requested a review from cachedout July 20, 2018 14:26

rallytime approved these changes Jul 20, 2018

View reviewed changes

garethgreenaway force-pushed the jids_in_logs branch 2 times, most recently from 8ee602b to 96cc656 Compare July 27, 2018 21:37

isbm approved these changes Jul 31, 2018

View reviewed changes

garethgreenaway added 15 commits July 31, 2018 09:19

Initial efforts to add jids in logs

85475f3

Current state. Almost works but JID is unavailable are seemingly rand…

1b595b9

…om times.

Pop the jid out of extra then add it to the logrecord. Swap out the i…

4856fe9

…mport for logging to import the salt.log.setup to ensure that we're using the customized salt logging.

Adding utils/ctx.py for RequestContext

e28d703

Adding RequestContext to minion.py so we can get the JID logging in t…

286c3bd

…he minion logs as well.

Introduce "log_fmt_jid" and ensure we are passing opts to the Request…

ffd1a8a

…Context so we have control the format of what the JID will look like in the logs.

Adding some notes to the Fluorine release notes. Adding the default f…

3b8a4ee

…ormat for log_fmt_jid.

Adding information about log_fmt_jid to logging documentation

cc14dff

Documentation fix.

16172dc

Adding an integration test to test that JIDs will show up in the logs…

ee1ef37

… if it is included in the log format string.

Adding integration/logging to runtests.py

ca4dd6c

Updaitng runtests.py for logging tests.

8f4d556

Fixing lint issues

534b13a

General cleanup of integration.logging.test_jid_logging test. Disabli…

18e7355

…ng for PY3.

Fixing lint error on six import.

c97e65e

garethgreenaway force-pushed the jids_in_logs branch from 6d3d0df to c97e65e Compare July 31, 2018 16:19

rallytime added the Fluorine label Jul 31, 2018

Merge branch 'develop' into jids_in_logs

121d68b

rallytime added the Tests-Passed label Aug 2, 2018

Merge branch 'develop' into jids_in_logs

f8f6ff8

cachedout approved these changes Aug 3, 2018

View reviewed changes

rallytime merged commit 8c5b369 into saltstack:develop Aug 3, 2018

This was referenced May 13, 2024

[master] Remove redundant run_func from salt.master.MWorker._handle_aes #66509

Merged

Remove redundant run_func from salt.master.MWorker._handle_aes openSUSE/salt#650

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[develop] Adding jids in master & minion logs #48660

[develop] Adding jids in master & minion logs #48660

Uh oh!

garethgreenaway commented Jul 18, 2018

Uh oh!

isbm left a comment

Uh oh!

gtmanfred left a comment

Uh oh!

rallytime left a comment

Uh oh!

rallytime Jul 20, 2018

Uh oh!

garethgreenaway commented Jul 27, 2018 •

edited

Loading

Uh oh!

isbm left a comment

Uh oh!

rallytime commented Jul 31, 2018

Uh oh!

garethgreenaway commented Aug 1, 2018

Uh oh!

cachedout left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[develop] Adding jids in master & minion logs #48660

[develop] Adding jids in master & minion logs #48660

Uh oh!

Conversation

garethgreenaway commented Jul 18, 2018

What does this PR do?

What issues does this PR fix or reference?

Commits signed with GPG?

Uh oh!

isbm left a comment

Choose a reason for hiding this comment

Uh oh!

gtmanfred left a comment

Choose a reason for hiding this comment

Uh oh!

rallytime left a comment

Choose a reason for hiding this comment

Uh oh!

rallytime Jul 20, 2018

Choose a reason for hiding this comment

Uh oh!

garethgreenaway commented Jul 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

isbm left a comment

Choose a reason for hiding this comment

Uh oh!

rallytime commented Jul 31, 2018

Uh oh!

garethgreenaway commented Aug 1, 2018

Uh oh!

cachedout left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

garethgreenaway commented Jul 27, 2018 •

edited

Loading