Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HPCC-23896 Missing ESP logging in cloud #13651

Merged
merged 1 commit into from
Apr 22, 2020
Merged

HPCC-23896 Missing ESP logging in cloud #13651

merged 1 commit into from
Apr 22, 2020

Conversation

richardkchapman
Copy link
Member

Signed-off-by: Richard Chapman rchapman@hpccsystems.com

Type of change:

  • This change is a bug fix (non-breaking change which fixes an issue).
  • This change is a new feature (non-breaking change which adds functionality).
  • This change improves the code (refactor or other change that does not change the functionality)
  • This change fixes warnings (the fix does not alter the functionality or the generated code)
  • This change is a breaking change (fix or feature that will cause existing behavior to change).
  • This change alters the query API (existing queries will have to be recompiled)

Checklist:

  • My code follows the code style of this project.
    • My code does not create any new warnings from compiler, build system, or lint.
  • The commit message is properly formatted and free of typos.
    • The commit message title makes sense in a changelog, by itself.
    • The commit is signed.
  • My change requires a change to the documentation.
    • I have updated the documentation accordingly, or...
    • I have created a JIRA ticket to update the documentation.
    • Any new interfaces or exported functions are appropriately commented.
  • I have read the CONTRIBUTORS document.
  • The change has been fully tested:
    • I have added tests to cover my changes.
    • All new and existing tests passed.
    • I have checked that this change does not introduce memory leaks.
    • I have used Valgrind or similar tools to check for potential issues.
  • I have given due consideration to all of the following potential concerns:
    • Scalability
    • Performance
    • Security
    • Thread-safety
    • Cloud-compatibility
    • Premature optimization
    • Existing deployed queries will not be broken
    • This change fixes the problem, not just the symptom
    • The target branch of this pull request is appropriate for such a change.
  • There are no similar instances of the same problem that should be addressed
    • I have addressed them here
    • I have raised JIRA issues to address them separately
  • This is a user interface / front-end modification
    • I have tested my changes in multiple modern browsers
    • The component(s) render as expected

Smoketest:

  • Send notifications about my Pull Request position in Smoketest queue.
  • Test my draft Pull Request.

Testing:

@richardkchapman
Copy link
Member Author

@jakesmith Please review

@hpcc-jirabot
Copy link

https://track.hpccsystems.com/browse/HPCC-23896
Jira updated

@richardkchapman
HPCC-23896 Missing ESP logging in cloud
d5c70a7 
Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
@HPCCSmoketest
Copy link
Contributor

Automated Smoketest: ✅
OS: centos 7.6.1810 (Linux 3.10.0-957.1.3.el7.x86_64)
GCC: gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
Host: ip-10-20-0-120.ca-central-1.compute.internal
Sha: d5c70a7
Build: success
Milestone:Install hpccsystems-platform-community_7.8.7-closedown0.el7.x86_64.rpm
HPCC Start: OK

Unit tests result:

Test total passed failed errors timeout elaps
unittest 120 120 0 0 0 54 sec
wutoolTest(Dali) 19 19 0 0 0 2 sec
wutoolTest(Cassandra) 19 19 0 0 0 7 sec

Regression test result:

phase total pass fail elaps
setup (hthor) 11 11 0 29 sec (00:00:29)
setup (thor) 11 11 0 43 sec (00:00:43)
setup (roxie) 11 11 0 23 sec (00:00:23)
test (hthor) 873 873 0 488 sec (00:08:08)
test (thor) 812 812 0 827 sec (00:13:47)
test (roxie) 953 953 0 380 sec (00:06:20)

HPCC Stop: OK
HPCC Uninstall: OK
Time stats:

Prep time Build time Package time Install time Start time Test time Stop time Summary
9 sec (00:00:09) 651 sec (00:10:51) 120 sec (00:02:00) 23 sec (00:00:23) 18 sec (00:00:18) 2030 sec (00:33:50) 18 sec (00:00:18) 2869 sec (00:47:49)

@HPCCSmoketest
Copy link
Contributor

Automated Smoketest: ✅
OS: centos 7.7.1908 (Linux 3.10.0-957.1.3.el7.x86_64)
GCC: gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
Host: ip-10-20-0-116.ca-central-1.compute.internal
Sha: d5c70a7
Build: success
Milestone:Install hpccsystems-platform-community_7.8.7-closedown0.el7.x86_64.rpm
HPCC Start: OK

Unit tests result:

Test total passed failed errors timeout elaps
unittest 120 120 0 0 0 54 sec
wutoolTest(Dali) 19 19 0 0 0 2 sec
wutoolTest(Cassandra) 19 19 0 0 0 9 sec

Regression test result:

phase total pass fail elaps
setup (hthor) 11 11 0 29 sec (00:00:29)
setup (thor) 11 11 0 44 sec (00:00:44)
setup (roxie) 11 11 0 20 sec (00:00:20)
test (hthor) 873 873 0 572 sec (00:09:32)
test (thor) 812 812 0 1010 sec (00:16:50)
test (roxie) 953 953 0 705 sec (00:11:45)

HPCC Stop: OK
HPCC Uninstall: OK
Time stats:

Prep time Build time Package time Install time Start time Test time Stop time Summary
9 sec (00:00:09) 915 sec (00:15:15) 124 sec (00:02:04) 22 sec (00:00:22) 18 sec (00:00:18) 2618 sec (00:43:38) 19 sec (00:00:19) 3725 sec (01:02:05)

jakesmith
jakesmith reviewed Apr 22, 2020
View reviewed changes
system/jlib/jlog.cpp
@@ -2350,6 +2351,11 @@ void setupContainerizedLogMsgHandler()
IPropertyTree * logConfig = queryComponentConfig().queryPropTree("logging");
if (logConfig)
{
if (logConfig->getPropBool(logQueueDisabledAtt, false))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps should also disable if traceLevel = 0?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

traceLevel is not something known about at jlib level (but yes, whole area of traceLevel vs loggingDetail is a bit of a mess)

jakesmith
jakesmith reviewed Apr 22, 2020
View reviewed changes
roxie/ccd/ccdmain.cpp
@@ -715,6 +715,8 @@ int CCD_API roxie_main(int argc, const char *argv[], const char * defaultYaml)
traceLevel = topology->getPropInt("@traceLevel", runOnce ? 0 : 1);
if (traceLevel > MAXTRACELEVEL)
traceLevel = MAXTRACELEVEL;
if (traceLevel && topology->hasProp("logging/@disabled"))
topology->setPropBool("logging/@disabled", false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a config file had disabled=true, I'd have thought that would override other logging settings like traveLevel.
But this means if traveLevel>0, it ignores @disabled flag.

Is this logic required?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's so that I can set traceLevel=1 when running a standalone program (where default logging is disabled)

@jakesmith
Copy link
Member

@richardkchapman - looks good, a couple of questions.

@richardkchapman
Copy link
Member Author

@jakesmith Questions answered

@ghalliday Please review/merge

@ghalliday ghalliday merged commit 23ef73f into hpcc-systems:candidate-7.8.x Apr 22, 2020
@richardkchapman richardkchapman deleted the missing-logs branch June 10, 2020 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jakesmith jakesmith jakesmith left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@richardkchapman @hpcc-jirabot @HPCCSmoketest @jakesmith @ghalliday