Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HPCC-17326 Rationalise the use of StTimeElapsed etc #10088

Merged
merged 1 commit into from Jun 16, 2017
Merged

HPCC-17326 Rationalise the use of StTimeElapsed etc #10088

merged 1 commit into from Jun 16, 2017

Conversation

ghalliday
Copy link
Member

Signed-off-by: Gavin Halliday gavin.halliday@lexisnexis.com

Type of change:

  • This change is a bug fix (non-breaking change which fixes an issue).
  • This change is a new feature (non-breaking change which adds functionality).
  • This change is a breaking change (fix or feature that will cause existing behavior to change).
  • This change alters the query API (existing queries will have to be recompiled)

Checklist:

  • My code follows the code style of this project.
    • My code does not create any new warnings from compiler, build system, or lint.
  • The commit message is properly formatted and free of typos.
    • The commit message title makes sense in a changelog, by itself.
    • The commit is signed.
  • My change requires a change to the documentation.
    • I have updated the documentation accordingly, or...
    • I have created a JIRA ticket to update the documentation.
    • Any new interfaces or exported functions are appropriately commented.
  • I have read the CONTRIBUTORS document.
  • The change has been fully tested:
    • I have added tests to cover my changes.
    • All new and existing tests passed.
    • I have checked that this change does not introduce memory leaks.
    • I have used Valgrind or similar tools to check for potential issues.
  • I have given due consideration to all of the following potential concerns:
    • Scalability
    • Performance
    • Security
    • Thread-safety
    • Premature optimization
    • Existing deployed queries will not be broken
    • This change fixes the problem, not just the symptom
    • The target branch of this pull request is appropriate for such a change.
  • There are no similar instances of the same problem that should be addressed
    • I have addressed them here
    • I have raised JIRA issues to address them separately
  • This is a user interface / front-end modification
    • I have tested my changes in multiple modern browers
    • The component(s) render as expected

Testing:

Run compiler regression suites, and submitted queries to eclccserver and eclserver. (eclccserver has problems with the types of some of the timings, but that will be addressed in HPCC-17693).

@hpcc-jirabot
Copy link

https://track.hpccsystems.com/browse/HPCC-17326
Jira updated

@ghalliday
HPCC-17326 Rationalise the use of StTimeElapsed etc
5fc13b7 
Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
@ghalliday
Copy link
Member Author

@shamser please review
See also https://github.com/hpcc-systems/LN/pull/845

@HPCCSmoketest
Copy link
Contributor

Automated Smoketest: ❌
Sha: 5fc13b7
Build: success
Build: success
ECL Watch: Rebuilding Site

errors warnings build time
0 75 43.778 seconds

Install hpccsystems-platform-community_6.5.0-trunk0.el7.x86_64.rpm
HPCC Start: OK

Unit tests result:

Test total passed failed errors timeout
unittest 94 94 0 0 0
wutoolTest(Dali) 19 19 0 0 0
wutoolTest(Cassandra) 19 19 0 0 0

Regression test result:

phase total pass fail
setup (hthor) 11 11 0
setup (thor) 11 11 0
setup (roxie) 11 11 0
test (hthor) 736 736 0
test (thor) 630 630 0
test (roxie) 765 764 1

Errors:

  • test
    • roxie
      • schedule2,

HPCC Stop: OK
HPCC Uninstall: OK

shamser
shamser reviewed Jun 12, 2017
View reviewed changes
Copy link
Contributor

@shamser shamser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should wutool->test suite need be updated with the new scope?

common/workunit/workunit.cpp
@@ -5960,11 +5953,6 @@ void CLocalWorkUnit::_loadStatistics() const
IConstWUStatisticIterator& CLocalWorkUnit::getStatistics(const IStatisticsFilter * filter) const
{
CriticalBlock block(crit);
//This should be deleted in version 6.0 when support for 4.x is no longer required
legacyTimings.loadBranch(p,"Timings");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ghalliday Does the removal of support for legacy timings need to be documented?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so - it would be very unusual to have a workunit from 4.0 in a 7.0 system. If they did the only problem would be timings didn't appear any more. @richardkchapman do we document things like this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No

ecl/eclcc/eclcc.cpp
@@ -2525,8 +2531,6 @@ void EclCC::processBatchedFile(IFile & file, bool multiThreaded)
if (info.wu &&
(info.wu->getDebugValueBool("generatePartialOutputOnError", false) || info.queryErrorProcessor().errCount() == 0))
{
WuStatisticTarget statsTarget(info.wu, "eclcc");
gatherTransformStats(statsTarget);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ghalliday Another change in behaviour.. require documentation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has now been made conditional rather than unconditional, and the option has been added to the eclcc help.
@shamser what kind of documentation do you suggest?

shamser
shamser reviewed Jun 12, 2017
View reviewed changes
ecl/hqlcpp/hqlnlp.cpp
}

gatherExplicitMatched(expr);
doBuildParseTransform(instance->startctx, expr); // also gathers all the MATCHED() definitions.
doBuildParseSearchText(instance->startctx, expr);
doBuildParseValidators(instance->nestedctx, expr);
doBuildParseExtra(instance->startctx, expr);
noteFinishedTiming("compile:generate PARSE:prepare", startPrepareCycles);
if (options.timeTransforms)
noteFinishedTiming("compile:PARSE:prepare", startPrepareCycles);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is 'PARSE' capitalized? Seems inconsistant with other timers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it corresponds to the PARSE keyword.
I could change to lower case, but then it might be confused with the eclcc parser, so I would keep it as is, even though it isn't consistent.

shamser
shamser reviewed Jun 12, 2017
View reviewed changes
ecl/hqlcpp/hqlnlp.cpp
@@ -727,7 +728,8 @@ ABoundActivity * HqlCppTranslator::doBuildActivityParse(BuildCtx & ctx, IHqlExpr
WARNING1(CategoryEfficiency, HQLWRN_GrammarIsAmbiguous, instance->activityId);

doBuildParseCompiled(instance->classctx, buffer);
noteFinishedTiming("compile:generate PARSE:compile", startCompileCycles);
if (options.timeTransforms)
noteFinishedTiming("compile:PARSE:compile", startCompileCycles);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, not sure why 'PARSE' is capitalized?

shamser
shamser reviewed Jun 12, 2017
View reviewed changes
ecl/hqlcpp/hqlnlp.cpp
@@ -760,7 +762,8 @@ ABoundActivity * HqlCppTranslator::doBuildActivityParse(BuildCtx & ctx, IHqlExpr
nlpParse = NULL;
buildInstanceSuffix(instance);
buildConnectInputOutput(ctx, instance, boundDataset, 0, 0);
noteFinishedTiming("compile:generate PARSE", startCycles);
if (options.timeTransforms)
noteFinishedTiming("compile:PARSE", startCycles);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, not sure why 'PARSE' is capitalized?

shamser
shamser reviewed Jun 12, 2017
View reviewed changes
Copy link
Contributor

@shamser shamser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ghalliday Reviewed... a few comments added.

shamser
shamser approved these changes Jun 13, 2017
View reviewed changes
Copy link
Contributor

@shamser shamser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@richardkchapman Code reviewed.. Looks good

@ghalliday
Copy link
Member Author

@shamser see replies.

@ghalliday
Copy link
Member Author

I don't think there is any need to change compile:parseTime in wutool

@shamser
Copy link
Contributor

shamser commented Jun 13, 2017

@ghalliday @richardkchapman Code reviewed.. Looks good

@richardkchapman richardkchapman merged commit daf34d4 into hpcc-systems:master Jun 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richardkchapman richardkchapman richardkchapman left review comments

@shamser shamser shamser approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@ghalliday @hpcc-jirabot @HPCCSmoketest @shamser @richardkchapman