Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New module which filters on logical expressions of Paths #20028

Merged
merged 2 commits into from Aug 4, 2017

Conversation

wddgit
Copy link
Contributor

@wddgit wddgit commented Aug 2, 2017

No description provided.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 2, 2017

A new Pull Request was created by @wddgit (W. David Dagenhart) for master.

It involves the following packages:

FWCore/Modules

@cmsbuild, @smuzaffar, @Dr15Jones can you please review it and eventually sign? Thanks.
@Martin-Grunewald this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

@wddgit
Copy link
Contributor Author

wddgit commented Aug 2, 2017

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 2, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22026/console Started: 2017/08/02 21:23

virtual bool filter(StreamID, Event&, EventSetup const&) const override final;

private:
std::unique_ptr<pathStatusExpression::Evaluator> evaluator_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see of edm::propagate_const can be used here.

}

private:
std::unique_ptr<Evaluator> operand_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::propagate_const here as well.

std::unique_ptr<Evaluator> operand_;
};

class AndOperator : public Evaluator {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could templatize the binary operator so you can share code between And and Or.

public:
EvaluatorType type() const { return Not; }

virtual void setLeft(std::unique_ptr<Evaluator> && v) { operand_ = std::move(v); }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

override

public:
EvaluatorType type() const { return And; }

virtual void setLeft(std::unique_ptr<Evaluator> && v) { left_ = std::move(v); }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

override on both of these

}

private:
std::unique_ptr<Evaluator> left_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::propagate_const

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 2, 2017

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 2, 2017

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20028/22026/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 25
  • DQMHistoTests: Total histograms compared: 2625965
  • DQMHistoTests: Total failures: 46700
  • DQMHistoTests: Total nulls: 211
  • DQMHistoTests: Total successes: 2578873
  • DQMHistoTests: Total skipped: 181
  • DQMHistoTests: Total Missing objects: 0
  • Checked 102 log files, 14 edm output root files, 25 DQM output files

@Martin-Grunewald
Copy link
Contributor

What is the difference wrt the TriggerResultsFilter ?

@Dr15Jones
Copy link
Contributor

@Martin-Grunewald TriggerResultsFilter must wait until the TriggerResults is created which happens after all Paths have finished. This new module uses the individual PathStatus which are available once the Path has finished. This allows one to use the new module to use the results of one Path to influence the results of another Path. That is not something that TriggerResultsFilter can do.

@Martin-Grunewald
Copy link
Contributor

Ah, OK.

What happens if one instance is on path A asking for path B and one instance is on path B asking for path A - how is the deadlock resolved? Or A->B->C->D->A?

@Dr15Jones
Copy link
Contributor

What happens if one instance is on path A asking for path B and one instance is on path B asking for path A - how is the deadlock resolved? Or A->B->C->D->A?

The framework will throw an 'unrunnable schedule' exception before it starts to process data.

@Martin-Grunewald
Copy link
Contributor

I suppose the same throw if an instance asks for a non-existing path, right?

@Dr15Jones
Copy link
Contributor

I assume under that case you'd get a 'product missing' exception at run time.

The 'unrunnable schedule' is something the framework would throw anytime two modules say they depend on each other.

The PathStatus object is just threated like any other data product in the Event.

@wddgit
Copy link
Contributor Author

wddgit commented Aug 3, 2017

Yes a Product Not Found exception is what you would get with the current version of this (although I could probably change that if we wanted some other error)

@Martin-Grunewald
Copy link
Contributor

OK, thanks for the clarifications!

@wddgit
Copy link
Contributor Author

wddgit commented Aug 3, 2017

A few more differences. This new filter does not support prescalers. It does not support reading path results from prior processes, only the current one. It does not get L1 trigger results. And there are some other options in TriggerResultsFilter that it does not support. The new filter does not support the XOR operator. (These things could be added, but they are not in the current version of the new filter and as far as I know we do not plan to add them)

One other comment I will add. I could be completely wrong about this because I did not test it or spend a lot of time studying it. I likely just didn't understand the code. But in reading through it, I got the impression that TriggerResultsFilter did not get precedence order of the operators correct unless you used parentheses and that the NOT operator would not work. But most likely I just didn't understand the code because I didn't spend enough time studying it. Are there tests for those things somewhere?

@Martin-Grunewald
Copy link
Contributor

Martin-Grunewald commented Aug 3, 2017

@fwyzard
re @wddgit git comment on issues with the TriggerResultsFilter

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2017

Pull request #20028 was updated. @cmsbuild, @smuzaffar, @Dr15Jones can you please check and sign again.

@wddgit
Copy link
Contributor Author

wddgit commented Aug 3, 2017

The commit I just pushed should resolve all the comments from Chris. All good suggestions and I implemented them all.

@wddgit
Copy link
Contributor Author

wddgit commented Aug 3, 2017

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22078/console Started: 2017/08/03 21:03

@Dr15Jones
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2017

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @davidlange6, @smuzaffar (and backports should be raised in the release meeting by the corresponding L2)

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2017

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2017

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2017

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20028/22078/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 25
  • DQMHistoTests: Total histograms compared: 2643105
  • DQMHistoTests: Total failures: 29993
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2612931
  • DQMHistoTests: Total skipped: 181
  • DQMHistoTests: Total Missing objects: 0
  • Checked 102 log files, 14 edm output root files, 25 DQM output files

@davidlange6
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 3177825 into cms-sw:master Aug 4, 2017
@wddgit wddgit deleted the pathStatusFilter branch November 29, 2017 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants