Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Recursive Driver Into Distributed Driver #1351

Open
wants to merge 77 commits into
base: master
from

Conversation

Projects
None yet
6 participants
@alenaizan
Copy link
Contributor

commented Nov 10, 2018

Needs Py36

Description

Starting PR. Change the Psi4 driver from being recursive to being based on compute classes that have planning, computing and querying functions. Then, integrate the new driver with the QCArchive project for parallelizing nbody, CBS and findif.

Todos

Notable points (developer or user-interest) that this PR has or will accomplish.

  • Create compute classes for nbody, CBS and findif
  • Integrate QCArchive

Questions

  • Question1

Checklist

Status

  • Ready for review
  • Ready for merge
@PeterKraus

This comment has been minimized.

Copy link
Contributor

commented Nov 10, 2018

This looks great. It's good to have the large monolithic chunks of driver code split into smaller parts, printing functions tucked away and not scattered across calculations. Counterpoise-corrected CBS is something I wanted to do for a while!

One point I'd like to make quite early in this PR: the whole pydantic validation mechanism is rather obscure. I understand that each Computer inherits from the base class, but I am not 100% sure how things fit together and launching what will get me where. The two tests are currently only of little help.

@dgasmith

This comment has been minimized.

Copy link
Member

commented Nov 10, 2018

This is pretty early days on this PR. The short is that a Computer can be CBS/FindIf/N-Body/Single/etc (?) so that you can nest them and where a Single is the only object that calls quantum chemistry. For example, a N-Body can be made up of Single computations or CBS computations where a CBS computation will be made up for two or more Single computations. In this case, calling compute on N-Body will call compute on all of its children which if is a Single will call a quantum chemistry computation or if CBS will call 2 or more single computations (or FindIf!) and then construct that data into an extrapolated energy which will be given to N-Body.

The key here is that the Single can have three modes 1) a continuous computation on a single node 2) a sow/reap mode using files and 3) a sow/reap mode using QCFractal. There will assuredly be many more docs.

@loriab loriab added this to the Psi4 1.4 milestone Dec 6, 2018

@loriab loriab force-pushed the dgasmith:recursive branch from 63b11c0 Dec 10, 2018

@loriab loriab force-pushed the dgasmith:recursive branch to ab6b312 Dec 19, 2018

@dgasmith dgasmith force-pushed the dgasmith:recursive branch from ab6b312 to fb38c2c Jan 1, 2019

@loriab loriab force-pushed the dgasmith:recursive branch from fb38c2c to 6752897 Jan 3, 2019

Show resolved Hide resolved psi4/driver/driver_nbody.py Outdated

@alenaizan alenaizan force-pushed the dgasmith:recursive branch from bb55d69 to 59b34f1 Jan 13, 2019

@dgasmith dgasmith referenced this pull request Jan 15, 2019

Closed

Add GenericCollection #111

@loriab loriab force-pushed the dgasmith:recursive branch from 4448c8a to 11da5f6 Feb 20, 2019

@loriab loriab referenced this pull request Mar 6, 2019

Merged

more qcvars -- mcscf, chemps2, psimrcc #1570

7 of 7 tasks complete

@JonathonMisiewicz JonathonMisiewicz referenced this pull request Mar 6, 2019

Merged

dertype and gradient passing #1551

4 of 4 tasks complete

@loriab loriab force-pushed the dgasmith:recursive branch 2 times, most recently from 16e1628 to 4e10c56 Mar 11, 2019

@loriab loriab changed the title Refactor Recursive Driver Refactor Recursive Driver Into Distributed Driver Mar 17, 2019

@loriab loriab referenced this pull request Mar 18, 2019

Merged

stdout shenanigans #56

@loriab loriab force-pushed the dgasmith:recursive branch from d77c496 to 68983ee May 20, 2019

@jeffschriber

This comment has been minimized.

Copy link
Contributor

commented May 24, 2019

I have some small changes to make the multilevel driver work with QCFractal. Could I get push permission, or should I open a PR to the branch?

@dgasmith

This comment has been minimized.

Copy link
Member

commented May 24, 2019

Invite in your mailbox.

@loriab

This comment has been minimized.

Copy link
Member

commented May 24, 2019

hooray, lgtm PR analysis is working! this may be a landslide.

@lgtm-com

This comment has been minimized.

Copy link

commented May 24, 2019

This pull request introduces 23 alerts and fixes 14 when merging 5c73c74 into 2edf6b7 - view on LGTM.com

new alerts:

  • 14 for Unused import
  • 7 for Unused local variable
  • 1 for Except block handles 'BaseException'
  • 1 for Syntax error

fixed alerts:

  • 5 for Unused local variable
  • 5 for Unused import
  • 3 for 'import *' may pollute namespace
  • 1 for Except block handles 'BaseException'

Comment posted by LGTM.com

loriab and others added some commits Jan 13, 2019

ddd: better options for json-running. both stashing/restoring options…
… in driver and flat-list serializing/loading (now locals, too) for the json job

@jeffschriber jeffschriber force-pushed the dgasmith:recursive branch from 5c73c74 to e11bb21 Jun 25, 2019

@lgtm-com

This comment has been minimized.

Copy link

commented Jun 25, 2019

This pull request introduces 23 alerts and fixes 14 when merging e11bb21 into d4681c9 - view on LGTM.com

new alerts:

  • 14 for Unused import
  • 7 for Unused local variable
  • 1 for Except block handles 'BaseException'
  • 1 for Syntax error

fixed alerts:

  • 5 for Unused local variable
  • 5 for Unused import
  • 3 for 'import *' may pollute namespace
  • 1 for Except block handles 'BaseException'
@codecov-io

This comment has been minimized.

Copy link

commented Jun 26, 2019

Codecov Report

Merging #1351 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1351   +/-   ##
=======================================
  Coverage    70.9%    70.9%           
=======================================
  Files        1468     1468           
  Lines      316368   316368           
=======================================
  Hits       224327   224327           
  Misses      92041    92041

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4681c9...e11bb21. Read the comment docs.

@lgtm-com

This comment has been minimized.

Copy link

commented Jul 18, 2019

This pull request introduces 24 alerts and fixes 14 when merging 8c0afe6 into 6e67529 - view on LGTM.com

new alerts:

  • 14 for Unused import
  • 8 for Unused local variable
  • 1 for Except block handles 'BaseException'
  • 1 for Syntax error

fixed alerts:

  • 5 for Unused local variable
  • 5 for Unused import
  • 3 for 'import *' may pollute namespace
  • 1 for Except block handles 'BaseException'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.