Release 0 0 6 #151

eirrgang · 2018-07-22T14:24:53Z

gmxpi 0.0.6 release

Interface and feature updates

Updates to :mod:gmx.version module
Automatically set and restore from MD simulation checkpoints in the session working directory.
Allow control of whether simulation output is appended or truncated (PR Allow control of whether output is truncated or appended. #126).
Allow plugins to issue a stop signal to MD simulations (reference Stop condition hook #62 for gromacs-gmxapi and sample_restraint repos).
Changes to gmx.exceptions
Allow full CMake-driven install
Updated example notebooks in sample_restraint repository.

Internal

Improved CI testing
Unique work spec identification #64 Unique work spec identification.

Bug fixes

Docker does not access current gmxpy version #66 Docker does not access current gmxpy version.
possible race condition in session closing #123 Race condition in session closing.

Remaining tasks

README.md should have citation information for gmxapi (bioinformatics 2018).
check for embedded "to do"s
remove shared_data_maker and test code, discussion, to one or more of checkpoint for data nodes #34, more execution graph node types #36, Eigen #37, Formal AllReduce operation #45, Generic data structures for graph edges and state #69
reword gmx.Status description, and cite appropriate tracked issue or design document.
reconsider comment lines used to hint text-block separation in workflow.py docstring.

Begin 0.0.6 development cycle

Fix some typos and provide better documentation for determining or recalculating Python module install location.

Remove target directory to ensure expected move behavior.

* Add __hash__ and serialize() instance methods to WorkSpec. * Add deserialize() class method to WorkSpec. * Add uid() instance method to WorkSpec to generate a consistent, concise and sufficiently unique string to identify a gmxapi_workspec_0_1 dataset. * Update WorkSpec documentation. * Add a CompatibilityError exception.

For simulations run in the local context, this commit allows access to the potentials used in the context after the simulation has run and while the context is still alive. This allows the Python script to interact with the object implementing the potential with any additional interface it may have. This is a temporary form of access until other means of interacting with workflow objects are devised and implemented.

Add badges for development branch.

allow full CMake-driven build and install. Fixes #60

The devel branch of gmxapi now requires the devel branch of gromacs-gmxapi

Create only the new directory we are going to use on the current rank. Restore current directory to original state at end of session.

Update Dockerfile to include documentation with the image.

* normalize and update some quotes * Add documentation to gmx.version * Make some notes about Status class, which needs some work.

* Add and fix some missing or broken documentation. * Clean up some comments. * Build docs as part of Travis-CI builds * Reformat some code and examples for readability. * Install sphinx and rtd-theme to build docs on Travis-CI.

Relates to issue #40 Functionally, add a boolean `append_output` property to MD simulations that defaults to `true`. Implemented with an optional keyword / param that, if set `false`, gives the `mdrun -noappend` behavior.

Relates to issue #40 and to bug #130 Resolves task in #130 when merged.

Allow control of whether output is truncated or appended.

Addresses #103 Use caching and build matrix stages to try to optimize time. Use default compiler for MPI, upgraded gcc for tMPI, but mpi4py still builds with default compiler. No clang yet. Need to build our own MPI stuff in the future.

Convert serialized data to (utf-8 encoded) byte sequence. Adds a little bit of extra code when using the json module to make pretty formatted output, but removes ambiguity and improves implementation across Python 2 and Python 3.

Handle both Python 2 and Python 3

We are observing sporadic Travis-CI job failures that look like binary incompatibilities in compiled Python modules. The `pip` cache on Travis-CI may not handle distinctions between Pythonversions, so we will disable it.

Only test against gcc-4.8, but build gromacs with and without mpi. Also, further reduce caching of packages built by pip. There seems to be some fairly onerous caching that mixes up built packages for different python versions. We should look into how best to just specify different cache directories for each of Python 2 and 3 when everything else is equal and Travis-CI gives the job the same container cache.

* Clarify binding order of Python references in ensemble_update facility. * Fix hanging MPI calls in certain configurations. * hide dead code pending redesign work.

* Clean up caching, build stages, and cross-repo compatibility testing. * Move some complexity into supporting scripts. * Switch to "python" language CI builds for better matrix management. * Improve build step behavior. partially resolves #54

[ci skip]

peterkasson

README.md should have citation information for gmxapi (bioinformatics 2018).

peterkasson · 2018-07-22T19:57:57Z

setup.py

@@ -210,7 +210,7 @@ def build_extension(self, ext):
        if build_gromacs:
            # TODO! We need to distinguish dev branch builds from master branch builds or always build with the
            # master branch of the dependency. For one thing, as is, this line needs to be toggled for every release.
-            gromacs_url = "https://github.com/kassonlab/gromacs-gmxapi/archive/dev_5.zip"
+            gromacs_url = "https://github.com/kassonlab/gromacs-gmxapi/archive/devel.zip"


Is there a compelling reason to use a static zip file for gromacs_url rather than a git branch (or tag) checkout?

It is much faster than cloning the repo and uses standard Python modules. More recently, I've found the fastest thing is git clone -depth 1 ..., but when we move off of readthedocs.org to a push from Travis-CI, we will not need to work so hard to shave seconds off the build time.

peterkasson

A few specific comments, otherwise LGTM.

peterkasson · 2018-07-22T20:00:15Z

src/gmx/context.py

+# def shared_data_maker(element):
+#     """Make a shared data element for use by dependent nodes.
+#
+#     Note: outdated. This code is in limbo and should not be used.


What is the meaning of "in limbo" here? Deprecated? In the process of being refactored?

Ah. Thanks. I should have tracked that better. There is a lot of code in the various repos that is currently unused or completely dead. I'm in the process of removing the dead code where it is untested, but for unused code that is tested, I have been content to make minimal changes. For this particular section, I think the code was tested and error free, but unused and inconsistent with forward-looking design. I'll take a closer look at it before letting it propagate further.

To do

make better use of "to do"s

remove shared_data_maker and test code, discussion, to one or more of checkpoint for data nodes #34, more execution graph node types #36, Eigen #37, Formal AllReduce operation #45, Generic data structures for graph edges and state #69

There's not really a good place to stash ideas for posterity that I can find with GitHub. Unless maybe that's what Gist is for? I would want to stash the commented out bits of context.py and of test_mpiarraycontext.py. As it is, I tried to be more consistent with the commented out code and to include comments referring to tracked issues with commit 848e5c1. If you have other ideas, please let me know.

peterkasson · 2018-07-22T20:06:04Z

src/gmx/status.py

@@ -1,7 +1,9 @@
 class Status(object):
    """Ultimately hold status for API operation.

-    This should probably inherit from gmx.core.Status.
+    Either this should be inherited by gmx.core.Status or a reference to it or it gets


syntax of this comment is a little hard to parse.

Todo:

reword comment.

cite appropriate tracked issue or design document.

peterkasson · 2018-07-22T20:08:04Z

src/gmx/workflow.py

@@ -36,6 +17,8 @@
    >>> with gmx.context.Context(md.workspec) as session:
    ...    session.run()

+.. This comment line seems to be necessary to help parsing in some cases.


There's probably a better way to express that... I couldn't get Sphinx to parse the docstring in a sane way without adding a comment line to kick the parser in a way I don't understand. I'll see if I can quickly come up with a solution and, if not, reword the necessary comment to seem like it was meant to be there. If someone with RST skilz has a better answer, please let me know.

reword or replace bizarre comment line.

Commented out some dead code more completely, but have not removed it entirely because it captures some ideas that are still being explorered, and there doesn't seem to be a good way to attach sample source code to GitHub issues. References to the appropriate tracked issue have been added. Maybe this is something that "Gist" is good for.

Remove dead code. Resolve "to-do"s and/or reference issue tracker. Prepare for release.

peterkasson

LGTM, thanks. Nit: might add a note on the zip source retrieval similar to our discussion (this is planned to transition to git source retrieval at some point in the future but is currently at zip retrieval to speed CI builds).

eirrgang · 2018-07-23T17:09:02Z

The conversation referenced by @peterkasson relates to the use of a GitHub release archive URL (zip file) rather than git based source retrieval. The PR has been approved, so follow up to the conversation has been moved to #153

peterkasson · 2018-07-23T18:00:27Z

My primary concern is just that if we pull a zip file, then unless that is automatically created every time then we have the potential for the zip file and the source tree to get out of sync. If we're pulling a designated tag from the source tree, then it's clearly defined.

eirrgang and others added 30 commits May 8, 2018 16:46

Begin 0.0.6 development cycle

6f12522

Merge pull request #59 from eirrgang/newdevel

49937ae

Begin 0.0.6 development cycle

allow full CMake-driven build and install.

dc84da5

gmxapi-60 Allow CMake-only builds

31389a3

Fix some typos and provide better documentation for determining or recalculating Python module install location.

gmxapi-66 Docker does not access current gmxpy version

002ff97

Remove target directory to ensure expected move behavior.

Update README.md

eb79ca3

Add badges for development branch.

Update CI builds and documentation.

68cb9b4

Merge pull request #61 from eirrgang/gmxapi-60

b02b6e5

allow full CMake-driven build and install. Fixes #60

Clean up some exception naming and deprecate some.

e0bc2d4

gmxapi-108 Dockerfile out of date

442c7da

The devel branch of gmxapi now requires the devel branch of gromacs-gmxapi

gmxapi-109 race condition in context.py. Fixes #109

6c57464

Create only the new directory we are going to use on the current rank. Restore current directory to original state at end of session.

Fix some errors and omissions in gmxapi docs.

6883c75

Update Dockerfile to include documentation with the image.

Minor documentation updates.

8b8c834

* normalize and update some quotes * Add documentation to gmx.version * Make some notes about Status class, which needs some work.

hot fix: typo in version.in

ec13836

Additional documentation updates.

897e57b

* Add and fix some missing or broken documentation. * Clean up some comments. * Build docs as part of Travis-CI builds * Reformat some code and examples for readability. * Install sphinx and rtd-theme to build docs on Travis-CI.

Allow control of whether output is truncated or appended.

d5edc12

Relates to issue #40 Functionally, add a boolean `append_output` property to MD simulations that defaults to `true`. Implemented with an optional keyword / param that, if set `false`, gives the `mdrun -noappend` behavior.

Clarify keyword argument behavior for append and steps.

2be078d

Relates to issue #40 and to bug #130 Resolves task in #130 when merged.

Merge pull request #126 from eirrgang/appendoutput

6f7d893

Allow control of whether output is truncated or appended.

gmxapi-103 Travis-CI should test larger matrix

d0e54b5

Addresses #103 Use caching and build matrix stages to try to optimize time. Use default compiler for MPI, upgraded gcc for tMPI, but mpi4py still builds with default compiler. No clang yet. Need to build our own MPI stuff in the future.

Merge pull request #132 from eirrgang/gmxapi-103

23b9015

Handle both Python 2 and Python 3

Disable pip cache on Travis-CI

34aa8ba

We are observing sporadic Travis-CI job failures that look like binary incompatibilities in compiled Python modules. The `pip` cache on Travis-CI may not handle distinctions between Pythonversions, so we will disable it.

Improve MPI management.

fff04af

* Clarify binding order of Python references in ensemble_update facility. * Fix hanging MPI calls in certain configurations. * hide dead code pending redesign work.

Update CI testing.

8b55d1b

* Clean up caching, build stages, and cross-repo compatibility testing. * Move some complexity into supporting scripts. * Switch to "python" language CI builds for better matrix management. * Improve build step behavior. partially resolves #54

Specify tagged 0.0.6 releases.

b672966

[ci skip]

Merge branch 'master' into devel

1f55ebd

Incorporate miscellaneous documentation updates.

9bb1e7f

eirrgang added the gmxapi pertains to this repository and the Python module support label Jul 22, 2018

eirrgang assigned peterkasson and jmhays Jul 22, 2018

peterkasson reviewed Jul 22, 2018

View reviewed changes

eirrgang added 6 commits July 23, 2018 11:55

Wrap long lines.

4a80e37

Add publication reference.

090aad7

De-obfuscate gmx.status.Status docstring.

b485ae2

Revise RST syntax for examples in workflow.py

7968aef

Clean up.

b796a98

Remove dead code. Resolve "to-do"s and/or reference issue tracker. Prepare for release.

peterkasson approved these changes Jul 23, 2018

View reviewed changes

eirrgang merged commit 6232476 into master Jul 24, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release 0 0 6 #151

Release 0 0 6 #151

eirrgang commented Jul 22, 2018 •

edited

Loading

peterkasson left a comment •

edited by eirrgang

Loading

peterkasson Jul 22, 2018

eirrgang Jul 22, 2018

peterkasson left a comment

peterkasson Jul 22, 2018

eirrgang Jul 23, 2018 •

edited

Loading

eirrgang Jul 23, 2018

peterkasson Jul 22, 2018

eirrgang Jul 23, 2018 •

edited

Loading

peterkasson Jul 22, 2018

eirrgang Jul 23, 2018 •

edited

Loading

peterkasson left a comment

eirrgang commented Jul 23, 2018 •

edited

Loading

peterkasson commented Jul 23, 2018

Release 0 0 6 #151

Release 0 0 6 #151

Conversation

eirrgang commented Jul 22, 2018 • edited Loading

peterkasson left a comment • edited by eirrgang Loading

Choose a reason for hiding this comment

peterkasson Jul 22, 2018

Choose a reason for hiding this comment

eirrgang Jul 22, 2018

Choose a reason for hiding this comment

peterkasson left a comment

Choose a reason for hiding this comment

peterkasson Jul 22, 2018

Choose a reason for hiding this comment

eirrgang Jul 23, 2018 • edited Loading

Choose a reason for hiding this comment

eirrgang Jul 23, 2018

Choose a reason for hiding this comment

peterkasson Jul 22, 2018

Choose a reason for hiding this comment

eirrgang Jul 23, 2018 • edited Loading

Choose a reason for hiding this comment

peterkasson Jul 22, 2018

Choose a reason for hiding this comment

eirrgang Jul 23, 2018 • edited Loading

Choose a reason for hiding this comment

peterkasson left a comment

Choose a reason for hiding this comment

eirrgang commented Jul 23, 2018 • edited Loading

peterkasson commented Jul 23, 2018

eirrgang commented Jul 22, 2018 •

edited

Loading

peterkasson left a comment •

edited by eirrgang

Loading

eirrgang Jul 23, 2018 •

edited

Loading

eirrgang Jul 23, 2018 •

edited

Loading

eirrgang Jul 23, 2018 •

edited

Loading

eirrgang commented Jul 23, 2018 •

edited

Loading