Thank you for the detailed proposal.

I am in favor of moving the light-weight IMDReader into MDAnalysis.coordinates as part of our effort to make MDAnalysis fully streaming capable.

What are additional dependencies that would come with IMDReader, i.e., which packages outside the standard dependencies that MDAnalysis already pulls in?

Do we already have a conda-forge package for IMDClient?

For quick overview, see @ljwoods2 's presentation 03-Streaming_MDAnalysis_Functionality-Woods.ipynb from the IMD Streaming Workshop.

@orbeckst We wouldn't be introducing any new dependencies- the imdclient package uses nothing except numpy arrays and the python standard library. In the future, we may want to rewrite parts of the IMDClient in cython (if needed after benchmarking), but that's nothing new.

@hcho38 is currently working on conda, will check in with him

Here is the link to the conda-forge recipe:

https://github.com/hcho38/imd-staged-recipes/tree/develop

Here is the link to the pull request:

conda-forge/staged-recipes#28554

@ljwoods2 @amruthesht can you guys leave a comment on the PR confirming you are willing to be listed as one of the maintainers?

Thank you for getting the conda-forge package up, @hcho38 !

• https://github.com/conda-forge/imdclient-feedstock
• https://anaconda.org/conda-forge/imdclient

It looks like with the IMDReader you'll have to pull utils.parse_host_port along with it, but nothing else in the MDAKit is referencing it so it shouldn't be an issue, so relatively clean break! I expect you'll need this function in MDAnalysis for testing a dummy server (but I could be wrong). Although, since the idea is that IMDClient could be used independently from MDAnalysis, do you also need a copy in IMDClient for those users?

It seems like parse_host_port is specific to the client address for the IMDReader. Is it's worth considering how alternatives to IMDClient could leverage the IMDReader in MDAnalysis and what the requirements for that interface looks like? Or is making this aspect generalizable not a priority?

Just a side note, I love InThreadIMDServer from the tests! It’s making testing so much smoother for me on macOS without running simulations :) I think it’d be a good idea to add some documentation for other developers to ultilize it---or even rescuing it from tests?

@jaclark5 Good point with parse_host_port- I would suggest keeping it in the imdclient package and importing it into MDA, though since it is a fairly trivial method I'd be okay with just duplicating it as well.

As for generalizing the IMDReader to use another client other than IMDClient (if I'm understanding you right), I don't think that's flexibility I see us needing. Is that what you meant?

@yuxuanzhuang I like the idea of being able to serve any (even prewritten) trajectories via IMD, so I'm open to this idea. As-is, it's fairly test-specific (with methods like expect_packet) and a little janky with hardcoded placeholder data, so I'd ideally want to make a similar class for that use case.

Opened Becksteinlab/imdclient#48 for this

Hi all, sorry it has taken me so long to get to this, and thanks for the comments @jaclark5 and @yuxuanzhuang.

I agree with basically everything said so far. The rapid iteration part of developing the IMDReader is largely over and I think adopting a base class + IMDReader into core is the right move, allowing generalisation, tight integration and make streaming a first class citizen.

As already pointed out I think it seems to be a fairly clean break, with minimal deps introduced. I am happy to go ahead with this as the plan. I think the main work for core will be in deciding on a path forward for #4828

A few points

• Once we merge into core we probably want to avoid thrashing on the interface between the two codes, as this will involve repeated PRs to core, which can be a bit slow. This means that we should sort out known issues or changes we would like to make before making the transition.
• Compatibility should be marked with a release.
• Aim to keep testing burden low, the core testsuite is already large.
• Lets wait for a bit of discussion on Make the AnalysisBase compatible with live simulation streams #4828 first, as we may need to make some changes to the way reader works for streaming / AnalysisBase compatibility.