[File format] Moltemplate export capability

[jewettaij]

Is your feature request related to a problem? Please describe.
Currently moltemplate users have to run ltemplify.py on the files created by VIPSTER, by typing this into the terminal/shell:

ltemplify.py LAMMPS_DATA_FILE > MOLTEMPLATE_FILE

Describe the solution you'd like

Can we add a new entry ("Moltemplate LT") to the list of file formats that users can select from when saving (exporting) a molecule? It would behave identically to the "LAMMPS Data File" format, with the same list of options. However when VIPSTER saves the file, it runs the ltemplify.py script on this file to automatically convert the result into moltemplate format (using the syntax above).

This will either require users to have also installed moltemplate (since "ltemplify.py" is distributed with moltemplate), OR we could copy the current version of the "ltemplify.py" code into VIPSTER. (This would require adding "ltemplify.py", "ttree_lex.py", and "lttree_styles.py" to VIPSTER. This is 6785 lines of code, but this code is currently relatively stable. These files do not change very often these days.) I am happy either way.

Describe alternatives you've considered
Alternatively, you could instead add a new "IO-Config Preset" for the existing LAMMPS file format (instead of creating a new entry in the list of available file formats). That's okay too. However, if you do that, perhaps we could change the name of the entry from "LAMMPS Data File" to "LAMMPS File", or something vague like that (because moltemplate files have a different format from LAMMPS data files).

It's possible I may add one or two additional options that are specific to moltemplate (to allow the user to specify the force field they want, for example). But we can worry about this later.

Please let me if there is anything I can do to make this easier for you.

[sgsaenger]

Coupling with moltemplate

If you check out the latest commits on testing, you'll see that a python terminal is now available in the GUI, which is enabled as soon as both the PYTHON and DESKTOP variables are enabled in cmake. I'll happily include moltemplate in future builds. I had a look at your dependencies, but couldn't find anything besides the python standard library, is this correct?

Implementation-wise, i would include your repository as a git submodule as i have done for my other dependencies, this way it's easy to stay up to date with upstream.

File support

There are a few things to consider with file-support.
Straight up calling ltemplify.py is not very ideal, and i fear not very portable. This assumes a working python installation and a working moltemplate installation, as well as a properly configured path. Very probable on Linux, very unlikely on Windows.
If you're interested, we could try reorganizing ltemplify.py in a more functional way, so it can be called "directly" from vipster. This would circumvent having to go through the OS and be way more robust.

Also, there needs to be a way to read in .lt files, at least simpler ones as generated from ltemplify.py. Do you have any suggestions in this regard? Is there a reasonable subset of your templating language that could be defined as being available intermediately, so it can be read in directly? Maybe with a moltemplate-lite wrapper?

[jewettaij]

Coupling with moltemplate

If you check out the latest commits on testing, you'll see that a python terminal is now available in the GUI, which is enabled as soon as both the PYTHON and DESKTOP variables are enabled in cmake. I'll happily include moltemplate in future builds. I had a look at your dependencies, but couldn't find anything besides the python standard library, is this correct?

Yes! Moltemplate currently has no external dependencies.

In the future, some optional features of moltemplate might depend on numpy, but I'll make sure that moltemplate can be installed without numpy (and ltemplify.py will never depend on numpy).

(Actually, I just discovered that one of the executable scripts that is bundled with moltemplate, interpolate_mt.py, already depends on numpy, but this script is not included when you run "import moltemplate" in python. That script is a stand-alone executable script which is run from the terminal. It is not needed by ltemplify.py or any other python file in moltemplate. I would not worry that it depends on numpy. I don't think anybody besides myself has ever used it.)

Implementation-wise, i would include your repository as a git submodule as i have done for my other dependencies, this way it's easy to stay up to date with upstream.

Sounds great to me.

File support

There are a few things to consider with file-support.
Straight up calling ltemplify.py is not very ideal, and i fear not very portable. This assumes a working python installation and a working moltemplate installation, as well as a properly configured path. Very probable on Linux, very unlikely on Windows.
If you're interested, we could try reorganizing ltemplify.py in a more functional way, so it can be called "directly" from vipster. This would circumvent having to go through the OS and be way more robust.

I'm working on changing ltemplify.py and moltemplate/init.py in order to make it possible to load ltemplify as a module from within python. (Currently "import moltemplate" works, but "ltemplify" is not visible. I'm working on this now.)

Also, there needs to be a way to read in .lt files, at least simpler ones as generated from ltemplify.py. Do you have any suggestions in this regard? Is there a reasonable subset of your templating language that could be defined as being available intermediately, so it can be read in directly? Maybe with a moltemplate-lite wrapper?

Are you thinking about including moltemplate import capability to VIPSTER? That would be cool. I wish I had thought about that.

Importing moltemplate files

To import data from moltemplate files, I the easiest way would be to ask moltemplate to generate a LAMMPS data file that VIPSTER can read. Moltemplate generate DATA files already include comments in the "Masses" section that contain the atom type strings. (If you like, I can also add comments following the "N bond types", "N angle types", "N dihedral types", and "N improper types" containing the type strings for these objects as well. We can worry about that later.)

** Does that sound reasonable? **
I discuss the alternative at the end of this (rather long) post.

Current plans

For now, I will just focus on export capability. In the next day or two, I will figure out how to interact with moltemplate by accessing the main() functions for the various stand-alone executable scripts (like ltemplify.py) This will enable you to run ltemplify.py from within python without interacting with the shell. You just have to supply main with an argv argument (a python list) containing a single string which is the name of the DATA file VIPSTER has created:

import moltemplate
output_text = moltemplate.ltemplify.main('LAMMPS_DATA_FILE')

(where LAMMPS_DATA_FILE is the name of the file created by VIPSTER)

Does this sound reasonable to you?

(If you really want to avoid using the file-system entirely, perhaps we can consider allowing non-string entries in the argv list and use StringIO perhaps. But I wasn't planning to do this unless you want me to. I'm open to other ideas too.)

Details (long, feel free to skip)

I hate to admit this embarrassing fact, but moltemplate was never intended be run from within python. This was an oversight on my part, because moltemplate (ttree.py) is nearly my first python program. I only intended it to be used from the command line to generate files for LAMMPS to read. I did this because, (as far as I know) the LAMMPS MD program has no way to interact with python other than by reading DATA files. Unlike hoomd-blu or OpenMM, a python user within LAMMPS cannot send the atom coordinates, topology information, and force field parameters to LAMMPS directly from within its python environment, without writing this to a data file (and using the LAMMPS' "read_data" command). That's also why part of moltemplate was writtin in BASH.

To really make moltemplate python scriptable would require rewriting significant portions of it. Actually, this is something I am thinking of doing some time next year, because we also need this. (This may also augment the moltemplate syntax.) But this is not yet a high priority for us now.

Making system calls in windows

It's true. In linux you can either run "ltemplify.py LAMMPS_FILE.data > MOLTEMPLATE_FILE" in the terminal, or use os.system() or the subprocess() module. The tricky part is to handle the ">" stdout redirection to a file. Although this is irrelevant to us, for my own records, the way you would do this in python is:

import subprocess
result = subprocess.run(['ltemplify.py', 'LAMMPS_FILE.data'], stdout=subprocess.PIPE)
moltemplate_file_contents = result.stdout

...but, as you say, this will not work in a windows environment unless the user has already installed python. In the process of writing this reply, I realized how unreasonable it would be to make this a requirement. (I thought windows 10 now includes python by default, but I was wrong.)

To get around this, we should load moltemplate.ltemplify as a module, which is what you suggested. I'm working on making all of the scripts in moltemplate available as modules. This might take a day, as I am inexperienced with python packaging.

[sgsaenger]

Yes! Moltemplate currently has no external dependencies.

In the future, some optional features of moltemplate might depend on numpy, but I'll make sure that moltemplate can be installed without numpy (and ltemplify.py will never depend on numpy).

Don't limit yourself too much :) It would be useful to ship a useful scientific base-system anyways, Numpy would be the first candidate for this.

To import data from moltemplate files, I the easiest way would be to ask moltemplate to generate a LAMMPS data file that VIPSTER can read. Moltemplate generate DATA files already include comments in the "Masses" section that contain the atom type strings. (If you like, I can also add comments following the "N bond types", "N angle types", "N dihedral types", and "N improper types" containing the type strings for these objects as well. We can worry about that later.)

This sounds easiest, yes. I'm not sure if this can reasonably integrated with vipster, if the preparation process is taking for too long (going from the hours you mention for your vesicle example) that's quite unreasonable for simply "loading a file". Maybe this is better left for the user to execute explicitely?
.lt files obviously are semantically different from all other supported formats, so maybe they shouldn't be treated exactly the same way.

I'll have to learn more about them, maybe i get an idea how to reasonably treat a subset of your language.

I'm working on changing ltemplify.py and moltemplate/init.py in order to make it possible to load ltemplify as a module from within python. (Currently "import moltemplate" works, but "ltemplify" is not visible. I'm working on this now.)
[...]
For now, I will just focus on export capability. In the next day or two, I will figure out how to interact with moltemplate by accessing the main() functions for the various stand-alone executable scripts (like ltemplify.py) This will enable you to run ltemplify.py from within python without interacting with the shell. You just have to supply main with an argv argument (a python list) containing a single string which is the name of the DATA file VIPSTER has created:

import moltemplate
output_text = moltemplate.ltemplify.main('LAMMPS_DATA_FILE')

That sounds great!
I've opened jewettaij/moltemplate#17 with a few suggestions.

(If you really want to avoid using the file-system entirely, perhaps we can consider allowing non-string entries in the argv list and use StringIO perhaps. But I wasn't planning to do this unless you want me to. I'm open to other ideas too.)

I'd really prefer avoiding the filesystem, yes. IIRC, StringIO can be used as a drop-in replacement for file objects in python, so hopefully the only change necessary is to encapsulate the parts of the current main() that operate on the file and pass this as an argument, then vipster could call this function with a StringIO object.

I hate to admit this embarrassing fact, but moltemplate was never intended be run from within python. This was an oversight on my part, because moltemplate (ttree.py) is nearly my first python program. I only intended it to be used from the command line to generate files for LAMMPS to read. I did this because, (as far as I know) the LAMMPS MD program has no way to interact with python other than by reading DATA files. Unlike hoomd-blu or OpenMM, a python user within LAMMPS cannot send the atom coordinates, topology information, and force field parameters to LAMMPS directly from within its python environment, without writing this to a data file (and using the LAMMPS' "read_data" command). That's also why part of moltemplate was writtin in BASH.

JFYI, since some time ago there's the PyLammps interface. It exposes much of lammps in a pythonic way.

[jewettaij]

Hi Sebastian
I implemented many of your suggestions from the ltemplify.py issue you opened.
If you are still willing to consider adding moltemplate-file export capability, here is the procedure:

To invoke ltemplify.py from within python use:

import moltemplate

argument_list = [DATA_FILE_NAME, '-name', MOLECULE_NAME]

# DATA_FILE_NAME is the name of the data file (internally) created by VIPSTER.
# MOLECULE_NAME is the name of the molecule (or system) as it will be referred later
# by moltemplate users.  (It is distinct from the name of the moltemplate file.)
# Would it be possible to add dialogue box somewhere in the UI  (when users save
# their files in moltemplate format) allowing them to specify this MOLECULE_NAME?

ltemp = moltemplate.ltemplify.Ltemplify(argument_list)

# Using these settings, convert these files into a single file.
# Send the content that file to sys.stdout.
import sys
ltemp.Convert(sys.stdout)
# (To send to a string, I think you can use io.StringIO instead of sys.stdout.)

(I tested this recently, but please let me know if any of this fails to work.)

Incidentally, I finally converted the
documentation for "ltemplify.py" to markdown format
An explanation of what the arguments do is located in a table near the beginning of the file.

There are also some examples. (Keep in mind that most of the examples use the "-mol" or "-id" arguments, this is because I was assuming that ltemplify.py users are extracting the molecule from a big LAMMPS data file that contains many molecules. In that case they need to specify which atoms we want in the final output file. I assume VIPSTER users would be drawing only one molecule, so we don't have to bother with this.)

[jewettaij]

I'd really prefer avoiding the filesystem, yes. IIRC, StringIO can be used as a drop-in replacement for file objects in python, so hopefully the only change necessary is to encapsulate the parts of the current main() that operate on the file and pass this as an argument, then vipster could call this function with a StringIO object.

My apologies. Oh dear. I just noticed this.

Unfortunately, in my previous post, I suggested writing the VIPSTER output to a DATA file, and then invoking ltemplify.Ltemplify.init() with an argument list which includes the name of this data file.
That doesn't sound like what you want.

However if you like, I can generalize it so that it can accept arguments that are file streams instead of file names. Would that be helpful? Something like this perhaps?

import moltemplate

import io
data_file = io.StringIO(DATA_FILE_CONTENTS)

# DATA_FILE_CONTENTS is a string containing the data file created by VIPSTER

input_files = [data_file]
args = ['-name', MOLECULE_NAME]

# DATA_FILE_NAME is the name of the data file (internally) created by VIPSTER.
# MOLECULE_NAME is the name of the molecule (or system) as it will be referred later
# by moltemplate users.  (It is distinct from the name of the moltemplate file.)
# Would it be possible to add dialogue box somewhere in the UI  (when users save
# their files in moltemplate format) allowing them to specify this MOLECULE_NAME?

ltemp = moltemplate.ltemplify.Ltemplify(input_files, args)

ltemp.Convert(sys.stdout)

Or do you have a different way of doing this in mind?

[sgsaenger]

Yes, that would be exactly it!

Regarding properties, what would you suggest to export?
Does the MOLECULE_NAME commonly differ from the file-name?
You mentioned generating angles and dihedrals inside of moltemplate, would it be enough to export templates with bonds-only or should we expose vipster's topology?
What about charges/forces?

[jewettaij]

Yes, that would be exactly it!
Great! I'll get to work on making this small change. I'll have this ready by the weekend.
I've decided to change the order slightly. I will use this instead:

ltemp = moltemplate.ltemplify.Ltemplify(argv=['-name', MOLECULE_NAME])

ltemp.convert(out_file=NEW_FILE, input_data_file=DATA_FILE)

where NEW_FILE and DATA_FILE can either be a string (filename) or a file object (eg io.StringIO), and DATA_FILE contains the contents of the file created by VIPSTER.

Regarding properties, what would you suggest to export?
Does the MOLECULE_NAME commonly differ from the file-name?

Moltemplate is sort of like a (dumb) object-oriented programming language for building simulations.

1. Users must define Molecule-Objects (which are sort of like class definitions in python or C++). These are the building-blocks people can use later to build a simulation. I imagine that VIPSTER could be a really convenient tool to create individual molecule objects. (And users could build larger structures with it as well.)

2. Then people can use these molecule objects to make bigger systems. In moltemplate, when you want to build something, you typically make many copies of these Molecule-Objects (building blocks).
   They could make many copies of these molecules, move them to their desired locations, modify them, and (optionally) bond them together and/or use them to define bigger objects. For example, a moltemplate user could use VIPSTER to create a "styrene.lt" file containing a definition of the "Styrene" monomer, which is a molecule-object. Then, that person could use these commands to build a polystyrene molecule:

import "styrene.lt"    #<-- define the "Styrene" object we will use later

and use a command like this:

polystyrene = new Styrene[40]

...to make 40 copies of the "Styrene" monomer. They can then link them together with bonds to create a polystyrene polymer (from within moltemplate).

In summary, in this example, "styrene.lt" is the file defining the object, and "Styrene" is the MOLECULE_NAME, which is the way moltemplate users will refer to this object inside their code (when using moltemplate later).

I realize this example is confusing in because the MOLECULE_NAME is actually the name of a monomer ("Styrene"), not the molecule (polystyrene). The MOLECULE_NAME is the name of the object that moltemplate users will refer to later.

(Again, alternatively users could build the entire polymer in VIPSTER, and save the entire thing as a moltemplate-object. That is fine too. We can give users the freedom they want.)

You mentioned generating angles and dihedrals inside of moltemplate, would it be enough to export templates with bonds-only or should we expose vipster's topology?

Just as you currently do with LAMMPS DATA files, leave it up to the user. Both kinds of molcules are allowed in moltemplate. Sometimes the user will want to include this information, and sometimes they won't. (See discussion below.)

Details (TL;DR Feel free to skip)

Molecule-Objects in moltemplate may or may not include force field information.

a) Here's an example of a molecule-object which does include angle topology and force-field information (including force-field styles and force-field parameters)
http://moltemplate.org/examples/martini_DPPC_bilayer/lipid.lt
In the interests of being thourough, here is the context in which this file appears:
http://moltemplate.org/visual_examples.html#martini_DPPC_bilayer

b) And here's an example which uses an external 3rd-party force-field (OPLSAA) instead. In this example, the user only needs to specify the atom coordinates, types, charges, and a list of pairs of bonds. They can leave out the angles, dihedrals, impropers and force-field parameters. Consequently, the definition of the "Ethylene" molecule below is much more compact:
http://moltemplate.org/examples/ethylene+benzene/ethylene.lt
Context:
http://moltemplate.org/visual_examples.html#ethylene+benzene

What about charges/forces?

Great questions!

I propose that users should be free to select the atom_style they want, just as they normally do when saving LAMMPS data files. If that atom_style supports charge, then you can add a column to the data file for charges (and leave it 0.0 by default). (For moltemplate users, at least, however, I think the default atom_style should be "full". I think it is currently "atom" by default.)

Similarly users can select whether or not they want to include angles, dihedrals, and impropers in the moltemplate file VIPSTER creates (just as they can select whether or not to include this information in the DATA file). Let users choose whether or not they want to include this information in their files. This way the burden of choice is on them, not you.

Future features:

DON'T WORRY ABOUT THIS NOW. WE CAN WORRY ABOUT THIS LATER!

In addition to letting the user choose the atom style, and whether to include angles, dihedrals, or impropers, it would be nice if users could :

1. specify the name of the force-field they want to use. (For example, "OPLSAA".)

Implementation: If the user types any text into this box, then append the following string to the end of the MOLECULE_NAME variable:

MOLECULE_NAME += ' inherits ' + FORCE_FIELD_NAME

2. If there was a check box to "omit bond types?".

Implementation If the user clicks on this box, then add '-ignore-bond-types' to the list of args, which will tell ltemplify.py to use a different type of Bonds list which omits bond types.

args.append('-ignore-bond-types')

(Why? Bond types are typically determined by the force-field, so theirs no need to specify these types. Adding '-ignore-bond-types' to the list of argunents will tell ltemplify.py and moltemplate to choose their own bond types when creating the final DATA file which LAMMPS will read, according to the force-field rules. I would not have this box checked by default, unless the user has typed a non-empty string into the FORCE_FIELD_NAME text-box.)

My apologies for writing such long, tedious replies. Thank you for reading them.

[jewettaij]

I think I have all of this working now in version 2.12.4 (Uploaded a few minutes ago.)

Note: Since I posted my comment 24 hours ago, I made some small changes to the Ltemplify API.
I edited the comment above to reflect these changes. To run ltemplify.py from within python you would use:

ltemp = moltemplate.ltemplify.Ltemplify(argv=['-name', MOLECULE_NAME])

ltemp.convert(out_file=NEW_FILE, input_data_file=DATA_FILE)

(where NEW_FILE and DATA_FILE can be file names or StringIO objects)
For now, we can set the MOLECULE_NAME equal to the file name (without the ".lt" extension). Eventually, though I'd love to have a dialogue box where the user can specify MOLECULE_NAME.

Let me know if there is anything else you need.

(Don't worry about the "Future Features" I mentioned above. Let's see if we can just get basic moltemplate file export working using the commands above.)

[sgsaenger]

Thanks for your work. I'm busy this week, so I won't be able to implement it for now, but it looks good so far.

[jewettaij]

Hi Sebastian. I was re-reading this conversation, and I found it long and confusing.
If it helps, I created an example of how to invoke ltemplify.py from within python and posted it
here

In your case, you can probably just use simple arguments, like: '-name', 'Ethylene'.
(You can omit the other arguments)

(Somebody else wanted to write a program that would export moltemplate files, so this pushed me to finally cleaned up the documentation.)

Thank you very much again for your recent bug fix.

[sgsaenger]

Hi Andrew,

sorry for letting this issue sit dormant for so long.

The good news:
there is now an interface for python-based file plugins. It's a pretty rough implementation that may cause a few unnecessary copies, but at least it's working.

An example implementation of a moltemplate plugin may look like this (saved as `~/.config/vipster/plugins/moltemplate.py):

from moltemplate.ltemplify import Ltemplify
import vipster as vp
from io import StringIO

name = "moltemplate" # display-name
extension = "lt" # default extension
command = "mol" # to specify plugin explicitely on CLI or in python

# parameters may hold the coeffs
# dict-value must be a 2-tuple with a `str`/`[str]`/`{str:str}` as the real value
# and a `str` as documentation
parameters = {"test_str": ("default_str", "doc_str"),
              "test_vec": (["v1", 'v2', 3], "doc_vec"),
              "test_map": ({'a': 5, 'b': 'c'}, "doc_map")
              }
# preset may control output, i.e. toggle bond inclusion
# dict-value must be a 2-tuple with a `bool` or a named-enum-shim as the real value
# (int as default value, tuple of strings to give allowed values with their names)
# and a `str` as documentation
preset = {"test_bool": (True, "doc_bool"),
          "test_enum": ((0, ('a', 'b', 'c')), 'doc_enum')
          }

def writer(mol, p, c, idx):

    # enforce fitting preset for lmp-Plugin intermediate
    lmp_preset = vp.Presets[vp.Plugins.lmp]['default']
    lmp_preset['style'] = "full"
    # TODO: fill from own preset
    lmp_preset['bonds'] = False
    lmp_preset['angles'] = False
    lmp_preset['dihedrals'] = False
    lmp_preset['impropers'] = False
    lmp_preset['coeff'] = True

    # setup parameters
    lmp_param = vp.Parameters[vp.Plugins.lmp]['default']
    # TODO: fill from own parameters
    lmp_param['Pair Coeff']['C'] = 'Null'

    # write lmp datafile
    tmp = vp.writeString(vp.Plugins.lmp, mol, param=lmp_param,
                         config=lmp_preset, index=idx)
    tmp = StringIO(tmp)

    # convert intermediate via ltemplify
    res = StringIO()
    ltemp = Ltemplify([])
    ltemp.Convert(out_file=res, input_data_file=tmp)
    res.seek(0)
    return res.read()

Please note the param/preset sections that need to be designed still,
to expose relevant moltemplate options.

The bad news is that I'm still unsure on how to include this.
My current best bet is to add a system-path where plugins are loaded from,
which is low on code-overhead, but may incur some build-system and distribution implications.

[jewettaij]

I had forgotten that VIPSTER can also generate angle, dihedral, and improper interactions, as well as Lennard-Jones parameters for the atomic elements. (Moltemplate users have other ways of generating that information automatically, but I like that they can specify it manually in VIPSTER as well.)

Anyway, thank you for remembering this feature Sebastian.
I'm glad to see progress on this topic.
Let me know if there is anything regarding moltemplate I can do to help.

For what it's worth, I spent a few minutes discussing VIPSTER when I gave a workshop on moltemplate last year at the LAMMPS conference.
https://lammps.sandia.gov/workshops/Aug19/talk_jewett_moltemplate.pdf#page=149

Even if your software still has bugs or issues, I think you should apply to go to the next conference in 2021. I remember two talks that presented in previous conferences (in 2013 and 2017 on "fix bond/react" and "Molecular Dynamics Studio") that described software whose development was still in progress.

[jewettaij]

Regarding dependencies:

I don't know if this helps, but moltemplate is maintained on pypi. That means that "pip install moltemplate" works reliably now. If this were a pure python project, you could include the 'moltemplate' dependency in the "install_requires" section of your "setup.py" file:

  install_requires=[
      'moltemplate', ...
  ],

... I think this means that this would install moltemplate any time people use pip to install your software. However, vipster cannot be installed using "pip", so I suppose this suggestion is not very helpful. But perhaps there is a way to put "pip install moltemplate" in your cmake files? (I haven't learned how to use cmake yet.)

Suggestion: Enclose the import Ltemplify command in a try block:

try:
    from moltemplate.ltemplify import Ltemplify
except ImportError:
    sys.stderr.write('Error: The moltemplate LT export feature requires moltemplate to be installed.\n'
                             Please install moltemplate using pip.\n')

This way VIPSTER will exit gracefully if people did not install moltemplate correctly. I don't feel like it should be your responsibility to install moltemplate on their computers.

Cheers
Andrew

[sgsaenger]

Let me know if there is anything regarding moltemplate I can do to help.

It would be great to get your opinion on the parameter and io-preset templates.

• The IO-preset controls how the data is generated/formatted.
  This could e.g. contain a NamedEnum with forcefields supported by moltemplate and set the angle/dihedral/improper flags for the lammps plugin as needed (or just pipe through the separate flags, as you deem more useful), or a toggle for using an input script.
• The parameters control extra data to be supplied to the plugin, and could e.g. hold a lammps-input script, or instructions on how to generate one.

I had forgotten that VIPSTER can also generate angle, dihedral, and improper interactions, as well as Lennard-Jones parameters for the atomic elements. (Moltemplate users have other ways of generating that information automatically, but I like that they can specify it manually in VIPSTER as well.)

See above, this is imho the most urgent part of the IO-preset/Plugin design.
Is it useful to use the lists provided by vipster or would it be preferable to offload this to moltemplate?

Regarding dependencies:

Sorry for the confusion, i was being imprecise in my last comment.
The problem is inclusion of the plugin-file, as there's currently no mechanism for installing data-files in vipster's build system (at least not in a way that is accessible from python), and the python-bindings may be installed seperately from the GUI, so having a separate file may lead to version clashes.
I'm investigating embedding the python code, will update when this issue is solved.

The current plan on including moltemplate:

• as a direct dependency in setup.py (so solvable by pip)
• installed in the bundled python-environment in official binary releases (which ship a standalone py-env)
• as a recommended optional dependency for manual installs and package builders
  Comments?

[jewettaij]

Hi Sebastian.
I have had two big deadlines over the last month and I did not see your post. I just discovered it before going to bed tonight. I am sending this message to remind myself to follow up on your message. I'm sorry for the really, really long delay. I will reply within a day or two.

-Andrew

[jewettaij]

Hi Sebastian
Thank you for your patience. It has been an exhausting couple months with many deadlines and I almost lost funding.

Let me know if there is anything regarding moltemplate I can do to help.

It would be great to get your opinion on the parameter and io-preset templates.

• The IO-preset controls how the data is generated/formatted.
  This could e.g. contain a NamedEnum with forcefields supported by moltemplate and set the angle/dihedral/improper flags for the lammps plugin as needed (or just pipe through the separate flags, as you deem more useful), or a toggle for using an input script.
• The parameters control extra data to be supplied to the plugin, and could e.g. hold a lammps-input script, or instructions on how to generate one.

I'm sorry. I realize there was no way for you to proceed until I provided you more information how to handle molecules that use force-fields. So I have listed all of the possible ways that VIPSTER users may want to export to moltemplate, along with the corresponding python code below.

Currently, the Ltemplify constructor currently accepts a list of string arguments that control its behavior. (Instead of a dictionary of name:value pairs.) These strings are the same command-line arguments which are normally passed to the "ltemplify.py" program when invoked from the shell. (I was lazy.) Currently, you will have to convert the information in the preset and parameters variables into this format. (...a list of strings. If you don't like the API I am using for the Ltemplify constructor, I am open to suggestions.)

I will now tell you exactly what this list of strings should be for each of the force-fields that moltemplate supports:

1. By default (if no force field was specified), the Ltemplify constructor should be invoked with these arguments

ltemp = Ltemplify(args=['-atomstyle', atom_style, '-name', name])

where
-atom_style='full' by default
-name='System' by default. (If the user has drawn a single ethylene molecule, they might set name='Ethylene'. Otherwise, a generic name like 'System' will suffice.).
2) If the user wants to use the COMPASS force field, the Ltemplify constructor should be invoked with these arguments:

ltemp = Ltemplify(args=['-atomstyle', atom_style, 
                 '-name', name+' inherits COMPASS',
                 '-preamble', 'import "compass_published.lt"',
                 '-ignore-angles',  '-ignore-bond-types', '-ignore-masses', '-ignore-coeffs'])

3. If the user wants to use the OPLSAA force field, the Ltemplify constructor should be invoked with these arguments:

ltemp = Ltemplify(args=['-atomstyle', atom_style, 
                 '-name', name+' inherits OPLSAA',
                 '-preamble', 'import "oplsaa.lt"',
                 '-ignore-angles',  '-ignore-bond-types', '-ignore-masses', '-ignore-coeffs'])

4. If the user wants to use the LOPLSAA force field, the Ltemplify constructor should be invoked with these arguments:

ltemp = Ltemplify(args=['-atomstyle', atom_style, 
                 '-name', name+' inherits OPLSAA',
                 '-preamble', 'import "oplsaa.lt"',
                 '-preamble', 'import "loplsaa.lt"',
                 '-ignore-angles',  '-ignore-bond-types', '-ignore-masses', '-ignore-coeffs'])

5. If the user wants to use the GAFF force field, the Ltemplify constructor should be invoked with these arguments:

ltemp = Ltemplify(args=['-atomstyle', atom_style, 
                 '-name', name+' inherits GAFF',
                 '-preamble', 'import "gaff.lt"',
                 '-ignore-angles',  '-ignore-bond-types', '-ignore-masses', '-ignore-coeffs'])

6. If the user wants to use the GAFF2 force field, the Ltemplify constructor should be invoked with these arguments:

ltemp = Ltemplify(args=['-atomstyle', atom_style, 
                 '-name', name+' inherits GAFF2',
                 '-preamble', 'import "gaff2.lt"',
                 '-ignore-angles',  '-ignore-bond-types', '-ignore-masses', '-ignore-coeffs'])

7. If the user wants to use the TraPPE force field, the Ltemplify constructor should be invoked with these arguments:

ltemp = Ltemplify(args=['-atomstyle', atom_style, 
                 '-name', name+' inherits TraPPE',
                 '-preamble', 'import "trappe1998.lt"',
                 '-ignore-angles',  '-ignore-bond-types', '-ignore-masses', '-ignore-coeffs'])

Other force fields might be added in the future, but if they are, I will issue a pull request. (Alternatively if you want to just let the user specify the correct arguments to Ltemplify manually, I suppose that's okay with me also. If you are curious, there are several other ltemplify.py arguments which are explained here, but I don't think you need to worry about them.)

I don't know how you want to encode this information in the preset and parameters variables for the moltemplate plugin. Again, if you don't like the API for the Ltemplify constructor, I am open to suggestions how to make it better.

I had forgotten that VIPSTER can also generate angle, dihedral, and improper interactions, as well as Lennard-Jones parameters for the atomic elements. (Moltemplate users have other ways of generating that information automatically, but I like that they can specify it manually in VIPSTER as well.)

See above, this is imho the most urgent part of the IO-preset/Plugin design.
Is it useful to use the lists provided by vipster or would it be preferable to offload this to moltemplate?

Regarding dependencies:

Sorry for the confusion, i was being imprecise in my last comment.
The problem is inclusion of the plugin-file, as there's currently no mechanism for installing data-files in vipster's build system (at least not in a way that is accessible from python), and the python-bindings may be installed seperately from the GUI, so having a separate file may lead to version clashes.
I'm investigating embedding the python code, will update when this issue is solved.

The current plan on including moltemplate:

• as a direct dependency in setup.py (so solvable by pip)
• installed in the bundled python-environment in official binary releases (which ship a standalone py-env)
• as a recommended optional dependency for manual installs and package builders
  Comments?

Sounds good to me.

Thanks for your patience for my slow replies.

Andrew

[sgsaenger]

Commits 49d519a and 0cdaaa0 implement a writer and parser, respectively.

The writer uses ltemplify.py as discussed and should thus be pretty stable.
The parser uses lttree.py and reproduces its main() functions parsing stage.
After parsing, there are two possible further behaviors:

• If the .lt file actually creates atoms, the output strings as produced by ExecCommands are used to create an intermediate lammps-file and parse it into vipster. This should be fairly stable, except for features missing from lttree.py itself.
• If the .lt file only defines templates, i manually parse the command tree to produce the intermediate lammps-file. This depends on assumptions about moltemplates internal trees and my naive interpretation, so may be fragile.

Especially the last aspect is a bit hacky, and i'm not sure if this is production ready.
Looking forward to your feedback!