-
Notifications
You must be signed in to change notification settings - Fork 109
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Action classes & parameter I/O #10
Comments
Yes -- here we are coming into the much delayed issue of parameter I/O, IDLs, protocol buffers, When constructing Action objects, or building a factory we have to munge parameters from somewhere. I've been delaying a decision on this, and also would prefer to bring my US colleagues into some reasonable level of agreement on what is decided. "Lighter than Chroma" is definitely a design goal. The lighter the better. I see you are munging XML We need a goal of enabling "normal" people to add actions! |
XML for iroiro++ was a design decision several years ago, I had no idea of protocol buffers, JSON, YAML or else. Now I'm leaning towards one of JSON or YAML. I cannot see the advantage of protocol buffers still, I should read/experiment more. The first 2 remain more human readable to me. As an example case. When we define the HMC with multilevel actions XML works very nicely in showing the levels, so would do JSON and YAML, in a cleaner way. In protocol buffers this can be cumbersome to implement, in my understanding. This would be a design request. Let me write some simple test codes that use these three methods to parse data and then choose the one who fits best our needs. |
Notice that anyway an XML parser is needed in the code to handle ILDG format. |
the advantage of protocol buffers is an interface definition language. |
One drawback I see in protocol buffers is that they would introduce a major dependency on what people do at Google. Not only the library but also the tools that you need to install to generate the c+ files from the .proto files. Not so nice for portability, I think. My experience is that QCD codes do not need complex parsers. My parser in IroIro is very simple with very few functions. Writing a parser of standard data serialization format, starting from one of the multiple available libraries, it is quite simple. I agree that we can have a discussion during the lattice conference. Better to prepare some examples in advance. |
http://en.wikipedia.org/wiki/Comparison_of_data_serialization_formats is interesting. Apache AVRO has a JSON idl. My enthusiasm is for an IDL that constrains a mapping between internal data types and external file formats, not for protocol buffers per se. Might also be interesting to see what ROOT has done. |
I know the page. AVRO idl is interesting. we should write down some of the essential requirements for the configuration files and the internal handling of it. I'll start:
Please correct/extend this list. Also there is no need of a full fledged messaging system, just need to read/write configure/output files. |
Probably worth mentioning, in UKhadron I took to an ugly C preprocessor macro to set up It evaded an IDL (as there wasn't any for XML at the time) and still enforced the It was easy to maintain but ugly. |
Having a common definition in one single place would be quite elegant. We tried to simplify things in this sense with parameters classes related to the bigger class that were responsible for reading them from the XML too. Everything in a small portion of the code easily manageable. But then the classes themselves need to be abstracted and composed and this is the role of factories. |
Not that I want to betray weakness, but I'm starting to come round to your use of Pugi xml with no At least people know XML by now, agree YAML looks cleaner. I do very much want to limit On another note, OpenQCD has a comment/ref which I haven't chased up yet claiming that closed form |
We can also be more advanced and give support to both xml and yaml eventually, leaving the user the choice. For the features we need the two are similar in structure, not in readability though. Then I can write an upper layer that masks the real markup language from the Grid code, assuming only that it is structured (i.e. has nesting and key-value-tags). Let me see the OpenQCD claim, but people may like having more than the square root. Some staggered fermions HMC code need different exponents. But if we can get rid of GMP and MPFR would be nice. |
Hi, Why not consider Lua as the configuration language? It's plain C, lightweight and embeddable. You gain the On 06/01/2015 10:00 PM, Guido Cossu wrote:
|
I did wonder about that and looked at the Lua pages a few times in the last week, but didn't get as far Another thought that occurred to me was to define the parameters in a ubiquitous dynamically typed scripting language, such as python or whatever, and use that language to generate the C++ headers and parsers. Effectively generating our own IDL by starting from a language that provides the RTTI [ A random dream is that if the data structures are sufficiently well defined to cross languages, Also... wouldn't it be nice to draw the correlation functions in something resembling Jaxodraw, and have |
I have some experience with using Python as a code generator. See for example: https://github.com/mspraggs/pyQCD/blob/master/pyQCD/templates/core/types.hpp Jinja2 in Python is a very easy to use templating package, and once you've got the header templates set up correctly it's very easy to generate new code. If you want to get more sophisticated there's also libclang bindings for Python, but I haven't tried this myself. I think the Lua route is something that USQCD have worked on, no? They gave a talk at Lattice 2014 about QLua or some such. If the "backend" part of Grid has sufficient encapsulation and provides a nice clean interface, then in principle this would give others the option of writing bindings or an interface of their own choosing. Of course some standard would need to be settled on within a particular collaboration or some such, but it's always nice to have the flexibility. |
I have read about LUA as configuration files and I really like the idea. http://www.ibm.com/developerworks/library/l-embed-lua/ http://stackoverflow.com/questions/15298253/why-lua-for-configuration-plugins http://www.lua.org/pil/25.html https://github.com/vstakhov/libucl (Universal Configuration Language) http://webserver2.tecgraf.puc-rio.br/~celes/tolua/tolua-3.2.html (kind of IDL) http://lua-users.org/wiki/BindingCodeToLua (about the speed of parsers (JSON, YAML, XML, or whatever): this is not a problem since the configuration files are few hundred kb at most. There exist small parsers that can be included completely in the code without external dependencies, too). About the python: I'd like to understand how it works from the final user point of view. How does a configuration file looks like? I also like the idea of writing Grid that is unaware of the configuration interface, leaving coders the ability to write their own, or providing different choices. |
Hello, FWIW, I've looked into formats discussed above a few month ago when we were starting to discuss "new CPS", and I ended up liking JSON at the time. Currently my preference I think is somewhere between JSON and YAML. Both seem to have good balance of human readability and 'writability'. I seem to remember the lua interpreter had to be modified (slightly, I think) for either FUEL or Qlua. Anyone remembers this? I'll try to find out. |
I'm closing this issue -- the issue on serialisation presents the resolution of a lot of the object I/O issues. The action fragments are in an HMC already running thanks Guido (Quenched) and my doing the fermion force. While I may revisit the HMC area to tidy up a little, I see no reason to keep this issue open any more, with the dangling aspects subsumed by the serialisation track. Even the serialisation seems to only now be a question if implementing a virtual/abstract reader/writer |
Feature/gpt sycl
I can start working on the action parts (gauge and fermions).
We have to decide the target design functionality and style.
I have in mind something similar to what I have in IroIro++, a kind of lightweight version of the Chroma ones.
You can have a look here
https://github.com/coppolachan/IroIro/tree/master/lib/Action
and here for the corresponding HMC implementation (leapfrog, 2MN, multilevel)
https://github.com/coppolachan/IroIro/tree/master/lib/HMC
The text was updated successfully, but these errors were encountered: