-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Redesigning ly.music #3
Comments
This is a great and important effort, and I'll try to join it as soon as possible. Although I have to admit that I will have to build experience with XML processing more or less from scratch. One thing I'd like to add (which is partly OT but not completely): One of my interests in this topic is the idea to create an interface between LilyPond and the MEI format/initiative because I see that as a huge potential for Lilypond in the academic editing domain, particularly for digital edition concepts. @PeterBjuhr , Mike (Solomon) and I will present a paper at the Music Encoding Conference in May 2015, and the progress on this effort is something we'll want and have to report there. |
Note:
|
I'm not sure that creating the tree structure in But you emphasize other advantages, and in that case redesigning is perhaps worth the effort! |
It is Christopher Antila'a strong conviction that doing this change will make processing significantly faster, and that's crucial because we don't only want to enable import/export but actually the idea of having multiple document formats as live "views" on a DOM. |
Adding to that, I would like to get rid of the many subclasses of Item that have their own methods. So I want the |
What I believe is that an ElementTree-backed data structure will be faster and more flexible than the current all-Python implementation with node.py, especially considering the requirements Wilbert mentioned above, and that Urs and I have in mind for the LilyPond-to-MEI conversion. Though I do believe better solutions may exist, I have experience with Python's ElementTree API, so for me at least it's the fastest way to get started (no doubt that's a somewhat selfish justification!) ElementTree will not by any means solve all our problems---we still need intelligent algorithms. However, the key to its usefulness is realizing that, as described in the Python documentation, ElementTree is simply a hierarchic data structure. Yes, it provides facilities to import and export XML documents, but that's secondary functionality. (It may be a useful way for Frescobaldi to avoid re-parsing a document every time it's opened too). Both the MEI and MusicXML converters, and any other ones we write, will probably need to significantly restructure the hierarchy of elements, as Peter rightly notes. There is simply no way to avoid that, but I hope the ElementTree API's XPath capabilities will make that task significantly easier. Other advantages, some mentioned above:
We should be able to fulfill Wilbert's eighth requirement too, "for a tree that is created from tokenized source: update the tree from the position where source text changes without recreating the whole tree." If every element has its source-document position available in its attributes, we will be able to find it with a relatively simple XPath query. By the way, every time I mention "XPath" query, I'm referring to the functionality provided by the find() and findall() methods. Again, I'm not convinced this is the best solution. It's just one with software I know how to use, and that I believe can meet our initial requirements. I have started experimental work on a bit-by-bit migration from node.py to ElementTree. Since there seems to be interest in at least seeing how the experiment turned out, I'll try to complete it enough to upload this week so you can see. Thanks for considering the move! |
@crantila , thank you for your ideas and help! |
A question. Which ElementTree implementation do we choose? I tend to favour/prefer This would require the |
To reduce the dependency burden, my plan was to try to use the standard library first, and only move to To find a node's parent node, I was imagining an XPath query like this:
That should select the parent of an element that has the "id" attribute set to "1234." This means we'll have to keep the document's root element in a module-accessible place, which isn't great. Maybe we could have a module-level wrapper object that offers a "find the parent" method. What do you think? Mostly I'm just wary of building with (Also, I hope to have a small trial sample finished by the end of the weekend). |
I think it's perfectly fine to use xml.etree. We'll probably always have the root available in some object that queries or traverses the tree. With careful coding we can al get around the getparent stuff. |
@crantila , I am brainstorming about converting Just some ideas: We basically wil be using the normal methods (without subclassing anything) to create the We need, for the iteration, a fast way to distinguish the main types of LilyPond content: music, markup, scheme etc. Currently, to iterate over a tree to look for music, we can just skip items that are not an instance of Or shall we use the tag name to described the main type. E.g. a piece of e.g. <music type="relative">
<pitch-arg note="c" octave="1"/> <!-- a pitch argument that is not music in itself -->
<music type="sequentialmusic">
<note name="a" octave="0"/>
<note name="b" octave="0"/>
<note name="n" octave="0"/>
</music>
</music> And <override>
<context name="Staff"/> <!-- an optional element -->
<grob name="InstrumentName"/>
<property-path>
<property name="extra-offset"/>
</property-path>
<value> <!-- the thing after the '=' -->
<scheme>
<quote>
<pair>
<integer value="1">
<integer value="2">
</pair>
</quote>
</scheme>
</value>
</override> Maybe a good departing point is that every token results in an element. That way we can easily store the token that generated elements in the token attribute. Maybe some elements, like music or markup lists, also have an ending token, which could be stored in the end_token attribute. Of course the tokens are not used when building a tree from scratch. |
Just a shot in the dark: I mean, MEI is the most comprehensive xml based data format to encode music, so it could be a good idea to go that way? Maybe it would even be possible to design our data format (partially) as a super/subset of MEI? |
Interesting and tempting thought, but as the tree structure in Nevertheless, @crantila to examine this idea and to teach us more about MEI could you give us some clues about how the examples of @wbsoft above could be represented in a MEI(-like) structure? |
@uliska, No, |
Ok. As I said, just a shot in the dark |
thinking about the "every token results in an element" idea, this is another rendering of the <override>
<context name="Staff"/> <!-- an optional element -->
<dot/>
<grob name="InstrumentName"/>
<dot/>
<property name="extra-offset"/>
<equalsign/>
<scheme>
<quote>
<list>
<integer value="1"/>
<dot/>
<integer value="2"/>
</list>
</quote>
</scheme>
</override> In this example the idea is that |
<markup>
<bold>
<markuplist>
<markupword text="two"/>
<markupword text="words"/>
</markuplist>
</bold>
</markup> |
I'm starting (finally) to read something in the MEI documentation, and still somewhat influenced by @uliska 's idea I don't see why we can't have: <note pname="a" oct="0"/> instead of: <note name="a" octave="0"/> If that would be in any way helpful... But to follow the "every token results in an element" idea, perhaps the note name and the octave token should be distinct elements!? |
@PeterBjuhr and @uliska , indeed we could borrow common naming from MEI.
@PeterBjuhr , yes that's thinkable, although I'm not completely sure. Our "LilyXML" tree should just be there to create easily from parsed LilyPond source (very much like the current When a LilyXML tree is built from a document, all elements should point to their position in the source. It should be possible to edit the tree and then see what changes are to be made to the source document. It is in fact not needed to store the tokens themselves. Just the position (from:to, or pos:length) needs to be set for every token. Adding When we have read a pitch from a document, it is represented there as e.g. |
In the end we should not only be able to convert LilyXML to MEI but also MEI to LilyXML. Following @uliska idea somewhat, maybe it would be more helpful to look in the import direction and see what changes need to be made in the conversion to LilyXML. That said, I want to again emphasize that we need to be careful not to be too influenced by the MEI syntax and loose track of other important considerations... |
Yes. I think one major goal is to be able to perform operations like those from the quick insert panel or applying graphical edits to the source code. That's the most important objective as that would open up tons of possibilities. |
It seems to me like there are three issues. I'll touch on each in turn. 1.) What the ly.music internal document structure should be like, in particular regarding similarity to MEI. Internal Document Structure(Interesting bit above about why Wilbert originally acquired the Frescobaldi domain). I've been quietly thinking about how MEI-like Frescobaldi's ElementTree should be, since after all my personal interest in this project is in using ly.music for bi-directional converters between MEI and LilyPond. On the one hand, it seems obvious that Frescobaldi should model a LilyPond document with a LilyPond-like data structure, or else we'll probably lose something in the conversion. There's no way to represent a LilyPond Now I've thought about it a bit more, especially reading comments in this issue. Now I think we should seriously consider parsing LilyPond documents straight into MEI. (Still not convinced it's what we should do---but let's consider it). For one, we haven't entirely decided how to represent a LilyPond document in an ElementTree, and it looks like we're basically going to have to develop an XML-capable data format to do this. Even if we retain the current "Yes but..." is the answer to that. Yes, but even though MEI is capable of representing more notational features than MusicXML, it's still not sufficient to represent the highly-specific complexities of a LilyPond document, especially in the way Frescobaldi requires. True. So far. One of MEI's biggest selling-points over MusicXML is its extensibility, so we can actually add any required features to MEI. If we choose this approach, we only have to invent the LilyPond-specific aspects of the data internal ElementTree. Thus Wilbert's example of
We might represent
And to remember source document locations, we might do this, which I think is self-explanatory:
If we're going to experiment anyway, this might be worth trying out. Connection to Source DocumentsJust above, I proposed a very simple strategy to maintain connections to the source document. One of our goals is to generate a LilyPond document from the ly.music ElementTree, which shouldn't be particularly difficult once we decide on a data format. To know where the ElementTree elements ended up, at first we could just import the generated LilyPond document. The problem of whitespace remains. Ideally, we could import a LilyPond document to the internal format, then export the internal data into an identical LilyPond document. A more feasible, but much slower, approach to document generation (or more accurately, to modification of existing documents) will have us search for every modified token, one at a time, updating the LilyPond source file to reflect modifications to the tree. Yet even that's difficult, since our modifications to the source file may change the positions of other tokens, so we would also have to update the token positions after every modification. I'm not sure this is the best approach, but it seems workable to convert
Tab, space, and newline characters will be thusly preserved, allowing us to (hopefully?) regenerate identical source files. Admittedly, it looks like a mess, but these files aren't meant for human consumption anyway. Migrating or Rewriting?My original idea was to "migrate" ly.music to the ElementTree API, bit by bit. First, Node would subclass Element. Then we'll get rid of the token-specific classes. Then we'll get rid of Node, and so on. Now it seems that, no matter which strategy we choose for internal representation, we're going to end up rewriting everything in ly.music anyway, so we should start from scratch? In any case, I'm going to hold off on preparing a sample by myself. Let's agree on an internal data format, then maybe Wilbert can lay out a source-file skeleton, and we can work together to fill it out? |
I might opt for a rewrite, but I really think I need to be able to express a LilyPond source file in the There are some issues which spring into view when thinking in a Mei-like xml format:
Unfortunately, I am not yet well-versed into Mei, but very well in LilyPond. I think that probably the best way is that I design a close-as-possible meaningful tree representation for the LilyPond input, that is very easily readable and in an event-handling traversed. Then building a Mei or MusicXML document could be done by iterating the tree or traversing its events (a structure defined by LilyPond) and build the MusicXML or Mei tree from scratch. The other way around (Mei->LilyPond or even MusicXML->LilyPond) is also done by understanding that data structure and convert it to LilyPond. When I write a helper module with functions that create the most important constructs immediately, it will not be difficult to write LilyPond documents from scratch. Also the score wizard in Frescobaldi would use it. (And then we can drop A final note: we will never be able to correcly export really complicated LilyPond music, because we can't export music that's built by Scheme functions. The best LilyPond→MusicXML or Mei convert would need to access the Scheme music tree data structure inside LilyPond.... |
@crantila great to learn more about MEI! But even if we can extend MEI with the LilyPond and Scheme specific stuff, I must agree with @wbsoft and say that I'm not convinced that will take us far enough. As stated before, the xml tree we're after must represent the LilyPond code without compromises in the form of premature conversions to another xml syntax. Surely MEI can't be that flexible!? |
To follow up on a previous posting, to learn more about MEI and to give a little more concrete example I've tried to create a tiny MEI example (perhaps it's too simplistic to be correct!?): <score>
<section>
<measure>
<staff>
<note pname="e" oct="5" dur="8" tuplet="i1"/>
<note pname="d" oct="5" dur="8" tuplet="m1"/>
<note pname="c" oct="5" dur="8" tuplet="t1"/>
</staff>
</measure>
</section>
</score> The to me nearest conceivable conversion to our proposed LilyXML would be: <score>
<section type="sequential">
<new />
<context name="Staff"/>
<section type="sequential">
<tuplet fraction="3/2">
<section type="sequential">
<note pname="e" oct="2" dur="8"/>
<note pname="d" oct="2" dur="8"/>
<note pname="c" oct="2" dur="8"/>
</section>
</tuplet>
</section>
</section>
</score> And in pure LilyPond it would amount to: \score {
\new Staff {
\tuplet 3/2 {
e''8 d''8 c''8
}
}
} The differences may be small, but to me they seem substantial even in this simple example. And I'm not convinced that it would be helpful to adapt to MEI naming conventions even where it can be done. |
Basically the problem is that the same music can be written in many different ways in LilyPond. We've been focusing on the conclusion that this means that we need to retain a strong connection between the tree structure and how the document is actually written; which I think is reason enough. But it could be added that this also means that exporting the music from the source code is quite complex. In fact it's impossible if you aren't willing to create yet another LilyPond/Scheme parser. I think the intermediate step that |
Am 31.01.2015 um 09:38 schrieb Peter Bjuhr:
I also think that we should agree with @wbsoft and make ly.music
I have to say that this is a subject I raised at the very beginning of
The latter seems more straightforward. By now (with my recent \annotate
I also think that when we want to use Frescobaldi as a live editor of a |
I think that the uitimate exporting backend should run in LilyPond. LilyPond builds a large Scheme data structure of the music. It should be possible to write a Scheme procedure (or LilyPond backend) that can read this data structure and convert it to something else. Here is an example:
Results in:
Note that every LilyPond Another example:
I think it is overkill if we add a parser for these scheme procedures to
to
And then run LilyPond, which will run our exporter. This then could be done by an editor. |
Building upon this, I have yet another idea. To write a small function in Scheme that maps a LilyPond music structure to XML directly. This structure (let's call it lilyxml) is then read by a different program which can convert it to anything else. The main manipulations are already done by LilyPond (like substituting variables, including files, setting durations etc.). This lilyxml could be the base for ly.music. The lilypond music structure did not change much over the last years. What is in I am willing to write such a scheme function. It might even exist already. Then we can experiment. |
Interesting thought! Will I remember that we looked into the information Nevertheless to be able to parse that information could open up new possibilities! |
This is something I just started to think about after your prevous comment :-) The only issue I see is that this structure will only be available after compilation. But for exporting to whatever target I think this should be a very promising approach. This could then be made a backend so one could simply selext xml instead of pdf... which would of course skip the whole layout processing. @crantila as an aside note wrt mei2ly: if we could write a converter from MEI to such a Scheme structure we could feed this directly into LilyPond, completely skipping the stage of LilyPond code. I have the impression this could be significantly more straightforward to be done. Maybe even in Scheme - which could lead to LilyPond reading MEI files directly. Maybe we should move that discussion to another place. |
Am 31. Januar 2015 11:46:12 MEZ, schrieb Peter Bjuhr notifications@github.com:
I think it wouldn't be plain \displayMusic but something else that is applied at a later stage of processing.
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. |
Note that this idea of conversion of the LilyPond music structure to XML is already some years old. But afaics, as of yet no-one stepped in to actually create such a scheme function. |
To set things into perspective, when the MusicXML exporter development started the goal was limited to being able to export only documents that had previously been imported (as a kind of exchange model). When |
I think that is possible. That is somewhat the behaviour of
It looses the |
I'm going to push a bit farther with the internally-held MEI idea, even though my own feeling is that a compromise between the full MEI spec and the existing There seems to be general agreement that the representation must reflect the source document quite closely. I agree with this in principle, but I also think that many of the Lily-XML samples follow the source document too closely. This obscures the document's meaning, and will make it harder to work with for any purpose. Consider
For me, this doesn't logically represent either the source document or its meaning. In Lily-XML I would rather write:
That's still a one-to-one mapping, but it collects everything about the
We will have to invent the markup element. Here's an alternative to @PeterBjuhr's MEI example, which more closely reflects the LilyPond code like this:
The mei, music, body, mdiv, score, and section elements are all required for valid MEI, but there's no reason we have to hold a fully-valid ElementTree internally. The layer element is also required by MEI. For us it represents a Voice context. Admittedly, adding the implied Voice takes us farther from the source document, so I invented the @lily.type attribute to indicate it's not written in the source document. Here's an example of where we could modify the MEI spec (to not require the layer element), or even just not bother to follow it exactly. It's also possible to resolve include files and allow for context-sensitive interpretation of the same music. Consider the following LilyPond code:
In my LilyPond-centric MEI, this could be:
I'm not sure I'm using the cross-reference attributes correctly because I've never done it before, but the point is that it's possible. To me, this snippet reflects the LilyPond source quite closely and transparently, and it's almost valid MEI. (The note elements should be inside a layer inside a staff). To deal with @PeterBjuhr: "Basically the problem is that the same music can be written in many different ways in LilyPond." That's a difficulty for parsing both LilyPond and MEI files, but I don't think it's that big of a problem. If anything, the ambiguity of both formats suggests they're well-suited for each other. In the end we're going to have to make some compromises, no matter how we chose to parse the source file. The only true one-to-one representation of a LilyPond file is itself. However, we all already know this: "exporting the music from the source code is... impossible if you aren't willing to create yet another LilyPond/Scheme parser." (@PeterBjuhr) Isn't that exactly what we're trying to do? Even though we're focussing on the LilyPond syntax first, I guess I took it for granted that we were going to have to parse Scheme too. Consider a simple example:
As development continues, we'll convert more and more of the Scheme into MEI, in this case by adding a @color attribute to the note. Since MEI isn't even sort of good at encoding Scheme programs, we should keep all the scheme elements in the document tree, even after their meaning is known and imported correctly. But I don't understand why you're talking about using LilyPond's Scheme output. It is the best way to know how LilyPond will parse a file, but it doesn't really work for our goals with ly.music. In particular, it renders the source file nearly unrecoverable. |
But wait! If LilyPond's hierarchic Scheme code is basically what we're planning to use ElementTree for, why don't we parse LilyPond files into an ElementTree based on the same hierarchy? So that this in LilyPond...
... becomes something like this in ly.music:
We could attach attributes with line numbers and character positions too. Just a thought. |
Or Scheme's XML, called SXML? https://www.gnu.org/software/guile/manual/html_node/SXML.html |
That's exactly what I am currently trying :-) We could use SXML but currently I just write xml by hand. There are some issues: a command like But the first step is to have a reliable XML conversion of the LilyPond scheme music and markup structures. Let me try and figure for some time, some premature code is in |
I follow the development in But it was also very sudden, I'm still trying to straight out a few things. I'll do it here and see if there still is any questions remaining. Hopefully it can be helpful for the overall discussion: The translation to XML now has to be triggered by an include and a LilyPond compilation. I guess the include can be made in the background in the same way as for the layout control. And the compilation could be made in a similar way as the auto engrave when the tree structure is needed (here could be a speed issue though). |
If we take an example like this: rhythm =
#(define-music-function (parser location p) (ly:pitch?)
#{ \tuplet 3/2 { $p 8 8 8 } #})
music = {
\rhythm c'
} the close connection with the document is lost. It would suddenly be possible to export the music correctly, which is great. And perhaps |
Peter, actually I want both: build the xml by hand or by LilyPond. I will clearly specify the differences. But to settle on the format I'll first implement the xml mapping from within LilyPond. |
Ah, I see! So a tree structure could still be created by I think that settles my questions! |
If this is what we end up doing, it means we need to build, test, and optimize a Scheme-to-XML program, and test and optimize the existing lexer and tokenizer. Plus, a fully correct runtime parse into ly.music requires coordinating the input of both sources. Do I have this right? Or are you just building the Scheme-to-XML program in order to help determine what |
I am experimenting (for fun also :-)
That's indeed what I am trying first. Of course there are large differences: E.g. |
If the road ahead for the redesign has become clearer, perhaps this issue can be closed? |
Clearer, but not clear. Shouldn't we close the issue only when we're either finished, or we've created a milestone with a set of smaller issues as a "to-do" list? @wbsoft: "I am experimenting (for fun also :-)" Two other ideas. The Contributor's Guide offers the LilyPond grammar as outputted "from the parser" (GNU Bison). Could we feed this into a Python parser generator? Or, since the licences work, we could simply copy the LilyPond parser, bringing it into Python as a C extension module. Taking this to the next step, since libxml2 is a C library, is there a way to get data from LilyPond into libxml2 without going through Python first? I know LilyPond is written in C++, but it still might work. If nobody knows whether either of these are possible, I'd be happy to investigate. |
I write code for fun! :-) I am a professional musician (organist, choir conductur and carillonist). Experimenting and finding things out gives me lot of fun, but many times there is no spare time at all and I can do less. I get paid incidentally to engrave music using LilyPond, and wrote the new Dutch church hymnbook that way (www.liedboek.nl).
Let it open for brainstorming, etc. I am now just experimenting how creating XML from LilyPond looks. Then I'll have a look how we can formalize stuff and make clear how we can manually build such trees that are understandable as well. XML generation from within LilyPond is quite promising. Once the format settles, it might become viable for further processing by other tools without needing to add those to LilyPond or write them entirely in Scheme.
Two very nice ideas, esp. having a LilyPond parser as Python extension module. The |
Yes, we should create a milestone. But currently it is not completely clear in my mind what exactly the new |
Hi, I'm just wondering if there has been progress or decisions about the direction of this issue or #8 ? If it helps, I tried out my idea of using LilyPond's actual parser as a C extension module, and it seems far more complicated than hoped. Also, I may be able to recruit someone with experience building parsers and lexers. We're both eager to start helping out! |
I've started to make a simple translator from REAPER selected items, formatted in Reaper's notation view, and discovered the same issue as described here. Can the team consider using the original LilyPond repo implementation inside this library? |
I think python-ly is essentially deprecated. The |
@wbsoft , |
quickly is not yet complete, it is rather in its infancy and very much in flux, but I have some time to devote to it, and eventually it will become the new engine for Frescobaldi's understanding of LilyPond text input. |
Thanks, maybe I'll find time to help with it. As I much more prefer to have a package-like tool on PyPi that can be mentioned in requirements than a single-folder from the big LilyPond repository))) |
Dear friends, I want to take the lead for redesigning
ly.music
. This issue is to continue the thread at frescobaldi/frescobaldi#492, which I'll close.ly.music
currently has two shortcomings:It was initially designed to create a tree structure from an existing document, which is done in the
ly.music.read
module.@crantila emphasizes on the fact that basing
ly.music
aroundxml.etree
orlxml
would make it much faster as not a python object is created for every single tree item, and we would have powerful search possibilities for free.I am willing to design the
ly.music
tree such that it adheres to simple principles, maybe even without creating a class for every type of element.We need the following:
ly.document.Document
) and have all tree items know the originating tokens (and thus source position, as every token has the position in the source file in thepos
attribute)Functions like transpose or translate currently iterate over tokens, but they should operate on the tree in the future, and simply change the tree.
When the tree was built from a tokenized source (i.e. the originating tokens are attached to the tree nodes, while their current value has changed) a dedicated module should be able to find out which changes need to be done to the originating document to have it reflect whatever is changed in the tree. (We won't dump a new LilyPond document because that would bluntly overwrite things that a user might have done different, like whitespace or comments that may or may not be present in the tree, and also that would destroy markings or cursor positions for point-and-click when a document is overwritten inside an editor such as Frescobaldi.)
The text was updated successfully, but these errors were encountered: