diff --git a/shared/xsd/ldf.xsd b/shared/xsd/ldf.xsd index 7b3519f0..fb6b46d8 100644 --- a/shared/xsd/ldf.xsd +++ b/shared/xsd/ldf.xsd @@ -57,6 +57,7 @@ + @@ -204,7 +205,13 @@ - + + + + + + + @@ -237,116 +244,114 @@ - - - - - - - Whatever the authors deem to be important enough to be put on one of the first - pages. For example, in C# specifications Foreword is about the differences brought - to the language by the current standard, while in the Scheme specification - Foreword discusses programming languages design and demonstrates its principles - applied to the forthcoming document. - - - Technically speaking, Foreword is not a part of the specification. Instead, it - precedes the specification and introduces it by putting in the proper context. - - - - - - - The goals of language design are sometimes encounted being explicitly - stated in the language document in one of the informative sections - of the front matter part. - - - For example: - "C# is intended to be a simple, modern, general-purpose, object-oriented - programming language." - (from ECMA 334 3rd edition, page xvii) - - - - - - - Scope section explains the context for the language document. - - - - - - - Conformance section defines several levels of compliance by - explaining what is a conforming program and a conforming - implementation with respect to this standard. - - - Definitions for meta-terms like "shall" and "should" - and their relation to the compliance issue explained above. - - - - - - - Same as conformance - - - - - - - While conformance/compliance define how external artifacts should - conform to this standard, this section defines how this standard - complies with previously existing ones. - - - - - - - Notation section defines grammar definition formalism used in the document: - mostly it is about the EBNF dialect. - - - - - - - Formally lined up references to all other standards that are used or referenced to - from within the document. - - - - - - - This section informally describes how the document is organised, divided - into parts and chapters. Sometimes it explicitly states which sections are - normative and which are informative. - - - - - - - A list of changes brought to the language by the current specification - replacing the previous one. - - - - - - - - - + + diff --git a/shared/xsd/xbgf.xsd b/shared/xsd/xbgf.xsd index 3be4bb23..8ecd3427 100644 --- a/shared/xsd/xbgf.xsd +++ b/shared/xsd/xbgf.xsd @@ -54,8 +54,22 @@ - Here is a list of the XBGF transformations that perform - folding and unfolding activities. + Folding and unfolding activities are the most basic ones in grammar + transformation and the most used ones in grammar convergence. Since + grammar comparison is done in such a way that only applies very basic + algebraic laws in its endeavours to match the two grammars, many + more sophisticated manipulations need to be executed semi-automatically + in a programmable fashion. These manual steps help to establish a + stronger link between the convergence point and the original grammar + artifact since they aid to reveal some unapparent properties of those + grammars. + + + All these transformations are provably correct in the sense that + it is possible to prove that the languages generated by the grammars + before the transformation and after it are indeed the same. All + refactorings are easily reversible and usually introduced below in + pairs. @@ -123,9 +137,33 @@ - Here is a list of the XBGF transformations that - preserve abstract semantics, but do not preserve - (break) concrete semantics. + We may refer to the semantics of a grammar as the language (set of strings) + generated by the grammar, as it is common for formal languages --- for + context-free grammars, in particular. With the string-oriented semantics + in mind, all transformations mentioned above in folding and refactoring + sections are semantics-preserving. To give an example where different + semantics are needed consider the scenario of aligning a concrete and + an abstract syntax. + + + When necessary, we may apply the algebraic interpretation of a grammar, where + grammar productions constitute an algebraic signature subject to a term-algebraic model. + In this case, the terminal occurrences in any given production do no longer carry + semantic meaning; they are part of the function symbol. (Hence, abstract and concrete + syntaxes can be aligned now.) + + + Some transformations that were effortlessly semantics-preserving w.r.t. + the string-oriented semantics, require designated bijective mappings + w.r.t. the term-oriented semantics, e.g., fold/unfold manipulations, + but generally, the term-oriented semantics admits a larger class of + semantics-preserving transformations than the string-oriented one. + + + The following section gathers those transformations that would have been + considered refactorings if we only used term-oriented semantics. From + the string-oriented point of view they revise semantics and can be deemed + as neither grammar lenghtening nor grammar shortening transformations. @@ -137,8 +175,19 @@ - Here is a list of the XBGF transformations that - revise semantics, both abstract and concrete. + Some grammar differences may require more arbitrary replacements, + that cannot be expressed as semantics-preserving even in abstract + syntax. These include projections or injections of non-optional + nonterminals, adding definitions for bottom nonterminals, + performing volatile replacements. + + + Whenever a transformation from this group is used in a conergence + path, it signals either about a construction point where the grammar + engineer is having a temporary shortcut to be substituted later + with a longer sequence of more accurate manipulations, or an inevitable + error in the BGF that needs fixing (preferably in the original + artifact---specification, compiler sources, etc). @@ -153,8 +202,13 @@ - Here is a list of the XBGF transformations that perform - decorative tasks like adding or removing labels and selectors. + Last but not least, there are two pairs of refactorings that are + very specific to the BGF itself. Not all grammar definition formalisms + have labelled productions, but since we do, we also need two transformation + steps made possible: to designate an already available production with a + label, and to unlabel an existing labelled production. By anonymising + we refer to stripping selectors from subexpressions, and by deanonymising, + naturally, adding selectors. @@ -566,7 +620,7 @@ There are two expression arguments: one to be matched, and another one that replaces the matched expression. - One of them must be in a "massage relation" to the other. + One of them must be in a `massage relation' to the other. The scope of diff --git a/shared/xsd/xldf.xsd b/shared/xsd/xldf.xsd index 85f7a51b..b4162a0c 100644 --- a/shared/xsd/xldf.xsd +++ b/shared/xsd/xldf.xsd @@ -37,7 +37,8 @@ - + + @@ -66,6 +67,22 @@ + + + + TBD + + + + + + + + + + + + @@ -75,7 +92,7 @@ - + diff --git a/shared/xsl/bgf2bnf.xslt b/shared/xsl/bgf2bnf.xslt index 4c804733..1300c238 100644 --- a/shared/xsl/bgf2bnf.xslt +++ b/shared/xsl/bgf2bnf.xslt @@ -141,7 +141,7 @@ ( - | + | ) diff --git a/shared/xsl/ldf2tex.xslt b/shared/xsl/ldf2tex.xslt index 2c0f11cc..8036f401 100644 --- a/shared/xsl/ldf2tex.xslt +++ b/shared/xsl/ldf2tex.xslt @@ -15,8 +15,10 @@ - \documentclass{article} - \begin{document} +\documentclass{article} +\setcounter{secnumdepth}{2} +\setcounter{tocdepth}{2} +\begin{document} \title{ @@ -61,6 +63,9 @@ + + + @@ -73,6 +78,9 @@ + + + @@ -114,7 +122,7 @@ - + @@ -127,135 +135,165 @@ $$ - - - - + + + + + + \label{ + + } + + + - -

- -

+ -
    - -
  • - -
    • -     -
+ -
- - - -
+ + + + \begin{verbatim} + + \end{verbatim} + + + + + + - - - - \label{ - - - - } - + + \section{ - - + + - + } - - - - - - \section{ + + + \subsection{ - - + + - + } - - - - \label{ - - } - - - - - - \begin{verbatim} - - - \end{verbatim} - - - - - - - - - - - \subsection{ + + + \subsubsection{ - - + + - + } - + + + + + + + + \label{ + + } + + + + + + \begin{verbatim} + + \end{verbatim} + + + + + + + + + + + + + + + + + + + + + + + + + \label{ - + } - - - - \begin{verbatim} - - \end{verbatim} - - + + + + + + + + + + + + + + + + + + + + + + + + @@ -279,174 +317,7 @@ -

- - - -
-

- - - - - - Grammar productions - - (ver. ) - - - -

- -
- - - - - - - - - - - - - - - - - - - - - - - - - : - - - - - - - - - - - - - - ( - - ) - +

- - - - - - - - - -
-

- - - - - - Source code sample - - (ver. ) - - - -

-
- 
-        
-
-
- - - - -
-

- - - - - - Sample execution - - (ver. ) - - - -

-
- - - - # - - - - - - -
-

- - (Should return ) - -

-
- - - - - - - - - -

- -

- - - - - - - - - - - - -
- | - - - - - - - - - # - - - - - - - - + \ No newline at end of file diff --git a/topics/extraction/xsd2ldf/ldfgen.py b/topics/extraction/xsd2ldf/ldfgen.py index 98ab6ebb..ed102c32 100755 --- a/topics/extraction/xsd2ldf/ldfgen.py +++ b/topics/extraction/xsd2ldf/ldfgen.py @@ -83,7 +83,9 @@ def main(xsdfile,bgffile,ldffile): for p in nt.findall('./{%s}annotation/{%s}documentation' % (xsdns,xsdns)): pel = ET.SubElement(el,'p') pel.text = p.text - section.append(grammar[nt.attrib['name']]) + # Need to decide whether to put productions inside description subsections + #section.append(grammar[nt.attrib['name']]) + el.append(grammar[nt.attrib['name']]) # print grammar[nt.attrib['name']] ET.ElementTree(dtree).write(ldffile) diff --git a/topics/languedoc/complete.xldf b/topics/languedoc/complete.xldf index 64677589..d83aba98 100644 --- a/topics/languedoc/complete.xldf +++ b/topics/languedoc/complete.xldf @@ -4,11 +4,274 @@ xmlns:xbgf="http://planet-sl.org/xbgf" xmlns:xldf="http://planet-sl.org/xldf"> + + + + Vadim Zaytsev + +

+ XBGF operator suite was developed mainly for grammar convergence + activities. +

+ + + + + + + + + notation-section + Vadim Zaytsev + +

+ All BGFs and XBGFs are presented in a unified pretty-printed way. +

+ + + grammar + + + + + + + root + + nonterminal + + + + + + + + + production + + + + + + + + production + + + + + + + label + + label + + + + + + + + nonterminal + + nonterminal + + + + + expression + + + + + + expression + + + + + epsilon + + + + + + + + empty + + + + + + + + value + + value + + + + + + any + + + + + + + + terminal + + terminal + + + + + + nonterminal + + nonterminal + + + + + + selectable + + + + + selector + + selector + + + + + expression + + + + + + + + sequence + + + + expression + + + + + + + + marked + + expression + + + + + + choice + + + + expression + + + + + + + + optional + + expression + + + + + + plus + + expression + + + + + + star + + expression + + + + + + + + value + + + + + int + + + + + + + + string + + + + + + + + + + label + + string + + + + nonterminal + + string + + + + selector + + string + + + + terminal + + string + + + + + + +

There are two expression arguments: one to be matched, and another one that - replaces the matched expression. One of them must be in a "massage relation" to + replaces the matched expression. One of them must be in a `massage relation' to the other.

@@ -261,7 +524,7 @@ concreterevisingtransformation - Semantics revising transformations that preserve abstract semantics + Refactorings in term-oriented semantics
element-abstractize
@@ -281,7 +544,7 @@ abstractrevisingtransformation - Stronger semantics revising transformations + Semantics revising transformations
element-define
diff --git a/topics/transformation/xldf/xldf.py b/topics/transformation/xldf/xldf.py new file mode 100755 index 00000000..f4fb687f --- /dev/null +++ b/topics/transformation/xldf/xldf.py @@ -0,0 +1,289 @@ +#!/usr/bin/python +import sys +import string +import elementtree.ElementTree as ET + +ldfns = 'http://planet-sl.org/ldf' +xldfns = 'http://planet-sl.org/xldf' +bgfns = 'http://planet-sl.org/bgf' +xbgfns= 'http://planet-sl.org/xbgf' +#ldxns = 'http://planet-sl.org/ldx' +xsdns = 'http://www.w3.org/2001/XMLSchema' +htmlns= 'http://www.w3.org/1999/xhtml' + +ET._namespace_map[ldfns] = 'ldf' +ET._namespace_map[xldfns] = 'xldf' +ET._namespace_map[bgfns] = 'bgf' +ET._namespace_map[xbgfns]='xbgf' +#ET._namespace_map[ldxns] = 'ldx' +ET._namespace_map[xsdns] = 'xsd' +ET._namespace_map[htmlns]='html' + +acceptedtags = ('{'+xsdns+'}complexType','{'+xsdns+'}element','{'+xsdns+'}simpleType','{'+xsdns+'}group') + +def preparetext(s): + if not s: + return s + else: + return ' '.join(map(string.strip,s.split('\n'))) + +def xldf_insert(cmd,tree): + welookfor = preparetext(cmd.findtext('*/*')) + for cnt in tree.findall('.//content'): + passed = False + for eli in range(0,len(cnt)): + if preparetext(cnt[eli].text) == welookfor: + cnt_idx = eli + passed = True + if passed: + if cmd.findall('*')[0].tag=='after': + cnt_idx += 1 + print '[XLDF] insert(', + cx = 0 + ctag = '' + for el in cmd.findall('content/*'): + if cx: + if ctag==el.tag: + cx += 1 + else: + if cx==1: + print ctag, + else: + print ctag,'*',cx, + cx = 1 + ctag = el.tag + else: + cx = 1 + ctag = el.tag + #print el.tag, + cnt.insert(cnt_idx,ET.Element(el.tag,{})) + cnt[cnt_idx].text = el.text + cnt_idx += 1 + if cx: + if cx==1: + print ctag, + else: + print ctag,'*',cx, + print ')' + return + print '[----] xldf:insert failed, cannot find the designated target!' + return + +def findnode(tree,id): + for s in tree.findall('//*'): + if s.findall('id'): + if s.findtext('id')==id: + return s + return None + +def xldf_append(cmd,tree): + found = findnode(tree,cmd.findtext('where')) + if not found: + print '[----] xldf:append failed: target id',cmd.findtext('where'),'not found' + else: + print '[XLDF] append(',cmd.findtext('where'),',', + for p in cmd.findall('content/*'): + found[-1][-1].append(p) + print p.tag, + print ')' + return + +def xldf_move(cmd,tree): + #found = tree.findall('//core[id="'+cmd.findtext('./section')+'"]') + found = findnode(tree,cmd.findtext('section')) + if not found: + print '[----] xldf:move failed: source node not found!' + return + #found2 = tree.findall('//core[id="'+cmd.findtext('./inside')+'"]') + found2 = findnode(tree,cmd.findtext('inside')) + if not found2: + print '[----] xldf:move failed: target node not found!' + return + found.tag = 'subtopic' + found2.append(found) + tree.getroot().remove(found) + print '[XLDF] move('+cmd.findtext('section')+',',cmd.findtext('inside')+')' + return + +def xldf_rename(cmd,tree): + if cmd.findall('from/title'): + byid = False + welookfor = cmd.findtext('from/title') + else: + byid = True + welookfor = cmd.findtext('from/id') + found = False + for core in tree.findall('//core'): + if not byid and welookfor == core.findtext('title'): + core.findall('title')[0].text = cmd.findtext('to') + found = True + if byid and welookfor == core.findtext('id'): + core.findall('title')[0].text = cmd.findtext('to') + found = True + if not found: + print '[----] xldf:rename failed:', + if byid: + print 'id','"'+welookfor+'"', + else: + print 'title','"'+welookfor+'"', + print 'not found!' + else: + print '[XLDF] rename('+welookfor,',',cmd.findtext('to')+')' + return + +def xldf_add_section(cmd,tree): + success = False + for s in cmd.findall('front/*'): + tree.findall('//frontMatter')[0].append(s) + print '[XLDF] add-section to front' + success = True + for s in cmd.findall('list/*'): + tree.findall('//lists')[0].append(s) + print '[XLDF] add-section to front' + success = True + for s in cmd.findall('lexical/*'): + tree.findall('//lexicalPart')[0].append(s) + print '[XLDF] add-section to front' + success = True + for s in cmd.findall('core'): + tree.getroot().append(s) + print '[XLDF] add-section to front' + success = True + if not success: + print '[----] add-section failed' + return + +def main(xldffile,inldffile,outldffile): + grammar={} + xtree = ET.parse(xldffile) + ltree = ET.parse(inldffile) + + for cmd in xtree.findall('*'):#'//{%s}*'%xldfns): + cmdname = cmd.tag.replace('{'+xldfns+'}','') + if cmdname == 'insert': + xldf_insert(cmd,ltree) + elif cmdname == 'move': + xldf_move(cmd,ltree) + elif cmdname == 'rename': + xldf_rename(cmd,ltree) + elif cmdname == 'append': + xldf_append(cmd,ltree) + elif cmdname == 'add-section': + xldf_add_section(cmd,ltree) + else: + print '[----] Unknown XLDF command:',cmdname + + ltree.write(outldffile) + return + + cx=0 + for prod in gtree.findall('//{%s}production' % bgfns): + cx+=1 + grammar[prod.findtext('nonterminal')]=prod + print 'Found', cx, 'productions' + + dtree.set('xmlns:ldf',ldfns) + dtree.set('xmlns:bgf',bgfns) + #dtree.set('xmlns:ldx',ldxns) + dtree.set('xmlns:html',htmlns) + + section = ET.SubElement(dtree,'titlePage') + el = ET.SubElement(section,'author') + el.text = 'XSD2LDF generator' + el = ET.SubElement(section,'topic') + el.text = stree.findall('/{%s}annotation/{%s}documentation' % (xsdns,xsdns))[0].text + el = ET.SubElement(section,'version') + el.text = '1.0' + el = ET.SubElement(section,'status') + el.text = 'unknown' + el = ET.SubElement(section,'date') + # generate!!! + el.text = '2008-02-21' + + section = ET.SubElement(dtree,'frontMatter') + el = ET.SubElement(section,'foreword') + el = ET.SubElement(el,'content') + for p in stree.findall('/{%s}annotation/{%s}documentation' % (xsdns,xsdns))[1:]: + pel = ET.SubElement(el,'p') + pel.text = p.text + + if stree.findall('/{'+xsdns+'}import'): + el = ET.SubElement(section,'normativeReferences') + el = ET.SubElement(el,'content') + el = ET.SubElement(el,'list') + for p in stree.findall('/{'+xsdns+'}import'): + pel = ET.SubElement(el,'item') + pel.text = p.attrib['schemaLocation'] + + #el = copymixedcontent(dtree,'title',stree,'/{%s}annotation/{%s}documentation' % (xsdns,xsdns)) + #el = ET.SubElement(dtree,'author') + #el.text = 'XSD2LDF generator' + #el = ET.SubElement(dtree,'abstract') + #el.text = '...abstract...' + #content = ET.SubElement(dtree,'content') + + for nt in stree.findall('/*'): + if nt.tag not in acceptedtags: + continue + section = ET.SubElement(dtree,'core') + el = ET.SubElement(section,'id') + el.text = nt.tag.replace('{'+xsdns+'}','')+'-'+nt.attrib['name'] + el = ET.SubElement(section,'title') + el.text = nt.attrib['name'] + el = ET.SubElement(section,'description') + el = ET.SubElement(el,'content') + for p in nt.findall('./{%s}annotation/{%s}documentation' % (xsdns,xsdns)): + pel = ET.SubElement(el,'p') + pel.text = p.text + section.append(grammar[nt.attrib['name']]) + # print grammar[nt.attrib['name']] + + ET.ElementTree(dtree).write(ldffile) + return + + for nt in stree.findall('/{%s}group' % (xsdns)): + s = ET.SubElement(content,'section') + el = ET.SubElement(s,'title') + el.text = nt.attrib['name'] + con2 = ET.SubElement(s,'content') + el = copymixedcontent(con2,'text',nt,'./{%s}annotation/{%s}documentation' % (xsdns,xsdns)) + el = ET.SubElement(con2,'grammar') + el.set('language',xbgfns) + el.append(grammar[nt.attrib['name']]) + #print 'found group!' + + for nt in stree.findall('/{%s}element' % (xsdns)): + s = ET.SubElement(content,'section') + el = ET.SubElement(s,'title') + el.text = nt.attrib['name'] + con2 = ET.SubElement(s,'content') + el = copymixedcontent(con2,'text',nt,'./{%s}annotation/{%s}documentation' % (xsdns,xsdns)) + el = ET.SubElement(con2,'grammar') + el.set('language',xbgfns) + el.append(grammar[nt.attrib['name']]) + + ET.ElementTree(dtree).write(ldffile) + +def bgfns_(element): + return '{'+bgfns+'}'+element + +def copymixedcontent (parent, name, stree, xpath): + element = ET.SubElement(parent, name) + if not stree.findall(xpath): + return None + mixed = stree.findall(xpath)[0] + if mixed.text: + element.text=mixed.text + for tag in mixed: + element.append(tag) + return element + +if __name__ == "__main__": + if len(sys.argv) == 4: + apply(main,sys.argv[1:4]) + else: + print '''LDF transformation engine + +Usage:''' + print ' ',sys.argv[0],'','','' + sys.exit(1)