Browse files

processing models added (very rough)

  • Loading branch information...
1 parent 5b0f06d commit a2b268ccdf18013b4614903498aae51fb25c960d sefftinge committed Jun 1, 2010
Showing with 37,896 additions and 7,198 deletions.
  1. +114 −0 plugins/org.eclipse.xtext.doc/doc/016-working-with-Xtext-models.textile
  2. +11 −6 plugins/org.eclipse.xtext.doc/doc/019-emf.textile
  3. +0 −264 plugins/org.eclipse.xtext.doc/doc/210-migrating-from-oaw.textile
  4. +0 −28 plugins/org.eclipse.xtext.doc/doc/240-Breaking-Changes_07_08.textile
  5. BIN plugins/org.eclipse.xtext.doc/doc/images/getting-started-grammar.png
  6. +9,809 −0 plugins/org.eclipse.xtext.doc/doc/images/xtext_data_structures.graffle
  7. BIN plugins/org.eclipse.xtext.doc/doc/images/xtext_data_structures.png
  8. +2 −1 plugins/org.eclipse.xtext.doc/doc/xtext-index.txt
  9. +47 −0 plugins/org.eclipse.xtext.doc/help/CustomizingPoints.html
  10. +1 −1 plugins/org.eclipse.xtext.doc/help/DSL.html
  11. +22 −0 plugins/org.eclipse.xtext.doc/help/Howdoesitwork.html
  12. +2 −2 plugins/org.eclipse.xtext.doc/help/MWE2SyntaxReference.html
  13. +2 −2 plugins/org.eclipse.xtext.doc/help/MigratingStepByStep.html
  14. +0 −17 plugins/org.eclipse.xtext.doc/help/MigrationSupport.html
  15. +0 −91 plugins/org.eclipse.xtext.doc/help/NewFeatures.html
  16. +4 −82 plugins/org.eclipse.xtext.doc/help/Overview.html
  17. +1 −1 plugins/org.eclipse.xtext.doc/help/Wherenext.html
  18. +19 −0 plugins/org.eclipse.xtext.doc/help/WhoisbehindXtext.html
  19. +18 −0 plugins/org.eclipse.xtext.doc/help/WhousesXtext.html
  20. +0 −104 plugins/org.eclipse.xtext.doc/help/Xtend_based_apis.html
  21. +18 −0 plugins/org.eclipse.xtext.doc/help/Xtextishighlyconfigurable.html
  22. +26 −14 plugins/org.eclipse.xtext.doc/help/antlr_ip_issue.html
  23. +28 −13 plugins/org.eclipse.xtext.doc/help/concepts.html
  24. +5 −5 plugins/org.eclipse.xtext.doc/help/configuration.html
  25. +2 −2 plugins/org.eclipse.xtext.doc/help/contentAssist.html
  26. +1 −1 plugins/org.eclipse.xtext.doc/help/dependencyInjection.html
  27. +0 −271 plugins/org.eclipse.xtext.doc/help/differences.html
  28. +3 −3 plugins/org.eclipse.xtext.doc/help/emf_integration.html
  29. +1 −1 plugins/org.eclipse.xtext.doc/help/emf_resource.html
  30. +3 −3 plugins/org.eclipse.xtext.doc/help/encoding.html
  31. +129 −53 plugins/org.eclipse.xtext.doc/help/formatting.html
  32. +0 −35 plugins/org.eclipse.xtext.doc/help/from_oaw_to_tmf.html
  33. +50 −35 plugins/org.eclipse.xtext.doc/help/getting-started-xpand.html
  34. +636 −128 plugins/org.eclipse.xtext.doc/help/getting-started.html
  35. BIN plugins/org.eclipse.xtext.doc/help/images/getting-started-editor1.png
  36. BIN plugins/org.eclipse.xtext.doc/help/images/getting-started-grammar.png
  37. BIN plugins/org.eclipse.xtext.doc/help/images/getting-started-wizard.png
  38. +9,809 −0 plugins/org.eclipse.xtext.doc/help/images/xtext_data_structures.graffle
  39. BIN plugins/org.eclipse.xtext.doc/help/images/xtext_data_structures.png
  40. +109 −0 plugins/org.eclipse.xtext.doc/help/jvmtypes.html
  41. +5 −5 plugins/org.eclipse.xtext.doc/help/linking.html
  42. +9 −9 plugins/org.eclipse.xtext.doc/help/metamodelInference.html
  43. +1 −1 plugins/org.eclipse.xtext.doc/help/migrating_from_0_7.html
  44. +0 −27 plugins/org.eclipse.xtext.doc/help/migration_overview.html
  45. +258 −0 plugins/org.eclipse.xtext.doc/help/processing_Xtext_models.html
  46. +1 −1 plugins/org.eclipse.xtext.doc/help/projectwizard.html
  47. +7 −7 plugins/org.eclipse.xtext.doc/help/scoping.html
  48. +167 −52 plugins/org.eclipse.xtext.doc/help/serialization.html
  49. +36 −36 plugins/org.eclipse.xtext.doc/help/syntax.html
  50. +2 −2 plugins/org.eclipse.xtext.doc/help/templates.html
  51. +135 −18 plugins/org.eclipse.xtext.doc/help/validation.html
  52. BIN plugins/org.eclipse.xtext.doc/html/images/getting-started-editor1.png
  53. BIN plugins/org.eclipse.xtext.doc/html/images/getting-started-grammar.png
  54. BIN plugins/org.eclipse.xtext.doc/html/images/getting-started-wizard.png
  55. +9,809 −0 plugins/org.eclipse.xtext.doc/html/images/xtext_data_structures.graffle
  56. BIN plugins/org.eclipse.xtext.doc/html/images/xtext_data_structures.png
  57. +6,565 −5,841 plugins/org.eclipse.xtext.doc/html/xtext.html
  58. BIN plugins/org.eclipse.xtext.doc/manual/xtext.pdf
  59. +29 −36 plugins/org.eclipse.xtext.doc/toc.xml
View
114 plugins/org.eclipse.xtext.doc/doc/016-working-with-Xtext-models.textile
@@ -0,0 +1,114 @@
+
+h2(#processing_Xtext_models). Processing Xtext models
+
+p. Now that we know how to create a language we should talk about processing the parsed models somehow. Text files parsed by Xtext are represented as
+object graphs in memory. We call these object graphs _Abstract Syntax Tree (AST)_, _semantic model_ or simply _model_ interchangeably. In Xtext
+models are implemented using the _Eclipse Modeling Framework (EMF)_, which can be seen as a very powerful version of JavaBeans. It not only provides
+the typical getter and setter methods for the different features of a model element but also comes with an long list of advanced concepts and semantics.
+
+h3(#resource). Loading Xtext models
+
+For instance, EMF models can be persisted by the means of a so called _Resource_. Xtext languages implement _Resource_ which is why you can use the EMF API to
+load a model into memory:
+
+bc.. new DomainmodelStandaloneSetup().createInjectorAndDoEMFRegistration();
+
+ResourceSet rs = new ResourceSetImpl();
+Resource resource = rs.getResource(URI.createURI("./mymodel.dmodel"), true);
+EObject eobject = resource.getContents().get(0);
+
+p. The first line initializes the language infrastructure to run in _standalone_ mode. That is EMF is designed to work in Eclipse and therefore makes use of
+Equinox extension points, in order to register factories and the like. In a vanilla Java project there is no Equinox, hence we need to do the registrartion programmatically.
+The generated _MyLanguageStandaloneSetup_ class does that. You don't need to do this kind of initialization when you run your plugins within Eclipse, since in that case
+the extension point declarations are used.
+
+The other thing the StandaloneSetup takes care of is creating a Guice _injector_. The use of Guice and Dependency Injection is explained here [TODO].
+
+Now that the language infrastructure is intialized and the different contributions to EMF are registered, we want to load a _Resource_. To do so we first create a
+_ResourceSet_, which as the name suggests represents a set of Resources. If one _Resource_ references another _Resource_, EMF will automatically load that other _Resource_
+into the same _ResourceSet_ as soon as the cross reference is resolved. Resolution of cross references is done lazy, i.e. on first access.
+
+The 4th line loads the _Resource_ using the resource set. We pass in a URI which points to the file in the file system.
+EMF's URI is a very important concept since, it supports a lot of different schemes like loading resources from file system, web sites, jars, OSGi bundles or even
+the class path. And if that is not enough you can come up with your own schemes. Also a URI not only can point to a resource but also to any _EObject_ in a resource.
+This is done by appending a so called _URI fragment_ to the URI.
+
+The second parameter denotes whether the resource should automatically be loaded if it wasn't before. Alternatively we could have written
+
+bc.. Resource resource = rs.getResource(URI.createURI("./mymodel.dmodel"), false);
+resource.load(null);
+
+p. The load method optionally takes a map of properties, which allows to define a contract between a client and the specific implementation. In Xtext, for instance, we use
+the map to state whether cross references should be eagerly resolved.
+
+The last line (_EObject eobject = resource.getContents().get(0);_) assigns the root element to a local variable.
+Actually it is the first element from the contents list of a _Resource_, but in Xtext a _Resource_ always has just one root element.
+
+h3(#EMF_reflection). Working with EMF models
+
+p. We have previously talked about Ecore models, which effectively is an _EPackage_ containing a any number of _EClasses_ with _EAttributes_ and _EReferences_.
+Ecore defines additional concepts but they are not so important here. An _EObject_ is an instance of an _EClass_. For instance, the root element
+of a domain model would be an instance of the EClass _DomainModel_, as defined in the grammar in the last chapter. EMF provides a reflection layer to work
+with EObjects in a generic way.
+
+bc.. assertEquals("DomainModel", eobject.eClass().getName());
+
+p. It is also possible to create new instances or get and set values using the reflection layer. That reflection layer very helpful when creating generic libraries for EMF models
+but if you know what kind of model you are working with, it's much nicer to program against the generated Java classes. As we know that the root element
+is an instance of DomainModel we can just cast it to the Java interface generated by EMF:
+
+bc.. DomainModel dm = (DomainModel) eobject;
+
+p. The generated Java types provide getter and setter for the _EAttributes_ and _EReferences_, so that you can easily navigate the model using Java:
+
+bc.. EList<AbstractElement> elements = domainModel.getElements();
+for (AbstractElement abstractElement : elements) {
+ if (abstractElement instanceof Entity) {
+ Entity entity = ((Entity)abstractElement);
+ System.out.println("entity "+entity.getName()+" extends "+entity.getSuperType().getName());
+ }
+}
+
+p. You'll find a lot of convenience API in EMF's EcoreUtil and Xtext's EcoreUtil2.
+
+h3(#node_model). Working with the parse tree (aka node model)
+
+p. In many situations the information from the _AST_ is sufficient, but in some situations you need the additional syntactical information. In Xtext not only
+an _AST_ is constructed while parsing but also a so called _parse tree_, which contains all the textual information chunked in so called tokens. The _parse tree_ in Xtext is
+also called _node model_ and consists of two different kinds of nodes. _LeafNodes_ as the name suggests represent the leafs of the parse tree. Each leaf node represents one
+token, the lexer has scanned. If you go through all leaf nodes of a parse tree and concatenate all the tokens to one string, you'll end up with the whole textual representation
+of the model, including all hidden tokens such as whitespace and comments. The following code does exactly that:
+
+bc.. CompositeNode node = NodeUtil.getNode(domainModel);
+Iterable<AbstractNode> contents = NodeUtil.getAllContents(node);
+StringBuffer text = new StringBuffer();
+for (AbstractNode abstractNode : contents) {
+ if (abstractNode instanceof LeafNode)
+ text.append(((LeafNode)abstractNode).getText());
+}
+System.out.println(text);
+
+p. In addition to the text _LeafNodes_ also holds information about the line, the offset and the length of that token.
+
+The other node type is called _CompositeNode_ and is created by the parser for a grammar element. A composite node can contain other composite nodes and leaf nodes.
+The super type of both node types is _AbstractNode_.
+
+h3(#working_with_grammar_model). Working with the grammar
+
+p. Also the grammar previously defined in Xtext's grammar language is represented as an EMF model and can be used in Java. In fact each node of the _node model_ references
+the element from the grammar which was responsible for parsing or lexing that node:
+
+bc.. DomainModel domainModel = (DomainModel) eObject;
+CompositeNode node = NodeUtil.getNode(domainModel);
+ParserRule parserRule = (ParserRule) node.getGrammarElement();
+assertEquals("DomainModel", parserRule.getName());
+
+h3. Summary
+
+p. Let's summarize what we have learned. An Xtext model is loaded by an EMF Resource. The main model is represented as an instance of so called _EClasses_, which are themselves
+declared within Ecore models. A _parse tree_ is created as well, which effectively acts as a tracing model between the text, the _AST_ and the grammar. The following
+diagram illustrates the four different kinds of models.
+
+!images/xtext_data_structures.png!
+
+
View
17 plugins/org.eclipse.xtext.doc/doc/019-emf.textile
@@ -1,18 +1,23 @@
h1(#concepts). Xtext and EMF
-In this section we try to introduce the basic features of the "Eclipse Modeling Framework (EMF)":#http://www.eclipse.org/emf in the context of Xtext. As EMF provides the base infrastructure for Xtext, it is useful to have a basic understanding of EMF, so we give a rough outline here.
+In this section we introduce the basic concepts of the "Eclipse Modeling Framework (EMF)":#http://www.eclipse.org/emf in the context of Xtext.
+As Xtext integrates with and uses EMF, it is important to have a basic understanding of EMF.
If you want to learn more about EMF, we recommend reading the "EMF book":#http://www.eclipse.org/emf.
-h2(#model_metamodel). Model, AST, Meta Model, and Ecore
+h2(#model_metamodel). Model, Ecore Model, and Ecore
-When you run the workflow in your Xtext project, Xtext will generate a bunch of artifacts that make up the infrastructure for your language. Among other useful things, it will create a parser. This parser takes models, i.e. files in your language, as an input and produce a graph of objects as output. This graph is called the _Abstract Syntax Tree_ (AST). Given the example model from the "introduction":#getting_started_xtext, the AST should look like this
+When Xtext uses EMF models as the in-memory representation of any parsed text files. This in-memory object graph is called the _Abstract Syntax Tree_ (AST).
+Depending on the community this concepts is also called _document object graph (DOM)_, _semantic model_, or simply _model_. We use _model_ and _AST_ interchangeably.
+Given the example model from the "introduction":#getting_started_xtext, the AST looks similar to this
!images/ast.png(Sample AST)!
-In terms of modeling, we also refer to the AST tree as the _semantic model_, as it does no longer carry any additional syntactical information. Usually it is only the semantic model that is processed by later processing steps, i.e. in the execution by a code generator or interpreter.
-
-The structure of the semantic model is defined by the so called _meta model_. You can think of the meta model as the schema for the models. In Xtext, meta models are either inferred from the grammar or predefined by the user (see the section on "package declarations":#package_declarations for details). The next diagram shows the meta model of our example:
+The _AST_ should contain the essence of your textual models and abstract over syntactical information. It is used by later processing steps,
+such as validation, compilation or interpretation. In EMF a model is made up of instances of _EObjects_ which are connected
+and an _EObject_ is an instance of an _EClass_.
+A set of _EClasses_ if contained in a so called _EPackage_, which are both concepts of _Ecore_
+In Xtext, meta models are either inferred from the grammar or predefined by the user (see the section on "package declarations":#package_declarations for details). The next diagram shows the meta model of our example:
!images/metamodel.png(Sample meta model)!
View
264 plugins/org.eclipse.xtext.doc/doc/210-migrating-from-oaw.textile
@@ -1,264 +0,0 @@
-
-h1(#from_oaw_to_tmf). From oAW to TMF
-
-TMF Xtext is a complete rewrite of the Xtext framework previously released with openArchitectureWare 4.3.1 (oAW). We refer to the version from oAW as oAW Xtext whereas the current Xtext version that is hosted at Eclipse.org will be called TMF Xtext to avoid confusion.
-oAW Xtext has been around for about 2 years before TMF Xtext was released in June 2009 and has been used by many people to develop little languages and corresponding Eclipse-based IDE support.
-
-TMF Xtext has been improved in many aspects compared to the former version. While it integrates far better into EMF, it offers new fundamental features as well. The overhauled architecture leads to better performance when working with large models and since the whole framework is wired via dependency injection it is highly customizable.
-Last not least a test coverage of more than 2.000 unit tests provide confidence in the overall quality of TMF Xtext. We have been using the framework in production environments since one of the earlier milestones.
-
-In this document we want to share the experience we made when migrating existing Xtext projects. The document describes the differences between oAW Xtext and TMF Xtext and is intended to be used as a guide to migrate from oAW Xtext to TMF Xtext.
-For people already familiar with the concepts of oAW Xtext it should also serve as a shortcut to learn TMF Xtext.
-
-h2(#why_rewrite). Why a rewrite?
-
-The first thing you might wonder about is why we decided to reimplement the framework from scratch as opposed to use the existing code base and enhance it further on.
-We decided so because we had learned a lot of lessons from oAW Xtext. Although we wanted to stick with many proven concepts
-we found the implementation was lacking a solid foundation (the author of these lines is the original author of that non-solid code btw. :-)). The first version of oAW Xtext was basically a proof of concept which was so well received that it had been extended with all kinds of features (some were good, some were bad). Unfortunately code quality, clean and orthogonal concepts and test coverage did not receive the necessary focus.
-
-In addition to this aspects of quality, oAW Xtext suffers from some severe performance problems. The extensive and naive use of Xtend (see next section) prevented many users to use oAW Xtext for growing real-world models.
-
-h2(#migration_overview). Migration overview
-
-Although a couple of things have changed we tried to keep good ideas and left many things unchanged. At the same time we wanted to clean up poor concepts and solve the main problems we and you had with oAW Xtext.
-From a bird's eye view if you want to migrate an existing oAW Xtext project to TMF Xtext, you mainly just need to rename the old grammar from *.xtxt to *.xtext and add two lines to the beginning of that document (see below for details). You might also have to change a few keywords, but all in all this is pretty easy and we've migrated a couple of oAW Xtext projects this way without problems.
-The other aspect where lots of code might have been written for is validation. In oAW Xtext we used Xpand's Check language to define constraints on the Ecore model. Even though this has been one major reason for the lack of scalability in Xtext we decided to keep the Check language as an option for compatibility reasons. Therefore, you do not need to translate your existing checks to a different language. Even better, you can overcome some performance issues by leveraging the newly introduced hooks to control the time of validation (while you type, on save, or on triggering an explicit action).
-Anyway, if you want to provide a slick user experience validation should run fast while you type. Therefore, we strongly encourage you to implement validation using our "declarative Java approach":#custom_validation.
-
-We've developed and reviewed a lot of oAW Xtext projects and saw that most of the work was done in the grammar and in the validation view point.
-Other aspects such as outline view, label provider or content assist have been customized too, but they usually do not contain complicated Xtend code.
-In some projects the exception was linking and content assist which in oAW Xtext usually forces one to write a lot of duplicated code. While working on this we came up with a new concept called "scopes" that not only streamlines implementation in terms of redundancy. Scopes also increase the overall performance of Xtext.
-But since the concept of scopes was not carved out in oAW Xtext one usually implemented a cluttered and duplicated poor copy through linking and content assist. For obvious reasons, we didn't manage to come up with a good compatibility layer.
-So this is where most of the migration effort will go into if implemented customized linking. But we think the notion of scopes is such a valuable addition that it is worth the refactoring. Also, when looking at existing oAW Xtext projects we found that most projects either didn't change the default linking that much or they came up with their own linking framework anyway.
-
-However, if we have completely misunderstood the situation and your oAW Xtext project cannot be migrated in a reasonable amount of time, please tell us. We want to help you!
-
-h2(#Xtend_based_apis). Where are the Xtend-based APIs?
-
-One of the nice things with oAW Xtext was the use of Xtend to allow customizing different aspects of the generated language infrastructure.
-Xtend is a part of the template language Xpand, which is shipped with oAW (and now is included in M2T Xpand). It provides a nicer expression syntax than Java.
-Especially the existence of higher-order functions for collections is extremely handy when working with models.
-In addition to the nice syntax, it provides dynamic polymorphic dispatch, which means that declaring e.g. label computation for an Ecore model is very convenient and type safe at the same time.
-In Java one usually has to write instanceof and cast orgies.
-
-h3(#Xtend_is_hard_to_debug). Xtend is hard to debug
-
-While the aforementioned features allow the convenient specification of label and icon providers, outline views, content assist and linking, Xtend is interpreted and therefore hard to debug. Because of that Xpand is shipped with a special debugger facility. Unfortunately, this debugger cannot be used in the context of Xtext since it implies that the Xtend functions have to be called from a workflow. This is not and cannot be the case for Xtext Editors.
-As a result one has to debug his way through the interpreter, which is hard and inconvenient (even for us, who have written that interpreter).
-
-h3(#Xtend_is_slow). Xtend is slow
-
-But the problematic debugging in the context of Xtext was not the main reason why there are no Xtend-based APIs beside Check in TMF Xtext. The main reason is that Xtend is too slow to be evaluated "inside" the editor again and again while you type.
-While Xtend's performance is sufficient when run in a code generator, it is just too slow to be executed on keystroke (or 500ms after the last keystroke, which is when the reconciler reparses, links and validates the model). Xtend is relatively slow, because it supports polymorphic dispatch (the cool feature mentioned above), which means that for each invocation it matches at runtime which function is the best match and it has to do so on each call. Also Xtend supports a pluggable typesystem, where you can adapt to other existing type systems such as JavaBeans or Ecore. This is very powerful and flexible but introduces another indirection layer. Last but not least the code is interpreted and not compiled. The price we pay for all these nice features is reduced performance.
-
-In addition to these scalability problems we have designed some core APIs (e.g. scopes) around the idea of Iterables, which allows for lazy resolution of elements. As Xtend does not know the concept of Iterators you would have to work with lists all the time. Copying collections over and over again is far more expensive than chaining Iterables through filters and transformers like we do with Google Collections in TMF Xtext.
-
-h3(#Convenient_Java). Convenient Java
-
-To summarize the dilemma we had to find a way to allow for convenient, scalable and debuggable APIs. Ultimately we wanted to provide neat DSLs for every view point, which provide all these things. However, we had to prioritize our efforts with the available resources in mind.
-As a result we found ways and means to tweak Java as good as possible to allow for relatively convenient, high performing implementations.
-
-Java is fast and can easily be debugged but ranks behind Xtend regarding convenience. We address this with different approaches to make Java development in the context of Xtext as comfortable as possible.
-
-Most of the APIs in TMF Xtext use polymorphic dispatching, which mimics the behavior known from Xtend. Another valuable feature of Xtend while working with oAW Xtext is static type checking while working with the inferred Ecore model whereas in Java the work with dynamic Ecore classes was rather cumbersome. Since TMF Xtext generates static Ecore classes per default you get static typing in Java as well. Additionally, the use of "Google Collections":http://code.google.com/p/google-collections reduces the pain when navigating over your model to extract information.
-
-With these techniques an ILabelProvider that handles your own EClasses @Property@ and @Entity@ can be written like this:
-
-bc..
-public class DomainModelLabelProvider extends DefaultLabelProvider {
-
- String label(Entity e) {
- return e.getName();
- }
-
- String image(Property p) {
- return p.isMultiValue() ? "complexProperty.gif": "simpleProperty.gif";
- }
-
- String image(Entity e) {
- return "entity.gif";
- }
-
-}
-
-p. As you can see this is very similar to the way one describes labels and icons in oAW Xtext, but has the advantage that it is easier to test and to debug, faster and can be used everywhere an ILabelProvider is expected in Eclipse.
-
-h3(#conclusion). Conclusion
-
-Just to get it right, Xtend is a very powerful language and we still use it for its main purpose: code generation and model transformation. The whole generator in TMF Xtext is written in Xpand and Xtend and its performance is at least in our experience sufficient for that use case. Actually we were able to increase the runtime performance of Xpand by about 60% for the Galileo release of M2T Xpand. But still, live execution in the IDE and on typing is very critical and one has to think about every millisecond in this area.
-
-As an alternative to the Java APIs we also considered other JVM languages. We like static typing and think it is especially important when processing typed models (which evolve heavily). That's why Groovy or JRuby were no alternatives. Using Scala would have been a very good match, but we didn't want to require knowledge of Scala so we didn't use it and stuck to Java.
-
-h2(#differences). Differences
-
-In this section differences between oAW Xtext and TMF Xtext are outlined and explained. We'll start from the APIs such as the grammar language and the validation and finish with the different hooks for customizing linking and several UI aspects, such as outline view and content assist. We'll also try to map some of the oAW Xtext concepts to their counterparts in TMF Xtext.
-
-h3(#differenced_grammar_language). Differences in the grammar language
-
-When looking at a TMF Xtext grammar the first time it looks like one has to provide additional information which was not necessary in oAW Xtext. In oAW Xtext *.xtxt files started with the first production rule where in TMF Xtext one has to declare the name of the language followed by declaration of one or more used/generated Ecore models:
-
-TMF Xtext heading information
-
-bc..
- grammar my.namespace.Language with org.eclipse.xtext.common.Terminals
- generate myDsl "http://www.namespace.my/2009/MyDSL"
-
- FirstRule : ...
-
-p. In oAW Xtext this information was provided through the generator (actually it is contained in the *.properties file) but we found that these things are very important for a complete description of a grammar. Therefore we made that information becoming a part of the grammar language in order to have self-describing grammars and allow for sophisticated static analysis, etc..
-
-Apart from the first two lines the grammar languages of both versions are more or less compatible. The syntax for all the different EBNF concepts (alternatives, groups, cardinalities) is similar.
-Also assignments are syntactically and semantically identical in both versions.
-However in TMF Xtext some concepts have been generalized and improved:
-
-h4(#differences_datatype_rules). String rules become Datatype rules
-
-The very handy String rules are still present in TMF Xtext but we generalized them so that you don't need to write the 'String' keyword in front of them and at the same time these rules can not only produce EStrings but (as the name suggests) any kind of EDataType. Every parser rule that neither includes assignments nor calls any other that does, returns an EDataType containing the consumed data. Per default this is an EString but you can now simply create a parser rule returning other EDataTypes as well (see "Datatype rules":#datatypeRules).
-
-bc.
- Float returns ecore::EDouble : INT ('.' INT)?;
-
-h4(#differences_enum_rules). Enum rules
-
-Enum rules have not changed significantly. The keyword has changed to be all lower case ('enum' instead of 'Enum').
-Also the right-hand side of the assignment is now optional. That is in oAW Xtext:
-
-bc..
- Enum MyEnum : foo='foo' | bar='bar';
-
-p. becomes
-
-bc..
- enum MyEnum : foo='foo' | bar='bar';
-
-p. and because the name of the literal equals the literal value one can omit the right-hand side in this case and write:
-
-bc..
- enum MyEnum : foo | bar;
-
-h4(#differences_native_rules). Native rules
-
-Another improvement is that we could replace the blackbox native rules with full-blown EBNF syntax.
-That is native rules become terminal rules and are no longer written as a string literal containing ANTLR syntax.
-
-Example :
-
-bc.
- Native FOO : "'f' 'o' 'o'";
-
-p. becomes
-
-bc.
- terminal FOO : 'f' 'o' 'o';
-
-p. See the "reference documentation":#TerminalRules for all the different expressions possible in terminal rules.
-
-h4(#differences_no_URI_token). No URI terminal rule anymore
-
-We decided to remove the URI terminal. The only reason for the existence was to mark the model somehow so that the framework knows what information to use in order to load referenced models. Instead we decided to solve this similar to how we imply other defaults: by convention.
-
-So instead of using a special token which is syntactically a STRING token, the default import mechanism now looks for EAttributes of type EString with the name 'importURI'.
-That is if you've used the URI token like this:
-
-bc.
- Import : 'import' myReference=URI;
-
-you'll have to rewrite it that way
-
-bc.
- Import : 'import' importURI=STRING;
-
-Although this changes your Ecore model, one usually never used this reference explicitly as it was only there to be used by the default import mechanism. So we assume and hope that changing the reference is not a big deal for you.
-
-h4(#differences_return_types). Return types
-
-The syntax to explicitly declare the return type of a rule has changed. In oAW Xtext (where this was marked as 'experimental') the syntax was:
-
-bc.
- MyRule [MyType] : foo=ID;
-
-in TMF Xtext we have a keyword for this :
-
-bc.
- MyRule returns MyType : foo=ID;
-
-This is a bit more verbose, but at the same time more readable. And as you don't have to write the return type in most situations, it's good to have a more explicit, readable syntax.
-
-h3(#differences_linking). Differences in Linking
-
-The linking has been completely redesigned. In oAW Xtext linking was done in a very naive way: To find an element one queries a list of all 'visible' EObjects, then filters out what is not needed and tries to find a match by comparing the text written for the crosslink with the value returned by the @id()@ extension. As a side-effect of @link_feature()@ the reference is set.
-
-The code about selecting and filtering @allElements()@ usually has been duplicated in the corresponding content assist function, so that linking and content assist are semantically in sync. If you're good (we usually were not) you externalized that piece of code and reused the same extension in content assist and linking.
-
-To put it bluntly this approach could be summarized in two steps:
-# Give me the whole universe including every unregarded object in the uncharted backwaters of the unfashionable end of the western spiral arm of the galaxy and squeeze it into an Arraylist
-# From this, select the one I need
-
-This was not only very expensive but also lacks an important abstraction: the notion of scopes.
-
-h4(#differences_linking_sopes). The idea of scopes
-
-In TMF Xtext we've introduced scopes and scope providers that are responsible for creating scopes. A scope is basically a set of name->value pairs. Scopes are implemented upon Iterables and are nested to build a hierarchy. With scopes we declare "visible" objects in a lazy and cost-saving way where the linker only navigates as far as necessary to find matching objects. The content assist reuses this set of visible objects to offer only reachable objects.
-
-When the linking has to be customized scoping is where most of the semantics typically goes into. By implementing an ${org.eclipse.xtext/src/org.eclipse.xtext.scoping.IScopeProvider} for your language linking and content assist will automatically be kept in sync since both use the scope provider.
-
-The provided default implementation is semantically mostly identical to how the default linking worked in oAW Xtext:
-# Elements which have an attribute 'name' will be made visible by their name
-# Referenced resources will be put on the (outer) scope by using the 'importURI'- naming convention and will only be loaded if necessary
-# The available elements are filtered by the expected type (i.e. the type of the reference to be linked)
-
-h4(#differences_linking_migration). Migration
-
-We expect the migration of linking to be very simple if you've not changed the default semantics that much. We've already migrated a couple of projects and it wasn't too hard to do so.
-If you have changed linking (and also content assist) a lot, you'll have to translate the semantics to the IScopeProvider concept. This might be a bit of work, but it is worth the effort as this will clean up your code base and better separate concerns.
-
-h3(#differences_ui). Differences in UI customizing
-
-In oAW Xtext several UI services such as content assist, outline view or the label provider have been customized using Xtend. In TMF Xtext there is no Xtend API for these aspects. Extensive model computations for the content assist is most probably not necessary anymore- it reuses scopes. And since we provide a declarative Java API that mimics the polymorphic dispatch and relies on static Ecore classes you will gain nearly the same expressiveness as before while increasing maintainability and performance.
-
-Beside the API change in favor of Java we have to mention that in TMF Xtext the outline view does not support multiple view points so far. This is just because we didn't manage to get this included. We don't think that view points are a bad idea in general, but we decided that other things were more important.
-
-h2. New Features
-
-This section provides an overview of new possibilities with TMF Xtext compared to oAW Xtext. Please note that this list is neither complete nor does it explain every aspect in detail to keep this document tight.
-
-h3. Dependency Injection with Google Guice
-
-Beyond the mentioned architectural overhaul that carve out separate concerns in a meaningful way these different classes of TMF Xtext are wired using "Google Guice":http://code.google.com/p/google-guice and can easily be replaced by or combined with your own implementation. We could have foreseen some common needs for adaption but with this mechanism you can virtually change every aspect of Xtext without duplicating an unmanageable amount of code.
-
-h3. Improvements on Grammar Level
-
-The Xtext grammar language introduces some new features, too. Read the chapter "grammar language":#grammarLanguage to understand the details about all the improvements that have been implemented.
-
-"Grammar mixins":#grammarMixins allow you to extend existing languages and change their concrete and abstract syntax. However the abstract syntax (i.e. the Ecore model) can only be extended. This allows you to reuse existing validations, code generators, interpreters or other code which has been written against those types.
-
-In oAW Xtext common terminals like ID, INT, STRING, ML_COMMENT, SL_COMMENT and WS (whitespace) were hard coded into the grammar language and couldn't be removed and hardly overridden. In TMF Xtext these terminals are imported through the newly introduced grammar mixin mechanism from the shipped grammar @org.eclipse.xtext.common.Terminals@ per default. This means that they are still there but reside in a library now. You don't have to use them and you can come up with your own set of reusable rules.
-
-"Reusing existing Ecore models":#grammarImport in oAW Xtext didn't work well and we communicated this by flagging this feature as 'experimental'. In TMF Xtext importing existing Ecore models is fully supported. Moreover, it is possible to import a couple of different EPackages and generate some at the same time, so that the generated Ecore models extend or refer to the existing Ecore models.
-
-The grammar language gained one new concept that is of great value when writing left-factored grammars (e.g. expressions). With actions one can do minor AST rewritings within a rule to overcome degenerated ASTs. You will find an "in-depth explanation of actions":#grammarActions in the dedicated chapter in the reference documentation.
-
-h3(#differences_validation). Fine-grained control for validation
-
-In order to make more expensive validations possible without slowing down the editor, TMF Xtext supports three different validation hooks.
-
-# FAST constraints are triggered by the reconciler (i.e. 500 ms after the last keystroke) and on save.
-# NORMAL constraints are executed on save only.
-# EXPENSIVE constraints are executed through an action which is available through the context menu.
-
-Please note that when using Xtext models for code generation the checks of all three categories will be performed.
-
-Beside this it is now possible to add information about the feature which is validated.
-
-bc.
- context Entity#name ERROR "Name should start with a capital "+this.name+"." :
- this.name.toFirstUpper() == this.name;
-
-If you add the name of a feature prepended by a hash ('#') in Check, the editor will only mark the value of the feature (name), not the whole object (Entity). Both concepts, control over validation time as well as pointing to a specific feature, complement one another in Check and Java based validation.
-
-h2. Migration Support
-
-In this document we tried to explain why we decided to change some aspects of Xtext's architecture. We consider most changes as minor but when it comes to scopes you will face a conceptual enhancement that did not exist in oAW Xtext. We tried to explain why it is not easily possible to come up with an adapter for scoping.
-
-That said you might not have the time to do the migration and wished to have advice for migrating, especially from oAW linking to TMF linking. You're welcome to ask any questions in the newsgroup and we'll try to help you as much as possible in order to get your projects migrated. Also, if you don't want to do the migration yourself we (itemis AG) can do the work for you or help you with that.
-
View
28 plugins/org.eclipse.xtext.doc/doc/240-Breaking-Changes_07_08.textile
@@ -1,28 +0,0 @@
-
-h1(#breaking_changes). Breaking API changes between the different versions
-
-
-h2(#breaking_changes_08). Changes in Xtext 0.8.0
-
-*Removed org.eclipse.xtext.log4j*
-
-* Replace plug-in dependency to org.eclipse.xtext.log4j with a package dependency org.apache.log4j
-
-*Moved validation classes to dedicated package*
-
-* org.eclipse.xtext.ui.core.editor.IDiagnosticConverter -> org.eclipse.xtext.ui.core.editor.validation.IDiagnosticConverter
-* org.eclipse.xtext.ui.core.editor.DefaultDiagnosticConverter -> org.eclipse.xtext.ui.core.editor.validation.DefaultDiagnosticConverter
-
-* org.eclipse.xtext.ui.core.editor.IXtextResourceChecker -> org.eclipse.xtext.ui.core.editor.validation.IXtextResourceChecker
-* org.eclipse.xtext.ui.core.editor.DefaultXtextResourceChecker -> org.eclipse.xtext.ui.core.editor.validation.DefaultXtextResourceChecker
-
-*Removed IScopeProvider.getScope(EObject,EClass)*
-
-*Renamed DefaultScopeProvider to SimpleNameScopeProvider*
-
-*Renamed and moved org.eclipse.xtext.concurrent to org.eclipse.xtext.util.concurrent*
-
-*Java Packages in generated UI plug-in now have an additional subpackage "ui"*
-
-* Old versions in the src folder are not deleted and ignored. You have to transpose your changes in the src folder to the classes in the right packages. That especially holds for additional bindings in the XXXUiModule.
-
View
BIN plugins/org.eclipse.xtext.doc/doc/images/getting-started-grammar.png
Deleted file not rendered
View
9,809 plugins/org.eclipse.xtext.doc/doc/images/xtext_data_structures.graffle
9,809 additions, 0 deletions not shown because the diff is too large. Please use a local Git client to view these changes.
View
BIN plugins/org.eclipse.xtext.doc/doc/images/xtext_data_structures.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
3 plugins/org.eclipse.xtext.doc/doc/xtext-index.txt
@@ -2,6 +2,7 @@
010-concepts.textile
015-getting-started.textile
+016-working-with-Xtext-models.textile
017-getting-started-xpand.textile
019-emf.textile
@@ -35,6 +36,6 @@
199-jvmtypes.textile
200-Appendix.textile
-210-migrating-from-oaw.textile
215-migrating-from-0.7.textile
230-antlr-ip-issue.textile
+
View
47 plugins/org.eclipse.xtext.doc/help/CustomizingPoints.html
@@ -0,0 +1,47 @@
+<html>
+<head>
+<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
+<title>Customizing Points</title>
+<link href="book.css" rel="stylesheet" type="text/css">
+<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
+<link rel="home" href="index.html" title="Xtext User Guide">
+<link rel="up" href="jvmtypes.html" title="Referring Java Types">
+<link rel="prev" href="jvmtypes.html" title="Referring Java Types">
+<link rel="next" href="migrating_from_0_7.html" title="Migrating from TMF Xtext 0.7.x">
+</head>
+<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
+<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Customizing Points</h1>
+<p>There are several customization hooks in the runtime layer and on the editor side as well:</p>
+<p>The
+ <span class="emphasis"><em>AbstractTypeScopeProvider</em></span> can be used to create scopes for members with respect to
+ the override semantics of the Java language. Of course it is possible to use this implementation to
+ create scopes for types as well.
+ </p>
+<p>As the Java VM types expose a lot of information about visibility, parameter- and return types, generic,
+ available annotations or enumeration literals, it is very easy to define constraints for the referred types.</p>
+<p>The
+ <span class="emphasis"><em>ITypesProposalProvider</em></span> can be used to provide optimized proposals based on various filter criteria.
+ The most common selector can be used directly via
+ <span class="emphasis"><em>createSubTypeProposals</em></span>. The implementation is optimized
+ and uses the JDT Index directly to minimize the effort for object instantiation. The class
+ <span class="emphasis"><em>TypeMatchFilters</em></span>
+ provides a comprehensive set of reusable filters that can be easily combined to reduce the list of proposals
+ to a smaller number of valid entries.
+ </p>
+<div class="literallayout">
+<p>
+<code class="code">import&nbsp;static&nbsp;org.eclipse.xtext.common.types.xtext.ui.TypeMatchFilters.*;&nbsp;<br>
+..<br>
+..<br>
+
+<br>
+&nbsp;&nbsp;proposalProvider.createSubTypeProposals(factory,&nbsp;context,<br>
+ &nbsp;&nbsp;and(not(canInstantiate()),&nbsp;isPublic()),&nbsp;acceptor);&nbsp;<br>
+
+<br>
+
+</code>
+</p>
+</div>
+</body>
+</html>
View
2 plugins/org.eclipse.xtext.doc/help/DSL.html
@@ -6,7 +6,7 @@
<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
<link rel="home" href="index.html" title="Xtext User Guide">
<link rel="up" href="Overview.html" title="Overview">
-<link rel="prev" href="Overview.html" title="Overview">
+<link rel="prev" href="WhoisbehindXtext.html" title="Who is behind Xtext?">
<link rel="next" href="getting-started.html" title="Getting Started">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
View
22 plugins/org.eclipse.xtext.doc/help/Howdoesitwork.html
@@ -0,0 +1,22 @@
+<html>
+<head>
+<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
+<title>How does it work?</title>
+<link href="book.css" rel="stylesheet" type="text/css">
+<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
+<link rel="home" href="index.html" title="Xtext User Guide">
+<link rel="up" href="Overview.html" title="Overview">
+<link rel="prev" href="Overview.html" title="Overview">
+<link rel="next" href="Xtextishighlyconfigurable.html" title="Xtext is highly configurable">
+</head>
+<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
+<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">How does it work?</h1>
+<p>Xtext provides you with a set of domain-specific languages and modern APIs to describe the different aspects of your programming language.
+ Based on that information it gives you a full implementation of that language running on the JVM. The compiler part can be used in any java environment
+ and includes such things as parser, AST, parse tree, serializer, formatter, scoping framework, linking, compiler checks aka validation and last but not least a
+ code generator or interpreter. The runtime components integrate with and implement the Eclipse Modeling Framework (EMF), which effectively allows you to
+ use Xtext together with other EMF frameworks like for instance the Graphical Modeling Project.</p>
+<p>In addition to this nice runtime architecture, you will get a full blown Eclipse-IDE specific for your language. It already provides great default functionality for
+ all aspects and again comes with DSLs and APIs to let you configure or change the most common things very easily. And if that&rsquo;s not flexible enough there is Guice.</p>
+</body>
+</html>
View
4 plugins/org.eclipse.xtext.doc/help/MWE2SyntaxReference.html
@@ -32,7 +32,7 @@ <h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Syntax Reference</h1
identifiers. A valid id-segment in MWE2 starts with a letter or an
underscore and is followed by any number of letters, numbers or underscores.
An identifier is composed from one or more segments which are delimited
- by a &sbquo;.&rsquo; dot.</p>
+ by a &lsquo;.&rsquo; dot.</p>
<div class="literallayout">
<p>
<code class="code">Name:&nbsp;ID&nbsp;('.'&nbsp;ID)*;<br>
@@ -144,7 +144,7 @@ <h3 class="title">
<code class="code">add</code>-method of a Java class.
</p>
<p>Components can be named to make them referable in subsequent assignments.
- Following the &sbquo;:&rsquo; keyword, one can define an identifier for the
+ Following the &lsquo;:&rsquo; keyword, one can define an identifier for the
instantiated component. The identifier is locally visible in the module
and any assignment that is defined after the named component can refer
to this identifier and thereby point to exactly the instantiated object.</p>
View
4 plugins/org.eclipse.xtext.doc/help/MigratingStepByStep.html
@@ -56,10 +56,10 @@ <h3 class="title">
</div>
<p>We changed the naming pattern for artifacts in the
<code class="code">dsl.ui</code>-plug-in to match the OSGI conventions. The easiest way to update your existing projects is to apply a
-&bdquo; Rename Package&rdquo; refactoring on the packages in the
+ &ldquo;Rename Package&rdquo; refactoring on the packages in the
<code class="code">src</code>- and
<code class="code">src-gen</code> folder
- <span class="bold"><strong>before</strong></span> you re-run the workflow that regenerates your language. Make sure you ticked &bdquo;Rename sub-packages&rdquo; in the dialog. It is error-prone to enable the search in non-java files as this will perform incompatible changes in the manifest files. Furthermore, it is important to perform the rename operation in the
+ <span class="bold"><strong>before</strong></span> you re-run the workflow that regenerates your language. Make sure you ticked &ldquo;Rename sub-packages&rdquo; in the dialog. It is error-prone to enable the search in non-java files as this will perform incompatible changes in the manifest files. Furthermore, it is important to perform the rename operation in the
<code class="code">src-gen</code> folder, too. This ensures that the references in your manually written code are properly updated.
</p>
</div>
View
17 plugins/org.eclipse.xtext.doc/help/MigrationSupport.html
@@ -1,17 +0,0 @@
-<html>
-<head>
-<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
-<title>Migration Support</title>
-<link href="book.css" rel="stylesheet" type="text/css">
-<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
-<link rel="home" href="index.html" title="Xtext User Guide">
-<link rel="up" href="from_oaw_to_tmf.html" title="From oAW to TMF">
-<link rel="prev" href="NewFeatures.html" title="New Features">
-<link rel="next" href="migrating_from_0_7.html" title="Migrating from TMF Xtext 0.7.x">
-</head>
-<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
-<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Migration Support</h1>
-<p>In this document we tried to explain why we decided to change some aspects of Xtext&rsquo;s architecture. We consider most changes as minor but when it comes to scopes you will face a conceptual enhancement that did not exist in oAW Xtext. We tried to explain why it is not easily possible to come up with an adapter for scoping.</p>
-<p>That said you might not have the time to do the migration and wished to have advice for migrating, especially from oAW linking to TMF linking. You&rsquo;re welcome to ask any questions in the newsgroup and we&rsquo;ll try to help you as much as possible in order to get your projects migrated. Also, if you don&rsquo;t want to do the migration yourself we (itemis AG) can do the work for you or help you with that. </p>
-</body>
-</html>
View
91 plugins/org.eclipse.xtext.doc/help/NewFeatures.html
@@ -1,91 +0,0 @@
-<html>
-<head>
-<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
-<title>New Features</title>
-<link href="book.css" rel="stylesheet" type="text/css">
-<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
-<link rel="home" href="index.html" title="Xtext User Guide">
-<link rel="up" href="from_oaw_to_tmf.html" title="From oAW to TMF">
-<link rel="prev" href="differences.html" title="Differences">
-<link rel="next" href="MigrationSupport.html" title="Migration Support">
-</head>
-<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
-<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">New Features</h1>
-<p>This section provides an overview of new possibilities with TMF Xtext compared to oAW Xtext. Please note that this list is neither complete nor does it explain every aspect in detail to keep this document tight.</p>
-<div class="section" title="Dependency Injection with Google Guice">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="DependencyInjectionwithGoogleGuice"></a>Dependency Injection with Google Guice</h3>
-</div>
-</div>
-</div>
-<p>Beyond the mentioned architectural overhaul that carve out separate concerns in a meaningful way these different classes of TMF Xtext are wired using
- <a class="ulink" href="http://code.google.com/p/google-guice" target="_new">Google Guice</a> and can easily be replaced by or combined with your own implementation. We could have foreseen some common needs for adaption but with this mechanism you can virtually change every aspect of Xtext without duplicating an unmanageable amount of code.
- </p>
-</div>
-<div class="section" title="Improvements on Grammar Level">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="ImprovementsonGrammarLevel"></a>Improvements on Grammar Level</h3>
-</div>
-</div>
-</div>
-<p>The Xtext grammar language introduces some new features, too. Read the chapter
- <a class="link" href="grammarLanguage.html" title="The Grammar Language">grammar language</a> to understand the details about all the improvements that have been implemented.
- </p>
-<p>
-
-<a class="link" href="grammarMixins.html" title="Grammar Mixins">Grammar mixins</a> allow you to extend existing languages and change their concrete and abstract syntax. However the abstract syntax (i.e. the Ecore model) can only be extended. This allows you to reuse existing validations, code generators, interpreters or other code which has been written against those types.
- </p>
-<p>In oAW Xtext common terminals like ID, INT, STRING, ML_COMMENT, SL_COMMENT and WS (whitespace) were hard coded into the grammar language and couldn&rsquo;t be removed and hardly overridden. In TMF Xtext these terminals are imported through the newly introduced grammar mixin mechanism from the shipped grammar
- <code class="code">org.eclipse.xtext.common.Terminals</code> per default. This means that they are still there but reside in a library now. You don&rsquo;t have to use them and you can come up with your own set of reusable rules.
- </p>
-<p>
-
-<a class="link" href="grammarImport.html" title="Importing existing Ecore models">Reusing existing Ecore models</a> in oAW Xtext didn&rsquo;t work well and we communicated this by flagging this feature as &sbquo;experimental&rsquo;. In TMF Xtext importing existing Ecore models is fully supported. Moreover, it is possible to import a couple of different EPackages and generate some at the same time, so that the generated Ecore models extend or refer to the existing Ecore models.
- </p>
-<p>The grammar language gained one new concept that is of great value when writing left-factored grammars (e.g. expressions). With actions one can do minor AST rewritings within a rule to overcome degenerated ASTs. You will find an
- <a class="link" href="syntax.html#grammarActions" title="Assigned Actions">in-depth explanation of actions</a> in the dedicated chapter in the reference documentation.
- </p>
-</div>
-<div class="section" title="Fine-grained control for validation">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="differences_validation"></a>Fine-grained control for validation</h3>
-</div>
-</div>
-</div>
-<p>In order to make more expensive validations possible without slowing down the editor, TMF Xtext supports three different validation hooks. </p>
-<div class="orderedlist">
-<ol class="orderedlist" type="1">
-<li class="listitem">
-<p>FAST constraints are triggered by the reconciler (i.e. 500 ms after the last keystroke) and on save.</p>
-</li>
-<li class="listitem">
-<p>NORMAL constraints are executed on save only.</p>
-</li>
-<li class="listitem">
-<p>EXPENSIVE constraints are executed through an action which is available through the context menu.</p>
-</li>
-</ol>
-</div>
-<p>Please note that when using Xtext models for code generation the checks of all three categories will be performed. </p>
-<p>Beside this it is now possible to add information about the feature which is validated.</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;context&nbsp;Entity#name&nbsp;ERROR&nbsp;"Name&nbsp;should&nbsp;start&nbsp;with&nbsp;a&nbsp;capital&nbsp;"+this.name+"."&nbsp;:<br>
-&nbsp;&nbsp;&nbsp;&nbsp;this.name.toFirstUpper()&nbsp;==&nbsp;this.name;<br>
-
-</code>
-</p>
-</div>
-<p>If you add the name of a feature prepended by a hash (&sbquo;#&rsquo;) in Check, the editor will only mark the value of the feature (name), not the whole object (Entity). Both concepts, control over validation time as well as pointing to a specific feature, complement one another in Check and Java based validation.</p>
-</div>
-</body>
-</html>
View
86 plugins/org.eclipse.xtext.doc/help/Overview.html
@@ -7,7 +7,7 @@
<link rel="home" href="index.html" title="Xtext User Guide">
<link rel="up" href="index.html" title="Xtext User Guide">
<link rel="prev" href="index.html" title="Xtext User Guide">
-<link rel="next" href="DSL.html" title="What is a domain-specific language">
+<link rel="next" href="Howdoesitwork.html" title="How does it work?">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Overview</h1>
@@ -20,87 +20,9 @@ <h2 class="title" style="clear: both">
</div>
</div>
</div>
-<p>Xtext is a framework for the development of
- <a class="link" href="DSL.html" title="What is a domain-specific language">domain-specific languages</a> and other textual programming languages. It is tightly integrated with the
- <a class="ulink" href="http://www.eclipse.org/emf" target="_new">Eclipse Modeling Framework (EMF)</a> and leverages the Eclipse Platform in order to provide a language-specific integrated development environment (IDE).
- </p>
-<p>In contrast to common parser generators (like e.g. JavaCC or ANTLR), Xtext derives much more than just a parser and lexical analyzer (lexer) from an input grammar.
- The grammar language is used to describe and generate:</p>
-<div class="itemizedlist">
-<ul class="itemizedlist" type="disc">
-<li class="listitem">
-<p>an incremental, ANTLR 3 based parser and lexer to read your models from text,</p>
-</li>
-<li class="listitem">
-<p>Ecore models (optional),</p>
-</li>
-<li class="listitem">
-<p>a serializer to write your models back to text,</p>
-</li>
-<li class="listitem">
-<p>a linker, to establish cross links between model elements,</p>
-</li>
-<li class="listitem">
-<p>an implementation of the EMF Resource interface with full support for loading and saving EMF models, and</p>
-</li>
-<li class="listitem">
-<p>an integration of the language into your Eclipse IDE.</p>
-</li>
-</ul>
-</div>
-<p>Some of the IDE features, that are either derived from the grammar or easily implementable, are</p>
-<div class="itemizedlist">
-<ul class="itemizedlist" type="disc">
-<li class="listitem">
-<p>
-
-<a class="link" href="highlighting.html" title="Syntax Coloring">syntax coloring</a>,
- </p>
-</li>
-<li class="listitem">
-<p>
-
-<a class="link" href="hyperlinking.html" title="Hyperlinking">model navigation (F3, etc.)</a>,
- </p>
-</li>
-<li class="listitem">
-<p>
-
-<a class="link" href="contentAssist.html" title="Content Assist">code completion</a>,
- </p>
-</li>
-<li class="listitem">
-<p>
-
-<a class="link" href="validation.html#quickfixes" title="Quickfixes">quick fixes</a>
-
-</p>
-</li>
-<li class="listitem">
-<p>
-
-<a class="link" href="formatting.html" title="Formatting (Pretty Printing)">code formatting</a>
-
-</p>
-</li>
-<li class="listitem">
-<p>
-
-<a class="link" href="outline.html" title="Outline View">outline view</a>, and
- </p>
-</li>
-<li class="listitem">
-<p>
-
-<a class="link" href="templates.html" title="Template Proposals">code templates</a>.
- </p>
-</li>
-</ul>
-</div>
-<p>The generated artifacts are wired up through
- <a class="ulink" href="http://code.google.com/p/google-guice/" target="_new">Google Guice</a>, a dependency injection framework which makes it easy to exchange certain functionality in a non-invasive manner.
- </p>
-<p>Although Xtext aims at supporting fast iterative development of domain-specific languages, it can be used to implement IDEs for general purpose programming languages as well.</p>
+<p>No matter if you want to create a small textual domain-specific language (DSL) or you want to implement a full-blown general purpose programming language.
+ With Xtext you can create textual programming languages in a snap. Also if you already have an existing language but it lacks decent tool support, you can use Xtext to create
+ an Eclipse-based IDE providing editing experience known from modern Java IDEs in a surprisingly short amount of time. We call Xtext a language development framework. </p>
</div>
</body>
</html>
View
2 plugins/org.eclipse.xtext.doc/help/Wherenext.html
@@ -11,7 +11,7 @@
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Where next?</h1>
-<p>This is the end of the &bdquo;Getting Started&rdquo; chapter of this documentation. The next part of this document is more detailed and compact at he same time. It discusses technical topics not always in an introductory way but acts as a comprehensive reference. You will find more material that introduces Xtext at the
+<p>This is the end of the &ldquo;Getting Started&rdquo; chapter of this documentation. The next part of this document is more detailed and compact at he same time. It discusses technical topics not always in an introductory way but acts as a comprehensive reference. You will find more material that introduces Xtext at the
<a class="ulink" href="http://www.xtext.org" target="_new">Xtext website</a> .
</p>
</body>
View
19 plugins/org.eclipse.xtext.doc/help/WhoisbehindXtext.html
@@ -0,0 +1,19 @@
+<html>
+<head>
+<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
+<title>Who is behind Xtext?</title>
+<link href="book.css" rel="stylesheet" type="text/css">
+<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
+<link rel="home" href="index.html" title="Xtext User Guide">
+<link rel="up" href="Overview.html" title="Overview">
+<link rel="prev" href="WhousesXtext.html" title="Who uses Xtext?">
+<link rel="next" href="DSL.html" title="What is a domain-specific language">
+</head>
+<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
+<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Who is behind Xtext?</h1>
+<p>Xtext is a professional Open-Source project. We, the main developers and the project lead, work for itemis, which is a well known consulting company specialized on model-based development.
+ Therefore we are able to work almost full-time on the project. Xtext is an Eclipse.org project. Besides many other advantages this means that you don&rsquo;t have to fear any IP issues, because the Eclipse Foundation
+ has their own layers who take care that no intellectual property is violated. </p>
+<p>Where does the money for Open-Source development come from you ask? Well, we provide professional services around Xtext. Be it training or on-site consulting, be it development of prototypes or implementation of full-blown IDEs for programming languages. We not only know the framework very well but also are experts in programming and domain-specific language design. Don&rsquo;t hesitate to get in contact with us (www.itemis.com).</p>
+</body>
+</html>
View
18 plugins/org.eclipse.xtext.doc/help/WhousesXtext.html
@@ -0,0 +1,18 @@
+<html>
+<head>
+<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
+<title>Who uses Xtext?</title>
+<link href="book.css" rel="stylesheet" type="text/css">
+<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
+<link rel="home" href="index.html" title="Xtext User Guide">
+<link rel="up" href="Overview.html" title="Overview">
+<link rel="prev" href="Xtextishighlyconfigurable.html" title="Xtext is highly configurable">
+<link rel="next" href="WhoisbehindXtext.html" title="Who is behind Xtext?">
+</head>
+<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
+<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Who uses Xtext?</h1>
+<p>Xtext is used in many different industries. It is used in the field of mobile devices, automotive development, embedded or Java enterprise software projects and game development.
+ People use Xtext-based languages to generateJava, C, C++, C#, Objective C, Python, or Ruby code. Although the infrastructure runs on the JVM, you can compile Xtext languages to any existing target platform.
+ Xtext based languages are developed for well known Open-Source projects such as Maven, Eclipse B3, the Eclipse Webtools platform or Google&rsquo;s Protocol Buffers and the framework is also widely used in research projects. </p>
+</body>
+</html>
View
104 plugins/org.eclipse.xtext.doc/help/Xtend_based_apis.html
@@ -1,104 +0,0 @@
-<html>
-<head>
-<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
-<title>Where are the Xtend-based APIs?</title>
-<link href="book.css" rel="stylesheet" type="text/css">
-<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
-<link rel="home" href="index.html" title="Xtext User Guide">
-<link rel="up" href="from_oaw_to_tmf.html" title="From oAW to TMF">
-<link rel="prev" href="migration_overview.html" title="Migration overview">
-<link rel="next" href="differences.html" title="Differences">
-</head>
-<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
-<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Where are the Xtend-based APIs?</h1>
-<p>One of the nice things with oAW Xtext was the use of Xtend to allow customizing different aspects of the generated language infrastructure.
- Xtend is a part of the template language Xpand, which is shipped with oAW (and now is included in M2T Xpand). It provides a nicer expression syntax than Java.
- Especially the existence of higher-order functions for collections is extremely handy when working with models.
- In addition to the nice syntax, it provides dynamic polymorphic dispatch, which means that declaring e.g. label computation for an Ecore model is very convenient and type safe at the same time.
- In Java one usually has to write instanceof and cast orgies. </p>
-<div class="section" title="Xtend is hard to debug">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="Xtend_is_hard_to_debug"></a>Xtend is hard to debug</h3>
-</div>
-</div>
-</div>
-<p>While the aforementioned features allow the convenient specification of label and icon providers, outline views, content assist and linking, Xtend is interpreted and therefore hard to debug. Because of that Xpand is shipped with a special debugger facility. Unfortunately, this debugger cannot be used in the context of Xtext since it implies that the Xtend functions have to be called from a workflow. This is not and cannot be the case for Xtext Editors.
- As a result one has to debug his way through the interpreter, which is hard and inconvenient (even for us, who have written that interpreter).</p>
-</div>
-<div class="section" title="Xtend is slow">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="Xtend_is_slow"></a>Xtend is slow</h3>
-</div>
-</div>
-</div>
-<p>But the problematic debugging in the context of Xtext was not the main reason why there are no Xtend-based APIs beside Check in TMF Xtext. The main reason is that Xtend is too slow to be evaluated &bdquo;inside&rdquo; the editor again and again while you type.
- While Xtend&rsquo;s performance is sufficient when run in a code generator, it is just too slow to be executed on keystroke (or 500ms after the last keystroke, which is when the reconciler reparses, links and validates the model). Xtend is relatively slow, because it supports polymorphic dispatch (the cool feature mentioned above), which means that for each invocation it matches at runtime which function is the best match and it has to do so on each call. Also Xtend supports a pluggable typesystem, where you can adapt to other existing type systems such as JavaBeans or Ecore. This is very powerful and flexible but introduces another indirection layer. Last but not least the code is interpreted and not compiled. The price we pay for all these nice features is reduced performance.</p>
-<p>In addition to these scalability problems we have designed some core APIs (e.g. scopes) around the idea of Iterables, which allows for lazy resolution of elements. As Xtend does not know the concept of Iterators you would have to work with lists all the time. Copying collections over and over again is far more expensive than chaining Iterables through filters and transformers like we do with Google Collections in TMF Xtext.</p>
-</div>
-<div class="section" title="Convenient Java">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="Convenient_Java"></a>Convenient Java</h3>
-</div>
-</div>
-</div>
-<p>To summarize the dilemma we had to find a way to allow for convenient, scalable and debuggable APIs. Ultimately we wanted to provide neat DSLs for every view point, which provide all these things. However, we had to prioritize our efforts with the available resources in mind.
- As a result we found ways and means to tweak Java as good as possible to allow for relatively convenient, high performing implementations. </p>
-<p>Java is fast and can easily be debugged but ranks behind Xtend regarding convenience. We address this with different approaches to make Java development in the context of Xtext as comfortable as possible.</p>
-<p>Most of the APIs in TMF Xtext use polymorphic dispatching, which mimics the behavior known from Xtend. Another valuable feature of Xtend while working with oAW Xtext is static type checking while working with the inferred Ecore model whereas in Java the work with dynamic Ecore classes was rather cumbersome. Since TMF Xtext generates static Ecore classes per default you get static typing in Java as well. Additionally, the use of
- <a class="ulink" href="http://code.google.com/p/google-collections" target="_new">Google Collections</a> reduces the pain when navigating over your model to extract information.
- </p>
-<p>With these techniques an ILabelProvider that handles your own EClasses
- <code class="code">Property</code> and
- <code class="code">Entity</code> can be written like this:
- </p>
-<div class="literallayout">
-<p>
-<code class="code">public&nbsp;class&nbsp;DomainModelLabelProvider&nbsp;extends&nbsp;DefaultLabelProvider&nbsp;{<br>
-
-<br>
-&nbsp;&nbsp;String&nbsp;label(Entity&nbsp;e)&nbsp;{<br>
-&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;e.getName();<br>
-&nbsp;&nbsp;}<br>
-
-<br>
-&nbsp;&nbsp;String&nbsp;image(Property&nbsp;p)&nbsp;{<br>
-&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;p.isMultiValue()&nbsp;?&nbsp;"complexProperty.gif":&nbsp;"simpleProperty.gif";<br>
-&nbsp;&nbsp;}<br>
-
-<br>
-&nbsp;&nbsp;String&nbsp;image(Entity&nbsp;e)&nbsp;{<br>
-&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;"entity.gif";<br>
-&nbsp;&nbsp;}<br>
-
-<br>
-}<br>
-&nbsp;&nbsp;<br>
-
-</code>
-</p>
-</div>
-<p>As you can see this is very similar to the way one describes labels and icons in oAW Xtext, but has the advantage that it is easier to test and to debug, faster and can be used everywhere an ILabelProvider is expected in Eclipse.</p>
-</div>
-<div class="section" title="Conclusion">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="conclusion"></a>Conclusion</h3>
-</div>
-</div>
-</div>
-<p>Just to get it right, Xtend is a very powerful language and we still use it for its main purpose: code generation and model transformation. The whole generator in TMF Xtext is written in Xpand and Xtend and its performance is at least in our experience sufficient for that use case. Actually we were able to increase the runtime performance of Xpand by about 60% for the Galileo release of M2T Xpand. But still, live execution in the IDE and on typing is very critical and one has to think about every millisecond in this area. </p>
-<p>As an alternative to the Java APIs we also considered other JVM languages. We like static typing and think it is especially important when processing typed models (which evolve heavily). That&rsquo;s why Groovy or JRuby were no alternatives. Using Scala would have been a very good match, but we didn&rsquo;t want to require knowledge of Scala so we didn&rsquo;t use it and stuck to Java. </p>
-</div>
-</body>
-</html>
View
18 plugins/org.eclipse.xtext.doc/help/Xtextishighlyconfigurable.html
@@ -0,0 +1,18 @@
+<html>
+<head>
+<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
+<title>Xtext is highly configurable</title>
+<link href="book.css" rel="stylesheet" type="text/css">
+<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
+<link rel="home" href="index.html" title="Xtext User Guide">
+<link rel="up" href="Overview.html" title="Overview">
+<link rel="prev" href="Howdoesitwork.html" title="How does it work?">
+<link rel="next" href="WhousesXtext.html" title="Who uses Xtext?">
+</head>
+<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
+<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Xtext is highly configurable</h1>
+<p>Xtext uses Google Guice to wire up the whole language as well as the IDE infrastructure. A central, external module is used to configure the dependency injection container.
+ As already mentioned, Xtext comes with decent default implementations and DSLs and APIs for the most often changed aspects. But if you need something completely different,
+ the use of Google Guice gives you the power to exchange every little class in a non-invasive way.</p>
+</body>
+</html>
View
40 plugins/org.eclipse.xtext.doc/help/antlr_ip_issue.html
@@ -10,35 +10,47 @@
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">The ANTLR IP issue (or which parser to use?)</h1>
-<p>In order to be able to parse models written in your language, Xtext needs to provide a special parser for it.
- The parser is generated from the language grammar. </p>
-<p>Currently it is recommended to use the
- <a class="ulink" href="http://www.antlr.org" target="_new">ANTLR-based</a> parser.
- ANTLR is a very sophisticated parser generator framework based on a so called LL(*) algorithm.
+<p>In order to be able to parse models written in your language, Xtext needs to provide a special parser
+ for it. The parser is generated from the self defined language grammar. </p>
+<p>It is recommended to use the
+ <a class="ulink" href="http://www.antlr.org" target="_new">ANTLR</a> based parser.
+ ANTLR is a very sophisticated parser generator framework which implements a so called LL(*) algorithm.
It is fast, simple and at the same time has some very nice and sophisticated features. Especially its
support for error recovery is much better than what other parser generators provide.
</p>
-<p>ANTLR comes in two parts: the runtime and the generator.
+<p>Xtext uses ANTLR 3 which comes in two parts: the runtime and the generator.
Both are shipped under the BSD license and have a clean intellectual property history.
- However the generator is still implemented in an older version of ANTLR (v 2.x), where it was not possible for the Eclipse Foundation
- to be sure where exactly every line of code originated. Therefore ANTLR v 2.x didn&rsquo;t get the required approval.
- Eclipse has a strict IP policy, which makes sure that everything provided by Eclipse can be consumed under the terms of the Eclipse Public License.
- The details are described in
- <a class="ulink" href="http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf" target="_new">this document</a>.
+ However the ANTLR parser generator is unfortunately still implemented in an older version of itself
+ (v 2.x), where it was not possible for the Eclipse Foundation to ensure where exactly every line of
+ code originated. Therefore ANTLR 2 didn&rsquo;t get the required IP approval. Eclipse has a strict IP policy,
+ which makes sure that everything provided by Eclipse can be consumed under the terms of the Eclipse
+ Public License. The details are described in
+
+ <a class="ulink" href="http://www.eclipse.org/org/documents/Eclipse_IP_Policy.pdf" target="_new">this document</a>. That is why we are not
+ allowed to ship Xtext with the ANTLR generator but only with the IP approvied runtime components. We
+ have to provide it separately and for your convenience you&rsquo;ll be asked to download the ANTLR generator
+ when you run your language generator for the first time. You can even download it directly or install
+ an additional plugin into Eclipse:
</p>
-<p>Unfortunately as the generator of ANTLR V3 needs ANTLR V2 it is as well not yet IP approved.
- That is why we are not allowed to ship Xtext with the ANTLR generator (the runtime is IP approved), but have to provide it separately via update-site at:</p>
<div class="itemizedlist">
<ul class="itemizedlist" type="disc">
<li class="listitem">
<p>
-<a class="ulink" href="http://download.itemis.com/updates/" target="_new">http://download.itemis.com/updates/</a>
+<a class="ulink" href="http://download.itemis.com/antlr-generator-3.0.1.jar" target="_new">http://download.itemis.com/antlr-generator-3.0.1.jar</a>
+
+</p>
+</li>
+<li class="listitem">
+<p>or use the update site at
+ <a class="ulink" href="http://download.itemis.com/updates" target="_new">http://download.itemis.com/updates</a>
</p>
</li>
</ul>
</div>
+<p>The workflow will not bother you with this issue on subsequent executions as the archive will be stored
+ in your workspace root directory and can thereby be reused during the next run. </p>
<p>
<span class="bold"><strong>IMPORTANT</strong></span> :
View
41 plugins/org.eclipse.xtext.doc/help/concepts.html
@@ -11,23 +11,31 @@
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Xtext and EMF</h1>
-<p>In this section we try to introduce the basic features of the
- <a class="link" href="">Eclipse Modeling Framework (EMF)</a> in the context of Xtext. As EMF provides the base infrastructure for Xtext, it is useful to have a basic understanding of EMF, so we give a rough outline here.
+<p>In this section we introduce the basic concepts of the
+ <a class="link" href="">Eclipse Modeling Framework (EMF)</a> in the context of Xtext.
+ As Xtext integrates with and uses EMF, it is important to have a basic understanding of EMF.
If you want to learn more about EMF, we recommend reading the
<a class="link" href="">EMF book</a>.
</p>
-<div class="section" title="Model, AST, Meta Model, and Ecore">
+<div class="section" title="Model, Ecore Model, and Ecore">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both">
-<a name="model_metamodel"></a>Model, AST, Meta Model, and Ecore </h2>
+<a name="model_metamodel"></a>Model, Ecore Model, and Ecore </h2>
</div>
</div>
</div>
-<p>When you run the workflow in your Xtext project, Xtext will generate a bunch of artifacts that make up the infrastructure for your language. Among other useful things, it will create a parser. This parser takes models, i.e. files in your language, as an input and produce a graph of objects as output. This graph is called the
- <span class="emphasis"><em>Abstract Syntax Tree</em></span> (AST). Given the example model from the
- <a class="link" href="">introduction</a>, the AST should look like this
+<p>When Xtext uses EMF models as the in-memory representation of any parsed text files. This in-memory object graph is called the
+ <span class="emphasis"><em>Abstract Syntax Tree</em></span> (AST).
+ Depending on the community this concepts is also called
+ <span class="emphasis"><em>document object graph (DOM)</em></span>,
+ <span class="emphasis"><em>semantic model</em></span>, or simply
+ <span class="emphasis"><em>model</em></span>. We use
+ <span class="emphasis"><em>model</em></span> and
+ <span class="emphasis"><em>AST</em></span> interchangeably.
+ Given the example model from the
+ <a class="link" href="">introduction</a>, the AST looks similar to this
</p>
<p>
@@ -37,11 +45,18 @@ <h2 class="title" style="clear: both">
<p>
</p>
-<p>In terms of modeling, we also refer to the AST tree as the
- <span class="emphasis"><em>semantic model</em></span>, as it does no longer carry any additional syntactical information. Usually it is only the semantic model that is processed by later processing steps, i.e. in the execution by a code generator or interpreter.
- </p>
-<p>The structure of the semantic model is defined by the so called
- <span class="emphasis"><em>meta model</em></span>. You can think of the meta model as the schema for the models. In Xtext, meta models are either inferred from the grammar or predefined by the user (see the section on
+<p>The
+ <span class="emphasis"><em>AST</em></span> should contain the essence of your textual models and abstract over syntactical information. It is used by later processing steps,
+ such as validation, compilation or interpretation. In EMF a model is made up of instances of
+ <span class="emphasis"><em>EObjects</em></span> which are connected
+ and an
+ <span class="emphasis"><em>EObject</em></span> is an instance of an
+ <span class="emphasis"><em>EClass</em></span>.
+ A set of
+ <span class="emphasis"><em>EClasses</em></span> if contained in a so called
+ <span class="emphasis"><em>EPackage</em></span>, which are both concepts of
+ <span class="emphasis"><em>Ecore</em></span>
+ In Xtext, meta models are either inferred from the grammar or predefined by the user (see the section on
<a class="link" href="syntax.html#package_declarations" title="EPackage declarations">package declarations</a> for details). The next diagram shows the meta model of our example:
</p>
<p>
@@ -90,7 +105,7 @@ <h2 class="title" style="clear: both">
<p>The superclass of EAttibutes and EReferences is
<span class="emphasis"><em>EStructuralFeature</em></span> and allows to define a name and a cardinality by setting
<code class="code">lowerBound</code> and
- <code class="code">upperBound</code>. Setting the latter to -1 means &sbquo;unbounded&rsquo;.
+ <code class="code">upperBound</code>. Setting the latter to -1 means &lsquo;unbounded&rsquo;.
</p>
<p>The common supertype of EDataType and EClass is
<span class="emphasis"><em>EClassifier</em></span>. An
View
10 plugins/org.eclipse.xtext.doc/help/configuration.html
@@ -41,7 +41,7 @@ <h3 class="title">
</div>
</div>
</div>
-<p>MWE2 allows to compose object graphs declaratively in a very compact manner. The nice thing about it is that it just instantiates Java classes and the configuration is done through public setter and adder methods as one is used to from Java Beans encapsulation principles. An in-depth documentation can be found in the chapter &bdquo;MWE2&rdquo;#MWE2.</p>
+<p>MWE2 allows to compose object graphs declaratively in a very compact manner. The nice thing about it is that it just instantiates Java classes and the configuration is done through public setter and adder methods as one is used to from Java Beans encapsulation principles. An in-depth documentation can be found in the chapter &ldquo;MWE2&rdquo;#MWE2.</p>
<p>Given the following simple Java class (POJO):</p>
<div class="literallayout">
<p>
@@ -130,11 +130,11 @@ <h3 class="title">
into this workflow as well to make it convenient to instantiate actual workflows and components, but these ones are covered
in depth in the appropriate
<a class="link" href="MWE2.html" title="MWE2">chapther</a>. The constructed objects are furthermore configured according to the declaration
- in the module, e.g. a second instance of Person will be created and added to the list of children of &bdquo;Grandpa&rdquo; while the third person
- - the class is inferred from the assigned feature &ndash; becomes a child of &bdquo;Father&rdquo;. All three instances will have their respective
+ in the module, e.g. a second instance of Person will be created and added to the list of children of &ldquo;Grandpa&rdquo; while the third person
+ - the class is inferred from the assigned feature &ndash; becomes a child of &ldquo;Father&rdquo;. All three instances will have their respective
<code class="code">name</code>
assigned via a reflective invocation
- <code class="code">setName</code>. If one wants to add another child to &bdquo;Father&rdquo;, she
+ <code class="code">setName</code>. If one wants to add another child to &ldquo;Father&rdquo;, she
can simply repeat the child assignment:
</p>
<div class="literallayout">
@@ -319,7 +319,7 @@ <h3 class="title">
</div>
</div>
<p>In the following table the most important standard generator fragments are listed. Please refer to the Javadocs for more detailed documentation.</p>
-<table id="N10974">
+<table id="N10BFA">
<tr>
<th>Class</th>
View
4 plugins/org.eclipse.xtext.doc/help/contentAssist.html
@@ -73,7 +73,7 @@ <h3 class="title">
<p>Clients who want to customize the behavior may override the methods from the
<code class="code">AbstractProposalProvider</code> or introduce new methods with specialized parameters. The framework dispatches method calls according to the current context to the most concrete implementation, that can be found.
</p>
-<p>It is important to know, that for a given offset in a model file, many possible grammar elements exist. The framework dispatches to the method declarations for any valid element. That means, that a bunch of '&sbquo;complete.*&rsquo;' methods may be called.</p>
+<p>It is important to know, that for a given offset in a model file, many possible grammar elements exist. The framework dispatches to the method declarations for any valid element. That means, that a bunch of '&lsquo;complete.*&rsquo;' methods may be called.</p>
</div>
<div class="section" title="Sample Implementation">
<div class="titlepage">
@@ -84,7 +84,7 @@ <h3 class="title">
</div>
</div>
</div>
-<p>To provide a dummy proposal for the description of a model object, you may introduce a specialization of the generated method and implement it as follows. This will give &sbquo;Description for model #7&rsquo; for a model with the intAttribute '7'</p>
+<p>To provide a dummy proposal for the description of a model object, you may introduce a specialization of the generated method and implement it as follows. This will give &lsquo;Description for model #7&rsquo; for a model with the intAttribute '7'</p>
<div class="literallayout">
<p>
<code class="code">public&nbsp;void&nbsp;completeModel_StringDescription&nbsp;(<br>
View
2 plugins/org.eclipse.xtext.doc/help/dependencyInjection.html
@@ -50,7 +50,7 @@ <h3 class="title">
<a class="link" href="configuration.html#generator" title="The Generator">generator fragments</a> to automatically generate service implementations for a grammar. Thereby, Xtext strives to provide meaningful implementations out of the box and to allow customization wherever needed. Developers are encouraged to subclass existing services and configure them for their languages in their
<a class="link" href="dependencyInjection.html#guicemodules" title="Modules">modules</a>.
</p>
-<p>When Guice instantiates an object, it also supplies this instance with all its dependent services. All a service does is to request &bdquo;some implementation for a certain interface&rdquo; using the <code class="code">@Inject</code>-annotation. Based on the
+<p>When Guice instantiates an object, it also supplies this instance with all its dependent services. All a service does is to request &ldquo;some implementation for a certain interface&rdquo; using the <code class="code">@Inject</code>-annotation. Based on the
<a class="link" href="dependencyInjection.html#guicemodules" title="Modules">modules configuration</a> Guice decides which class to instantiate or which object to reuse.
</p>
<p>For example, Guice can automatically initialize member variables with the needed services. </p>
View
271 plugins/org.eclipse.xtext.doc/help/differences.html
@@ -1,271 +0,0 @@
-<html>
-<head>
-<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
-<title>Differences</title>
-<link href="book.css" rel="stylesheet" type="text/css">
-<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
-<link rel="home" href="index.html" title="Xtext User Guide">
-<link rel="up" href="from_oaw_to_tmf.html" title="From oAW to TMF">
-<link rel="prev" href="Xtend_based_apis.html" title="Where are the Xtend-based APIs?">
-<link rel="next" href="NewFeatures.html" title="New Features">
-</head>
-<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
-<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Differences</h1>
-<p>In this section differences between oAW Xtext and TMF Xtext are outlined and explained. We&rsquo;ll start from the APIs such as the grammar language and the validation and finish with the different hooks for customizing linking and several UI aspects, such as outline view and content assist. We&rsquo;ll also try to map some of the oAW Xtext concepts to their counterparts in TMF Xtext.</p>
-<div class="section" title="Differences in the grammar language">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="differenced_grammar_language"></a>Differences in the grammar language</h3>
-</div>
-</div>
-</div>
-<p>When looking at a TMF Xtext grammar the first time it looks like one has to provide additional information which was not necessary in oAW Xtext. In oAW Xtext *.xtxt files started with the first production rule where in TMF Xtext one has to declare the name of the language followed by declaration of one or more used/generated Ecore models:</p>
-<p>TMF Xtext heading information</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;grammar&nbsp;my.namespace.Language&nbsp;with&nbsp;org.eclipse.xtext.common.Terminals<br>
-&nbsp;&nbsp;generate&nbsp;myDsl&nbsp;"http://www.namespace.my/2009/MyDSL"<br>
-&nbsp;<br>
-&nbsp;&nbsp;FirstRule&nbsp;:&nbsp;...<br>
-
-<br>
-
-</code>
-</p>
-</div>
-<p>In oAW Xtext this information was provided through the generator (actually it is contained in the *.properties file) but we found that these things are very important for a complete description of a grammar. Therefore we made that information becoming a part of the grammar language in order to have self-describing grammars and allow for sophisticated static analysis, etc.. </p>
-<p>Apart from the first two lines the grammar languages of both versions are more or less compatible. The syntax for all the different EBNF concepts (alternatives, groups, cardinalities) is similar.
- Also assignments are syntactically and semantically identical in both versions.
- However in TMF Xtext some concepts have been generalized and improved:</p>
-<div class="section" title="String rules become Datatype rules">
-<div class="titlepage">
-<div>
-<div>
-<h4 class="title">
-<a name="differences_datatype_rules"></a>String rules become Datatype rules</h4>
-</div>
-</div>
-</div>
-<p>The very handy String rules are still present in TMF Xtext but we generalized them so that you don&rsquo;t need to write the &sbquo;String&rsquo; keyword in front of them and at the same time these rules can not only produce EStrings but (as the name suggests) any kind of EDataType. Every parser rule that neither includes assignments nor calls any other that does, returns an EDataType containing the consumed data. Per default this is an EString but you can now simply create a parser rule returning other EDataTypes as well (see
- <a class="link" href="syntax.html#datatypeRules" title="Data type rules">Datatype rules</a>).
- </p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;Float&nbsp;returns&nbsp;ecore::EDouble&nbsp;:&nbsp;INT&nbsp;('.'&nbsp;INT)?;<br>
-
-</code>
-</p>
-</div>
-</div>
-<div class="section" title="Enum rules">
-<div class="titlepage">
-<div>
-<div>
-<h4 class="title">
-<a name="differences_enum_rules"></a>Enum rules</h4>
-</div>
-</div>
-</div>
-<p>Enum rules have not changed significantly. The keyword has changed to be all lower case (&sbquo;enum&rsquo; instead of &sbquo;Enum&rsquo;).
- Also the right-hand side of the assignment is now optional. That is in oAW Xtext:</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;Enum&nbsp;MyEnum&nbsp;:&nbsp;foo='foo'&nbsp;|&nbsp;bar='bar';<br>
-&nbsp;&nbsp;<br>
-
-</code>
-</p>
-</div>
-<p>becomes</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;enum&nbsp;MyEnum&nbsp;:&nbsp;foo='foo'&nbsp;|&nbsp;bar='bar';<br>
-&nbsp;&nbsp;&nbsp;<br>
-
-</code>
-</p>
-</div>
-<p>and because the name of the literal equals the literal value one can omit the right-hand side in this case and write:</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;enum&nbsp;MyEnum&nbsp;:&nbsp;foo&nbsp;|&nbsp;bar;<br>
-
-<br>
-
-</code>
-</p>
-</div>
-</div>
-<div class="section" title="Native rules">
-<div class="titlepage">
-<div>
-<div>
-<h4 class="title">
-<a name="differences_native_rules"></a>Native rules</h4>
-</div>
-</div>
-</div>
-<p>Another improvement is that we could replace the blackbox native rules with full-blown EBNF syntax.
- That is native rules become terminal rules and are no longer written as a string literal containing ANTLR syntax.</p>
-<p>Example :</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;Native&nbsp;FOO&nbsp;:&nbsp;"'f'&nbsp;'o'&nbsp;'o'";<br>
-
-</code>
-</p>
-</div>
-<p>becomes</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;terminal&nbsp;FOO&nbsp;:&nbsp;'f'&nbsp;'o'&nbsp;'o';<br>
-
-</code>
-</p>
-</div>
-<p>See the
- <a class="link" href="syntax.html#TerminalRules" title="Terminal Rules">reference documentation</a> for all the different expressions possible in terminal rules.
- </p>
-</div>
-<div class="section" title="No URI terminal rule anymore">
-<div class="titlepage">
-<div>
-<div>
-<h4 class="title">
-<a name="differences_no_URI_token"></a>No URI terminal rule anymore</h4>
-</div>
-</div>
-</div>
-<p>We decided to remove the URI terminal. The only reason for the existence was to mark the model somehow so that the framework knows what information to use in order to load referenced models. Instead we decided to solve this similar to how we imply other defaults: by convention.</p>
-<p>So instead of using a special token which is syntactically a STRING token, the default import mechanism now looks for EAttributes of type EString with the name &sbquo;importURI&rsquo;.
- That is if you&rsquo;ve used the URI token like this:</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;Import&nbsp;:&nbsp;'import'&nbsp;myReference=URI;<br>
-
-</code>
-</p>
-</div>
-<p>you&rsquo;ll have to rewrite it that way</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;Import&nbsp;:&nbsp;'import'&nbsp;importURI=STRING;<br>
-
-</code>
-</p>
-</div>
-<p>Although this changes your Ecore model, one usually never used this reference explicitly as it was only there to be used by the default import mechanism. So we assume and hope that changing the reference is not a big deal for you. </p>
-</div>
-<div class="section" title="Return types">
-<div class="titlepage">
-<div>
-<div>
-<h4 class="title">
-<a name="differences_return_types"></a>Return types</h4>
-</div>
-</div>
-</div>
-<p>The syntax to explicitly declare the return type of a rule has changed. In oAW Xtext (where this was marked as &sbquo;experimental&rsquo;) the syntax was:</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;MyRule&nbsp;[MyType]&nbsp;:&nbsp;foo=ID;&nbsp;&nbsp;<br>
-
-</code>
-</p>
-</div>
-<p>in TMF Xtext we have a keyword for this :</p>
-<div class="literallayout">
-<p>
-<code class="code">&nbsp;&nbsp;MyRule&nbsp;returns&nbsp;MyType&nbsp;:&nbsp;foo=ID;<br>
-
-</code>
-</p>
-</div>
-<p>This is a bit more verbose, but at the same time more readable. And as you don&rsquo;t have to write the return type in most situations, it&rsquo;s good to have a more explicit, readable syntax.</p>
-</div>
-</div>
-<div class="section" title="Differences in Linking">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="differences_linking"></a>Differences in Linking</h3>
-</div>
-</div>
-</div>
-<p>The linking has been completely redesigned. In oAW Xtext linking was done in a very naive way: To find an element one queries a list of all &sbquo;visible&rsquo; EObjects, then filters out what is not needed and tries to find a match by comparing the text written for the crosslink with the value returned by the
- <code class="code">id()</code> extension. As a side-effect of
- <code class="code">link_feature()</code> the reference is set.
- </p>
-<p>The code about selecting and filtering
- <code class="code">allElements()</code> usually has been duplicated in the corresponding content assist function, so that linking and content assist are semantically in sync. If you&rsquo;re good (we usually were not) you externalized that piece of code and reused the same extension in content assist and linking.
- </p>
-<p>To put it bluntly this approach could be summarized in two steps:</p>
-<div class="orderedlist">
-<ol class="orderedlist" type="1">
-<li class="listitem">
-<p>Give me the whole universe including every unregarded object in the uncharted backwaters of the unfashionable end of the western spiral arm of the galaxy and squeeze it into an Arraylist </p>
-</li>
-<li class="listitem">
-<p>From this, select the one I need</p>
-</li>
-</ol>
-</div>
-<p>This was not only very expensive but also lacks an important abstraction: the notion of scopes.</p>
-<div class="section" title="The idea of scopes">
-<div class="titlepage">
-<div>
-<div>
-<h4 class="title">
-<a name="differences_linking_sopes"></a>The idea of scopes</h4>
-</div>
-</div>
-</div>
-<p>In TMF Xtext we&rsquo;ve introduced scopes and scope providers that are responsible for creating scopes. A scope is basically a set of name-&gt;value pairs. Scopes are implemented upon Iterables and are nested to build a hierarchy. With scopes we declare &bdquo;visible&rdquo; objects in a lazy and cost-saving way where the linker only navigates as far as necessary to find matching objects. The content assist reuses this set of visible objects to offer only reachable objects. </p>
-<p>When the linking has to be customized scoping is where most of the semantics typically goes into. By implementing an
- <a class="ulink" href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext/src/org/eclipse/xtext/scoping/IScopeProvider.java?root=Modeling_Project&view=co" target="_new">IScopeProvider</a> for your language linking and content assist will automatically be kept in sync since both use the scope provider.
- </p>
-<p>The provided default implementation is semantically mostly identical to how the default linking worked in oAW Xtext: </p>
-<div class="orderedlist">
-<ol class="orderedlist" type="1">
-<li class="listitem">
-<p>Elements which have an attribute &sbquo;name&rsquo; will be made visible by their name</p>
-</li>
-<li class="listitem">
-<p>Referenced resources will be put on the (outer) scope by using the &sbquo;importURI&rsquo;- naming convention and will only be loaded if necessary</p>
-</li>
-<li class="listitem">
-<p>The available elements are filtered by the expected type (i.e. the type of the reference to be linked)</p>
-</li>
-</ol>
-</div>
-</div>
-<div class="section" title="Migration">
-<div class="titlepage">
-<div>
-<div>
-<h4 class="title">
-<a name="differences_linking_migration"></a>Migration</h4>
-</div>
-</div>
-</div>
-<p>We expect the migration of linking to be very simple if you&rsquo;ve not changed the default semantics that much. We&rsquo;ve already migrated a couple of projects and it wasn&rsquo;t too hard to do so.
- If you have changed linking (and also content assist) a lot, you&rsquo;ll have to translate the semantics to the IScopeProvider concept. This might be a bit of work, but it is worth the effort as this will clean up your code base and better separate concerns.</p>
-</div>
-</div>
-<div class="section" title="Differences in UI customizing">
-<div class="titlepage">
-<div>
-<div>
-<h3 class="title">
-<a name="differences_ui"></a>Differences in UI customizing</h3>
-</div>
-</div>
-</div>
-<p>In oAW Xtext several UI services such as content assist, outline view or the label provider have been customized using Xtend. In TMF Xtext there is no Xtend API for these aspects. Extensive model computations for the content assist is most probably not necessary anymore- it reuses scopes. And since we provide a declarative Java API that mimics the polymorphic dispatch and relies on static Ecore classes you will gain nearly the same expressiveness as before while increasing maintainability and performance.</p>
-<p>Beside the API change in favor of Java we have to mention that in TMF Xtext the outline view does not support multiple view points so far. This is just because we didn&rsquo;t manage to get this included. We don&rsquo;t think that view points are a bad idea in general, but we decided that other things were more important.</p>
-</div>
-</body>
-</html>
View
6 plugins/org.eclipse.xtext.doc/help/emf_integration.html
@@ -1,7 +1,7 @@
<html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
-<title>Integration with EMF</title>
+<title>Integration with other EMF editors</title>
<link href="book.css" rel="stylesheet" type="text/css">
<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
<link rel="home" href="index.html" title="Xtext User Guide">
@@ -10,7 +10,7 @@
<link rel="next" href="encoding.html" title="Encoding in Xtext">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
-<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Integration with EMF</h1>
+<h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Integration with other EMF editors</h1>
<p>Xtext relies heavily on EMF internally, but it can also be used as the serialization back-end of other EMF-based tools.</p>
<div class="section" title="XtextResource Implementation">
<div class="titlepage">
@@ -110,7 +110,7 @@ <h3 class="title">
<p>To illustrate how to build a graphical editor on top of an
<code class="code">XtextResource</code> we have provided an example. It consists of a number of plug-ins
</p>
-<table id="N112CC">
+<table id="N11673">
<tr>
<td>Plug-in</td>
View
2 plugins/org.eclipse.xtext.doc/help/emf_resource.html
@@ -57,7 +57,7 @@ <h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">EMF Resources</h1>
</ul>
</div>
<p>Model elements (EObjects) within a resource are referred to by a so called
- <span class="emphasis"><em>fragment</em></span>. The fragment must be unique inside the resource. The URI of an EObject consists of the URI of its resource followed by a hash mark &sbquo;#&rsquo; and its fragment. The default EMF strategy for calculating fragments is to build a path expression, similar to XPath. In Xtext, you can override that by binding a new
+ <span class="emphasis"><em>fragment</em></span>. The fragment must be unique inside the resource. The URI of an EObject consists of the URI of its resource followed by a hash mark &lsquo;#&rsquo; and its fragment. The default EMF strategy for calculating fragments is to build a path expression, similar to XPath. In Xtext, you can override that by binding a new
<code class="code">IFragmentProvider</code>.
</p>
<p>As the name suggests, a
View
6 plugins/org.eclipse.xtext.doc/help/encoding.html
@@ -6,7 +6,7 @@
<meta content="DocBook XSL Stylesheets V1.75.1" name="generator">
<link rel="home" href="index.html" title="Xtext User Guide">
<link rel="up" href="RuntimeConcepts.html" title="Runtime Concepts">
-<link rel="prev" href="emf_integration.html" title="Integration with EMF">
+<link rel="prev" href="emf_integration.html" title="Integration with other EMF editors">
<link rel="next" href="MWE2.html" title="MWE2">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
@@ -15,7 +15,7 @@ <h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Encoding in Xtext</h
<span class="emphasis"><em>Encoding</em></span>, AKA
<span class="emphasis"><em>character set</em></span>, describes the way characters are encoded into bytes and vice versa.
- Famous standard encodings are &bdquo;UTF-8&rdquo; or &bdquo;ISO-8859-1&rdquo;. The list of available encodings can be determined
+ Famous standard encodings are &ldquo;UTF-8&rdquo; or &ldquo;ISO-8859-1&rdquo;. The list of available encodings can be determined
by calling
<code class="code">java.nio.Charset.availableCharsets()</code>. There is also a list of encodings and their
canonical Java names in the
@@ -160,7 +160,7 @@ <h3 class="title">
</div>
</div>
</div>
-<p>The source code of the Xtext framework itself is completely encoded in &bdquo;ISO 8859-1&rdquo;, which is necessary
+<p>The source code of the Xtext framework itself is completely encoded in &ldquo;ISO 8859-1&rdquo;, which is necessary
to make the Xpand templates work everywhere (they use french quotation markup). That encoding is hard
coded into the Xtext generator code. You are likely never going to change that. </p>
</div>
View
182 plugins/org.eclipse.xtext.doc/help/formatting.html
@@ -40,11 +40,13 @@ <h1 xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">Formatting (Pretty P
<div>
<div>
<h3 class="title">
-<a name="declarativeformatter"></a>Declarative Formatter </h3>
+<a name="declarativeformatter"></a>Declarative Formatter</h3>
</div>
</div>
</div>
-<p>A declarative formatter can be implemented by sub-classing {org.eclipse.xtext/src/org.eclipse.xtext.formatting.impl.AbstractDeclarativeFormatter}, as shown in the following example:</p>
+<p>A declarative formatter can be implemented by sub-classing
+ <a class="ulink" href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext/src/org/eclipse/xtext/formatting/impl/AbstractDeclarativeFormatter.java?root=Modeling_Project&view=co" target="_new">AbstractDeclarativeFormatter</a>, as shown in the following example:
+ </p>
<div class="literallayout">
<p>
<code class="code">public&nbsp;class&nbsp;ExampleFormatter&nbsp;extends&nbsp;AbstractDeclarativeFormatter&nbsp;{<br>
@@ -55,27 +57,39 @@ <h3 class="title">
&nbsp;&nbsp;&nbsp;&nbsp;ExampleLanguageGrammarAccess&nbsp;f&nbsp;=&nbsp;getGrammarAccess();<br>
&nbsp;&nbsp;&nbsp;&nbsp;<br>
&nbsp;&nbsp;&nbsp;&nbsp;c.setAutoLinewrap(120);<br>
+&nbsp;&nbsp;&nbsp;&nbsp;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;find&nbsp;common&nbsp;keywords&nbsp;an&nbsp;specify&nbsp;formatting&nbsp;for&nbsp;them<br>
+&nbsp;&nbsp;&nbsp;&nbsp;for&nbsp;(Pair&lt;Keyword,&nbsp;Keyword&gt;&nbsp;pair&nbsp;:&nbsp;f.findKeywordPairs("(",&nbsp;")"))&nbsp;{<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;c.setNoSpace().after(pair.getFirst());<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;c.setNoSpace().before(pair.getSecond());<br>
+&nbsp;&nbsp;&nbsp;&nbsp;}<br>
+&nbsp;&nbsp;&nbsp;&nbsp;for&nbsp;(Keyword&nbsp;comma&nbsp;:&nbsp;f.findKeywords(","))&nbsp;{<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;c.setNoSpace().before(comma);<br>
+&nbsp;&nbsp;&nbsp;&nbsp;}<br>
<br>
-&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;Line<br>
+&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;formatting&nbsp;for&nbsp;grammar&nbsp;rule&nbsp;Line<br>
&nbsp;&nbsp;&nbsp;&nbsp;c.setLinewrap(2).after(f.getLineAccess().getSemicolonKeyword_1());<br>
&nbsp;&nbsp;&nbsp;&nbsp;c.setNoSpace().before(f.getLineAccess().getSemicolonKeyword_1());<br>
&nbsp;&nbsp;&nbsp;&nbsp;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;TestIndentation<br>
-&nbsp;&nbsp;&nbsp;&nbsp;c.setIndentation(<br>
-&nbsp;&nbsp;&nbsp;&nbsp; f.getTestIndentationAccess().getLeftCurlyBracketKeyword_1(),<br>
+&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;formatting&nbsp;for&nbsp;grammar&nbsp;rule&nbsp;TestIndentation<br>
+&nbsp;&nbsp;&nbsp;&nbsp;c.setIndentationIncrement().after(<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f.getTestIndentationAccess().getLeftCurlyBracketKeyword_1());<br>
+&nbsp;&nbsp;&nbsp;&nbsp;c.setIndentationDecrement().before(<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f.getTestIndentationAccess().getRightCurlyBracketKeyword_3());<br>
&nbsp;&nbsp;&nbsp;&nbsp;c.setLinewrap().after(<br>
-&nbsp;&nbsp;&nbsp;&nbsp; f.getTestIndentationAccess().getLeftCurlyBracketKeyword_1());<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f.getTestIndentationAccess().getLeftCurlyBracketKeyword_1());<br>
&nbsp;&nbsp;&nbsp;&nbsp;c.setLinewrap().after(<br>
-&nbsp;&nbsp;&nbsp;&nbsp; f.getTestIndentationAccess().getRightCurlyBracketKeyword_3());<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f.getTestIndentationAccess().getRightCurlyBracketKeyword_3());<br>
&nbsp;&nbsp;&nbsp;&nbsp;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;Param<br>
+&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;formatting&nbsp;for&nbsp;grammar&nbsp;rule&nbsp;Param<br>
&nbsp;&nbsp;&nbsp;&nbsp;c.setNoLinewrap().around(f.getParamAccess().getColonKeyword_1());<br>
&nbsp;&nbsp;&nbsp;&nbsp;c.setNoSpace().around(f.getParamAccess().getColonKeyword_1());<br>
&nbsp;&nbsp;&nbsp;&nbsp;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;comments<br>
-&nbsp;&nbsp;&nbsp;&nbsp;c.setNoLinewrap().before(f.getSL_COMMENTRule());<br>
+&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;formatting&nbsp;for&nbsp;Comments&nbsp;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;cfg.setLinewrap(0,&nbsp;1,&nbsp;2).before(g.getSL_COMMENTRule());<br>
+&nbsp;&nbsp;&nbsp;&nbsp;cfg.setLinewrap(0,&nbsp;1,&nbsp;2).before(g.getML_COMMENTRule());<br>
+&nbsp;&nbsp;&nbsp;&nbsp;cfg.setLinewrap(0,&nbsp;1,&nbsp;1).after(g.getML_COMMENTRule());<br>
&nbsp;&nbsp;}<br>
}<br>
@@ -85,11 +99,11 @@ <h3 class="title">
</p>
</div>
<p>The formatter has to implement the method
- <code class="code">configureFormatting(...)</code> which is supposed to declaratively set up a
+ <code class="code">configureFormatting(...)</code> which declaratively sets up a
<a class="ulink" href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext/src/org/eclipse/xtext/formatting/impl/FormattingConfig.java?root=Modeling_Project&view=co" target="_new">FormattingConfig</a>.
</p>
<p>The
- <a class="ulink" href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext/src/org/eclipse/xtext/formatting/impl/FormattingConfig.java?root=Modeling_Project&view=co" target="_new">FormattingConfig</a> consist general settings and a set of rules:
+ <a class="ulink" href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext/src/org/eclipse/xtext/formatting/impl/FormattingConfig.java?root=Modeling_Project&view=co" target="_new">FormattingConfig</a> consist general settings and a set of formatting instructions:
</p>
<div class="section" title="General FormattingConfig Settings">
<div class="titlepage">
@@ -105,64 +119,45 @@ <h4 class="title">
<li class="listitem">
<p>
-<code class="code">setAutoLinewrap(int)</code> defines the amount of characters after which a line-break should be dynamically inserted between two tokens. The rule
- <code class="code">setNoLinewrap()</code> can be used to suppress this behavior locally. The default is 80.
- </p>
-</li>
-<li class="listitem">
-<p>
-
-<code class="code">setIndentationSpace(String)</code> defines the string which is used for a single degree of indentation. The default is two whitespace.
- </p>
-</li>
-<li class="listitem">
-<p>
-
-<code class="code">setWhitespaceRule(AbstractRule)</code> defines the grammar rule which is used to match whitespace. This is needed by the formatter to identify whitespace and to insert whitespace. The default is the rule named
- <code class="code">WS</code>.
- </p>
-</li>
-<li class="listitem">
-<p>
-
-<code class="code">setIndentation(start, end)</code> increases the level of indentation when the element
- <code class="code">start</code> is matched and decreases the level when element
- <code class="code">end</code> is matched. The matching of elements happens in the same way as it does for formatting rules.
+<code class="code">setAutoLinewrap(int)</code> defines the amount of characters after which a line-break should be dynamically inserted between two tokens. The instructions
+ <code class="code">setNoLinewrap().???()</code>,
+ <code class="code">setNoSpace().???()</code> and
+ <code class="code">setSpace(space).???()</code> suppress this behavior locally. The default is 80.
</p>
</li>
</ul>
</div>
</div>
-<div class="section" title="FormattingConfig Rules">
+<div class="section" title="FormattingConfig Instructions">
<div class="titlepage">
<div>
<div>
<h4 class="title">
-<a name="FormattingConfigRules"></a>FormattingConfig Rules</h4>
+<a name="FormattingConfigInstructions"></a>FormattingConfig Instructions</h4>
</div>
</div>
</div>
<p>Per default, the
- <a class="link" href="formatting.html#declarativeformatter" title="Declarative Formatter">Declarative Formatter</a> inserts one whitespace between two tokens. Rules can be used to specify a different behavior. They consist of two parts:
- <span class="emphasis"><em>When</em></span> to apply the rule and
+ <a class="link" href="formatting.html#declarativeformatter" title="Declarative Formatter">Declarative Formatter</a> inserts one whitespace between two tokens. Instructions can be used to specify a different behavior. They consist of two parts:
+ <span class="emphasis"><em>When</em></span> to apply the instruction and
<span class="emphasis"><em>what</em></span> to do.
</p>
<p>To understand
- <span class="emphasis"><em>when</em></span> a rule is applied think of a stream of tokens whereas each token is associated with the corresponding grammar element. The rules are matched against these grammar elements. The following matching criteria exist.
+ <span class="emphasis"><em>when</em></span> an instruction is applied think of a stream of tokens whereas each token is associated with the corresponding grammar element. The instructions are matched against these grammar elements. The following matching criteria exist:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" type="disc">
<li class="listitem">
<p>
-<code class="code">after(ele)</code>: The rule is executed after the grammar element
- <code class="code">ele</code> has been matched. For example, if your grammar uses the keyword &bdquo;;&rdquo; to end lines, this can instruct the formatter to insert a line-break after the semicolon.
+<code class="code">after(ele)</code>: The instruction is applied after the grammar element
+ <code class="code">ele</code> has been matched. For example, if your grammar uses the keyword &ldquo;;&rdquo; to end lines, this can instruct the formatter to insert a line break after the semicolon.
</p>
</li>
<li class="listitem">
<p>
-<code class="code">before(ele)</code>: The rule is executed before the matched element. For example, if your grammar contains lists which separate its values with keyword &bdquo;,&rdquo;, you can instruct the formatter to suppress the whitespace before the comma.
+<code class="code">before(ele)</code>: The instruction is executed before the matched element. For example, if your grammar contains lists which separate its values with keyword &ldquo;,&rdquo;, you can instruct the formatter to suppress the whitespace before the comma.
</p>
</li>
<li class="listitem">
@@ -178,15 +173,17 @@ <h4 class="title">
<code class="code">between(ele1, ele2)</code>: This matches if
<code class="code">ele2</code> directly follows
- <code class="code">ele1</code>. There may be no other elements in between.
+ <code class="code">ele1</code> in the document. There may be no other tokens in between
+ <code class="code">ele1</code> and
+ <code class="code">ele2</code>.
</p>
</li>
<li class="listitem">
<p>
<code class="code">bounds(ele1, ele2)</code>: This is the same as
- <code class="code">before(ele1)</code> combined with
- <code class="code">after(ele2)</code>.
+ <code class="code">after(ele1)</code> combined with
+ <code class="code">before(ele2)</code>.
</p>
</li>
<li class="listitem">
@@ -196,23 +193,39 @@ <h4 class="title">
<code class="code">ele1</code> is matched, and disabled when
<code class="code">ele2</code> is matched. Thereby, the rule is active for the complete region which is surrounded by
<code class="code">ele1</code> and
- <code class="code">ele2</code>.
+ <code class="code">ele2</code>.
</p>
</li>
</ul>
</div>
+<p>The term
+ <span class="emphasis"><em>tokens</em></span> is used slightly different here compared to the parser/lexer. Here, a token is a keyword or the string that is matched by a terminal rule, datatype rule or cross reference. In the terminology of the lexer a datatype rule can match a composition of multiple tokens.
+ </p>
<p>The parameter
<code class="code">ele</code> can be a grammar&rsquo;s
<code class="code">AbstractElement</code> or a grammar&rsquo;s
- <code class="code">AbstractRule</code>. However, only elements which represent a token in the textual representation can be matched. This are:
+ <code class="code">AbstractRule</code>. All grammar rules and almost all abstract elements can be matched. This includes rule calls, parser rules, groups and alternatives. The semantic of
+ <code class="code">before(ele)</code>,
+ <code class="code">after(ele)</code>, etc. for rule calls and parser rules is identical to when the parser would &ldquo;pass&rdquo; this part of the grammar. The stack of called rules is taken into account. The following abstract elements can
+ <span class="bold"><strong>not</strong></span> have assigned formatting instructions:
</p>
<div class="itemizedlist">
<ul class="itemizedlist" type="disc">
<li class="listitem">
-<p>terminal rules for comments.</p>
+<p>Actions. E.g.
+ <code class="code">{MyAction}</code> or
+ <code class="code">{MyAction.myFeature=current}</code>.
+ </p>
</li>
<li class="listitem">
-<p>keywords, assignments, call to terminal or data-type rules.</p>
+<p>Grammar elements nested in datatype rules. This is due to to the fact that tokens matched by a data type rule are treated as atomic by the serializer. To format the tokens themselfs, please implement a
+ <a class="link" href="valueconverter.html" title="Value Converter">ValueConverter</a>.
+ </p>
+</li>
+<li class="listitem">
+<p>Grammar elements nested in
+ <code class="code">CrossReferences</code>.
+ </p>
</li>
</ul>
</div>
@@ -222,13 +235,38 @@ <h4 class="title">
<li class="listitem">
<p>
-<code class="code">setLinewrap()</code>: Inserts a line-break at this position.
+<code class="code">setIndentationIncrement()</code> increments indentation by one unit at this position. Whether one unit consists of one tab-character or spaces is defined by
+ <a class="ulink" href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/plugins/org.eclipse.xtext/src/org/eclipse/xtext/formatting/IIndentationInformation.java?root=Modeling_Project&view=co" target="_new">IIndentationInformation</a>. The default implementation consults Eclipse&rsquo;s
+ <code class="code">PreferenceStore</code>.
+ </p>
+</li>
+<li class="listitem">
+<p>
+
+<code class="code">setIndentationDecrement()</code> decrements indentation by one unit.
+ </p>
+</li>
+<li class="listitem">
+<p>
+
+<code class="code">setLinewrap()</code>: Inserts a line-wrap at this position.
</p>
</li>
<li class="listitem">
<p>
-<code class="code">setLinewrap(int)</code>: Inserts the specified number of line-breaks at this position.
+<code class="code">setLinewrap(int count)</code>: Inserts
+ <code class="code">count</code> numbers of line-wrap at this position.
+ </p>
+</li>
+<li class="listitem">
+<p>
+
+<code class="code">setLinewrap(int min, int default, int max)</code>: If the amount of line-wraps that have been at this position before formatting can be determined (i.g. when a node model is present), then the amount of of line-wraps is adjusted to be within the interval [
+ <code class="code">min</code>,
+ <code class="code">max</code>] and is then reused. In all other cases
+ <code class="code">default</code> line-wraps are inserted. Example:
+ <code class="code">setLinewrap(0, 0, 1)</code> will preserve existing line-wraps, but won&rsquo;t allow more than one line-wrap between two tokens.
</p>
</li>
<li class="listitem">
@@ -240,12 +278,50 @@ <h4 class="title">
<li class="listitem">
<p>
+<code