UCOSPLogWarrenMarivel
Each UCOSP participant is asked to log all the difficulties they have getting set up to understand Umple. This includes understanding how to model in Umple, using UmpleOnline, code that is difficult to figure out, and development tools that are hard to figure out, or don't seem to work properly. The more log entries added the more we can better get a grip on the usability issues with Umple, and the more we can help future new contributors come up to speed.
April 2nd
left to do:
- initial tags and attributes
- write reference doc on scxml generator and importer
I had a bit of trouble with getting StateComparator to properly implement Comparator in Umple, but once I had this figured out translating the <initial> tag was easy. I just had to make sure that I defined the interface as external and used the isA statement.
The reference doc can be found here: https://code.google.com/p/umple/wiki/ScxmlImplementation?ts=1428016369&updated=ScxmlImplementation
Continuing my work on the umple importer.
To do today:
- file a bug reporting SAX Parser issue
- write the corresponding onentry/onexit faulty test and comment them out
- Make test that passes and ship actions
Turns out that the characters method issue is not a bug. Parameter ch always contains the full xml. You have to use start and length to locate the characters.
In the documentation: “The application must not attempt to read from the array outside of the specified range.”
Should be more careful while reading the documentation next time.
March 25th
Continuing my work on the umple importer.
What’s left to do:
- onentry
- onexit
- figure out what to do with executable content
- transition / state machine actions
Executable content:
Only parse <script> elements for now.
Looks like SAX Parser doesn’t consider text elements as xml elements. This is problematic because I need a way to get the children of <script> elements. Maybe they are somewhere in the attributes.
SAX parser’s DefaultHandler class has a character’s method we can override to get the content inside of tags.
According to this post, the characters method will read the content in chunks, so I’ll need to set up a buffer for reading the content of <script> tags.
Looks like the characters method is buggy If I write anything else than something like:
<script>
doSomething
</script>
In between script tags, the characters methods will be called on the whole scxml document, which causes the generated umple code to have the whole scxml source code inside the action.
Continuing my work on the umple importer.
I’ll need the importer to support nested states. However, the way the SAX parser works is by calling methods on certain events (start of document, start of new element, end of element ... ). This means I cannot just recursively go through the elements when encountering nested states.
Using a stack to save mimic the recursion solves this:
- On start of new state I push it on the stack
- n end of element, if it is a state I pop it out of the stack
- n start of any new element that’s known to be in a state, I can just peek in the stack to know which state element it belongs to, and add it accordingly.
I may have to extend the stack to accept objects other than states, but this is an easy refactoring. For now I’ll simplify things and only have it accept states.
Having trouble with indentation. Looks like I’ll have to keep track of how far deep am at all times.
To track the depth, I created a new abstract class UmpleImportSMElement that inherits from UmpleImportElement and contains a “depth” attribute. All state machines element inherit from this new class. I can now keep track of the depth in SCXMLImportHandler and pass it to elements so they generate code with the correct indentation.
I indeed had to change the stack to support different kinds of elements. As a consequence, I also refactored UmpleImportSMElement to have a list of nested elements. The code for generating stateMachine elements and state elements is now exactly the same, so I moved the generateUmple implementation to UmpleImportSMElement.
Now that we have states implemented, the next step is to work on transitions.
transitions are fairly simple to implement, with the exception that scxml transitions accept children element in the form of executable content. For now, I will ignore those elements. In the future, I will have to take that executable content and add it in appropriate places in the generated umple code.
Today I’ll get the foundation going for the importer.
TODO:
- make umple differentiate between xmi and scxml
- build the classes needed for the importer
- get a simple empty result going
To differentiate between xmi and scxml, I just added a condition that checks for filename’s extension. I kept it as simple and unintrusive so that it doesn’t break what’s already there: it will only consider the file as scxml if it ends in “.scxml”. All other files are treated the same way as before.
I also created the new SCXMLImportHanlder class
I’ll start working on the importer.
Looks like I’ll need to implement something like EcoreImportHandler that extends DefaultHandler.
Found this tutorial about sax parser.
Problem seems to be coming from StateMachine_Code.ump, static method createAutoTransition() in Event class. The counter in the method is not being reset after each test. I created the method setNextAutoTransitionId so that I can use it to reset the transition Id. This way the test will have a determined output and work in all cases.
Today I will look at auto transitions.
auto transitions:
auto transitions are used after “do” activities. We’ll need to support them to get auto transitions to work.
Do activities are spawned in a new thread when java code is generated. However, I generated a state machine with do activities in Php. In this case, the do activity is treated as an entry action. I’ll take this approach to get do activities in scxml.
Scxml documents have a <raise> tag that can be used to send events in the state machine. This will be useful to implement auto transitions.
Issue on tests:
Only run the scxml tests is fine
Run the full tests: expected autotransition1 but was autotransition3
Today I implemented having script code inside scxml documents. Since right now, we take the Java code and put it inside the scxml, I didn’t make the test check for the correctness of the code, but rather that it is picked up and placed in the correct spot.
Today I’ll look at the importer stuff and continue working on the code generator.
Looking at the code, I spotted the following classes: EcoreImportHandler UmpleImportModel UmpleConsoleMain
preModelOptionProcess in UmpleConsoleMain. We assume that we will be given xmi. Need to change this to check for xmi vs scxml files/input. will need to parse scxml the same way we’re parsing Ecore (EcoreImportHandler)
Today I’ll work on:
- take care of code outside of states (data, methods etc…)
- look at timed and auto transitions
code outside of state:
It’s actually code completely outside of the state machine. For example
In the first example in this link, all methods used in actions are defined outside of the state machine itself.
My idea to support this kind of code is to have all those methods in a <script></script> tag at the end of the scxml document. To make sure that this will work, I will create a small scxml document with a script in it defining simple functions and use my scxml implementation to see if it actually runs it.
I tested the scxml file I created and it seems to be ok: functions defined outside of states can be called by actions. I will go ahead with this approach then.
I’ve been having an issue since yesterday with a certain class not being recognized as a type. After a quick look in google, I found the solution: in eclipse, go to project -> clean and that fixes the issue.
What’s left to do:
- Actions and do Activities - Decided to keep java code for now. When we have the code, we can later make a converter to javascript and translate the code this way. Doesn’t look like do activities can be supported by scxml. Only entry and exit actions.
- Auto transitions and Timed transitions - Need to see how it’s represented in code. We can raise events in scxml, so this may help.
- Events with parameters
- Queued/Pooled state machines
Today I tried to set the guards in SCXML guard.getCondition needs a code translator. We don’t have a translator for any ECMAScript language. Will need to look into it and possibly start implementing one for javascript to support guards. Will also be useful for other things as well.
In order to implement guards, I will need to implement a new code generator that implements CodeTranslator interface.
Did some cleaning on the generated SCXML:
corrected some syntax mistakes to make the scxml runnable
now empty tags will be like this <tag/> instead of <tag></tag>
Looking for an implementation of scxml so that I can feed it the code I generate and see how it behaves.
It will also be good to look at what these implementations expect in terms of more advanced features that require ECMAScript capabilities so that I have a better idea of where I should be going with the data model and how to convert the advanced UMPLE state machine features into SCXML.
http://commons.apache.org/proper/commons-scxml/
https://github.com/jroxendal/PySCXML
PySCXML was easy to install and claims to be highly standard compliant, so I decided to go with it.
Issue I need to solve: The tests I write don’t clean up after themselves (leaving .scxml files after them). I need to find out why this is happening.
Going to look at other tests and see what they’re doing that I’m not.
Found the issue, I wasn’t calling SampleFileWriter.destroy() when tearing down the tests. I added the necessary calls.
The garbage .scxml files were commited to the repository, when I try to make a patch to delete the patch only modifies the file. Need to find a way to add the deletion to the patch.
Apparently patches don’t delete files, I’ll need to ask Andrew to delete them before committing the patch.
Back to Issue 653 (on Google Code): Trying to find where it is being labelled as a standAloneTransition so I can put a test here and generate an error. Can’t see where this standAloneTransition is being created in code.
There is no explicit new Token(“standAloneTransition” which makes it hard to trace. My intuition is that the .grammar files get loaded somewhere and used to generate the tokens, but I don’t know where exactly.
Looking at the umple_state_machine.grammar file, I can see what a valid standAloneTransition is. Let me try to modify the expression and see what impact it has on the code generator.
Modified the standAloneTransition: standAloneTransition : nothing;
Ran the buggy example against the new grammar and instead of the compiler hanging I got the following error: State machine syntax could not be processed. It has been considered as Extra Code.
We might want to output a more meaningful error, I’ll try to look for a syntax error message and see where it is generated so I have a better idea of where to modify code.
code 1502: bad class definition A search for “messageNumber=1502;” pointed me to Parser.java
Also looked for the error message State machine syntax could not be processed and found the related 1006 error code. Searching for this error code pointed me to UmpleInternalParser.java:
else if(sub.is("malformedStatemachine1")||sub.is("malformedStatemachine2"))
fromState in umple grammar file seems to be what we should modify. In grammar files, ~ is used to match alphanumeric characters only. I will try this as a tentative fix.
Today I will implement nested states and add the external/internal option to <transition>.
After this I will look into scxml data models and how to add scripting capabilities so that I can go ahead with implementing transitions with guards.
While looking at how to get nested states inside a state, I found that states have a method getNestedStateMachines that will give you a list stateMachine objects with nested states. However, reading on nested state machines in the wiki, it looks like you will always only have one nested state machine inside a state. The umple syntax looks like this:
sm {
s1 {
e1 -> s2; e2 -> s2b;
} s2 {
e1 -> s1; s2a {
e3 -> s2b;
} s2b {
e3 -> s2a;
}
} }
With s2a and s2b being the nested states. We don’t even define a state machine in the parent state, we just have to define the substates directly. That leads me to think that, inside a State object, the substates would be contained as a list of State object. However, they are wrapped inside a StateMachine object, which itself is contained inside a list. It looks unnecessary to me since, according to the umple syntax, we really only have nested states inside a parent state. It may have been implemented this way because it serves a purpose that I do not know of. However, in my case, I will just assume that there can only be substates in a state, since scxml doesn’t allow statemachine tags inside state tags. This means that, for each State object, I will just cycle through its list of nested StateMachine objects and generate scxml for the substates.
Looking back at the type (internal/external) option in transitions, it doesn’t look like we ever set it to “internal” in Umple. I may have mistaken the isInternal property inside the Transition class for something it is not. hat property is present in many other classes. I have to dig deeper to see what it’s about.
A quick look at in umple.ump show this:
“Boolean isInternal = false; // for code generation only e.g. ouside scope of nested state”
isInternal has nothing to do with the internal/external type option for transitions. In conclusion, I will leave it blank and it will default to external.
Looking at the data models in SCXML, it looks like compliant SCXML processors will only accept ECMAScript languages for the ECMAScript data model. Many of UMPLE state machine features rely on Java code that cannot be incorporated into SCXML.
While I get more clarification on how to approach this, I will go in the issues and look for a bug I can work on.
Going to tackle bug 653 (on Google Code):
Tried running example code java -jar umple.jar 653.ump and it does take up 100% CPU.
Set up test and tried debugging to see where it’s coming from:
UmpleInternalParser->analyzeStateMachine->populateStateMachine->standAloneTransition
Looks like the transition is being being incorrectly labelled as a stand alone transition.
Next step now is to implement simple transition declarations in states. We need to look at how they are declared in the umple grammar, and how we will translate them to scxml.
Transitions in umple are declared the following way:
transition -> newState
Transitions in umple may contain transition actions as well as guards.
In SCXML, <state> elements allow <transition> elements inside of them, and it is how transitions are defined. <transitions> contain the following optional attributes:
- event: the list of events, separated by spaces, that trigger this transition.
- cond: the guard condition for this transition.
- target: the target state.
- type: determines whether the source state is exited in transitions whose target state is a descendant of the source state.
Apart from transition actions, which I am not sure of yet, all the possible constructs present in umple transitions are also supported by scxml.
First step is to write an umple file with a simple transition, the corresponding scxml file and write a test that evaluates them. Once we have this simple transition implement, we can focus on supporting guards and external vs. internal transitions.
While writing my second test, I realized that, other than my own judgement, there is no way to automatically validate the scxml that I write for the tests. For now I will be looking for an online scxml parser to validate the scxml I write for the tests. Integrating an scxml validator in umple that can be used for unit testing might be a good project in the future. Turns out there aren’t any good validators online. Will trust on my own judgement for now then and be extra careful about the scxml I write.
Now that simple transition is implemented, we can look at special cases like guards and internal transitions. According to scxml, the default type for a transition is set to “external”, so I will only set the type attribute if the transition is internal ie: transition.isIsInternal == true
For guards, I’ll leave it empty for now. Looks like it’s related to being able to have executable code in the scxml as well as a data model, which we don’t have yet.
I’m trying to find a way to set a transition type to internal in umple, but there is no documentation about it. I checked where Transition:setIsInternal is used, and the only place where it is used to set isInternal to true in in GeneratorHelper .java -> prepareNestedStateMachine.
It might be better to implement nested state machines next, then come back to internal vs. external transitions.
I started writing the first test as well as first step towards generating scxml. The first test is just a simple statemachine with one state and no transitions. I wrote the first test and it is failing, now let’s go to the implementation. Reading the grammar on statemachines, it looks like we can define a statemachine the following way:
statemachine {
/*statemachine’s content*/
}
However, it doesn’t seem to be the way code is generated. I looked at the code generator for Papyrus XMI and it always looks for a class at the top level. Sure enough, if I try to generate a statemachine that’s not inside a class I get null as an output. Moreover, all statemachine examples in umple online are always contained inside a class. So for this first step I’ll follow that rule and make sure I define my state machine inside a class when testing.
It took me some time to get used to working with umple templates, but they ended being useful and helped me cut down on some code.
One issue I had while developing was that I wanted to see my changes in umpleonline as I made changes. However, the only way I could get my changes into umple online was by first doing a full build, then copying umplesync.jar and umple.jar to umpleonline/scripts. It was not convenient since a full build takes time and I didn't want to have to copy the jars every time. Looking at build.xml, I was able to get the options I wanted for building the appropriate jars and moving them to umpleonline:
ant -Dmyenv=local -f build.umple.xml umpleSelf compile packageJars
ant -DshouldPackageUmpleOnline=true -Dmyenv=local -f build.umple.xml packageUmpleonline
I added both commands as an alias, and now I can easily see my changes through umpleonline as I make them.
Today I kept looking at what scxml is and what its features are. I was able to get a better grasp on how I would go about implementing the code generator by drawing parallels between scxml and how state machines are described in umple.
For example if I want to define a transition in umple I would write something like this:
eventA -> targetA
But in scxml, the same can be achieved with the following:
<transition event=”eventA” target=”targetA”>
I had some trouble figuring out how I would go about integrating the scxml w3c test suite into umple's unit tests. In order to have a better idea of how I would do this, I decided to look at existing scxml implementations.
One of those implementations is uscxml. Looking at their unit tests, I realized that all they did was run the scxml files provided by w3c and evaluate whether the state machine landed on the pass state. It then became clear to me that this test suite were conformance tests for scxml processors. They were not tests that validated scxml documents. Going back to the scxml specification document, I found this section that describe what was necessary for a document to be considered valid scxml.
Since I do not rely on the test suite anymore, I have to change my approach to implement scxml code generation. Instead of having all the tests laid out in advance, then improving the code generator until all of them pass, I will now make small incremental changes to implement all umple state machines features into the equivalent valid scxml code. This implies that I will have to make my own test suite as I keep improving the code generator.
The first patch for issue 551 (on Google Code) was added to the project. Umple is now properly generating code for keys inherited from a parent class. Now I have to take care of the warnings generated when a key member is not defined in a class, but still inherited from a parent class. The warnings are generated in UmpleInternalParser_CodeClass.ump.
While debugging in the tests I set up before, I noticed that, when keys are analyzed in UmpleInternalParser.analyzeKey, the class object the key belongs to has no extends property despite having a isA statement in the umple code. Therefore I cannot get the inherited attributes at this point in the code and use this to avoid generating the warning. I have yet to find a solution to this issue.
After discussing my semester project with Andrew, this is the plan we put together:
SCXML
Generate a hello-world parser that simply generates
For "any" model.
Add a dropdown to Umpleonline to support SCXML output, with a note saying "Experimental"
Investigate testing against w3c standards. http://www.w3.org/Voice/2013/scxml-irp/
Integrate tests into "cruise.umple.implementation.scxml"
ALLOW "FAILING" in the actual Junit tests
w3c standards test ==> Assert all the "result" (0/40)
@Test public void testNestedSTates() {}
@Test public void testGuards() {}
@Test public void testW3cSTandard() {
int numPassing = 0 numPassing += runW3CTest355() ; 1 if passing, 0 else numPassing += runW3CTest355() numPassing += runW3CTest355() numPassing += runW3CTest355() numPassing += runW3CTest355() numPassing += runW3CTest355() numPassing += runW3CTest355() numPassing += runW3CTest355() numPassing += runW3CTest355() numPassing += runW3CTest355()
Assert.assertEquals(0, numPassing); }
Now, improve the XML generation.
Andrew helped me with setting up the boiler plate and also added the SCXML option in umple online, so what I have to do next is get all the validation tests from w3c in umple. Once they are all there, keep improving the code generation for SCXML until they all pass.
Issue 551 (on Google Code)
Reproduced the example and I tried to look for a failing test I could add to start debugging. I tried to assertParse on the example at first, but the test wouldn't fail. It's because assertParse will not fail on warnings, and the bug only issues a warning error level. After looking at other tests involving keys, I found the assertion I would have to use to create the right test: assertNoWarningsParse. Once I had the test ready, I started looking at the place where the warning is generated and disabled it to see if not issuing the warning is enough to solve the bug. It turns out that, even if the warning is not issued, the generated code is still incorrect.
With the help of Andrew, I set up new tests asserting the correctness of the code generated and used them as a starting debug point. After a good time spent debugging and looking around the code to understand what it's doing, I noticed that in JavaGenerator, the inherited attribute is not present when preparing the model's classes. With the help of Andrew, I set up a new method UmpleClass::getAllAttributes that would also fetch inherited ones, along with test cases for them. The tests are now in place, but are still failing because getAllAttributes is not implemented yet. I implemented the method and tried to use it in JavaGenerator to fix the bug, but the code still wasn't generating right. I concluded that I was not looking at the right place and kept looking at where the code for keys was being generated. After a bit of digging, I arrived at JavaClassGenerator::getEqualsCode and noticed that it was using UmpleClass::getAttribute to generate the keys equals code. I modified getAttribute to use UmpleClass::getAllAttributes instead of the attributes member variable.
Summary of bug fix:
- Create method UmpleClass::getAllAttributes to get the inherited attributes as well.
- Where the warning is generated, create check to ensure that attribute is inherited, and don’t issue warning if it is.
- In UmpleClass::getAttribute, use UmpleClass::getAllAttributes instead of member variable.
I don't have the full patch ready yet. I have yet to create a proper check for inherited attributes where the warning is being generated. Since this will probably make use of UmpleClass::getAllAttributes, I will have to deploy this part of the bug in another patch, after that method was integrated in the umple compiler.
I also discussed with Andrew about what I wanted to do for this semester. I will be working on issue 639 (on Google Code): Implement SCXML in Umple
Andrew helped me set up a local cruise control server so I can have a better understanding of the build results. He then went over the UMPLE project in detail and we got an introduction of its architecture. I also started to look at Issue 551 (on Google Code).
I was able to set up and have umple compile. All the tests are passing now. I ran into the following issues while attempting to get it to compile:
- While doing a full build for the first time:
[taskdef]Could not load definitions from resource net/sf/antcontrib/antcontrib.properties. It could not be found. After a quick google search, I found this link and was able to solve the problem by downloading the ant-contrib jar and copying it to /usr/local/Cellar/ant/1.9.4/libexec/lib/ - While running the tests on eclipse, I noticed that only one test was failing: executeSimpleAntScript. After a good amount of time trying to figure out what was going wrong, I noticed that if I replaced the command "ant" with its full path in the test, it would run properly. I concluded that eclipse's Java runtime did not have PATH set properly. A friend then pointed me to this link and setting the path in eclipse solved the issue.
Progress on the checklist:
- Write a brief welcome message to umple-dev@googlegroups.com.
Done.
- Re-create a model from a past assignment (big or small) in umpleonline and paste the saved URL in your log
- Run a full build (required or creating a catch), all tests should pass, how many are there
3866 tests, 0 failed, 199 skipped
- Add a test to umple project (any test), and create a valid patch (please send to aforward@gmail.com)
Done.
- Load the umple projects into eclipse, and provide a screenshot in your log
- Change the welcome comment in the generated Java code (UmpleToJava? project), create a new version of Umple, re-generate your code from #1 to observe the changes
Have yet to try this.
- Find an easy bug in bugs.umple.org, resolve it and send a patch
I had a quick look at the different bug, but I have not started working on one yet.