UCOSPLogJohnZwiep

TCLethbridge edited this page Aug 29, 2015 · 2 revisions
Clone this wiki locally

Each UCOSP participant is asked to log all the difficulties they have getting set up to understand Umple. This includes understanding how to model in Umple, using UmpleOnline, code that is difficult to figure out, and development tools that are hard to figure out, or don't seem to work properly. The more log entries added the more we can better get a grip on the usability issues with Umple, and the more we can help future new contributors come up to speed.

Log Entries in reverse chronological order

12 April 2015

  • Modified umplifyonline, and the glue scripts with the new directory format (<org>/<proj>/<branch>/<ver>/<src>)
  • Updated the associated docs
  • Refactored dlproj and its tests

11 April 2015

  • Worked on dlproj
    • Finished implementing the 'versions' funtionality: dlproj now downloads the first X versions of a repository (where 'x' can be specified when calling 'download()')
    • Wrote tests to cover the new functionality
    • Changed the directory structure of the downloaded projects to accommodate the new versions: <org>/<proj>/<branch>/<version>/src
    • My next steps:
      • Refactor
      • Change the build process to 'bake-in' dlproj's dependencies (for portability)
      • Help Andrew deploy dlproj and the Umplificator to production

7 April 2015

  • Worked on dlproj
    • Fixed a couple issues:
      • The ant build script wasn't fully cleaning out files which had be causing test failures
      • Pushed an old change the destination directory so we could separate source files from metadata
      • Refactored the tests to improve readability: I had previously put all variables in a shared 'helper' class, but that made it difficult to see what was actually being tested. I moved them back out at Andrew's suggestion
      • Added a real test project (we were just using one of Kevin's before) so that we can add files, commits, and tags.
    • Added support for multi-version, but haven't tested it yet. I needed to deal with the above issues before I could start writing tests for them, but getting multi-versions of the same project will go roughly like this:
      • Clone the project
      • Get all version tags and pick the first X
      • For each of the X version tags
        • Create a directory <org>/<proj>/X/src
        • Copy the project repo into this new directory
        • Update the copied repo to the commit id specified by the tag

6 April 2015

  • Worked on dlproj:
    • Looked into modifying dlproj to further split the repo into separate version. Essentially: clone the repository, get all the available tags, 'split off' each of the first 5 or so tags
    • Other things to consider:
      • Repos with no code/tags
      • 'Updating' a set of projects

5 April 2015

26 March 2015

Hi John,

I have added an option where you can specify that generation of config file is not required.

tracer log4j noconfig;

With this, log4j xml file will not be generated and overwrite your existing config file.

<------------------------>

  • Tracer feature request
    • For the Umplificator project we needed more options for logging with log4j than the 'tracer' feature allowed. We wanted debug level file logging, but we needed it to use a rollingfile appender so that each individual file didn't get too large (some runs the log file was 2GB+). Looking through the Umple docs I couldn't see any way to get that through 'tracer'. Instead I had tracer generate a 'dummy' log4j config file, and then used my own when packaging the application.
    • This is the tracer line tracer log4j root=debug (info)=(file, console);
    • This is the log4j file it generates
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
  <Appenders>
    <Console name="console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
    </Console>
    <File name="file" fileName="logs/trace.log" immediateFlush="false" append="false">
      <PatternLayout pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
    </File>
  </Appenders>
  <Loggers>
    <Root level="debug">
      <AppenderRef ref="file" level="info"/>
      <AppenderRef ref="console" level="info"/>
    </Root>
  </Loggers>
</Configuration>
* This is the configuration we wanted
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
  <Appenders>
    <Console name="console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
    </Console>
    <RollingFile name="trace" fileName="logs/trace.log" filePattern="logs/trace.log.%i" immediateFlush="false" append="false">
      <PatternLayout pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="50 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="3"/>
    </RollingFile>
    <File name="error" fileName="logs/error.log" immediateFlush="false" append="false">
      <PatternLayout pattern="%d{yyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
    </File>
  </Appenders>
  <Loggers>
    <Root level="debug">
      <AppenderRef ref="error" level="warn"/>
      <AppenderRef ref="trace" level="debug"/>
      <AppenderRef ref="console" level="info"/>
    </Root>
  </Loggers>
</Configuration>
* I'd love the ability to specify more logging options from the tracer statement (i.e. `tracer log4j root=debug (info)=(rollingFile:3:50, console);`) or even better, have the option to turn off log4j file generation and just use my own (so that I can keep my log config file in the src dir without Umple overwriting it)
  * To expand on this, I'd like to use a command similar to this:
tracer log4j:noconfig;
    * Specifies that I'm using my own log4j config file and that Umple shouldn't try to generate one for me
    * Makes the changes specific to log4j
    * Allow users full control over how the log appenders are handled
    * Avoids muddying up the syntax for others who don't use log4j tracers
  * Or, if we _must_ generate the file through umple, something like this:
tracer log4j root=debug (info)=(rollingFile:3:50, console);
    * Read as: rollingFile up to a max of 3 rollover files 50MB in size each
    * This satisfies my exact needs, but could lead to a lot of clutter if we were to support all of log4j's options

22 March 2015

  • Worked on the umplificator (Issue 688 (on Google Code))
    • Fixed a few issues with the umplificator score script
      • Log generating and copying to capture stderr messages that would have otherwise not been logged (i.e. unhandled exceptions thrown by the Umplificator)
      • Fixed the file test to skip projects that have already been umplified
    • Added a check for projects with no valid source files. The umplificator now outputs an error message and exits with 1

21 March 2015

  • Worked on the umplificator (Issue 687 (on Google Code))
    • Looked into using log4j tracers in more complex ways (i.e. rolling file appenders). The online docs are sparse, but from the looks of it only the simplest log4j2.xml config files are generated by the tracer.
      • Disabling the tracer in Master.ump ended up breaking logging, as it looks like the tracer is inserting the log4j trace points itself. To solve this I simply left the tracer in, but ignore the log4j2.xml file it outputs, instead I use one I've written myself
    • Changed the log level of the console output to INFO as to not spam stdout
    • Changed the original file appender to rolling appender. The idea being to log enough to give context to any errors, but not enough that each project generates GBs of logs
    • Added a WARN and above logger to better isolate errors
    • Added stderr redirection to the umplificator score script in order to log any uncaught exceptions
    • Cleaned up some of the log messages
    • Added exception handling to catch a common NullPointerException I've been seeing

19 March 2015

  • Worked on dlproj
    • To separate the source and 'meta' files in each downloaded projects, I changed dlproj and the score script to both use /{org}/{proj}/{ver}/src as the source directory.
    • Tested and pushed my changes for dlproj, and tested and updated the UmplifyProjectList wiki with my changes to the score script
  • Worked on Issue686 (on Google Code)
    • Added a 'P' for pending score to denote that the umplificator is still being run against a project
    • Looked into adding an 'E' score to denote unexpected umplificator failures (to distinguish it from an expected failure) but it doesn't look like the umplificator makes any distinction between them
    • Fixed an issue where the script was halting on first failure instead of continuing on with other projects
    • Tested and submitted a patch with these changes to Andrew

17 March 2015

  • Helped Andrew deploy the umplificator score script to umplifyonline
  • Planned the next steps for the project with Andrew's input

16 March 2015

  • Looked into how logging was done in the Umplificator
    • To make the logs useful, a few changes need to be made (as the logs after umplifying a project were a single text file 2.5GB in size)
      • Add a rolling file appending to the existing logs
      • Ensure that logs are created on a per-project basis vs. a central set of logs
      • Console output should show only WARN and above
      • Separate file for ERROR and above messages (for easy parsing)
    • Also, since the log4j config file is packaged on build, it might be nice to include a verbosity option from the command line

10 March 2015

  • Worked on umplifyonline
    • Made a few changes to the umplify all projects script
    • Still needs a couple of things before it's useful
      • Umplificator stdout is still too verbose, I'll talk to Andrew/Miguel to see if I can add a flag to change the default log level to higher than DEBUG
      • The script should do something useful with the stdout and stderr, that way we can look at why projects are failing at certain levels
      • The script should have an option to simply ignore existing umplify files. If the option is used, then the script tries umplifying each project regardless of previous attempts. This would be useful when trying out new versions of the umplificator
      • Better progress status output. Just something so we know the script isn't hanging

8 March 2015

  • Worked on umplifyonline
    • Wrote a script to crawl the project directory and umplify all un-umplified projects. For now I'm keeping it on the UmplifyProjectList page. It's 90% finished, it just needs to make the actual call to the umplifier.
    • Using Miguel's feedback (and changes), I'm trying the umplificator again, hopefully getting umplify successes to test my script with.

4 March 2015

  • Worked on umplifyonline
    • I designed the first pass at reporting umplifier scores through the file system. I'm going to do it with an empty file at the root level of the specific project version (i.e. <umplify_dir>/<org>/<project>/<version>/<myfile>). Where <myfile> will be the following:
      • (Doesn't Exist) -- If the umplifier hasn't even been run on that version yet
      • F.Umplify.Score -- If the umplifier failed to even umplify the version to level 0
      • [012].Umplify.Score -- If the umplifier succeeds on a specific level, a file starting with that level number (a project can potentially have all 3 levels and therefor can have 3 of these files)
    • Added 'getUmplifierScore' helper method to the project model
    • Added tests and test data for the new method
    • Added and populated a new data column in umplify online to display the umplifier score

2 March 2015

  • Worked on umplifyonline
    • Updated the project's README

1 March 2015

  • Worked on umpifyonline
    • Refactored my unit tests for Project.php
    • Added a test data set
    • I'm considering generating the test data set at runtime versus keeping them in the repo (waiting to hear back from Andrew)
    • I'm also going to add a couple short notes in the project's readme about how to use umplifyonline and how to run the test cases for it.

25 February 2015

  • Worked on umplifyonline
    • Added methods to Project.ump to iterate over projects and versions
    • Added tests for the new methods
    • Looking into reporting Umplifier results to umplifyonline via the filesystem
  • Meeting with SFU's UCOSP coordinator today to talk about semester progress

18 February 2015

  • Worked on umplifyonline
    • Updated my environment PHP version to 5.5 to support the PHP webserver
    • Cloned a local copy of umplify online
    • Started working on populating the project and version columns of the project list (an extension of Andrew's work). I'm able to directory crawl for that information.

l7 February 2015

15 February 2015

  • Compiled a list of 100 java projects from GitHub. I just took the top 100 starred projects, removing any 'projects' that weren't actually code. Otherwise I haven't curated the list.
  • Wrote a glue script in bash to download each of the 100 projects using dlproj jar. Tested it in my local setup. I'll commit it once I've cleaned up the output a little bit
  • Looking into UmplificatorUsage for my second task.
    • Built the Umplificator and Umplevalidator locally with ant -Dmyenv=local -f build.umple.xml build-umplificator
    • Ran the Umplificator against a recent copy of JHotDraw (a project that has passed all 3 levels in the past). It ran for about 40min before eventually hitting a null pointer exception:
2015-02-15 19:37:33.586 [main] DEBUG cruise.umplificator.FileHelper - --UmpleClass has been created for./output/AbstractLineDecoration.ump--
2015-02-15 19:37:33.586 [main] DEBUG cruise.umplificator.FileHelper - --UmpleClass has been created for./output/BorderRectangleFigure.ump--
cruise.umple.compiler.exceptions.UmpleCompilerException: There was a problem with generating UmpleSelf code.java.lang.NullPointerException
    at cruise.umplificator.FileHelper.writeFile(FileHelper.java:135)
    at cruise.umplificator.core.Umplificator.umplify(Umplificator.java:250)
    at cruise.umplificator.UmplificatorMain.main(UmplificatorMain.java:80)
Caused by: java.lang.NullPointerException
    at cruise.umplificator.formatter.UmpleCodeFormatter.beautifyCode(UmpleCodeFormatter.java:57)
    at cruise.umplificator.generator.UmpleToUmpleGenerator.getUmpleClassAllCode(UmpleToUmpleGenerator.java:144)
    at cruise.umplificator.FileHelper.writeFile(FileHelper.java:121)
    ... 2 more
* I've found out the basics of running it, and the expected returns on success and failure. I'll dig into the null pointer exception a little bit before moving on.
* Tried another project, Netflix/Hystrix, on levels 0 and 2
  * Level 2 I got an OutOfMemoryError
2015-02-15 20:50:00.482 [main] DEBUG cruise.umplificator.rules.RuleRunner - RuleRunner.insertUmpleClass - Insert uClass into working memory
2015-02-15 20:50:00.482 [main] DEBUG cruise.umplificator.rules.RuleRunner - RuleRunner.insertJavaElements - Insert Java elements into working memory
2015-02-15 20:50:00.484 [main] DEBUG cruise.umplificator.rules.RuleRunner - RuleRunner.fireAllRules - Fire rules
2015-02-15 20:50:00.484 [main] DEBUG cruise.umplificator.rules.RuleRunner - RuleRunner.fireAllRules - Fire rules LEVEL3
2015-02-15 20:50:00.484 [main] DEBUG cruise.umplificator.rules.RuleRunner - RuleRunner.fireAllRules - Fire rules LEVEL2
2015-02-15 20:50:00.484 [main] DEBUG cruise.umplificator.rules.RuleRunner - RuleRunner.fireAllRules - Fire rules LEVEL2.1
2015-02-15 20:50:00.484 [main] DEBUG cruise.umplificator.rules.RuleRunner - RuleRunner.fireAllRules - Fire rules LEVEL1
java.lang.OutOfMemoryError: GC overhead limit exceeded
    at org.drools.core.reteoo.EvalConditionNode.createLeftTuple(EvalConditionNode.java:237)
    at org.drools.core.phreak.PhreakJoinNode.doLeftInserts(PhreakJoinNode.java:100)
    at org.drools.core.phreak.PhreakJoinNode.doNode(PhreakJoinNode.java:60)
    at org.drools.core.phreak.RuleNetworkEvaluator.switchOnDoBetaNode(RuleNetworkEvaluator.java:547)
    at org.drools.core.phreak.RuleNetworkEvaluator.evalBetaNode(RuleNetworkEvaluator.java:533)
    at org.drools.core.phreak.RuleNetworkEvaluator.innerEval(RuleNetworkEvaluator.java:334)
    at org.drools.core.phreak.RuleNetworkEvaluator.outerEval(RuleNetworkEvaluator.java:161)
    at org.drools.core.phreak.RuleNetworkEvaluator.evaluateNetwork(RuleNetworkEvaluator.java:116)
    at org.drools.core.phreak.RuleExecutor.evaluateNetwork(RuleExecutor.java:53)
    at org.drools.core.common.DefaultAgenda.evaluateEagerList(DefaultAgenda.java:960)
    at org.drools.core.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:921)
    at org.drools.core.common.DefaultAgenda.fireAllRules(DefaultAgenda.java:1200)
    at org.drools.core.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:957)
    at org.drools.core.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:931)
    at org.drools.core.impl.StatefulKnowledgeSessionImpl.fireAllRules(StatefulKnowledgeSessionImpl.java:256)
    at cruise.umplificator.rules.RuleRunner.fireAllRules(RuleRunner.java:242)
    at cruise.umplificator.core.Umplificator.umplifyJava(Umplificator.java:344)
    at cruise.umplificator.core.Umplificator.umplifyElement(Umplificator.java:263)
    at cruise.umplificator.core.Umplificator.umplify(Umplificator.java:245)
    at cruise.umplificator.UmplificatorMain.main(UmplificatorMain.java:80)
    * Level 0 I got another NullPointerException
2015-02-15 21:09:00.098 [main] DEBUG cruise.umplificator.visitor.JavaClassVisitor - -Visiting PackageDeclaration-com.netflix.hystrix.contrib.javanica.annotation
2015-02-15 21:09:00.098 [main] DEBUG cruise.umplificator.visitor.JavaClassVisitor - Visiting ImportDeclarations-com.netflix.hystrix.HystrixCollapser.Scope
2015-02-15 21:09:00.098 [main] DEBUG cruise.umplificator.visitor.JavaClassVisitor - Visiting ImportDeclarations-java.lang.annotation.Documented
2015-02-15 21:09:00.098 [main] DEBUG cruise.umplificator.visitor.JavaClassVisitor - Visiting ImportDeclarations-java.lang.annotation.ElementType
2015-02-15 21:09:00.098 [main] DEBUG cruise.umplificator.visitor.JavaClassVisitor - Visiting ImportDeclarations-java.lang.annotation.Retention
2015-02-15 21:09:00.098 [main] DEBUG cruise.umplificator.visitor.JavaClassVisitor - Visiting ImportDeclarations-java.lang.annotation.RetentionPolicy
2015-02-15 21:09:00.098 [main] DEBUG cruise.umplificator.visitor.JavaClassVisitor - Visiting ImportDeclarations-java.lang.annotation.Target
java.lang.NullPointerException
    at cruise.umplificator.core.Umplificator.umplifyJava(Umplificator.java:322)
    at cruise.umplificator.core.Umplificator.umplifyElement(Umplificator.java:263)
    at cruise.umplificator.core.Umplificator.umplify(Umplificator.java:245)
    at cruise.umplificator.UmplificatorMain.main(UmplificatorMain.java:80)

12 February 2015

  • Updated the UCOSPUmplificatrix page with the tasks Andrew and I talked about yesterday. The goal is to get the first 3 done by the end of this weekend.

11 February 2015

  • Worked on dlproj
    • Moved all test related constants into their own Helper class
    • Refactored a few of the test methods
  • Met with Andrew to talk about and re-scope the Umplificatrix project

9 February 2015

  • Live coding session with Andrew. He covered some TDD basics using my downloader project as an example. I noted down a couple things that need to be changed:
    • Add tests to cover command line input
    • Centralize test data into a helper class

8 February 2015

  • Worked on dlproj
    • Added argument handling to DlprojMain using JOpt. The command arguments will be mainly used for testing, eventually dlproj will just be used in other components.
    • Added JOpt into the project and into the ant script. It took me a while to figure out how to add the relative classpath to the jar packaging section. It turned out that ant needed the classpath in specific format, so I used the <manifestclasspath/> task to convert the project lib path into an appropriate manifest path.
    • Stubbed out the Download method as well as the main project test. The Download method is ~90% finished already, but it needs a helper method to build the gitURI. The tests I'll leave until after. I'm not sure where to start and we have a live coding session on Monday that covers it.

7 February 2015

  • Worked on dlproj
    • Restructured the project to be more in line with Andrew's idea: the downloader itself will be dumb and github specific. We should be able to use like this: dlproj -d <local_dir> <project_name> <branch> to perform a one-time download of a project
    • Looked into using JOpt Simple to parse arguments for the script, and tested it out in the project (I haven't committed this part yet, still making a few tweaks)

4 February 2015

  • Worked on dlproj:
    • Fixed an issue where the git clone was failing. Looking through the JGit docs and a few stack overflow questions I found the problem to be a missing jsch dependency (as I was trying to clone via HTTPS). I added the dependency for now, but it may end up being better to use a different cloning method.
    • Planned out the test repo that I'll be using to unit-test the downloader. After I test with a for a bit on my local machine I'll add it to the Umple-ucosp GitHub group
    • Started looking through JGit for ways to leverage GitHub's version tags

2 February 2015

  • Tested out JGit
  • Worked on dlproj:
    • Partially implemented download method in GitProject class. I need to work out a better way to specify target local path. Right now there isn't enough feedback, and having it be specifiable might not even be necessary
    • Created a test class for GitProject. Class partially verifies constructor, but doesn't verify Download method yet. I'll need to talk to Andrew about how complete test coverage needs to be.
    • Added JGit jar to the repo and included it in the classpath of ant's build.xml
  • Attended Andrew's live coding demo of state tables

28 January 2015

  • Joined the Umple GitHub org
  • Tried using PHP's built-in web server like Andrew requested, but I have PHP 5.3.10 and the web server requires PHP 5.4+. I'll leave my environment as-is for now, if I look at anything related to UmpleOnline I'll update my PHP. I updated the SettingUpLocalUmpleOnlineWebServer page with the version info.
  • Started the dlproj in the Umple GitHub group
  • Made my first push to dlproj

24/25 January 2015

Finished setting up my Ubuntu development environment. The Ruby errors I was getting were caused by an older version of Ruby. After a lot of time spent trying to debug it and some help from Andrew I fixed the problem by updating to Ruby 1.9.3

I set up Eclipse and I'm looking into JGit as my starting point to the umplificatrix downloader (as recommended by Kevin)

23 January 2015

I finally switched my development environment over from Windows to an Ubuntu flavor (specifically Windows 8.1 to Elementary OS 0.2). I'm about 3/4 of the way through getting the Umple development environment up and running. I can build, but I'm getting 'rake aborted' errors run running the unit tests.

21 January 2015

19 January 2015

18 January 2015

  • Last day of code kickoff
    • Talked with Andrew about my semester project: a subset of Issue 661 (on Google Code)
      • First steps are to write up the project overview as I understand it
    • Finished off my work for Issue 668 (on Google Code) but I'm not sure if it should be closed, or assigned to Tim/Andrew for review
    • Created a new issue Issue 672 (on Google Code) to cover the whitespace issue I saw with windows

17 January 2015

  • Second day of code kickoff
    • Looked into cleaning up my fix for Issue 668 (on Google Code). As it is right now, tests no longer consider whitespace when comparing when comparing actual and expected code. Ideally I'd like to just modify the generator so it creates the same amount of whitespace regardless of platform.
    • Talked to Andrew about potential semester project ideas
    • Started looking into Docker as a deployment helper
      • Had a rough overview of the requirements with Andrew
      • Got Boot2Docker running in my environment
      • Started working through the documentation

16 January 2015

  • First day of code kickoff
    • Morning:
      • Got my build working, but still having 11 tests fail. Andrew gave an overview of the Umple infrastructure and architecture and started helping me debug my test failures.
    • Afternoon:
      • With Andrew's help I submitted a patch for Issue 668 (on Google Code)
      • The patch fixes the problem, but I'd like to spend some time tomorrow cleaning it up

15 January 2015

  • Updated InstallAntWithAntContribs with my steps for my specific environment. The process isn't any different than I read from other student's logs, but now at least it's there explicitly.
  • Ran a successful build using ant -Dmyenv=wlocal
  • Still seeing 11 unit tests failing due to extra white-space

14 January 2015

  • Imported the cruise.umple, cruise.umple.nebula, jopt-simple, and lib
  • Cleaned the cruise.umple project and rebuilt it using the Umple plugin within Eclipse. No errors showing in the cruise.umple project
  • Ran through the JUnit tests. Managed to run 1835 out of the 3866 unit test (though 1100 of those failed) before the JUnit test execution gets stuck on cruise.umple.UmpleConsoleMainTest.MultiUmpleFile
    • I'll probably just do a full rebuild with ant and try again later today
  • After rebuilding Umple I could run all 3866 tests. This time only 12 failed. The failures look pretty benign, but the might be related to the Windows environment.

13 January 2015

  • Used Umple online to re-create a simple banking model
    • I tried to get the UML representation of the model looking good, but ended up just giving up. It's difficult to move the associate/generalization lines, and the undo/redo buttons never quite do what you expect them to.

12 January 2015

  • Installed Ant, following these instructions: http://ant.apache.org/manual/install.html
  • Built Umple using ant following the DevelopmentSetUp page
    • Just a note here, Step 2b could really use more detail. It doesn't get into how you should get ant, nor does it really talk about what that first ant command does.

6 January 2015

  • Played with a few Umple Online examples
  • Read over a few Umple tutorials
  • Setup Eclipse, JDK, SVN, and the Umple plugin on my windows environment (following Install_Eclipse_Plugin)