When compiling templates, do so separately from compiling the controllers that may call them

[bjorn-ali-goransson]

If I open up a newly created project using Play 2.2 with Eclipse for Java EE having also installed Scala IDE 3.0.2, then changing the index.scala.html message constructor parameter type to Int, then Play will throw the following error while trying to compile (manually or listening for source changes does not change this):

[error] /Users/owner/Desktop/play-test/app/controllers/Application.java:11: error: method render in class index cannot be applied to given types;
[error]         return ok(index.render("Your new application is ready."));
[error]                        ^
[error]   required: int
[error]   found: String
[error]   reason: actual argument String cannot be converted to int by method invocation conversion
[error] 1 error
[error] (compile:compile) javac returned nonzero exit code

Well, this may not be totally incorrect, but looking at my Application.java file, Eclipse shows no errors. Apparently the render method awaits a String.

And hovering over the index method confirms this. Looking at the compiled bytecode of /play-test/target/scala-2.10/classes_managed/views/html/index.class in fact proves that the changed template was not compiled because of the offending Application.java.

Now, the cause of this must be that the compiler tries to compile both the templates and the controllers in one pass, and I think this is not correct, strictly speaking, because the templates being successfully compiled should not be dependent upon other classes not trying to invoke them incorrectly.

Am I right?

[bjorn-ali-goransson]

Sorry, further details.

I have enabled in Eclipse Preferences / Workspace / Refresh using native hooks or polling.

And, as soon as I fix the Application.java error, the hovering over the index method changes signature and the Play console tells me that the compilation was successfully completed.

[bjorn-ali-goransson]

Confirmed for many of my students in class today... the errors become very confusing for beginners. "To compile new templates you must first see that your Controllers have no compiler errors".

One student renamed a template, and behold, the Controller that was calling it could not find it with the new name, and as such the old one wouldn't go away and the new one wouldn't be compiled. The solution was to temporarily return just null so that the templates could compile. Then reenter the template call.

FYI, I tried this on fresh eclipse (j2ee kepler + play plugin from scala-ide), fresh Play installation, (2.2.1), fresh Play application, ran eclipse from Play, imported into empty workspace...

[richdougherty]

Can you try reproducing the problem without Scala IDE open and starting
from a clean build?

[bjorn-ali-goransson]

Sure, just did, same error:

[bug-test] $ compile
[info] Compiling 1 Scala source to /Users/owner/Downloads/bug-test/target/scala-2.10/classes...
[info] Compiling 1 Scala source and 1 Java source to /Users/owner/Downloads/bug-test/target/scala-2.10/classes...
[error] /Users/owner/Downloads/bug-test/app/controllers/Application.java:11: error: method render in class index cannot be applied to given types;
[error]         return ok(index.render("Your new application is ready."));
[error]                        ^
[error]   required: int
[error]   found: String
[error]   reason: actual argument String cannot be converted to int by method invocation conversion
[error] 1 error
[error] (compile:compile) javac returned nonzero exit code
[error] Total time: 6 s, completed Mar 11, 2014 7:21:16 AM

(make new Play application, fresh eclipse jee kepler (no extra plugins), fresh workspace, play eclipse, import application, compiles successfully, remove @play20... call from index.scala.html (so @message is not used in the template) and change @message to Int like so:

@(message: Int)

@main("Welcome to Play") {

}

And vi-ola - the compiler throws up on Application.java, and the template is not recompiled.

To fix manually:

Remove the index.render call (insert null; // between return and ok(index.render("Your new application is ready."));) from Application.java and run play compile again, and the template is compiled.

Now, remove the null; // and Bob's your uncle, you have now a compiled template and an erroneous call in Application.java.

[bjorn-ali-goransson]

OK, retried now with Play 2.2.2 using only play and a simple text editor.

I am now trying a new way of convincing you that this is a bug: Comparing the changed dates of the generated class files inside target/scala-2.10/classes_managed/views/html/.

=== Proof that only recompiled classes have their modification date changed

From the Terminal, cd to your Desktop.
Type play new bug-test.
Press Enter to choose the name bug-test. Choose 2 for Java application.
Type cd bug-test.
Type play compile. Success.

Note the compilation date of the generated .class files index.class and main.class.
Wait two minutes.

In index.scala.html, remove the line @play20.welcome(message, style = "Java").
Run play compile. Success.

Note that only the index.class modification date has changed.

=== Proof that compilation errors occuring in calling class prevents templates from being compiled

In index.scala.html, change the first line @(message: String) to @(message: Int).
Run play compile. Fail.

Note that none of the .class files have had their modification dates updated.

Now, just to add to the frustration of this bug report, let's change the calling line in Application.java from return ok(index.render("Your new application is ready.")); to return null;.
Run play compile. Success.

Note that the index.class modification date has changed!!!

Tearing my hair here. Please acknowledge the existence of this, lol.

[bjorn-ali-goransson]

Actually, now I have an even more obvious way of proving the erroneous flow of action here.

New project.

In index.scala.html, change the message parameter to Int. Be sure to have removed the @play20... line as your template would then contain an error.

Now your template has no errors, and it (the template itself) should compile.

Now, run play clean. If the target directory ever existed, it should disappear.

Now, run play compile. The target directory reappears, and even the scala-2.10/src_managed, but no classes_managed is even created. And the reason for that is in Application.java - outside of the template.

That means that eclipse etc can't load the template to show meaningful info (most often crucial to solving the error - especially for beginners), AND it's stuck with the old version (if any) of the compiled template from the previous compile, further adding to the confusion.

[bjorn-ali-goransson]

Okay, more info. I've seen now that the generation (or is it just the copying) of .class files is done in sbt-plugin/src/main/scala/PlayCommands.scala, in the PostCompile method. Don't really have time to analyze it further, though.

Could this mean that the compiling aspect is OK, but the copying of generated template files to classes_managed is not, as it should not be dependent upon a successful compile i.e. should not be done in PostCompile?

[richdougherty]

Don't tear your hair out, at least one person is reading your messages. :)

This sounds like a bug to me. Thanks for isolating the problem from
Eclipse, that makes it easier. Eclipse can also create and copy class
files, so using Eclipse at the same time would confuse the issue a bit.
This bug reminds me of another possible problem (routes re-compilation)
that I have been meaning to investigate.

If you're in a hurry to get a solution you may wish to ask on the mailing
list and someone might be able to help you find a workaround. I imagine
there could be a line or two of SBT configuration that you could paste into
your build.sbt file to get things working again. Unfortunately I can't tell
you a workaround without spending some time investigating.

[bjorn-ali-goransson]

Note: The identical erroneous behaviour also exists when trying to add a new dependency.

For example, adding a call to JPA.em() will cause a compiler error, because javaJpa is not added as an application depencency (build.sbt).

If you now try to add javaJpa to your application dependencies, and restart Play and then hit compile, you will notice that Play! will refuse to add javaJpa to the build path unless you first remove the call to JPA.em(). (primal scream here)

This is very frustrating for me, but mostly for my students who are already struggling with other aspects of java web programming.

[bjorn-ali-goransson]

Question - will this happen when using the Typesafe Activator?

[jroper]

Hi @bornemix,

Looking over this, the behaviour you are describing seems to be completely expected. Let me first explain exactly how compilation works:

1. The template files are translated to scala files, these end up in files like target/scala-2.10/src_managed/main/views/html/index.template.scala
2. SBT then does "mixed mode" compilation of all Java and Scala files in the project. Mixed mode compilation means a two pass compilation, first the Scala compiler runs over all Java/Scala files, doing a rudimentary parse of Java files to establish method signatures and compiling Scala files to bytecode, then the Java compiler runs with the class files from the Scala compiler on its classpath.

The SBT mixed mode compliation allows Java code to depend on Scala code, and vice versa.

If the Scala compilation step fails, then class files will not be produced. However, the Scala files from the template compiler will be produced, allowing IDEs that understand Scala to use these. I'm not 100% sure how ScalaIDE works here, but do you have the Play plugin enabled? As I understand it, it should natively understand the Scala template language, know how to compile it, and therefore get everything right.

Your statement about the compilation errors in the calling class preventing the template from being compiled being wrong might sound logical, but is not how things can work in practice - compilers don't work by tracing call graphs and then compiling things as needed - if they did this, then you couldn't have circular dependencies between classes (but as you would know, there is no problem with two Java classes referencing each other in some way). The way compilers work is by compiling everything at once, using an incremental iterative approach that does caching and so on. What that means is effectively, from the outside, you can only view compilation as an all or nothing thing. So that means if one thing fails, everything fails. The caller failing to compile will prevent the callee from compiling. Now as a result of not using a transactional file system, you may find in practice that some class files are produced and updated even when compilation fails. This is not an intentional thing by the compilers, this is just when compilation fails, they don't bother to go to the effort of cleaning up after themselves because there's no real advantage in doing so.

So, perhaps there are some bugs in ScalaIDE here, or in the Play plugin for ScalaIDE. But it seems to me that everything is working as expected from the Play side.

[bjorn-ali-goransson]

Hi @jroper , thanks for your reply.

Know that Eclipse (with or without Play! plugin) is using the classes_managed directory to find errors - it is not parsing or compiling the scala template files.

(The icon for the classes_managed folder shows that it's loading and using its classes - its entry is found under Project properties -> Java Build Path -> Libraries amongst a bunch of JARs)

A simple proof:

For example, if I load the default Java project in Eclipse (the eclipse command forces a compile) and remove the main classes inside classes_managed/views/html, Eclipse will directly start showing errors (if you have "refresh workspace by using native hooks or polling" enabled - otherwise you need to hit Refresh on your project).

Another situation:

Load the default Java project in Eclipse, and write an x in the end of Application.java, e.g. after the last }. Create a new Play template and invoke it instead of the index template in the Application.index() action. You will get a "cannot find symbol" for the new view. Running Play compile will not help until you remove your x.

This means that the compiling everything at once strategy is problematic, at least for Eclipse users.

So, compiling templates becomes dependent upon other classes having no errors.

[bjorn-ali-goransson]

If my last comment is correct, then we have a serious (but not necessarily complex) problem.

Is Typesafe Activator using the same mechanism for compiling?

[jroper]

@bornemix Typesafe Activator just uses SBT.

I'm not sure if you answered this already, but do you have the Eclipse Play plugin installed? You can follow these instructions here to install it:

http://scala-ide.org/docs/tutorials/play/

The Play Eclipse integration shouldn't have these issues.

[bjorn-ali-goransson]

_I've tried this with and without the Eclipse Play plugin._

Kindly, @jroper , you are making me having to repeat myself as I have already mentioned this.

It's apparent that you have lots to do but this is not the way to handle a bug report - you should assign it to someone else if you're to busy to read the feedback (I'm specifically referring to my 2nd last comment which mentions the Play plugin).

[jroper]

Hi @bornemix,

Sorry for not seeing that. At this point, I'm still not seeing any clear bug in Play - everything you've explained to me so far - from a Play perspective - seems like expected behaviour. Perhaps I'm not understanding how Eclipse interacts with Play - I am not an Eclipse user. There are thousands of people using ScalaIDE with the Play plugin, both people like you in the community, and people who are paying us for support, and they aren't reporting these problems. Perhaps you should raise an issue on the ScalaIDE, because your problem does seem to be about its integration with Play, and not about Play itself:

https://www.assembla.com/wiki/show/scala-ide/bug_reporting

And please remember, I'm not responding here because I am assigned to this issue or because I have any obligation to do so, this isn't a support system, if I stop handling this, no one handles it, there's no assigning it to someone else. This issue tracker is a system where you can come to help an open source project, not the other way around.

[bjorn-ali-goransson]

Hi @jroper,

Sorry for assuming that you are being paid to do this. But this kind of replying (like your last reply) is only adding noise to the issue at hand.

The issue is not related to 1) wether the behaviour is expected. Also, clearly the issue is not 2) related to Play being functional without an IDE. Yet it is also not 3) related to the Play plugin for Scala. You address these points as if they were related to this issue. Once again you are making me repeat myself.

Sorry for sounding arrogant, but I am trying to be practical and getting this issue 1) clearly defined and 2) solved. This task does not get any easier by you asking questions and stating things that would not need to be mentioned would you have read more thoroughly.

Also, as for you stating that Play plugin compiles scala templates, adding further noise, Scala IDE website states the opposite. It loads the classes that Play compiles. ("The templates are transformed into Scala code by the Play framework. As Play has been started in auto-reloading mode in the background, templates are recompiled as soon as the file is saved." Google this and read the context)

For people not experiencing this, I am now googling a bit:

http://stackoverflow.com/questions/10910215/play2-framework-my-template-is-not-seen-package-views-html-doesn-not-exist (see comments to answers)
http://stackoverflow.com/questions/19776892/idea-play-framework-cannot-resolve-method-symbol (see answer)
http://stackoverflow.com/questions/14175460/does-idea-recognize-play-scala-template-changes?rq=1
https://groups.google.com/forum/#!topic/play-framework/J-dlxDJ9aIk (see 2nd reply by vladimir)
http://www.jamesward.com/2012/02/21/play-framework-2-with-scala-anorm-json-coffeescript-jquery-heroku#comment-451813748

As for expected Play functionality, I am reminded of this: twbs/bootstrap#3057

I can say that making a custom Scala template compiling (you say "understanding") routine (along with package resolution into "views.html"), combined with reimplementing already existant code completion in the template editor is a lot more work than using the classes_managed folder which is already there. This paragraph was about the Play plugin.

So, i may only suspect that there are other reasons for not contemplating this suggestion (the issue heading) - performance issues in generating/compiling templates in one pass, and including those classes in the class path while compiling the rest of the code in another pass?

[jroper]

You can't compile the templates in a separate pass though, and this is what I tried to explain earlier, because the templates depend on your application code. Consider this template:

user.scala.html

@(user: User)

<p>User: @user.name</p>

User.java

public class User {
  public String name;
  public String render() {
    return views.html.user.render(this);
  }
}

You can't compile user.scala.html first because it depends on the User class being compiled, and you can't compile User first because it depends on user.scala.html being compiled. You can only compile them both at the same time, and they either both compile, or they both don't compile.

The above example as it stands may be a little unusual, but in practice there are usually many circular dependencies between your templates code and your Java code as a whole, your templates usually depend on your model classes as arguments, and your controllers depend on your templates. Maybe we could compile model classes in one pass, then templates, then controllers, but that's not how compilers work, they don't see models, templates controllers, they see everything as one blob that could have dependencies in any direction, and we certainly don't want to add a new compilation mode that precludes such circular dependencies.

So the problem is in the Play plugin for Eclipse, because it is trying to treat Java code and templates code as something that can be compiled in separate passes, but that is simply not true, if it wants to provide a good experience for Play, then if it's going to compile one of them, it needs to also compile the other. The IntelliJ support I believe already does this.