Port code to Scala 2.8.0

[ijuma]

See issue #147: compilation problems with scala 2.8.0
http://github.com/ether/pad/issuesearch?state=open&q=scala#issue/147

[redhog]

This code seems to work on scala 2.8.*, but instead breaks 2.7 :( Is it at all possible to write code that works on both? We're in the situation that some users are on Linux distributions 2.8 and some on distros with 2.7 :S

Apart from that, great great work! Thanks so much!

If we can't get it to work on both, we'll just end up having to package scala 2.8 for Ubuntu, but that's not the end of the world...

[ijuma]

It's hard to write code that works on both as there were many changes in Scala 2.8. In general, most projects have either moved to 2.8 or use separate branches for 2.7 and 2.8. Note that Fedora 13 also includes Scala 2.8.0.

[redhog]

Ok, so latest fedora is no problem, but latest ubuntu is... Also, lots of people are still running old distros. I guess the problem araises when a distro does not come with 2.7...

So, what do you propose we do? It's a bit sad, because Scala's the only dependency that we've had such problems with (ok, AppJet did bundle some others to avoid that... that's actually even worse...)

[ijuma]

It's true that incompatible changes cause distribution issues for software that does not include its own dependencies (which is the normal approach for Linux packages). However, Scala is certainly not alone in this regard, see Python 3 and Ruby 1.9 for two examples. A mitigating factor for Scala is that it's much newer.

My suggestion is to move to 2.8.0 and package Scala 2.8.0 for Ubuntu and Debian. Scala 2.8 is a major improvement and no updates are planned for 2.7.x. Also, it seems like OpenSuse has 2.8.0:

http://download.opensuse.org/repositories/openSUSE:/Factory:/Contrib/openSUSE_11.1/noarch/scala-2.8.0-2.1.noarch.rpm

[redhog]

Ok, sounds like a reasonable plan, but I'll have to talk to the Ubuntu guys about it I think

[victornoel]

Also, since Scala just consists of a JAR library at runtime, nothing stops you to package it INSIDE the etherpad packages.

Or am I wrong? :/

[redhog]

Packaging stuff inside etherpad is a big packaging no-no, and it still wouldn't solve

apt-get source
cd package
deb-buildpackage

:(

[zamfi]

What's the status on this? I'm of the opinion that we should move to 2.8, because it's the future. :)

Can someone pull ijuma's 2.8.0 branch into the main ether/pad? Users who don't have scala 2.8 can (a) install scala 2.8, or (b) use the older release of etherpad. Right?

[kaechele]

I think you are right. But unfortunately the code in this pull request isn't compatible with 2.8.1. So I think before migrating to 2.8 we should first fix errors for 2.8.1.

[kakra]

In main.scala, there's a val startupExecutable somewhere near line 101 which accesses config.ajstdlibHome despite the config (which contains this path) has not yet been loaded (I tested the root parameter in the belonging class, it contains "null"). Making this a lazy value gets me past the java.lang.ExceptionInInitializerError, but still it does not work and I am now stuck again:

Exception in thread "main" java.lang.NullPointerException
        at net.appjet.oui.AutoUpdateFile.update(files.scala:261)
        at net.appjet.oui.DiskLibrary.updateExecutable(files.scala:306)
        at net.appjet.oui.DiskLibrary.executable(files.scala:321)
        at net.appjet.oui.main$.startupExecutable(main.scala:101)
        at net.appjet.oui.main$.runOnStartup(main.scala:103)
        at net.appjet.oui.main$.main(main.scala:267)
        at net.appjet.oui.main.main(main.scala)

Line numbers may not be exact because I added some println debug. But before making the value lazy, the code did not even enter the main function, it entered just the object initializer.

According to strace it tries system calls like these, and "null" does not belong there:

JarOrNotFile: root = ../infrastructure/framework-src/modules, fname = onstartup.js
[pid 20817] stat("../infrastructure/framework-src/modulesnullonstartup.js", 0x71441891e380) = -1 ENOENT (No such file or directory)
JarOrNotFile: root = ../infrastructure/framework-src/modules, fname = JS$onstartup.class
[pid 20817] stat("../infrastructure/framework-src/modulesnullJS$onstartup.class", 0x71441891ddc0) = -1 ENOENT (No such file or directory)
[pid 20817] stat("../infrastructure/framework-src/modulesnullJS$onstartup.class", 0x71441891df90) = -1 ENOENT (No such file or directory)
Made DiskLibrary on onstartup.js, with classFile: null
[pid 20817] stat("../infrastructure/framework-src/modulesnullonstartup.js", 0x71441891deb0) = -1 ENOENT (No such file or directory)

For some reason I cannot follow, "fileSep" evaluates to "null" in the following line:

val file = if (! isJar) FileCache.file(root+fileSep+fname, this) else null;

although it is set to another value three lines above. Confirmed by statically setting the concatenation like this:

val file = if (! isJar) FileCache.file(root+"/"+fname, this) else null;

Of course this failes later because the derived classes no longer use the intended fileSep value but at least Etherpad does something like booting then.

Using scala 2.8.1

[kakra]

Foxtrott unicorn charlie kilo... Making "val file" into "lazy val file" makes "fileSep" evaluated like intended...

Now it says

Startup execution failed with non-200 response: 500

But I'm sure this is a different issue.

[redhog]

@johnyma22 we need to test this on a modern Linux dist. Could you do that? If it works fine, then we merge (don't forget to update debian package dependencies!!!)

[kaechele]

I can provide a VM with Fedora 15 (Scala 2.8.1) for testing if that helps...

[redhog]

@kaechele Sounds good. What we need tested is really: Build a debian package and an RPM package. Install on clean Fedora, Ubuntu, and Debian machines, check that you can create a pad and edit it. Once that works for all three distros this can be merged. Check with latest version 8even if Ubuntu 11.04 is kind of buggy; it's still the latest version)

[kakra]

BTW: I'm testing this on a Gentoo machine with Scala 2.8.1... I'll make my changes into a patchset applyable to this branch.

[kakra]

I've pulled latest release 1.1 branch into my branch and it also seems to work with Scala 2.8.1, using Gentoo Linux.

[redhog]

Would you be able to try it on the other distros and/or update the packages?

[kakra]

Our server infrastructure is completely Gentoo based. But I'm sure other people on this thread have such test platforms. I'm also not familiar with how to package this software into deb or rpm.

[sesam]

1. Build a debian package and an RPM package. -- are those packages ready for download somewhere, or where do I find docs on how the packaging should be done?

(2. Install on clean Fedora, Ubuntu, and Debian machines, check that you can create a pad and edit it. I can probably drum up some VM:s too for testing. I mean, all it takes is a host with 1GB ram to spare during the testing phase.)

[redhog]

debian/ubuntu: Run (dpkg-buildpackage or) git-buildpackage in the etherpad root directory.
RedHat: rpmbuild --something --somethingelse redhat/etherpad.spec (I've forgotten the switches...)

[kakra]

BTW: My branch has been successfully been tested, running behind a varnish frontend. Multiple users in parallel edited and created pads, timelines are working, even the team pads are working, smtp mail notifications also work. There are some crashes in the admin backend, however, I consider them uncritical and not related to my patches in the branch - in other words: These have been there before. Actually, I was impressed how flawlessly it works now - based on the experiences I had when trying first to install this. ;-)

[JohnMcLear]

That's good news :)

[JohnMcLear]

Just a quick note for varnish users, ensure you use pipe on none static objects.. Your config will want to look something like...

if (req.http.host ~ "doc.etherpad.org" && req.url ~ "static"){set req.backend = epad;return(lookup);}
if (req.http.host ~ "doc.etherpad.org"){set req.backend = epad;return(pipe);}

[kakra]

@johnyma22 What difference will it make? It seems to work just fine for me without using pipes... Please explain...

[JohnMcLear]

Piping will stop any potential problems with varnish messing up the socket.io connectivity.

https://www.varnish-cache.org/trac/wiki/VCLExamplePipe

[kakra]

But shouldn't one send a “Connection: close” header then - as per the example?

[JohnMcLear]

I have never had to so I'm not sure.

[redhog]

@johnyma22 could you provide your varnish config in the contrib directory?

Does this mean this branch should be merged?

[sesam]

^^ bump

[kakra]

Works for me in production since two months now, no problems reported by users.

[vie]

Hi

I builded etherpad pro on ubuntu 10.4 with scala 2.8.1 and sun-
jdk-1.6.0_26. Jar is created, but during startup throws following
exception:

Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef
$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;
at net.appjet.oui.config$.(config.scala:209)
at net.appjet.oui.config$.(config.scala)
at net.appjet.oui.main$.(main.scala:58)
at net.appjet.oui.main$.(main.scala)
at net.appjet.oui.main.main(main.scala)

what i'm doing wrong?

[opoplawski]

Is this any closer to getting finally resolved?

[SteffanCline]

The last comment relates to Link.contains(). I posted the error and documentation on a new issue 313. In scala 2.8 up, it is the same error. I tried throwing additional scala libs in there and the errors nearly all go away except the one posted above. Weird!