Remove ant #5311

Merged
merged 2 commits into from Sep 5, 2016

Conversation

Projects
None yet
@szeiger
Contributor

szeiger commented Jul 29, 2016

The final steps for removing the ant build. Marking as on-hold for now until we're sure that nobody relies on it anymore (e.g. the community build still needs to be changed).

@szeiger szeiger added the on-hold label Jul 29, 2016

@szeiger szeiger added this to the 2.12.0-RC1 milestone Jul 29, 2016

@martijnhoekstra

This comment has been minimized.

Show comment
Hide comment
@martijnhoekstra

martijnhoekstra Aug 1, 2016

Contributor

Doesn't the Debian Scala package depend on the Ant build?

Contributor

martijnhoekstra commented Aug 1, 2016

Doesn't the Debian Scala package depend on the Ant build?

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 1, 2016

Member

@martijnhoekstra Good point, although I think that getting those usually completed outdated builds of Scala out of Debian is a plus. :)

Member

soc commented Aug 1, 2016

@martijnhoekstra Good point, although I think that getting those usually completed outdated builds of Scala out of Debian is a plus. :)

@martijnhoekstra

This comment has been minimized.

Show comment
Hide comment
@martijnhoekstra

martijnhoekstra Aug 1, 2016

Contributor

I can see that not having Scala in Debian - which also means no Scala in most downstreams like Ubuntu - is a sacrifice that one could be willing to make, due to how old those packages tend to be.

But SBT in Debian - and downstreams - would be really nice, even if it is an older version. And without Scala in Debian, all hope for eventually having SBT in Debian is ruled out.

Contributor

martijnhoekstra commented Aug 1, 2016

I can see that not having Scala in Debian - which also means no Scala in most downstreams like Ubuntu - is a sacrifice that one could be willing to make, due to how old those packages tend to be.

But SBT in Debian - and downstreams - would be really nice, even if it is an older version. And without Scala in Debian, all hope for eventually having SBT in Debian is ruled out.

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 1, 2016

Member

@martijnhoekstra Wouldn't the exceptions from the bootstrapping requirements not apply to SBT?

Member

soc commented Aug 1, 2016

@martijnhoekstra Wouldn't the exceptions from the bootstrapping requirements not apply to SBT?

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 1, 2016

Member

Having SBT in Debian would be great, although to really benefit from it we should probably change the versioning scheme to SBT 1.0.0.0-x so that it can receive updates. :-)

Member

soc commented Aug 1, 2016

Having SBT in Debian would be great, although to really benefit from it we should probably change the versioning scheme to SBT 1.0.0.0-x so that it can receive updates. :-)

@martijnhoekstra

This comment has been minimized.

Show comment
Hide comment
@martijnhoekstra

martijnhoekstra Aug 1, 2016

Contributor

@soc I asked around about SBT quite a while ago on the Debian packaging IRC channel on FreeNode, where it was indicated they insisted on a make build rather than a build that had to be bootstrapped. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=639910 is the request for packaging in Debian.

Reading through it again, I may be mistaken that Debian requires all build-deps to have a free Debian package - and the recursive build dependency may be allowed for at least Scala.

Contributor

martijnhoekstra commented Aug 1, 2016

@soc I asked around about SBT quite a while ago on the Debian packaging IRC channel on FreeNode, where it was indicated they insisted on a make build rather than a build that had to be bootstrapped. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=639910 is the request for packaging in Debian.

Reading through it again, I may be mistaken that Debian requires all build-deps to have a free Debian package - and the recursive build dependency may be allowed for at least Scala.

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 1, 2016

Member

I think the issue is that no one from the SBT team seemed to communicate how/if it is possible to compile SBT without internet access.
(A problem which is still unsolved, and manifests itself in many different instances, like SBT downloading dozens of jar files with a few dozen kBs instead of one package during first start.)

Member

soc commented Aug 1, 2016

I think the issue is that no one from the SBT team seemed to communicate how/if it is possible to compile SBT without internet access.
(A problem which is still unsolved, and manifests itself in many different instances, like SBT downloading dozens of jar files with a few dozen kBs instead of one package during first start.)

@martijnhoekstra

This comment has been minimized.

Show comment
Hide comment
@martijnhoekstra

martijnhoekstra Aug 1, 2016

Contributor

Back to the Scala Debian package itself though: is requiring sbt a blocker for that package, and is that enough reason not to merge the patch? Should the package maintainer be contacted for that?

Contributor

martijnhoekstra commented Aug 1, 2016

Back to the Scala Debian package itself though: is requiring sbt a blocker for that package, and is that enough reason not to merge the patch? Should the package maintainer be contacted for that?

@jodersky

This comment has been minimized.

Show comment
Hide comment
@jodersky

jodersky Aug 1, 2016

Member

Reading through it again, I may be mistaken that Debian requires all build-deps to have a free Debian package - and the recursive build dependency may be allowed for at least Scala.

I'm not sure about the free requirement, IIRC someone suggested uploading sbt (with a prebuilt sbt in the source package as to have no build-deps) to contrib or non-free. Bootsrapping into main could then happen from there on.
As @soc mentioned, I think the major blocker is the internet connectivity during build.

Considering that, I would also think that requiring an sbt build would effectively ban scala from debian until sbt is available as a package.

Since it appears that dropping internet connectivity for sbt is not so trivial, has anyone considered making scala and sbt available on the non-free repository? I'm not sure how to go about it but it seems to be a viable compromise, at least for a first step.

Member

jodersky commented Aug 1, 2016

Reading through it again, I may be mistaken that Debian requires all build-deps to have a free Debian package - and the recursive build dependency may be allowed for at least Scala.

I'm not sure about the free requirement, IIRC someone suggested uploading sbt (with a prebuilt sbt in the source package as to have no build-deps) to contrib or non-free. Bootsrapping into main could then happen from there on.
As @soc mentioned, I think the major blocker is the internet connectivity during build.

Considering that, I would also think that requiring an sbt build would effectively ban scala from debian until sbt is available as a package.

Since it appears that dropping internet connectivity for sbt is not so trivial, has anyone considered making scala and sbt available on the non-free repository? I'm not sure how to go about it but it seems to be a viable compromise, at least for a first step.

@mizdebsk

This comment has been minimized.

Show comment
Hide comment
@mizdebsk

mizdebsk Aug 2, 2016

Scala package in Fedora is built with Ant. Removing Ant build script would make things much more difficult for us and in worst case scenario in could even result in dropping Scala from the distribution.

mizdebsk commented Aug 2, 2016

Scala package in Fedora is built with Ant. Removing Ant build script would make things much more difficult for us and in worst case scenario in could even result in dropping Scala from the distribution.

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

Hi, I'm one of the volunteers currently maintaining Scala in Debian (and indirectly Ubuntu). Debian 8 currently ships a rather old 2.9.2 version, but the upcoming Debian 9 and the last 3 releases of Ubuntu have Scala 2.11.6. Scala 2.11.8 should arrive in Debian soon. So the package used to be outdated, but we're trying to do better nowadays.

The Debian package is built with Ant and dropping this build system would seriously impact us since we don't have SBT in Debian. We would be really grateful if you could keep supporting the Ant build (or consider alternative build systems supported in Debian like Maven or Gradle).

On a side note, we are rather short-staffed on people knowledgeable in Scala. If there are developers interested in improving the Scala ecosystem in Debian/Ubuntu please contact us on debian-java@lists.debian.org.

ebourg commented Aug 2, 2016

Hi, I'm one of the volunteers currently maintaining Scala in Debian (and indirectly Ubuntu). Debian 8 currently ships a rather old 2.9.2 version, but the upcoming Debian 9 and the last 3 releases of Ubuntu have Scala 2.11.6. Scala 2.11.8 should arrive in Debian soon. So the package used to be outdated, but we're trying to do better nowadays.

The Debian package is built with Ant and dropping this build system would seriously impact us since we don't have SBT in Debian. We would be really grateful if you could keep supporting the Ant build (or consider alternative build systems supported in Debian like Maven or Gradle).

On a side note, we are rather short-staffed on people knowledgeable in Scala. If there are developers interested in improving the Scala ecosystem in Debian/Ubuntu please contact us on debian-java@lists.debian.org.

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

I quickly checked the other Linux distributions, Gentoo, Arch and SUSE also build Scala with Ant.

ebourg commented Aug 2, 2016

I quickly checked the other Linux distributions, Gentoo, Arch and SUSE also build Scala with Ant.

@dwijnand

This comment has been minimized.

Show comment
Hide comment
@dwijnand

dwijnand Aug 2, 2016

Member

Of course they do, that's been the official build system for Scala until recently.

Personally I think we should remove the burden of supporting both build tools, and forward-fix whatever is necessary to continue to allow for a scala package in the various distributions.

Member

dwijnand commented Aug 2, 2016

Of course they do, that's been the official build system for Scala until recently.

Personally I think we should remove the burden of supporting both build tools, and forward-fix whatever is necessary to continue to allow for a scala package in the various distributions.

@martijnhoekstra

This comment has been minimized.

Show comment
Hide comment
@martijnhoekstra

martijnhoekstra Aug 2, 2016

Contributor

@dwijnand if forward-fixing entails helping the distros package SBT in so far they haven't been able to yet, and helping them package Scala under an SBT build, is there a reasonable guarantee of this actually happening?

Contributor

martijnhoekstra commented Aug 2, 2016

@dwijnand if forward-fixing entails helping the distros package SBT in so far they haven't been able to yet, and helping them package Scala under an SBT build, is there a reasonable guarantee of this actually happening?

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

Once the community build starts using pre-built Scala artifacts, the ant build will be essentially unsupported because there are no CI jobs using it anymore. PR validation and nightly builds are already ant-free. The goal is to get rid of ant as quickly as possible. We don't want to maintain multiple build systems (one is more than enough, considering the complexity of the build setup).

Contributor

szeiger commented Aug 2, 2016

Once the community build starts using pre-built Scala artifacts, the ant build will be essentially unsupported because there are no CI jobs using it anymore. PR validation and nightly builds are already ant-free. The goal is to get rid of ant as quickly as possible. We don't want to maintain multiple build systems (one is more than enough, considering the complexity of the build setup).

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

BTW, internet connectivity shouldn't be required for building with sbt. You can populate a local repository with all required artifacts and use that as the only bootstrap repository.

Contributor

szeiger commented Aug 2, 2016

BTW, internet connectivity shouldn't be required for building with sbt. You can populate a local repository with all required artifacts and use that as the only bootstrap repository.

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 2, 2016

Member

@szeiger I'm sure it's possible, but it's not straight-forward enough for anyone to depend on it. There should be one fat-jar that delivers everything the SBT launcher needs on first startup, even if it was just for performance reasons.

Member

soc commented Aug 2, 2016

@szeiger I'm sure it's possible, but it's not straight-forward enough for anyone to depend on it. There should be one fat-jar that delivers everything the SBT launcher needs on first startup, even if it was just for performance reasons.

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

Would it be possible to deprecate the Ant build in Scala 2.12 (using SBT as the main build system) and removing it in Scala 2.13? Debian 9 will be frozen at the end of the year, and unless someone steps up to package SBT it's unlikely we'll have it in time. This means we won't be able to ship Scala 2.12 in Debian 9.

ebourg commented Aug 2, 2016

Would it be possible to deprecate the Ant build in Scala 2.12 (using SBT as the main build system) and removing it in Scala 2.13? Debian 9 will be frozen at the end of the year, and unless someone steps up to package SBT it's unlikely we'll have it in time. This means we won't be able to ship Scala 2.12 in Debian 9.

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

@soc The artifacts required by the sbt launcher are not sufficient. You need additional plugins for the build plus the actual build dependencies. I don't see how an sbt fat jar would help with this. You still need to bootstrap your own setup through a proxy repo if you want offline builds.

Contributor

szeiger commented Aug 2, 2016

@soc The artifacts required by the sbt launcher are not sufficient. You need additional plugins for the build plus the actual build dependencies. I don't see how an sbt fat jar would help with this. You still need to bootstrap your own setup through a proxy repo if you want offline builds.

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 2, 2016

Member

@ebourg I think it would be better to package SBT(-launcher), as this has the chance of keeping itself updated. I'd rather not have to deal with questions from people who have Debian and run into bugs in prehistoric versions of Scala.

Member

soc commented Aug 2, 2016

@ebourg I think it would be better to package SBT(-launcher), as this has the chance of keeping itself updated. I'd rather not have to deal with questions from people who have Debian and run into bugs in prehistoric versions of Scala.

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 2, 2016

Member

@szeiger No one is denying that a project that defines dependencies needs those dependencies to compile. What I have in mind are the dozens of JAR files SBT needs regardless of any actual project.

Member

soc commented Aug 2, 2016

@szeiger No one is denying that a project that defines dependencies needs those dependencies to compile. What I have in mind are the dozens of JAR files SBT needs regardless of any actual project.

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

@soc Packaging the SBT launcher alone wouldn't help, because the Debian builds must run offline. We need SBT and all its runtime dependencies packaged first, and this isn't a trivial task.

ebourg commented Aug 2, 2016

@soc Packaging the SBT launcher alone wouldn't help, because the Debian builds must run offline. We need SBT and all its runtime dependencies packaged first, and this isn't a trivial task.

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

@soc Sure, I'm not saying it wouldn't be useful in general, it's just that it doesn't solve the problem at hand. We have the knowledge how to build a fat distribution, the Activator team has done it (Activator is sbt plus some extra plugins, available either as a minimum download like sbt or a complete distribution with all dependencies). You'd have to convince the sbt team that it would be beneficial to do it.

Or maybe a better solution is to document how to do it so that interested parties can do it themselves (like building your own Windows installation image with additional drivers). As far as sbt configuration goes, it's really simple. I have not yet tried to use a repository proxy with the goal of building a standalone repo (that can be accessed through a file: URL).

Contributor

szeiger commented Aug 2, 2016

@soc Sure, I'm not saying it wouldn't be useful in general, it's just that it doesn't solve the problem at hand. We have the knowledge how to build a fat distribution, the Activator team has done it (Activator is sbt plus some extra plugins, available either as a minimum download like sbt or a complete distribution with all dependencies). You'd have to convince the sbt team that it would be beneficial to do it.

Or maybe a better solution is to document how to do it so that interested parties can do it themselves (like building your own Windows installation image with additional drivers). As far as sbt configuration goes, it's really simple. I have not yet tried to use a repository proxy with the goal of building a standalone repo (that can be accessed through a file: URL).

@martijnhoekstra

This comment has been minimized.

Show comment
Hide comment
@martijnhoekstra

martijnhoekstra Aug 2, 2016

Contributor

I get that nobody wants to maintain multiple build systems, but I don't get putting a change on hold to make sure that nobody depends on the old situation, the discussion shows people do depend on the old situation, and then concluding that doesn't matter. Did I misunderstand the on-hold reason?

Contributor

martijnhoekstra commented Aug 2, 2016

I get that nobody wants to maintain multiple build systems, but I don't get putting a change on hold to make sure that nobody depends on the old situation, the discussion shows people do depend on the old situation, and then concluding that doesn't matter. Did I misunderstand the on-hold reason?

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 2, 2016

Member

@szeiger I think that repo maintainer having to mess around with local repositories and proxies is hardly an improvement. I think what they want is a simple tar.gz that contains what SBT needs to build Scala.

Member

soc commented Aug 2, 2016

@szeiger I think that repo maintainer having to mess around with local repositories and proxies is hardly an improvement. I think what they want is a simple tar.gz that contains what SBT needs to build Scala.

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 2, 2016

Member

@martijnhoekstra I'd be in favor of deprecating, but keeping the ant scripts for 2.12 to not increase the pain as current distros work on shipping their next release. I just don't want to keep things around forever, because the pressure to migrate to the SBT build might not be there until the ant scripts are finally gone.

Member

soc commented Aug 2, 2016

@martijnhoekstra I'd be in favor of deprecating, but keeping the ant scripts for 2.12 to not increase the pain as current distros work on shipping their next release. I just don't want to keep things around forever, because the pressure to migrate to the SBT build might not be there until the ant scripts are finally gone.

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

I opened this as on-hold as a wake-up call for everyone who may still depend on ant. The migration to sbt has been going for the last 10 months with the goal to complete it for 2.12.0. The conclusion from the discussion here is not that it doesn't matter, but neither is it "we won't do it". Since we're so close to 2.12.0-RC1 it may be reasonable to keep the ant build around (even though it is no longer tested) in the 2.12.x branch and do any further build improvements in 2.13.

Contributor

szeiger commented Aug 2, 2016

I opened this as on-hold as a wake-up call for everyone who may still depend on ant. The migration to sbt has been going for the last 10 months with the goal to complete it for 2.12.0. The conclusion from the discussion here is not that it doesn't matter, but neither is it "we won't do it". Since we're so close to 2.12.0-RC1 it may be reasonable to keep the ant build around (even though it is no longer tested) in the 2.12.x branch and do any further build improvements in 2.13.

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

It just occurred to me that the Scala modules are sbt-based. How does Debian deal with that? You already cannot build a Scala distribution without using sbt.

Contributor

szeiger commented Aug 2, 2016

It just occurred to me that the Scala modules are sbt-based. How does Debian deal with that? You already cannot build a Scala distribution without using sbt.

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

For the modules we use another build system. For example scala-xml and scala-parser-combinator are built with Maven (see https://sources.debian.net/src/scala-xml/1.0.3-3/debian/pom.xml/). This works well because the projects are rather simple. But this would be more challenging with more complicated SBT builds.

ebourg commented Aug 2, 2016

For the modules we use another build system. For example scala-xml and scala-parser-combinator are built with Maven (see https://sources.debian.net/src/scala-xml/1.0.3-3/debian/pom.xml/). This works well because the projects are rather simple. But this would be more challenging with more complicated SBT builds.

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

How do you deal with dependencies in Maven? It also downloads half the Internet in order to build anything.

Contributor

szeiger commented Aug 2, 2016

How do you deal with dependencies in Maven? It also downloads half the Internet in order to build anything.

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

For Maven we use a local repository under the /usr/share/maven-repo directory. Each package installs its artifacts in this repository (see for example the files installed with the scala-xml package: https://packages.debian.org/sid/all/scala-xml/filelist). At build time we tell Maven to run in offline mode and to use the local repository instead of Maven Central (by setting the maven.repo.local property).

ebourg commented Aug 2, 2016

For Maven we use a local repository under the /usr/share/maven-repo directory. Each package installs its artifacts in this repository (see for example the files installed with the scala-xml package: https://packages.debian.org/sid/all/scala-xml/filelist). At build time we tell Maven to run in offline mode and to use the local repository instead of Maven Central (by setting the maven.repo.local property).

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

Well, that's the same approach I'd use for sbt as well. The only missing part is a way to pre-populate this repository (and possibly a 2nd one with Ivy layout for the sbt plugins) with all binary dependencies. Maybe we can reuse whatever Activator uses to do that.

(I still don't understand the motivation for building it yourself though. Is this done for purely political reasons? It doesn't make any sense for Java code. You can bootstrap as much as you want, in the end you will still download some old Scala compiler to start the bootstrap process, and in the best case (if you didn't make any mistakes in the non-trivial bootstrap process) you end up with a build that is identical to the binaries you can download.)

Contributor

szeiger commented Aug 2, 2016

Well, that's the same approach I'd use for sbt as well. The only missing part is a way to pre-populate this repository (and possibly a 2nd one with Ivy layout for the sbt plugins) with all binary dependencies. Maybe we can reuse whatever Activator uses to do that.

(I still don't understand the motivation for building it yourself though. Is this done for purely political reasons? It doesn't make any sense for Java code. You can bootstrap as much as you want, in the end you will still download some old Scala compiler to start the bootstrap process, and in the best case (if you didn't make any mistakes in the non-trivial bootstrap process) you end up with a build that is identical to the binaries you can download.)

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

There are several motivations for rebuilding everything from sources:

  • Debian is all about open source. Open source means you give the freedom to modify and rebuild your software to your users. It should be possible anywhere, even on a remote planet without an internet connection (provided you have the code for the application and its dependencies of course). The least we can do is ensuring it's indeed true.
  • Some Java applications contain a native part (for example JNA or snappy-java). Rebuilding from sources allows us to support more architectures than the one supported upstream (Debian packages are built on 19 architectures).
  • Debian aims at making all packages reproducible. This means that anyone building a package will get the exact same result (identical packages at the byte level, the checksums will match). This is important to prove that a given set of source files produces a given binary. This is even more important for a compiler since it could be modified to inject malicious code in users' applications (see https://reproducible-builds.org for more information, the talk by Lunar at the Chaos Communication Camp 2015 presents this very well).

ebourg commented Aug 2, 2016

There are several motivations for rebuilding everything from sources:

  • Debian is all about open source. Open source means you give the freedom to modify and rebuild your software to your users. It should be possible anywhere, even on a remote planet without an internet connection (provided you have the code for the application and its dependencies of course). The least we can do is ensuring it's indeed true.
  • Some Java applications contain a native part (for example JNA or snappy-java). Rebuilding from sources allows us to support more architectures than the one supported upstream (Debian packages are built on 19 architectures).
  • Debian aims at making all packages reproducible. This means that anyone building a package will get the exact same result (identical packages at the byte level, the checksums will match). This is important to prove that a given set of source files produces a given binary. This is even more important for a compiler since it could be modified to inject malicious code in users' applications (see https://reproducible-builds.org for more information, the talk by Lunar at the Chaos Communication Camp 2015 presents this very well).
@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

@ebourg The 3rd point is actually the one that I'm worried about. If you're building with your own build system (like the modules) and your own build scripts (for bootstrapping), there's always a risk that you end up with a build that claims to be Scala 2.12.0 but is actually different in subtle ways. The closer you can stick to the standard build procedure the better. Ideally, this would mean running scripts/jobs/integrate/bootstrap in the proper environment. Plan A should therefore be to make that environment available to an offline build.

Contributor

szeiger commented Aug 2, 2016

@ebourg The 3rd point is actually the one that I'm worried about. If you're building with your own build system (like the modules) and your own build scripts (for bootstrapping), there's always a risk that you end up with a build that claims to be Scala 2.12.0 but is actually different in subtle ways. The closer you can stick to the standard build procedure the better. Ideally, this would mean running scripts/jobs/integrate/bootstrap in the proper environment. Plan A should therefore be to make that environment available to an offline build.

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

I hope we've done the packaging well, for Scala 2.11.x we did the following:

  1. Build a first scala package with the external dependencies downloaded from internet
  2. Build the dependencies with the initial scala package
  3. Rebuild the scala package with the packaged dependencies, no external downloads (the Ant build is patched to use the local Maven artifacts).
  4. Rebuild the dependencies with the "clean" scala package.

ebourg commented Aug 2, 2016

I hope we've done the packaging well, for Scala 2.11.x we did the following:

  1. Build a first scala package with the external dependencies downloaded from internet
  2. Build the dependencies with the initial scala package
  3. Rebuild the scala package with the packaged dependencies, no external downloads (the Ant build is patched to use the local Maven artifacts).
  4. Rebuild the dependencies with the "clean" scala package.
@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

Erratum: for the modules we don't use Maven, we use a simple Make script invoking scalac and jar: https://sources.debian.net/src/scala-xml/1.0.3-3/debian/rules/

ebourg commented Aug 2, 2016

Erratum: for the modules we don't use Maven, we use a simple Make script invoking scalac and jar: https://sources.debian.net/src/scala-xml/1.0.3-3/debian/rules/

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 2, 2016

Contributor

@ebourg The process looks correct, under the assumption that there are no bugs. Do you verify that the results are identical to the official artifacts? In our bootstrap build, we run the full test suite after step 3, plus an additional build (strap) which rebuilds the the compiler and library again with the one built in 3 (quick) and verifies that the class files are identical.

Contributor

szeiger commented Aug 2, 2016

@ebourg The process looks correct, under the assumption that there are no bugs. Do you verify that the results are identical to the official artifacts? In our bootstrap build, we run the full test suite after step 3, plus an additional build (strap) which rebuilds the the compiler and library again with the one built in 3 (quick) and verifies that the class files are identical.

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 2, 2016

Currently the tests are disabled during the build due to missing dependencies (pax, partest, scalacheck). But if it's possible to run the tests independently after the build I can check.

ebourg commented Aug 2, 2016

Currently the tests are disabled during the build due to missing dependencies (pax, partest, scalacheck). But if it's possible to run the tests independently after the build I can check.

@milessabin

This comment has been minimized.

Show comment
Hide comment
@milessabin

milessabin Aug 4, 2016

Contributor

What are the Scala applications which Debian wants to package? Or is it just other downstream Scala dependencies?

Contributor

milessabin commented Aug 4, 2016

What are the Scala applications which Debian wants to package? Or is it just other downstream Scala dependencies?

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 4, 2016

Member

@milessabin Things like netlogo, apache-spark, ... judging from the RFPs and ITPs.

Member

soc commented Aug 4, 2016

@milessabin Things like netlogo, apache-spark, ... judging from the RFPs and ITPs.

@milessabin

This comment has been minimized.

Show comment
Hide comment
@milessabin

milessabin Aug 4, 2016

Contributor

Does it really make sense for these to be packaged?I can't imagine recommending someone to use a distro-packaged Spark.

Contributor

milessabin commented Aug 4, 2016

Does it really make sense for these to be packaged?I can't imagine recommending someone to use a distro-packaged Spark.

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Aug 4, 2016

Member

@milessabin I have no idea. I don't think it makes any sense at all to even use the host OpenJDK for development work, but applications depend on it, and each of those application is basically another ping to package all the things underneath. (Scala in this case.)

Member

soc commented Aug 4, 2016

@milessabin I have no idea. I don't think it makes any sense at all to even use the host OpenJDK for development work, but applications depend on it, and each of those application is basically another ping to package all the things underneath. (Scala in this case.)

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger Aug 8, 2016

Contributor

We discussed this in the Lightbend Scala team and the consensus is that we don't want to wait until 2.13. The reason is that we expect to do substantial feature development in the 2.12.x branch after 2.12.0 is released, not only bug fixes.

Contributor

szeiger commented Aug 8, 2016

We discussed this in the Lightbend Scala team and the consensus is that we don't want to wait until 2.13. The reason is that we expect to do substantial feature development in the 2.12.x branch after 2.12.0 is released, not only bug fixes.

@ebourg

This comment has been minimized.

Show comment
Hide comment
@ebourg

ebourg Aug 8, 2016

Any hope to switch to SBT as the main build system but at least preserve the Ant build for the duration of 2.12?

ebourg commented Aug 8, 2016

Any hope to switch to SBT as the main build system but at least preserve the Ant build for the duration of 2.12?

@milessabin

This comment has been minimized.

Show comment
Hide comment
@milessabin

milessabin Aug 8, 2016

Contributor

Would a packaged sbt-extras with a dependency on a JDK be a good minimal dependency route to providing a smooth OOTB SBT experience?

Contributor

milessabin commented Aug 8, 2016

Would a packaged sbt-extras with a dependency on a JDK be a good minimal dependency route to providing a smooth OOTB SBT experience?

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Aug 15, 2016

Member

yep, I have a branch where dbuild resolves a pre-built scala: https://github.com/adriaanm/community-builds/tree/scala-resolve -- will integrate that this week

Member

adriaanm commented Aug 15, 2016

yep, I have a branch where dbuild resolves a pre-built scala: https://github.com/adriaanm/community-builds/tree/scala-resolve -- will integrate that this week

@dwijnand dwijnand referenced this pull request in lightbend/dbuild Aug 15, 2016

Closed

Remove references to ant in ScalaBuildSystem #185

@adriaanm adriaanm referenced this pull request in scala/scala-dev Aug 16, 2016

Closed

2.12 community build omnibus ticket #203

13 of 16 tasks complete

@SethTisue SethTisue self-assigned this Aug 16, 2016

@SethTisue SethTisue modified the milestones: 2.12.1, 2.12.0-RC1 Aug 23, 2016

@adriaanm adriaanm modified the milestones: 2.12.0-RC1, 2.12.1 Aug 24, 2016

@SethTisue

This comment has been minimized.

Show comment
Hide comment
@SethTisue

SethTisue Sep 2, 2016

Member

tracking the community build work at scala/community-builds#274 ; once that's done, this PR will be merged. (sorry, this wasn't the outcome some people preferred. we read your feedback and considered carefully.)

I'd suggest continuing discussion about support for Debian et al at https://issues.scala-lang.org/browse/SI-9299

Member

SethTisue commented Sep 2, 2016

tracking the community build work at scala/community-builds#274 ; once that's done, this PR will be merged. (sorry, this wasn't the outcome some people preferred. we read your feedback and considered carefully.)

I'd suggest continuing discussion about support for Debian et al at https://issues.scala-lang.org/browse/SI-9299

szeiger and others added some commits Jul 29, 2016

Remove the ant build
- Remove ant scripts.

- Remove shell scripts that were specific to the ant build or the old
  `*.desired.sha1` binary artifact management.

- Remove `build.number`.

- Remove `src/build/maven` and `src/build/bnd`. The POM and Manifest
  metadata is generated in a different way by sbt.

@adriaanm adriaanm removed the on-hold label Sep 3, 2016

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Sep 3, 2016

Member

This should be ready to merge! I've started another community build that should fix the last glitch in my previous run, but nothing fundamental blocking the removal of ant. https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/544/console

Member

adriaanm commented Sep 3, 2016

This should be ready to merge! I've started another community build that should fix the last glitch in my previous run, but nothing fundamental blocking the removal of ant. https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/544/console

@adriaanm adriaanm referenced this pull request in scala/community-builds Sep 5, 2016

Closed

Resolve scala from maven #275

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Sep 5, 2016

Member

It took a little detour, but the community build is now ant-free!

Member

adriaanm commented Sep 5, 2016

It took a little detour, but the community build is now ant-free!

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Sep 5, 2016

Member

LGTM

Member

adriaanm commented Sep 5, 2016

LGTM

@adriaanm adriaanm merged commit d29df86 into scala:2.12.x Sep 5, 2016

6 checks passed

cla @szeiger signed the Scala CLA. Thanks!
Details
combined All previous commits successful.
integrate-ide [3179] SUCCESS. Took 8 s.
Details
validate-main [3582] SUCCESS. Took 69 min.
Details
validate-publish-core [3513] SUCCESS. Took 5 min.
Details
validate-test [3094] SUCCESS. Took 63 min.
Details

@adriaanm adriaanm modified the milestone: 2.12.0-RC1 Oct 29, 2016

@adriaanm adriaanm added the 2.12 label Oct 29, 2016

@SethTisue

This comment has been minimized.

Show comment
Hide comment
@SethTisue

SethTisue Feb 4, 2017

Member

is there a ticket of record, somewhere else, on the Debian situation?

Member

SethTisue commented Feb 4, 2017

is there a ticket of record, somewhere else, on the Debian situation?

@SethTisue

This comment has been minimized.

Show comment
Hide comment
@zaxebo1

This comment has been minimized.

Show comment
Hide comment
@zaxebo1

zaxebo1 Feb 4, 2017

a lot of organizations out of these installations(including ours) have a convention that - they strictly do not install/depend any package which is not in debian and ubuntu repo.

#5311 (comment)
Those 900/6000 installations aren't to "build things on top of it", I'd wager they're just to have a quick repl, maybe sometimes call scalac. For everything else they're using sbt or some other build tool instead of the scala distribution

zaxebo1 commented Feb 4, 2017

a lot of organizations out of these installations(including ours) have a convention that - they strictly do not install/depend any package which is not in debian and ubuntu repo.

#5311 (comment)
Those 900/6000 installations aren't to "build things on top of it", I'd wager they're just to have a quick repl, maybe sometimes call scalac. For everything else they're using sbt or some other build tool instead of the scala distribution

@SethTisue

This comment has been minimized.

Show comment
Hide comment
@SethTisue

SethTisue Feb 4, 2017

Member

please take any further Debian discussion to the JIRA ticket

Member

SethTisue commented Feb 4, 2017

please take any further Debian discussion to the JIRA ticket

@scala scala locked and limited conversation to collaborators Feb 4, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.