Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement @main functions #6898

Merged
merged 13 commits into from Jul 30, 2019

Conversation


/** An annotation that designates a main function
*/
class main extends scala.annotation.Annotation {}

This comment has been minimized.

Copy link
@hepin1989

This comment has been minimized.

Copy link
@odersky

odersky Jul 21, 2019

Author Contributor

👍

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

Are we not making it final?

compiler/src/dotty/tools/dotc/ast/MainProxies.scala Outdated Show resolved Hide resolved
}

object Test2 {
@main val x = 2 // does nothing, should this be made an error?

This comment has been minimized.

Copy link
@hrhino

hrhino Jul 21, 2019

Contributor

I'd agree that @main val shouldn't be allowed.

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

I second this. I can't see any good use cases for tagging a val with @main, and thinking about its semantics already stretches the imagination beyond the result being reasonable.

}

class Foo {
@main def f = () // does nothing, should this be made an error?

This comment has been minimized.

Copy link
@hrhino

hrhino Jul 21, 2019

Contributor

Probably more trouble than it's worth.

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

Technically this is a correct program that does nothing. I think we should allow empty programs.

@soronpo

This comment has been minimized.

Copy link
Contributor

commented Jul 21, 2019

Where can I discuss the deprecation of App ?

@soronpo

This comment has been minimized.

Copy link
Contributor

commented Jul 21, 2019

In what way @main is a "special" feature and not just a private case for annotation macros (which don't exist yet in dotty)?

@odersky

This comment has been minimized.

Copy link
Contributor Author

commented Jul 21, 2019

@soronpo If we had annotation macros @main would indeed be expressible with them. But we don't have them, and will probably not get them for 3.0 (maybe later, there are no plans at the moment either way). At the same time we have to do something about the App situation. Current App requires DelayedInit which we don't have either. So we should already promote something else before we go into feature freeze. If @main methods can in the future be expressed with macro annotations so much the better!

A `@main` annotation on a method turns this method into an executable program.
Example:
```scala
@main def happyBirthday(age: Int, name: String, others: String*) = {

This comment has been minimized.

Copy link
@soronpo

soronpo Jul 21, 2019

Contributor

Should we encourage the use of : Unit here or not?

@soronpo

This comment has been minimized.

Copy link
Contributor

commented Jul 21, 2019

What is stopping us from annotating an object instead of/in addition to a def?
Currently App is treated differently in some DelayedInit fashion which I understand we wish to get rid of.

object MyApp extends App {
  //do something
}

I think we should also allow @main to work on an object with a def main so App can remain the same, but without extending DelayedInit:

@main object MyApp extends App {
  //do something
}

In my code base I have:

 trait DSLApp extends App { 
  //DSL Library stuff here
}

object DSLUserApp extends DSLApp {
  //User stuff here
}

I would very much like the ability to extend App (or something else), and it does not mean we need to carry the burden of DelayedInit.

@jducoeur

This comment has been minimized.

Copy link
Contributor

commented Jul 21, 2019

Along the lines of @soronpo's point: the FP community makes pretty heavy use of IOApp, IIRC. So this notion of wanting a specialized main entry point is pretty widespread -- we'd be well-advised to provide some means of supporting that.

@odersky

This comment has been minimized.

Copy link
Contributor Author

commented Jul 22, 2019

IOApp is unaffected by this change, you can still use it as is.

The general problem with annotating an object for main is that the main program is the run in the object's initializer code. The JIT tends to be less aggressive in optimizing such code. This is mainly a problem for micro-benchmarks, but that matters, since you get your reputation ruined quickly if people run the same trivial program in Java and Scala and the Scala version is x times slower. DelayedInit gets around that by placing the same code in a method, but that seriously contorts semantics, so we want to get rid of it.

Note that that's not a problem for IOApp, because that one requires you to place your code in the run method.

@odersky

This comment has been minimized.

Copy link
Contributor Author

commented Jul 22, 2019

I forgot the more important reason why we do not want to put initalization code in object initializers: It's prone to deadlocks if your program is multi-threaded.

package scala.util

/** A utility object to support command line parsing for @main methods */
object CommandLineParser {

This comment has been minimized.

Copy link
@julienrf

julienrf Jul 22, 2019

I understand the goal of simplifying the way to implement application entry points. In practice, most of the users just write a def main(args: Array[String]): Unit method, and manually parse the application arguments. For these cases, this PR definitely improves the developer experience.

That being said, by introducing a way to parse the program arguments in the compiler, we will also establish a “standard” way of doing that. And, I’m a little bit worried about the introduction of the CommandLineParser and FromString abstractions. For now, they are simple but I think we can already anticipate the following requests soon: can we also support optional parameters? Can we support named parameters? How could we customize the error message in case of invalid parameters? Can we print all the error messages instead of stopping at the first one? Should we use Either instead of relying on exceptions to model parsing failures? All these requests are legitimate, in my opinion, but if we try to fulfill them the size of the CommandLineParser and FromString modules will be 10x bigger. Then, the question will be “should we keep that thing in the compiler, or should we move it to the library land (like scopt, or optparse-applicative)?”. We can make the decision of keeping CommandLineParser simple and not honor these requests, “by design”, but then I’m worried about introducing a “standard” way of parsing command-line arguments which does not scale to advanced use cases.

Unfortunately, I don’t have a simple idea of how to decouple @main expansion from CommandLineParser (so that users could plug in the parsing + reporting logic they want). I still think the most flexible way to go is to require users to forward a plain old main method to whatever they want:

def happyBirthday(age: Int, name: String, others: String*): Unit =def main(args: Array[String]): Unit = {
  val happyBirthdayParser =// some code that relies on a library for arguments parsing
  happyBirthdayParser.run(args)(happyBirthday)
}

But I agree that this approach is not beginner-friendly…

This comment has been minimized.

Copy link
@odersky

odersky Jul 22, 2019

Author Contributor

I believe we can stay firm and keep CommandLineParser simple. If one wants something else, one can perfectly well define a traditional main method, or define the entry point like this:

@main def f(args: String*) = ...
@liufengyun

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2019

I find the following is more natural, no semantic magic with names:

The code

    @main class HappyBirthday(age: Int, name: String, others: String*) {
        // code
    }

expands to

    class HappyBirthday(age: Int, name: String, others: String*) {
	// code
    }

    object HappyBirthday {
      import scala.util.{CommndLineParser => CLP}
      def main(args: Array[String]): Unit = {
	try
	  new HappyBirthday(
	      CLP.parseArgument[Int](args, 0),
	      CLP.parseArgument[String](args, 1),
	      CLP.parseRemainingArguments[String](args, 2))
	catch {
	  case error: CLP.ParseError => CLP.showError(error)
	}
      }
    }
@liufengyun

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2019

I don’t see how benchmarking is an issue, as usage of JMH will requires the code to be in a method.

@soronpo

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2019

@odersky is supporting both forms (@main object ..., @main def ...) out of the question? The latter is used in most cases and the first to ease migration and allow the example I've given above.
Or alternatively if we can do @main class MyApp extends (Array[String] => Unit) {}, that is fine too I believe.

@odersky

This comment has been minimized.

Copy link
Contributor Author

commented Jul 22, 2019

@odersky is supporting both forms (@main object ..., @main def ...) out of the question?

For me, yes. We don't need more than one convenient way to do main programs. @main def has the advantage that it's what ammonite does, so we close the gap between scripting and normal programs.

@anatoliykmetyuk
Copy link
Contributor

left a comment

Looks fine to me, except the points specified below. Another thing: doesn't work with SBT.

From Dotty repo, on this PR branch, do dotty-bootstrapped/publishLocal. Then:

sbt new lampepfl/dotty.g8 --name mainmethods
cd mainmethods

In buld.sbt, set:

val dottyVersion = "0.18.0-bin-SNAPSHOT"

Then in Main.scala, write:

object Main {
  // def main(args: Array[String]): Unit = println("foo")

  @main def f(): Unit = {
    println("Hello world!")
    println(msg)
  }

  def msg = "I was compiled by dotty :)"

}

Then do sbt run:

[error] java.lang.RuntimeException: No main class detected.
[error] 	at scala.sys.package$.error(package.scala:26)
[error] (Compile / bgRun) No main class detected.
[error] Total time: 11 s, completed Jul 26, 2019 2:27:44 PM

However, if you compile with dotc instead, it works (I tested it from SBT console):

dotc ../ecosystem/mainmethods/src/main/scala/Main.scala
dotr f

(change the argument to dotc to the path to Main.scala on your machine).

Now in Main.scala, uncomment def main(args: Array[String]): Unit and run dotc again. Then from Dotty repo in bash, do:

tsf-428-wpa-0-238:dotty anatolii$ javap Main$ f
Compiled from "Main.scala"
public final class Main$ implements scala.Serializable {
  public static final Main$ MODULE$;
  public static {};
  public void main(java.lang.String[]);
  public void f();
  public java.lang.String msg();
}
Compiled from "Main.scala"
public final class f {
  public f();
  public static void main(java.lang.String[]);
}

I believe sbt may not be prepared to the fact that main methods may be static and/or come from classes and not objects (i.e. classes that end with $). IMO it is best to keep the new main methods compliant to the way the main methods are usually expressed in Scala.

}
```
**Note**: The `<static>` modifier above expresses that the `main` method is generated
as a static method of class `happyBirthDay`. It is not available for user programs in Scala. Regular "static" members are generated in Scala using objects instead.

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

Why do we make this design decision here? Looks like a limitation to me. E.g. I may want to write a library of utility functions that may depend on each other:

@main def disableFile(path: Path): Unit =
  // Change the `path`'s extension from `scala` to `disabled`

@main def enableFile(path: Path): Unit =
  // Change the `path`'s extension from `disabled` to `scala`

@main def filterFolder(folder: Path, fileNamePattern: Regex): Unit = {
  // Disable all the files except those the names of which match the given regex pattern
  val allFiles: List[Path] = // get all the files present in the folder
  allFiles.foreach(enableFile)  // Make sure we start with a clean folder, unaffected by previous runs of this or sister programs
  allFiles.filterNot(file => fileNamePattern.matches(file.name)).foreach(disableFile)
}

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

It is not available for user programs in Scala

Also, it doesn't seem that this limitation is true:

object Main {
  def main(args: Array[String]): Unit = { f(); println("foo") }

  @main def f(): Unit = {
    println("Hello world!")
    println(msg)
  }

  @main def g(): Unit = f()

  def msg = "I was compiled by dotty :)"

}

Compiles and runs fine.

This comment has been minimized.

Copy link
@hrhino

hrhino Jul 27, 2019

Contributor

Doesn't that mean that the synthetic method main can't be called, rather than that the @main method written in source can't be called?

the required types. If a check fails, the program is terminated with an error message.
Examples:
```
> scala happyBirthday 22

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

What is the exact lifecycle of such a program here? What's happyBirthday? Do we need to compile it with dotc and run scala from the folder in which the class files end up? Can we do scala happyBirthday.scala 22 – that is without the necessity to compile the source? I think we need to elaborate on this in the docs.


/** An annotation that designates a main function
*/
class main extends scala.annotation.Annotation {}

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

Are we not making it final?

library/src/scala/util/FromString.scala Show resolved Hide resolved
}

object Test2 {
@main val x = 2 // does nothing, should this be made an error?

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

I second this. I can't see any good use cases for tagging a val with @main, and thinking about its semantics already stretches the imagination beyond the result being reasonable.

}

class Foo {
@main def f = () // does nothing, should this be made an error?

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

Technically this is a correct program that does nothing. I think we should allow empty programs.

println("Hello, world!")

object A {
@main def foo(x: Int, y: String, zs: Float*) =

This comment has been minimized.

Copy link
@anatoliykmetyuk

anatoliykmetyuk Jul 26, 2019

Contributor

Is this code executed anywhere?

compiler/src/dotty/tools/dotc/ast/MainProxies.scala Outdated Show resolved Hide resolved

@odersky odersky force-pushed the dotty-staging:add-main-fn branch from a3fb089 to b935782 Jul 26, 2019

@smarter

This comment has been minimized.

Copy link
Member

commented Jul 26, 2019

I believe sbt may not be prepared to the fact that main methods may be static and/or come from classes and not objects (i.e. classes that end with $).

We control main class detection, not sbt:

if (sym.isStatic && defType == DefinitionType.Module && ctx.platform.hasMainMethod(sym)) {
_mainClasses += name
}

@odersky

This comment has been minimized.

Copy link
Contributor Author

commented Jul 26, 2019

Thanks for the hint @smarter! I tried a fix. @anatoliykmetyuk Can you test whether it works now with sbt?

@@ -230,7 +230,7 @@ private class ExtractAPICollector(implicit val ctx: Context) extends ThunkHolder

allNonLocalClassesInSrc += cl

if (sym.isStatic && defType == DefinitionType.Module && ctx.platform.hasMainMethod(sym)) {
if (sym.isStatic && ctx.platform.hasMainMethod(sym)) {

This comment has been minimized.

Copy link
@allanrenucci

allanrenucci Jul 26, 2019

Member

I think you need to drop sym.isStatic to allow classes to have main method.

@odersky odersky force-pushed the dotty-staging:add-main-fn branch from 8de209b to 8b2737d Jul 26, 2019

@smarter

This comment has been minimized.

Copy link
Member

commented Jul 26, 2019

We should add a scripted sbt test to verify the main class detection works: https://github.com/lampepfl/dotty/tree/master/sbt-dotty/sbt-test/sbt-dotty

@anatoliykmetyuk

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2019

As of 8b2737d, SBT detects the @main methods

@odersky odersky force-pushed the dotty-staging:add-main-fn branch 3 times, most recently from 18c2061 to 5cdf2d6 Jul 27, 2019

odersky added 13 commits Jul 20, 2019
Check @main annotations for applicability
Check applicability of annotations only once they are completed
Avoid IFT in Reporter#reportsErrorsFor
If we make this method take an implicit function rather than a normal one we get a build error with trace:
```
[error] ## Exception when compiling 9 sources to /Users/odersky/workspace/dotty/sbt-bridge/src/target/classes
[error] Type scala.ImplicitFunction1 not present
[error] sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:117)
[error] sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:125)
[error] sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
[error] sun.reflect.generics.repository.ConstructorRepository.getParameterTypes(ConstructorRepository.java:94)
[error] java.lang.reflect.Executable.getGenericParameterTypes(Executable.java:283)
[error] java.lang.reflect.Method.getGenericParameterTypes(Method.java:283)
[error] sbt.internal.inc.ClassToAPI$.parameterTypes(ClassToAPI.scala:566)
[error] sbt.internal.inc.ClassToAPI$.methodToDef(ClassToAPI.scala:318)
[error] sbt.internal.inc.ClassToAPI$.$anonfun$structure$1(ClassToAPI.scala:182)
[error] sbt.internal.inc.ClassToAPI$.$anonfun$mergeMap$1(ClassToAPI.scala:400)
[error] scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:240)
[error] scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32)
[error] scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29)
[error] scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:37)
[error] scala.collection.TraversableLike.flatMap(TraversableLike.scala:240)
[error] scala.collection.TraversableLike.flatMap$(TraversableLike.scala:237)
[error] scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
[error] sbt.internal.inc.ClassToAPI$.merge(ClassToAPI.scala:411)
[error] sbt.internal.inc.ClassToAPI$.mergeMap(ClassToAPI.scala:400)
[error] sbt.internal.inc.ClassToAPI$.structure(ClassToAPI.scala:182)
[error] sbt.internal.inc.ClassToAPI$.x$2$lzycompute$1(ClassToAPI.scala:133)
[error] sbt.internal.inc.ClassToAPI$.x$2$1(ClassToAPI.scala:133)
[error] sbt.internal.inc.ClassToAPI$.instance$lzycompute$1(ClassToAPI.scala:133)
[error] sbt.internal.inc.ClassToAPI$.instance$1(ClassToAPI.scala:133)
[error] sbt.internal.inc.ClassToAPI$.$anonfun$toDefinitions0$1(ClassToAPI.scala:140)
[error] xsbti.api.SafeLazyProxy$$anon$1.get(SafeLazyProxy.scala:26)
[error] xsbti.api.SafeLazy$Impl.get(SafeLazy.java:58)

```

@odersky odersky force-pushed the dotty-staging:add-main-fn branch from 16b3efe to eb374a1 Jul 29, 2019

@odersky

This comment has been minimized.

Copy link
Contributor Author

commented Jul 29, 2019

@anatoliykmetyuk I think everything we agreed on is done. Merge?

@anatoliykmetyuk anatoliykmetyuk merged commit b3826d0 into lampepfl:master Jul 30, 2019

2 checks passed

CLA User signed CLA
Details
continuous-integration/drone/pr Build is passing
Details

@anatoliykmetyuk anatoliykmetyuk deleted the dotty-staging:add-main-fn branch Jul 30, 2019

@anatoliykmetyuk anatoliykmetyuk added this to the 0.18 Tech Preview milestone Aug 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.