Mill BSP returns the same base directory for targets in different platforms

[azdrojowa123]

Hi !
Let me start by saying that I have no experience with Mill :(
In the IntelliJ scala plugin we got a ticket from the user related to the very long import of the project.
The project introduced in the ticket is a multi platform Mill project, and it is imported to IntelliJ with bsp.
I spent some time with this project, and noticed that all build targets (for different platforms) which are returned from the Mill server contain the same base directory. I mean, for example, all the targets listed below contain the same base directory:

• kubernetes-client[2.13.10].js
• kubernetes-client[2.13.10].jvm
• kubernetes-client[2.13.10].native

The situation is analogous with test targets (the base directory is the same but with the /test suffix).
Is it an expected behaviour from the mill server, or is it a bug on the server side?
I also consider that there may be some configuration error.
Additionally, I noticed that the server reports all generated sources separately, which results in hundreds of single source entries.
Why isn't it merged and reported as generated source directories?

[lefou]

Hi @azdrojowa123, thanks for reaching out.

Hi ! Let me start by saying that I have no experience with Mill :( In the IntelliJ scala plugin we got a ticket from the user related to the very long import of the project. The project introduced in the ticket is a multi platform Mill project, and it is imported to IntelliJ with bsp. I spent some time with this project, and noticed that all build targets (for different platforms) which are returned from the Mill server contain the same base directory.

If all these sub-modules share the same sources and resources, it make much sense to me to let them have the same base directory. The BSP spec does not prohibit this scenario.

Yes, I'm aware, that IJ has issues with multiple modules sharing the same directory, but I thought, this limitation does not exists in it's BSP support, only when using the classic Java / Scala setup. (Fun fact: In our non-BSP IDEA project generator, we have some option to override the base directory specifically for IDEA with intellijModulePath.)

Is it an expected behaviour from the mill server [..] ?

Yes, I would think so. Of course it depends on the concrete project setup. The BSP server is reporting the same base directory as Mill is using for each module.

I also consider that there may be some configuration error.

If it builds with Mill CLI, it's probably not an error but some mismatch of assumptions how Scala project need to be organized. At least, the BSP spec does not prohibit this scenario.

Additionally, I noticed that the server reports all generated sources separately, which results in hundreds of single source entries. Why isn't it merged and reported as generated source directories?

I can't tell you, as it's up to the specific project how it is reporting generated sources. Technically, in BSP it's a list of URIs representing file or directory locations. There is no specific requirement to choose one over the other and it should not make any difference. Also in Mill you return a Seq[PathRef], which can represent either files or directories. As Mill does not enforce strict separation of source directories for different languages (Java, Scala or Kotlin files can all live in the same source directory), returning just files might make sense sometime, although it's not commonly used, I think. Typically, when multiple files are generated by the same tool / target, those end up in the same directory and only that is returned.

/cc @lolgab, as it's his project which is referenced in the originating YouTrack ticket.

[lefou]

I checked the project, and indeed, the generatedSources returns each generated source file separately, although all files are located in the same directory. When generating the IDEA project with mill mill.idea.GenIdea/idea, you can also see the many generated source roots. Even as human, I'd prefer to just return the directories from generatedSources. (If you are interested in the effective source files used in compilation, you can still run mill show __.allSourceFiles.)

[azdrojowa123]

Thank you for checking.
But there are no plans to make a change on the server side?
In the background, I am working on optimising the import so that it does not take so much time. But the number of content roots will probably stay the same

[lefou]

I guess, there is not much we can do about it. It's rather simple to change the concrete generatedSources in this project, but rather hard to generically consolidate a bunch of files to some directory. Most and for all, it's semantically not correct, a directory isn't the same as it's including files. And also, we would need to apply lots of checks (Are there other files in the dir? Are there any semantic requirements due to the target language, e.g. Java requires proper package/directory correlations? And probably a lot more.). I think the BSP server should reflect the project structure as close as it is in Mill and intended by its authors/users.

[azdrojowa123]

I see your point