Check up to 5 nested project folders #2995

mkurz · 2023-02-28T23:21:25Z

Some projects set up plugins in project/project/plugins.sbt and even in project/project/project/plugins.sbt (I worked on such a private project, so I can't share details, but such things do exists 😉)

For example, Play and the sbt-web-build-base project put sbt-buildinfo in their project/project/plugins.sbt files:

However, as you can see, even though scala-steward is enabled and visiting those repos, that plugin never gets updated (latest release is 0.11.0, also see maven central)

Making scala-steward check and update those dependencies is actually pretty easy and straight forward:
We just need to set up the special scala-steward-StewardPlugin_<version>.scala plugin in the corresponding project/project/project/project/ and project/project/project/ folders so they are available as plugins for the parent "build" and then just enter those sub "build" projects via reloadPlugins again and also run stewardDependencies which also logs the dependencies of those projects.
Nothing special, just a couple of more lines logged and parsed by scala steward.
I also think the test that I adjusted in this pull request explains everything.

To make sure everything works 100% I set up following test repo:

https://github.com/mkurz/three-level-sbt-plugins-project

As you can see it has three levels of project folders including plugins.sbt:

So I added outdated plugins to each plugins.sbt and the build.sbt and ran scala steward with this pull request applied locally and voilà, scala steward opened all expected pull requests:

BTW:
If you want to test out what the output of the sbt command ran by sbt looks like, you can check out the steward-plugin-file branch of my test repo and ran the sbt command I show in the README. It is exactly what scala steward runs to figure out the versions. (For this to stewardDependencies command to work I manually added the StewardPlugin_1_3_11.scala files myself, see the README again).

codecov · 2023-02-28T23:41:05Z

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.04 🎉

Comparison is base (5357f28) 90.98% compared to head (ed1fae1) 91.02%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2995      +/-   ##
==========================================
+ Coverage   90.98%   91.02%   +0.04%     
==========================================
  Files         160      160              
  Lines        3305     3322      +17     
  Branches      297      303       +6     
==========================================
+ Hits         3007     3024      +17     
  Misses        298      298

Impacted Files	Coverage Δ
...a/org/scalasteward/core/buildtool/sbt/SbtAlg.scala	`98.55% <100.00%> (+0.24%)`	⬆️

... and 2 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

mkurz · 2023-03-01T12:37:10Z

Would be nice if this PR makes it into v0.23.0 😉

mkurz · 2023-03-06T15:16:48Z

Hey! I am pretty sure this pull request is good to go, reviews would be highly appreciated. Thanks!

fthomas · 2023-03-07T10:52:52Z

Thanks for the PR, @mkurz! I'm fine with updating dependencies in meta-meta-builds and even meta-meta-meta-builds but I think it would be better to check how many nested project directories there are before calling reload plugins; stewardDependencies thrice. My concern is that calling reload plugins can take a few seconds and we're slowing down Scala Steward unnecessarily for the majority of sbt projects.

I would check how many (nested) project directories there are and then only adding the StewardPlugin_*.scala to them if they exist. We would then only need to call reload plugins; stewardDependencies as much as needed.

mkurz · 2023-03-07T11:00:09Z

I get it. Will update when I find time, thanks!

mkurz · 2023-03-08T22:56:56Z

@fthomas Updated. It works exactly like you asked for, depending on the levels of meta builds. I tested that again on a real project: https://github.com/mkurz/three-level-sbt-plugins-project/pulls (I deleted the repo and recreated it so it was fresh again). As you can see a pull request for each meta build was created. After one run I even created one more level with the scalafmt plugin and on the second run scala-steward also created the according pull request (you can see that the sbt-scalafmt pull request was opened 4 minutes later).

fthomas · 2023-03-09T16:39:32Z

modules/core/src/main/scala/org/scalasteward/core/buildtool/sbt/SbtAlg.scala

+          .iterate(buildRootDir / project)(_ / project)
+          .takeWhile(p => p.exists && p.isDirectory)
+          .size,
+        1 // There is always at least one meta build, even if there is no project folder


I'm wondering if we should drop this Math.max call. If there is no project folder, there aren't any *.sbt files either which could contain dependencies (plugins) that Scala Steward could update. And if there is nothing to update, why loading the meta-build and calling stewardDependencies? 🤔

In theory yes. However removing Math.max will break many many tests because of lines like

scala-steward/modules/core/src/test/scala/org/scalasteward/core/buildtool/sbt/SbtAlgTest.scala

Line 29 in a6ff326

Cmd("write", s"$repoDir/project/project/scala-steward-StewardPlugin_1_3_11.scala"),

scala-steward/modules/core/src/test/scala/org/scalasteward/core/buildtool/sbt/SbtAlgTest.scala

Line 43 in a6ff326

Cmd("rm", "-rf", s"$repoDir/project/project/scala-steward-StewardPlugin_1_3_11.scala"),

which assume scala-steward always checks/updates the project folder. And I would keep all those tests how they are currently.
Also in real life I never saw a sbt project without a project folder (sbt even creates the project folder when running in an empty folder).

I would just keep it.

And I would keep all those tests how they are currently.

We can keep them if we use FileAlg#isDirectory for checking the project folders as I've done in 1a97fc5. Another benefit of using FileAlg is that the tests are now sensitive to the actual number of project folders.

Great! Thanks!

fthomas · 2023-03-09T16:39:59Z

Updated.

Thanks @mkurz!

This allows to remove the `Math.max(..., 1)` call since our tests actually count the `project` directories that are present in `MockState`.

mkurz · 2023-03-09T21:40:18Z

LGTM 👍

mkurz · 2023-03-09T21:50:19Z

Tested again with a real world repo, everything works great: https://github.com/mkurz/three-level-sbt-plugins-project/pulls

fthomas · 2023-03-09T22:02:26Z

Awesome, thanks for testing again!

mkurz force-pushed the check_3-level-plugins branch from ce44d63 to 9ab8af3 Compare February 28, 2023 23:34

This was referenced Feb 28, 2023

Make sure remote is always called "origin" when cloning git repo #2996

Merged

PRs not being sent in for various projects VirtusLab/scala-steward-repos#98

Open

mkurz mentioned this pull request Mar 1, 2023

Make use of sbt-dependency-submission playframework/play-meta#242

Open

Update up to 3 levels of plugins (project/project[/project/]plugins.sbt)

ae2a739

mkurz force-pushed the check_3-level-plugins branch from 9ab8af3 to ae2a739 Compare March 8, 2023 22:43

mkurz changed the title ~~Update up to 3 levels of plugins (project/project[/project/]plugins.sbt)~~ Check all nested project folders Mar 8, 2023

fthomas reviewed Mar 9, 2023

View reviewed changes

Use FileAlg#isDirectory in metaBuildsCount

1a97fc5

This allows to remove the `Math.max(..., 1)` call since our tests actually count the `project` directories that are present in `MockState`.

mkurz changed the title ~~Check all nested project folders~~ Check up to 5 nested project folders Mar 9, 2023

Typo

ed1fae1

fthomas added this to the 0.23.0 milestone Mar 10, 2023

fthomas added the enhancement New feature or request label Mar 10, 2023

fthomas approved these changes Mar 10, 2023

View reviewed changes

fthomas merged commit c322669 into scala-steward-org:main Mar 10, 2023

mkurz deleted the check_3-level-plugins branch March 10, 2023 06:51

mkurz mentioned this pull request Mar 23, 2023

sbt-buildinfo 0.11.0 (was 0.10.0) playframework/playframework#11718

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check up to 5 nested project folders #2995

Check up to 5 nested project folders #2995

mkurz commented Feb 28, 2023 •

edited

Loading

codecov bot commented Feb 28, 2023 •

edited

Loading

mkurz commented Mar 1, 2023 •

edited

Loading

mkurz commented Mar 6, 2023

fthomas commented Mar 7, 2023

mkurz commented Mar 7, 2023

mkurz commented Mar 8, 2023

fthomas Mar 9, 2023

mkurz Mar 9, 2023

fthomas Mar 9, 2023

mkurz Mar 9, 2023

fthomas commented Mar 9, 2023

mkurz commented Mar 9, 2023

mkurz commented Mar 9, 2023

fthomas commented Mar 9, 2023

Check up to 5 nested project folders #2995

Check up to 5 nested project folders #2995

Conversation

mkurz commented Feb 28, 2023 • edited Loading

codecov bot commented Feb 28, 2023 • edited Loading

Codecov Report

mkurz commented Mar 1, 2023 • edited Loading

mkurz commented Mar 6, 2023

fthomas commented Mar 7, 2023

mkurz commented Mar 7, 2023

mkurz commented Mar 8, 2023

fthomas Mar 9, 2023

Choose a reason for hiding this comment

mkurz Mar 9, 2023

Choose a reason for hiding this comment

fthomas Mar 9, 2023

Choose a reason for hiding this comment

mkurz Mar 9, 2023

Choose a reason for hiding this comment

fthomas commented Mar 9, 2023

mkurz commented Mar 9, 2023

mkurz commented Mar 9, 2023

fthomas commented Mar 9, 2023

mkurz commented Feb 28, 2023 •

edited

Loading

codecov bot commented Feb 28, 2023 •

edited

Loading

mkurz commented Mar 1, 2023 •

edited

Loading