Split swarm libraries #1069

xsebek · 2023-01-31T21:24:35Z

split libraries to enforce dependency policy
- add -Wunused-packages to keep the dependencies minimal
closes Split Swarm components into libraries #1043

This is how our dependencies look like and the libraries should enforce that we keep everything well ordered:

find src -name '*.hs' | xargs graphmod -q -p | dot -Tsvg -o swarm.svg

On a high level, the libraries look like this:

graph TD;
    swarm-language --> swarm-util;
    swarm-language-server --> swarm-util;
    swarm-language-server --> swarm-language;
    swarm-game --> swarm-util;
    swarm-game --> swarm-language;
    swarm-tui --> swarm-util;
    swarm-tui --> swarm-language;
    swarm-tui --> swarm-game;
    swarm-tui --> swarm-version;
    swarm-app --> swarm-util;
    swarm-app --> swarm-language;
    swarm-app --> swarm-game;
    swarm-app --> swarm-version;
    swarm-app --> swarm-tui;
    swarm-exe --> swarm-app;
    swarm-exe --> swarm-language;
    swarm-exe --> swarm-language-server;
    swarm-exe --> swarm-version;
    swarm-exe --> swarm-tui;

byorgey · 2023-02-01T03:46:36Z

We also need to make sure to update stack.yaml as well as the release action.

- Move the Location module to Game - Move Achievements modules to Game - Move Render and Model modules to TUI - split off from #1069 - part of #1043

src/swarm-language/Swarm/Language/Syntax.hs

This avoids using `Brick.AttrName` and we can remove the `-fno-warn-orphans` from `Attr.hs`. As a bonus, we can now specify a terrain attribute (`terrain_stone`) in a scenario along with the `default` attribute. - split off from #1069 - part of #1043

- remove the useless `DirInfo` type - move the `Direction -> Heading` functions to the `Location` module - split off from #1069 - part of #1043 - part of #707

xsebek · 2023-02-20T12:14:12Z

src/swarm-game/Swarm/Game/Util/Failure.hs

@@ -11,7 +11,7 @@
 --
 -- These failures are often not fatal and serve
 -- to create common infrastructure for logging.
-module Swarm.TUI.Model.Failure where
+module Swarm.Game.Util.Failure where


@kostmo FYI I had to move the module back to the Game just to compile, but the change in #1116 is better (no Util directory and less code in the Util module itself) so I will rebase once merged. 👍

It seems that I am also susceptible to making silly Util modules when desperate. 😅

Splitting the libraries with Cabal has been a great learning experience for me, so I hope not to repeat my mistake from #982 where I put this module in TUI in the first place. 🤦

swarm.cabal

xsebek · 2023-02-20T12:19:28Z

swarm.cabal

@@ -295,7 +625,7 @@ test-suite swarm-integration
 benchmark benchmark
  import:         stan-config, common, ghc2021-extensions
  main-is:        Benchmark.hs
-  hs-source-dirs: bench
+  hs-source-dirs: test/bench


@byorgey I moved the app and bench directories to src and test respectively.

Sometimes I forgot to recompile the benchmark until the CI told me, so I hope seeing it with other tests will remind me about it. 🙂

I did not save file before commit

xsebek · 2023-02-20T12:54:44Z

.github/workflows/haskell-ci.yml

      - name: checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@v3


Speaking of CI, it looks like regenerating this file will get rid of GitHub warnings about deprecated actions. 🙂

Let's hope it will all work nicely. 🤞

xsebek · 2023-02-20T15:06:49Z

@byorgey @kostmo this will be ready to merge after #1116, please take a look. 🙂

If there are any old PR you wish to resurrect that can not be rebased just because of this PR, I will happily rebase them myself rather than let this wait for all PRs to be slowly merged.

However, because this is almost purely moving modules into library folders, git rebase should handle most cases. 🤞

byorgey

I still mostly think that this is a good idea but I am still a bit concerned about the maintenance burden of managing package versions, doing releases, etc. What's our plan for updating the release action?

byorgey · 2023-02-20T15:28:54Z

swarm.cabal

+      StrictData
+
+
+library


Why doesn't this library have a name?

It did not before, it is simply swarm. I am not 100% clear about whether we need this or if swarm-app would be better or maybe if it could even merge with swarm-exe.

I do not think we need to do anything about this right now.

byorgey · 2023-02-20T15:30:38Z

swarm.cabal

@@ -79,12 +80,113 @@ common ghc2021-extensions
      -- Not GHC2021, but until we get \cases we use \case a lot
      LambdaCase

-library
+
+library boolexpr


What's our plan for boolexpr? I don't think we can have an internal library called boolexpr when there is already a package on Hackage with that name.

It turns out we can... at least until we try to publish it on Hackage. 😄

Do we want to rename it swarm-boolexpr and support it indefinitely @kostmo?

I like that it is a separate package so we at least know that it does not have any Swarm dependency and could be upstreamed.

@kostmo @byorgey let's discuss this in #934 👀

xsebek · 2023-02-20T16:11:42Z

@byorgey the release action can stay exactly as it is. All commands it uses stay the same, just as you can still use the same Stack/Cabal commands as before.

I am still a bit concerned about the maintenance burden of managing package versions, doing releases

package versions: once we thin out swarm-util the lowest library can set the package version range and the dependent libraries can remove them - this will result in two kinds of dependencies:
- common to basically all (base)
- specific to one package (unification-fd, brick)
we can use tooling like packdeps or cabal-bounds to ease the pain
we can now rely on -Wunused-packages to keep things minimal
ultimately we will not make changes too often
releases stay the same (write CHANGELOG entry, bump Cabal version, push git tag 🏖️ 🍹 )

byorgey · 2023-02-20T16:31:45Z

OHHHH, I just realized what my confusion was. I thought by doing this we were going to have to upload a bunch of separate packages to Hackage, like swarm-language, swarm-game, etc. But we can still just upload the one swarm package to Hackage, which happens to have a bunch of internal library components... right?

If that's true, then none of my concerns are valid (e.g. version number management, release tooling, and what we will do with boolexpr)!

xsebek · 2023-02-20T16:51:52Z

But we can still just upload the one swarm package to Hackage, which happens to have a bunch of internal library components... right?

First, yes. Second, I am not sure.

The Hackage release is still just $CABAL v2-sdist and we let haskell-actions/hackage-publish@v1 handle it.

But I think it will actually publish the libraries separately and also publish swarm. Think about it like the distinction between optics-core and optics. This means that I should probably use the reexported-modules field in swarm.cabal. It looks like Hackage displays the reexported modules now:
https://hackage.haskell.org/package/optics

byorgey · 2023-02-20T20:53:52Z

But I think it will actually publish the libraries separately and also publish swarm. Think about it like the distinction between optics-core and optics.

But optics-core and optics each have their own separate .cabal file. There is absolutely no way that publishing something with a single .cabal file could actually cause multiple separate packages to be uploaded to Hackage. How would each one get a version number? Would they all have the same .cabal file? etc.

byorgey · 2023-02-20T20:56:48Z

This means that I should probably use the reexported-modules field in swarm.cabal.

Yes, probably so. We want someone who depends on the swarm package to be able to use any of the modules from any of our internal packages (possibly except for a few "internal/private" ones).

byorgey · 2023-02-20T20:58:15Z

We should probably try running Haddock on this locally to make sure that it produces something reasonable.

xsebek · 2023-02-20T22:47:41Z

@byorgey I tested it with Haddock and it showed all modules. 🙂

Feel free to test it yourself and with Stack. Otherwise, I will have to download Stack to test it myself. 😄

byorgey · 2023-02-20T23:01:36Z

I tried it with stack, and the first thing that happens is it prints this concerning message:

SubLibrary dependency is not supported, this will almost certainly fail

...and then it proceeds to build everything perfectly. 🤷

Also: * adds a script to view locally-generated Haddocks. * Describes module organization as per #1069 (comment) in the toplevel cabal package description

restyled-io bot mentioned this pull request Jan 31, 2023

Restyle Split swarm libraries #1070

Merged

xsebek mentioned this pull request Feb 8, 2023

Move modules where they belong #1096

Merged

mergify bot pushed a commit that referenced this pull request Feb 9, 2023

Move modules where they belong (#1096)

38a569a

- Move the Location module to Game - Move Achievements modules to Game - Move Render and Model modules to TUI - split off from #1069 - part of #1043

xsebek mentioned this pull request Feb 9, 2023

Use internal attribute type in Display #1100

Merged

xsebek commented Feb 9, 2023

View reviewed changes

src/swarm-language/Swarm/Language/Syntax.hs Outdated Show resolved Hide resolved

xsebek mentioned this pull request Feb 11, 2023

Move Heading and conversions to Location module #1103

Merged

xsebek force-pushed the split-libs branch from fd4c644 to 8d77123 Compare February 18, 2023 18:44

xsebek added 4 commits February 18, 2023 19:46

move files

9667e15

wip cabal

ad568bd

wip

a19995d

wrap language server again in language

547a547

restyled-io bot mentioned this pull request Feb 18, 2023

Restyle Split swarm libraries #1119

Closed

xsebek added 2 commits February 18, 2023 19:58

fixup bench

e5e2aef

restyle

6df71f2

xsebek force-pushed the split-libs branch from 8d77123 to 6df71f2 Compare February 18, 2023 18:59

xsebek added 3 commits February 20, 2023 12:59

fixup import

fd7c1e8

remove tabs

50d8175

Regenerate Haskell CI

0ed6710

xsebek commented Feb 20, 2023

View reviewed changes

restore header notes

df459fe

I did not save file before commit

xsebek commented Feb 20, 2023

View reviewed changes

Clean up cabal file

c05f295

xsebek marked this pull request as ready for review February 20, 2023 14:12

xsebek requested review from byorgey and kostmo February 20, 2023 14:13

xsebek mentioned this pull request Feb 20, 2023

Add helpful high level modules with documentation and re-exports #1126

Open

byorgey reviewed Feb 20, 2023

View reviewed changes

Reexport modules

19809ff

xsebek closed this Jun 4, 2023

kostmo mentioned this pull request Sep 7, 2023

more documentation tweaks #1493

Merged

mergify bot pushed a commit that referenced this pull request Sep 7, 2023

more documentation tweaks (#1493)

98ebf74

Also: * adds a script to view locally-generated Haddocks. * Describes module organization as per #1069 (comment) in the toplevel cabal package description

byorgey mentioned this pull request Nov 19, 2023

Split Swarm components into libraries #1043

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split swarm libraries #1069

Split swarm libraries #1069

xsebek commented Jan 31, 2023 •

edited

Loading

byorgey commented Feb 1, 2023

xsebek Feb 20, 2023

xsebek Feb 20, 2023

xsebek Feb 20, 2023

xsebek commented Feb 20, 2023

byorgey left a comment

byorgey Feb 20, 2023

xsebek Feb 20, 2023

byorgey Feb 20, 2023

xsebek Feb 20, 2023

xsebek Feb 20, 2023

xsebek commented Feb 20, 2023

byorgey commented Feb 20, 2023

xsebek commented Feb 20, 2023

byorgey commented Feb 20, 2023

byorgey commented Feb 20, 2023

byorgey commented Feb 20, 2023

xsebek commented Feb 20, 2023

byorgey commented Feb 20, 2023

Split swarm libraries #1069

Split swarm libraries #1069

Conversation

xsebek commented Jan 31, 2023 • edited Loading

byorgey commented Feb 1, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xsebek commented Feb 20, 2023

byorgey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xsebek commented Feb 20, 2023

byorgey commented Feb 20, 2023

xsebek commented Feb 20, 2023

byorgey commented Feb 20, 2023

byorgey commented Feb 20, 2023

byorgey commented Feb 20, 2023

xsebek commented Feb 20, 2023

byorgey commented Feb 20, 2023

xsebek commented Jan 31, 2023 •

edited

Loading