Change layout of local library search path in order to be able to move `Round_Spec.enso` back to `Tests` #7634

radeusgd · 2023-08-22T15:35:30Z

Pull Request Description

Closes Improve experience of inter-project imports in local projects #7633
Moves Round_Spec.enso from published Standard.Test into our test/Tests project; the Table_Tests that depend on it, simply import enso_dev.Tests.
Changes the layout of the local libraries directory:
- It used to be root/<namespace>/<name>.
- Now it is root/<dir> - the namespace and name are now read from package.yaml instead.
Adds the parent directory of the current project to the default ENSO_LIBRARY_PATH.
- It is treated as a secondary path, so the default ENSO_HOME/lib still takes precedence.
- This allows projects to reference and load 'sibling' projects easily - the only requirement is for the project to enable prefer-local-libraries: true or add the other local project to its edition. The edition resolution logic is not changed.

Important Notes

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

The documentation has been updated, if necessary.
Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
All code follows the
Scala,
Java,
and
Rust
style guides. In case you are using a language not listed above, follow the Rust style guide.
All code has been tested:
- Unit tests have been written where possible.
- If GUI codebase was changed, the GUI was tested when built using ./run ide build.

radeusgd · 2023-08-24T11:58:35Z

As of commit e6dc959 I was able to run:

> $env:NO_LOG_MASKING=1
> enso run .\test\Table_Tests\src\Common_Table_Operations\Column_Operations_Spec.enso
...
[info] [2023-08-24T11:56:08.23Z] [enso.org.enso.interpreter.runtime.DefaultPackageRepository] Found library Standard.Test @ 0.0.0-dev at [C:\Users\progr\AppData\Local\enso\dist\0.0.0-dev\lib\Standard\Test\0.0.0-dev].
[info] [2023-08-24T11:56:08.241Z] [enso.org.enso.interpreter.runtime.DefaultPackageRepository] Found library enso_dev.Tests @ local at [C:\NBO\enso\test\Tests].
...
106 tests succeeded.
0 tests failed.
5 tests skipped.

Now to change the default search paths to make the env var override unnecessary.

…local libraries next to current project

…ocalLibraryManager (creating vs finding created libraries)

radeusgd · 2023-08-28T16:02:58Z

...nguage-server/src/test/scala/org/enso/languageserver/libraries/LocalLibraryManagerSpec.scala

+  }
+
+  "LocalLibraryManager" should {
+    "find the libraries it has itself created" in {


This test is partially redundant with

enso/engine/language-server/src/test/scala/org/enso/languageserver/websocket/json/LibrariesTest.scala

Line 55 in 5dc2c4c

"create a library project and include it on the list of local projects" in {

but it checks some more cases and is more a 'unit test' whereas the other is more an 'integration test'.

So IMO it may make sense to keep both - the role of the newly added one is to ensure the logic for creating a library is kept in-sync with the logic for resolving them. The other test is just an integration test checking library management.

radeusgd · 2023-08-28T16:05:36Z

test/Table_Tests/src/Common_Table_Operations/Column_Operations_Spec.enso

@@ -17,6 +16,8 @@ from Standard.Database.Errors import all
 from Standard.Test import Test, Problems
 import Standard.Test.Extensions

+import enso_dev.Tests.Data.Round_Spec


I'm really glad we could get this working 😀

I think this also opens up doors to more logic sharing where we can import parts of tests from Tests in other test suites like Table_Tests.

radeusgd · 2023-08-28T16:09:20Z

test/Table_Tests/package.yaml

 license: MIT
 author: enso-dev@enso.org
 maintainer: enso-dev@enso.org
+prefer-local-libraries: true


Another, IMO better, way to do this would be to declare the depended-on library in the edition override:

edition: extends: 2023.123-dev libraries: - name: enso_dev.Tests repository: local

However to do so, we need to specify some edition (like 2023.123-dev example), but we don't really have one to specify. I guess we should extend the edition resolution to be able to specify extends: default. Currently that is not possible, but it should be a relatively simple extension.

@jdunkerley @JaroslavTulach shall I create a ticket to allow extends: default? Seems +- aligned with the direction #7639 is going in.

shall I create a ticket .... aligned with the direction https://github.com/orgs/enso-org/discussions/7639 is going in.

#7639 discusses incompatibilities between engine & IDE. I am unsure how that relates and affects editions. If an existing concept like prefer-local-libraries works, it is easier to suggest to use it than think about consequences of changing extends mechanism.

JaroslavTulach · 2023-08-28T16:20:08Z

CHANGELOG.md

@@ -923,6 +923,8 @@
 - [Only use types as State keys][7585]
 - [Allow Java Enums in case of branches][7607]
 - [Notification about the project rename action][7613]
+- [Changed layout of local libraries directory, making it easier to 
+  reference projects next to each other][7634]


Supporting depedencies among sibling projects

radeusgd · 2023-08-28T16:40:21Z

...e/language-server/src/main/scala/org/enso/languageserver/libraries/LocalLibraryManager.scala

    if (Files.exists(libraryPath)) {
+      // TODO [RW] we could try finding alternative names (as directory name does not matter for local libraries), to find a free name
+      // This can be done as part of #1877
      throw new RuntimeException("Local library already exists")
    }


The issue #1877 was closed some time ago as part of a cleanup, but it indeed remains unsolved - there are a few TODOs referencing it and indeed the error handling in that part is not fully developed but a shortcut.

It may be worth to re-open the issue. However, this part of the API is not yet being used by the IDE. I guess we may wait until it starts being used to re-visit these improvements. But let's keep in mind there is some missing functionality there that should be filled-in once this is actually used.

# Conflicts: # CHANGELOG.md

…r projects interfering with expected values)

radeusgd linked an issue Aug 22, 2023 that may be closed by this pull request

Improve experience of inter-project imports in local projects #7633

Closed

7 tasks

radeusgd mentioned this pull request Aug 22, 2023

Improve experience of inter-project imports in local projects #7633

Closed

7 tasks

radeusgd force-pushed the wip/radeusgd/test-suite-dependencies branch from cfa30ad to e6dc959 Compare August 24, 2023 11:57

radeusgd force-pushed the wip/radeusgd/test-suite-dependencies branch from 3d4704c to 66dac58 Compare August 28, 2023 14:18

radeusgd changed the title ~~Move Round_Spec.enso back to Tests, try linking Tests from Table_Tests~~ Change layout of local library search path in order to be able to move Round_Spec.enso back to Tests Aug 28, 2023

radeusgd added 13 commits August 28, 2023 17:52

Move Round_Spec.enso back to Tests, try linking Tests from Table_Tests

7bdda0f

revert to default edition, but allow local lib resolution

420b311

flatten library search structure and use package.yaml instead

f4f1d9c

WIP: passing project root to LibraryLocations so that we can resolve …

e0ff029

…local libraries next to current project

pt2. allowing resolving projects next to current project as libraries

e3c51a9

Add tests for new semantics of local lib search

cf2cbd0

fix tests

83728ee

remove debug(?) log

7771d4a

add (failing) test for integration between LocalLibraryProvider and L…

dc64dbf

…ocalLibraryManager (creating vs finding created libraries)

fix that test

c297c83

formatting

25ddd88

changelog

e810382

update docs

b5bf679

radeusgd force-pushed the wip/radeusgd/test-suite-dependencies branch from 66dac58 to b5bf679 Compare August 28, 2023 15:53

radeusgd self-assigned this Aug 28, 2023

radeusgd marked this pull request as ready for review August 28, 2023 15:56

radeusgd requested review from jdunkerley, GregoryTravis, 4e6, JaroslavTulach, hubertp and Akirathan as code owners August 28, 2023 15:56

radeusgd commented Aug 28, 2023

View reviewed changes

cleanup

24ba65e

JaroslavTulach approved these changes Aug 28, 2023

View reviewed changes

radeusgd commented Aug 28, 2023

View reviewed changes

enso-bot bot mentioned this pull request Aug 29, 2023

Investigate usage of Python libraries #7388

Closed

GregoryTravis approved these changes Aug 29, 2023

View reviewed changes

jdunkerley approved these changes Aug 31, 2023

View reviewed changes

Merge branch 'develop' into wip/radeusgd/test-suite-dependencies

385dd4d

# Conflicts: # CHANGELOG.md

radeusgd added the CI: Ready to merge This PR is eligible for automatic merge label Aug 31, 2023

radeusgd added 15 commits August 31, 2023 17:10

Merge branch 'develop' into wip/radeusgd/test-suite-dependencies

f2407da

prettier: changelog

f472f7b

prettier: docs

5488afd

Merge branch 'develop' into wip/radeusgd/test-suite-dependencies

dffd99c

fixing LibrariesTests; avoid NoSuchFile exception

eb3933c

try aligning local library discovery

ab524be

Merge branch 'develop' into wip/radeusgd/test-suite-dependencies

93695fc

fix libraries test

becb9ec

make test more robust to side effects (other tests were creating othe…

a44aadb

…r projects interfering with expected values)

Merge branch 'develop' into wip/radeusgd/test-suite-dependencies

5868b2e

fix other test (missing implementation)

40af6a1

better diagnostic info on conflicts

05bf37d

align package name with its true name, avoiding a naming conflict

fe427bf

warn only once about missing path

a56b744

not fail tests on cleanup

06be852

mergify bot merged commit 87ce786 into develop Sep 1, 2023
24 checks passed

mergify bot deleted the wip/radeusgd/test-suite-dependencies branch September 1, 2023 20:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change layout of local library search path in order to be able to move `Round_Spec.enso` back to `Tests` #7634

Change layout of local library search path in order to be able to move `Round_Spec.enso` back to `Tests` #7634

radeusgd commented Aug 22, 2023 •

edited

Loading

radeusgd commented Aug 24, 2023

radeusgd Aug 28, 2023

radeusgd Aug 28, 2023

radeusgd Aug 28, 2023

JaroslavTulach Aug 29, 2023

JaroslavTulach Aug 28, 2023

radeusgd Aug 28, 2023

Change layout of local library search path in order to be able to move Round_Spec.enso back to Tests #7634

Change layout of local library search path in order to be able to move Round_Spec.enso back to Tests #7634

Conversation

radeusgd commented Aug 22, 2023 • edited Loading

Pull Request Description

Important Notes

Checklist

radeusgd commented Aug 24, 2023

radeusgd Aug 28, 2023

Choose a reason for hiding this comment

radeusgd Aug 28, 2023

Choose a reason for hiding this comment

radeusgd Aug 28, 2023

Choose a reason for hiding this comment

JaroslavTulach Aug 29, 2023

Choose a reason for hiding this comment

JaroslavTulach Aug 28, 2023

Choose a reason for hiding this comment

radeusgd Aug 28, 2023

Choose a reason for hiding this comment

Change layout of local library search path in order to be able to move `Round_Spec.enso` back to `Tests` #7634

Change layout of local library search path in order to be able to move `Round_Spec.enso` back to `Tests` #7634

radeusgd commented Aug 22, 2023 •

edited

Loading