Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): Accept structured imports [LNG-288] #989

Merged
merged 35 commits into from
Dec 13, 2023

Conversation

InversionSpaces
Copy link
Contributor

@InversionSpaces InversionSpaces commented Nov 23, 2023

Description

Allow specifying imports based on compiled file path prefix and import prefix to resolve issues with transitive dependencies (no way to have different versions of a transitive dependency).

Proposed Changes

Accept imports in form

{
  "<compiled-file-path-prefix>": {
    "<import-prefix>": "<import-location>" | ["<import-location>", ...],
    ...
  },
  ...
}

Implementation Details

Refactor AquaFileSources.

Checklist

  • Corresponding issue has been created and linked in PR title.
  • Proposed changes are covered by tests.
  • Documentation has been updated to reflect the changes (if applicable).
  • I have self-reviewed my code and ensured its quality.
  • I have added/updated necessary comments to aid understanding.

Reviewer Checklist

  • Tests have been reviewed and are sufficient to validate the changes.
  • Documentation has been reviewed and is up to date.
  • Any questions or concerns have been addressed.

Copy link

linear bot commented Nov 23, 2023

LNG-288 Accept structured imports in compiler API

In aqua-api, accept imports as the following structure:

{
  "<path-to-first-folder>": [
    "<path-to-first-import>",
    "<path-to-second-import>",
    ...
  ],
  "<path-to-second-folder>": [
    "<path-to-first-import>",
    "<path-to-second-import>",
    ...
  ],
}

While compiling a file, use imports from all entries that are prefixes for this file path (prioritize longer prefixes).

doc

@InversionSpaces InversionSpaces added the e2e Run e2e workflow label Nov 27, 2023
@InversionSpaces InversionSpaces marked this pull request as ready for review December 7, 2023 16:30
@InversionSpaces InversionSpaces enabled auto-merge (squash) December 7, 2023 16:30
/**
* Imports resolution configuration.
*/
final case class Imports(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is two same classes with the same structure. Maybe merge it somehow?

Copy link
Contributor Author

@InversionSpaces InversionSpaces Dec 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two of them so that api and io packages have clear dependencies.
If we put Imports to api, io would have to depend on api and this would create cyclic dependency. Also Imports define the logic of resolving imports and api does not feel like the proper place for it.
If we put Imports to io, then api would have to expose internal io type in interface. This does not feel right too.
So those two classes exist to decouple interface representation and representation for internal logic. If one of them changes, we would need to change only the conversion function. But now they are the same, yes.

}

// Get all files if the path is a directory or this path otherwise
// TODO: Test it or remove it. It is not used anywhere
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but it was used in AquaFileSources

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I meant folder walking functionality. Right now it is used on files only. Rephrased comment.


// Get all files if the path is a directory or this path otherwise
// TODO: Test it or remove it. It is not used anywhere
override def listAqua(folder: Path): EitherT[F, AquaFileError, Chain[Path]] =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if folder will be not a folder, but a file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If tests pass, it means this works, yes? Also from the source code it follows that the path itself is returned in stream too, independent of whether it is a folder or not:
https://github.com/typelevel/fs2/blob/aba387fec4841d575c7a66d65594f893b728b6ec/io/shared/src/main/scala/fs2/io/file/Files.scala#L504

settings.filter { case (prefix, _) =>
from.startsWith(prefix)
}.maxByOption { case (prefix, _) =>
prefix.toString.length
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the longest prefix be compared by the number of folders, not characters?

Copy link
Contributor Author

@InversionSpaces InversionSpaces Dec 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If two paths are prefixes of from, the longer one have at least the same amount of folder as the shorter one.
Also, they can have the same amount of folders, example:
from = /a/b/ccc/d.aqua, p1 = /a/b/c, p2 = /a/b/cc, p1 and p2 have the same amount of folders. So we should pick p2 because it is a longer prefix in terms of characters.

}

// Transform each inner string into an array
return Object.fromEntries(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check if imports === undefined

}

// Get all files if the path is a directory or this path otherwise
// TODO: Test it or refactor. Right now it is used on single files only
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is used with a directory in integration tests

@InversionSpaces InversionSpaces merged commit f7bfa83 into main Dec 13, 2023
10 checks passed
@InversionSpaces InversionSpaces deleted the feat/structured-imports-LNG-288 branch December 13, 2023 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
e2e Run e2e workflow
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants