Integrating Enso Cloud with the libraries (part 1...)

[jdunkerley]

Pull Request Description

• Add a File_For_Read type. Used for File_Format to read files.
• Added Enso_User representing the current user in Enso_Cloud.
  • Will be later able to list known users.
• Added Enso_Secret representing a value defined in Enso_Cloud.
  • Value not used within Enso only accessed within polyglot Java.
  • Integrated into Username_And_Password and can be used within JDBC connections.
  • Integrated into HTTP Headers so a secret can be used as a value.
  • New URI_With_Query with the same API as URI. Supporting secrets in the value.
  • Will be integrated with AWS credentials.
• Added Enso_File representing a file or a folder in the cloud.
  • Support the same API as File (like the S3_File).
  • Will support enso:// URI style access.

Minor fixes:

• Linting fixes.
• Fixed for web APIs:
  • Removed Http_Utils.java and folded functionality into Enso code.
  • Altered old URI API, just returning Nothing instead of throwing as an error.
  • Use HTTP 2 by default.
• Fix bug in JSON.into creating a Map.
• Fix bug in S3_File.
• Added type checking for Vector.+.
• Added conversion from Column to Vector, allowing the use of columns within Vector methods.
• Fix allowing using debug mode on Windows.
• Tweak to Source_Location so that stacktrace renders correctly in IDE.

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

• The documentation has been updated, if necessary.
• Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
• All code follows the
  Scala,
  Java,
  and
  Rust
  style guides. In case you are using a language not listed above, follow the Rust style guide.
• All code has been tested:
  • Unit tests have been written where possible.
  • If GUI codebase was changed, the GUI was tested when built using ./run ide build.

[radeusgd]

If I read this right, we do a full GET request, meaning we actually start downloading the file which may be costly if the file is big + will cost us data transfer.

Can we use HEAD option here instead?

[jdunkerley]

In the cloud API, the files GET doesn't give the file it just gives the metadata for the file.
I have asked for a HEAD method to be added to the API to do exists check.

[radeusgd]

We are using this construct more and more often, I wonder if we should think about adding some support for it, like something similar to assets.cast Enso_File (not sure about the name, as clashing with Table.cast is not ideal).

Maybe assets.ensure_items_type Enso_File.

Not a thing for this PR though, just a general comment about the pattern.

[radeusgd]

Suggested change

	authorization_basic : Text -> Text -> Header
	authorization_basic user pass =
	authorization_basic : Text -> Text|Enso_Secret -> Header
	authorization_basic (user : Text) (pass : Text | Enso_Secret) =

[radeusgd]

Ah ok I see that this would complicate the logic quite significantly.

I think useful to allow this (together with also allowing secrets in authorization_bearer) at some point, but probably not at this PR.

[radeusgd]

I think the overall structure looks great.

[radeusgd]

Looks like a great step forward.

I wonder how we can approach integration testing of the cloud parts (but ofc that's for the follow-ups).

[radeusgd]

I think it may be good to include a TODO here that JSON can actually contain the Unicode escapes - if we ever run into this being an issue it would be a valuable hint.

[GregoryTravis]

PRIVATE?