Refactor Enso_File to be path-based

[radeusgd]

Currently Enso_File contains metadata like the asset id. This makes it only possible to refer to Enso files that already exist. As we also want to be able to have a write operation that may create new files, that is not enough. We need to be able to refer files that do not yet exist.

• We should refactor Enso_File to be based on a path instead (much like S3_File works).
  • Change the constructor to hold the organization_name and path (these are parallel to S3 bucket and key).
  • We should consider if we can re-use the S3 Path logic.
  • Now asset_type needs to fetch. It should fail if the file currently does not exist.
  • Update other methods to resolve the asset id based on the path and then access the API (e.g. size, read etc.)
• Currently / throws Common.Not_Found if a file is not found. Now, it should just create a new path.
  • The error should be 'deferred' until the path is actually accessed, e.g. in read, size,
    • and it should be File_Error.Not_Found instead.
  • Update tests to the new behaviour - check creating nonexistent paths with nonexistent parents.
• We can now implement Enso_File.parent as it is now only path manipulation and does not need to perform any requests.
  • Add tests for that as well.

Writing will be implemented as a separate task.

[radeusgd]

I recall some mentions about possibly caching the paths.

Indeed, currently accessing deeply nested directories will be very costly as each operation will need as many requests as the level of nesting + 1.

However, there are fundamental problems to such caching - what if the user modifies the files from another Enso instance? We cannot tell when to revoke the cache. Moreover it feels like throwaway work, because once the Cloud APIs give us a way to work directly with paths, we would no longer need such caches - we will be able to perform every operation in just 1 request that will take the path - so I think we should instead go in that direction.

IF the performance is really bad and adding the path support to the APIs is not coming soon, it may be worth to reconsider such caching (as a separate ticket).

[enso-bot]

Radosław Waśko reports a new STANDUP for today (2024-03-29):

Progress: Mostly finished the refactor. The Enso File tests are working again while being path based. It should be finished by 2024-04-05.

Next Day: Next day I will be working on the #9289 task. Fix some regressions in other parts of cloud tests. Think of some optimizations (caching the asset info that is fetched now A LOT).

[enso-bot]

Radosław Waśko reports a new STANDUP for the provided date (2024-04-02):

Progress: Catching up on PR reviews, meetings, fixing tests and updating cloud integration. Created some followup tickets. Wrote tests for #8906 It should be finished by 2024-04-05.

Next Day: Next day I will be working on the #8906 task. Finish the small PR. Start work on integrating the Cloud path resolver.