[Architecture] Responsability of download client adapters

[T4g1]

In the context of download client adapters, the responsibility of these are not clearly identified. As a result, implementing and maintaining those is risky as we do not know which information are required or not and what it should produce precisely.

We should investigate the current features and limitations of the existing adapters. Two existing concerns:

• Adapters interface define two method to get informations about ongoing downloads: Only one of which seems to be used. We should commit to only one of those
• When giving informations about download, adapters produces QueueItem that can support RemotePath, LocalPath, ContentPath and SourceFiles. All of those are nullable fields with no clear orientation of what should be produced. In recent discussions, we suggested that an adapter should have the responsibility of producing source files but we suspect that some of them will be unable to always produce that and can instead only give a remote path.

For this last issue, we want to explore the following solutions:

• Either the remote path is given as ContentPath and SourceFiles is null, the download gateway is then responsible of mapping the path to local path and checking wether or not ContentPath is a file/directory and populate SourceFiles accordingly
• Either we allow Adapter to map path and produce SourceFiles accordingly

[therobbiedavis]

Future direction note: ideally the adapter should return import-ready SourceFiles. That means the adapter should use the shared remote path mapping helper/service, resolve file-vs-directory cases, normalize the paths, and log why it could not resolve anything.

Long term, the gateway should not be doing path mapping, ContentPath fallback scanning, or missing-path diagnostics as part of normal import resolution. It should mostly pass through the adapter’s resolved result.

The contract I’d like us to move toward is:

• SourceFiles = null means the adapter could not determine source files.
• SourceFiles = [] means the adapter determined there are no importable files.
• Non-empty SourceFiles should already be mapped, normalized, validated, and ready for the processor.
• ContentPath should become a legacy/fallback field, not the normal import path.

[T4g1]

I'm not a big fan of having the adapter handle the remote path mapping, in practice, such implementation will probably lead to a set of method to either:

• translate a remote to local path
• from a path, compute a list of source files
  Which each specific adapter will simply need to call so instead of one single point of application for remote path mapping, we now have several per adapter, this feels error prone to me and does not improve efficiency (the same processing is applied as if it was done in gateway) and doesn't bring useful informations that the adapter needs to perform it's task (ie: communicate with a third-party)

There is a limitation with the gateway approach though: If an adapter gives a list of files and directories to be used as source files without knowing which is which, the current design do not allow to handle that as we can only get one ContentPath. A better approach if we go that direction would be for the adapter to return raw path in source files only. Then gateway would recursively add files from given directories and single files into a cleaner list of local source files. This would remove the need for ContentPath entirely in fact and simplify the logic of SourceFiles = null -> check ContentPath into simply check any path in SourceFiles as a file-or-directory path

wdyt ?

(in the current merge request, i did not change much the current gateway for remote path mapping implementation, so if someone does a review of this, keep in mind that decision is not set in stone for that part)