New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use http.FileSystem for all the things #144

Open
wants to merge 4 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@coolaj86
Copy link
Contributor

coolaj86 commented Dec 21, 2018

This shows how it's possible to use http.FileSystem for:

  • file
  • packr
  • fileb0x
  • wedav
  • almost every other thing that works at all like a file system

coolaj86 added some commits Dec 20, 2018

coolaj86 added some commits Dec 21, 2018

@coolaj86

This comment has been minimized.

Copy link
Contributor

coolaj86 commented Dec 21, 2018

It looks like the build is failing due to Cassandra, not necessarily this change.

I’ve been able to use it locally in a project and it works as expected.

How would I be able to tell?

@dhui

This comment has been minimized.

Copy link
Member

dhui commented Dec 21, 2018

How would I be able to tell?

Add tests

@dhui

This comment has been minimized.

Copy link
Member

dhui commented Dec 21, 2018

Please address the issues I've raised here. It's fine to address those issues as a comment in this PR. e.g. doesn't need to be a comment in that github issue.

I don't see WebDAV source driver implementing the source.Driver interface, nor do I see any of the WebDAV sub-drivers being registered.

@coolaj86

This comment has been minimized.

Copy link
Contributor

coolaj86 commented Dec 21, 2018

@dhui

Add tests

I ran my own tests locally. I also ran your tests locally. The tests that are failing are something about a Cassandra timeout, which seems unrelated. Adding my own tests will not solve that problem as I don't use Cassandra and I wouldn't know how to test this specific functionality more than you and I already have.

Please address the issues I've raised here.

This PR is a direct response to those issues. Please take a look at the code. The purpose of the code is to illustrate that by using Go's http.FileSystem` interface you support almost every file module that exists - including Packr.

WebDAV, the network transfer protocol, is not useful for this. However, the webdav.FileSystem, from the official Go package, much like http.FileSystem is an interface that is used by many packages. Therefore, by supporting both interfaces, you eliminate the need for custom plugins (such as packr, fileb0x, vfsgen, and any others that support them).

@dhui

This comment has been minimized.

Copy link
Member

dhui commented Dec 21, 2018

We need tests for each source driver. Relying on existing tests to catch issues is not sufficient since we need to know if the driver we're depending on works or breaks with an update.

The purpose of the code is to illustrate that by using Go's http.FileSystem` interface you support almost every file module that exists - including Packr.

Ahh, I see now!
I like how using interfaces reduces the need for custom source driver code, but I'm not a fan of the new split logic in the file source driver. It would be cleaner to implement a source driver(s) that only consumes certain interfaces (e.g. named source.HttpFS and source.webdavFS) and re-implement source.File to use source.HttpFS.
Source drivers would still need to be registered but can be registered at the top level. e.g. we won't need to use "sub-drivers"

Are you familiar with other FS interfaces? I find it odd to use an FS interface from the http package since some of the FS won't the associated with HTTP. I did a cursory search for other FS interfaces but didn't find another built-in one. I also found this, but it's not a built-in interface so fewer packages are likely to implement it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment