Attempt to make tar work on Windows

[Fydon]

The tar commandline requires changes to the path for it to work on Windows. As a result I moved CompressSpecificFilesInDir and DecompressFileToDir into separate Windows and non Windows build files. I also split the tests into separate Windows and non Windows build files, so that they will be able to run build specific tests. All the tests in fileutil are passing on Windows.

I'm still new to Go, so please let me know how I can improve this.

As tar isn't available on Windows by default, so #9 is still advised, but these changes should enable the version of tar that comes with Git to work.

[cfdreddbot]

Hey Fydon!

Thanks for submitting this pull request! I'm here to inform the recipients of the pull request that you and the commit authors have already signed the CLA.

[charlievieth]

@Fydon Thanks for the PR. This isn't documented (and probably should be), but we added Windows support to bosh-utils in order to support the bosh-agent, which includes a BSD tar compiled for Windows (note: you will also need zlib.dll). I believe the source comes from libarchive - we have a story to start building tar in our CI.

We do not use the tar bundled with Git as it does not work with Windows file paths and the Unix paths it accepts are not valid Windows paths. It breaks interoperability with the Windows API and Go's standard library. For example:

Path	Windows API	BSD Tar	Git Tar
/Windows/Temp/foo	valid	valid	invalid
/c/Windows/Temp/foo	invalid	invalid	valid
C:\Windows\Temp\foo	valid	valid	invalid

We cannot support one without breaking the other and supporting both would require deducing the version/flavor of tar at runtime, which is not viable.

I think the best solution is for us to improve the documentation around tar on Windows, build tar in our CI and make that tar publicly available (and pointed to in the bosh-utils and bosh-agent docs).

Edit: This also break cygwin tar.

[charlievieth]

Closing as this PR makes bosh-utils dependent on the tar installed with git-bash and breaks compatibility with cygwin-tar and libarchive-tar (which we use and correctly handles Windows paths).

We created a story to document tar on Windows: 139998715.

[cppforlife]

i think we are not quite ready to close this PR. the reason why Fydon is making these changes is to make bosh-cli v2 work on windows (not on top of windows stemcells) which cannot rely on a specific tar being present. but you do bring a good point about bosh-utils also being used by the agent (which is good) which does run on a deterministic window image. so the question remains how do we get this to work on top of windows stemcells and arbitrary windows machines -- i think the answer to that is we need to use go std library to do the unpacking for windows impl (and possibly switch linux impl at some point as well).

…

Sent from my iPhone

On Feb 15, 2017, at 6:38 PM, Charlie Vieth ***@***.***> wrote: Closed #11. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[charlievieth]

Didn't realize this was related to the bosh-cli (it's mentioned in #9, but not here).

I would rather not re-open this PR as it would make the bosh-agent and bosh-cli dependent on the tar bundled with git-bash (GPL-2.0). This would also likely break the bosh-cli for those using the popular cygwin terminal. Maybe #9 or the bosh-cli is a better place for this issue.

Using Go's archive/tar lib would be great, but the lib is spartan. Maybe we could explore bundling the bsdtar provided by libarchive with the bosh-cli or bosh-utils.

[cfdreddbot]

Hey Fydon!

Thanks for submitting this pull request! I'm here to inform the recipients of the pull request that you and the commit authors have already signed the CLA.

[Fydon]

How about using archiver?

[jaresty]

Merge story #152974592

[bosh-admin-bot]

This pull request was marked as Stale because it has been open for 21 days without any activity. If no activity takes place in the coming 7 days it will automatically be close. To prevent this from happening remove the Stale label or comment below.

[bosh-admin-bot]

This pull request was closed because it has been labeled Stale for 7 days without subsequent activity. Feel free to re-open this pull request at any time by commenting below.