pre-create hook on Windows always return file not exists

[avvertix]

I'm trying file hooks on Windows 10 and I always get an error while the pre-create hook is executing

To Reproduce
Steps to reproduce the behavior:

1. Download version 0.11 for Windows
2. Start tusd with a hook folder tusd.exe --hooks-dir .\hooks\win\
3. Try to upload a file
4. See the error report on tusd stdout

[tusd] event="RequestIncoming" method="POST" path=""
[tusd] event="HookInvocationStart" type="pre-create" id=""
[tusd] event="HookInvocationError" type="pre-create" id="" error="exec: "C:\\...\\hooks\\win\\pre-create": file does not exist"
[tusd] event="ResponseOutgoing" status="500" method="POST" path="" error="pre-create hook failed: exec: "C:\\...\\hooks\\win\\pre-create": file does not exist

Expected behavior
Hook is executed

Setup details
Please provide following details, if applicable to your situation:

• Operating System: Windows 10
• Used tusd version: Version: 0.11.0, Commit: 27c9c4a Date: Mon Nov 19 16:47:02 UTC 2018
• Used tusd data storage: disk store
• Used tusd configuration: --hooks-dir .\hooks\win\
• Used tus client library: tus-js-client

[avvertix]

I tried to pull the code and verify what was happening. Seems that exec.Command in the hooks.go file tries to execute what is passed as a binary on the PATH, but cannot consider the hookPath as a valid binary name. If I change the exec.Command to

cmd := exec.Command("powershell", hookPath)

the script pointed by hookPath I don't get the error.

Since I'm new to Go, I don't know how to solve this situation, or how to go further

[Acconut]

Thanks for reporting this and sorry for the delayed response, I didn't have the time yet to debug this issue on my Windows machine. In the meantime I would have a few questions about your setup:

1. How are your file hooks called? Is it just pre-create without a file extension (tusd expects the hook files to have no extension)?
2. Are your hook files scripts (e.g. Powershell) or directly binary files?
3. Would you be able to share a simple version of your hook files with us, so we try them out?

The hook system was mostly designed with a Linux setup in mind and not enough effort was spent on testing it on Windows sadly (as we all are no Windows developers in our daily life).

[avvertix]

How are your file hooks called? Is it just pre-create without a file extension (tusd expects the hook files to have no extension)?

I called them just pre-create, post-create,... without extension

Are your hook files scripts (e.g. Powershell) or directly binary files?

powershell scripts, because I need to execute some commands (in particular call a php script)

Would you be able to share a simple version of your hook files with us, so we try them out?

the hook file contains just an "echo" to see if was working

echo ">>>>>>>>>>>>>>>>>>>>>>>>> hello"

I had time to experiment a bit more, and I found that:

• if I name the file without an extension and run it on the powershell directly, or as argument of the powershell executable I get a dialog that ask me to select which application to use to open the file

PS > .\.hooks\pre-create
PS > powershell.exe .\.hooks\pre-create

• If I name the file with the ps1 extension that pertain to powershell scripts it outputs the string in the echo command (no need to specify the file extension)

PS > .\.hooks\pre-create
>>>>>>>>>>>>>>>>>>>>>>>>> hello
PS > .\.hooks\pre-create.sh
>>>>>>>>>>>>>>>>>>>>>>>>> hello

[Acconut]

Oh, that's interesting.

If I name the file with the ps1 extension that pertain to powershell scripts it outputs the string in the echo command (no need to specify the file extension)

Do the hooks work in this situation (when you name the file pre-create.ps1)?

[avvertix]

I'm doing some tests on that side

[avvertix]

The testing scenario was tusd version 0.11 executed as follows

 .\tusd-win.exe --hooks-dir .\.hooks\

For the post-* hooks I was using the files without extension, while I was going to change the pre-create hook file. In this way I could see immediately an upload failure from the client side.

I started from

Do the hooks work in this situation (when you name the file pre-create.ps1)?

I tried with

• single file named pre-create.ps1
• two files, one named pre-create.ps1 and the other pre-create to see if the file existence check was somehow involved

the result was

pre-create hook failed: exec: "...\\github.com\\tus\\tusd\\.hooks\\pre-create": file does not exist

the second test

I decided to edit a bit the source code of tusd to impose the .ps1 file extension. I was still getting an error, but totally different

fork/exec .\tus\tusd\.hooks\pre-create.ps1: %1 not a valid Win32 application"

the third test

The second test error made me think, and I gave a try with a batch file instead of a powershell script.

I named the file pre-create.bat. I reverted the changes made to the source code for the previous step and added log.Printf("hookPath: %s", hookPath) before the exec.Command() call.

With my surprise it was working, here is the log trace

[tusd] Using '.\tus\tusd\.hooks' for hooks
[tusd] Using './data' as directory storage.
[tusd] Using 0.00MB as maximum size.
[tusd] Using 0.0.0.0:1080 as address to listen.
[tusd] Using /files/ as the base path.
[tusd] Using /metrics as the metrics path.
[tusd] Core: ✓ Terminater: ✓ Finisher: ✗ Locker: ✓ GetReader: ✓ Concater: ✓ LengthDeferrer: ✓
[tusd] event="RequestIncoming" method="POST" path=""
[tusd] event="HookInvocationStart" type="pre-create" id=""

2019/04/24 09:34:10 hookPath: .\tus\tusd\.hooks\pre-create

[tusd] event="HookInvocationFinish" type="pre-create" id=""
[tusd] event="UploadCreated" id="bd9d80091cba050c396e85e5455f4d8f" size="371852" url="http://localhost:1080/files/bd9d80091cba050c396e85e5455f4d8f"
[tusd] event="ResponseOutgoing" status="201" method="POST" path=""

I believe that exec.Command() uses the command prompt behind the scenes and not a Powershell.

[Acconut]

I named the file pre-create.bat. I reverted the changes made to the source code for the previous step and added log.Printf("hookPath: %s", hookPath) before the exec.Command() call.
With my surprise it was working, here is the log trace

Great, that's a nice find! So the hooks are working for you now if you are using Batch files?

I believe that exec.Command() uses the command prompt behind the scenes and not a Powershell.

Yes, Go's exec package does - on purpose - not use any system shell: https://golang.org/pkg/os/exec/#pkg-overview. You might be able to call Powershell scripts if you add .PS1 to the PATHEXT environment variable but that's just a guess.

[avvertix]

So the hooks are working for you now if you are using Batch files?

I will test deeply in a real application, but adding the .bat extension seems to solve the problem

[Acconut]

Perfect! It would be nice if you could report back your results, so we can update the documentation accordingly.

[avvertix]

Sorry for the delay.

I did a full test and I confirm that all hooks are executed if the file has a .bat extension. I believe this will work also if the hook files are executables with .exe extension.

[Acconut]

No worries, thank you very much for help! I updated the docs here: 12f102a

Are you issues now resolved and can we close this issue?

[avvertix]

Are you issues now resolved and can we close this issue?

yes