Add support for tunnelto with config file importer and file-based provisioning

[arunsathiya]

This PR adds support for tunnelto dev. Fixes #157.

Key points

• tunnelto supports API Key-based authentication. On the docs and on the CLI help commands, there are some references to "API Authentication Key" but for simplicity, I have stuck to "API Key"
• tunnelto set-auth -k key or tunnelto set-auth --key key is the command to set the API Key for authentication.
• The shell plugin supports importing of the API Key from the ~/.tunnelto/key.token path.
• Provisions the API Key to the same path for authentication.

Notes

Environment variable not supported

I am not seeing an environment variable to use for authentication. I reached out @agrinman on email to understand if it's possible. Until that's clear, I suggest we move forward with file-based provisioning.

1Password authentication needed for arg-less usage

If an user runs just tunnelto, 1Password authentication should be used because that command basically runs tunnelto --port 8000

Not sure how to handle set-auth flow

When someone sets up tunnelto after setting up the shell plugin, running tunnelto set-auth -k key or tunnelto set-auth --key key results in the API Key being stored to ~/.tunnelto/key.token on the computer.

And running tunnelto commands after that results in failure because 1Password tries to provision the credential to that path, but a file already exists at that path:

➜  ~ tunnelto set-auth -k XddpK7jZiQ0CpE3EXAMPLE
#########################################################################
# WARNING: 'tunnelto' is not from the official registry.                #
# Only proceed if you are the developer of 'tunnelto'.                  #
# Otherwise, delete the file at /Users/arun/.op/plugins/local/tunnelto. #
#########################################################################
Authentication key stored successfully!
➜  ~ cat ~/.tunnelto/key.token
➜  ~ tunnelto
#########################################################################
# WARNING: 'tunnelto' is not from the official registry.                #
# Only proceed if you are the developer of 'tunnelto'.                  #
# Otherwise, delete the file at /Users/arun/.op/plugins/local/tunnelto. #
#########################################################################
[ERROR] 2023/01/31 23:57:40 could not run plugin tunnelto.dev CLI[31;1m [test build][0m: file with path /Users/arun/.tunnelto/key.token already exists, please remove it before proceeding

I have already skipped 1Password authentication for -k and --key.

[arunsathiya]

Not sure how to handle set-auth flow

I thought about this a bit more. It makes sense to disable 1Password auth if the command contains set-auth as well. To elaborate a bit, a new tunnelto shell plugin user would start in one of these two ways:

• Already have the key.token file on their computer and run op plugin init tunnelto, in which case we offer to import the file. After doing that, running tunnelto commands fail because 1Password cannot create the key.token file at the same path until the existing key.token file is manually deleted by the user.
• User doesn't have the key.token file so far, and runs op plugin init tunnelto, in which case we prompt the user to enter the key manually and then store it to 1Password.

In either case, there's no requirement to use 1Password auth for set-auth command, so that update has been done in 726e0fc.

[hculea]

Looks good to me, thank you for this contribution! Only one question.

[hculea]

Is provisioning of host, port, scheme and subdomain also available via envvars? If so, I'd propose we add these as well as optional fields.

[AndyTitu]

a few of them are: https://github.com/agrinman/tunnelto/blob/06428f13c638180dd349a4c42a17b569ab51a25f/tunnelto/src/config.rs#L6. This would be a nice improvement indeed.

[AndyTitu]

Sadly they do not offer support for overriding the fixed path :(, so users of this plugin will have to move a potentially already existing file elsewhere. I opened an issue on their repo for this: agrinman/tunnelto#78

[AndyTitu]

lgtm. @hculea 's comment should be addressed before merging.

[arunsathiya]

Thanks for reviewing, Andi and Horia! Just wanted to note that this PR is on my radar, but it may be another two or three days before I can get back. 🙏🏼

[arunsathiya]

Hi @hculea and @AndyTitu, thank you for your patience! Do I understand correctly that a shell plugin can have only one provisioner? In our current format, we provision a TempFile but if we want to support envvar-based provisioning, I think I'd have to remove the TempFile provisioner. 🤔

I don't have much experience with rust code, but I am slowly exploring the tunnelto codebase (starting with the config.rs file) to understand how we can configure host, port, scheme and subdomain in the configuration file, if that's an option at all.

[hculea]

Hey there! Indeed, plugins can have at most one provisioner.

For now, we (mostly I 😂 ) have a preference towards envvar provisioners over file provisioners, because of some limitations that are inherent to the latter. Whenever both are an option, I advise to proceed with envvars.

From the looks of it, this tool offers envvar support as well, so let's explore that! I'm not very familiar with Rust myself, but let us know if we can help in any way 😄

[arunsathiya]

From the looks of it, this tool offers envvar support as well, so let's explore that!

Definitely happy to explore envvar-based provisioning! But I haven't exactly located the actual envvar that we'd use for the API Key. Only the config directory and config file names are available as envvars:

https://github.com/agrinman/tunnelto/blob/06428f13c638180dd349a4c42a17b569ab51a25f/tunnelto/src/config.rs#L14-L15

I have reached out to the author of tunnelto, @agrinman, on email for advise. I'll followup as soon as I know more.

[accraw]

I took a quick look around, and based on this file: https://github.com/agrinman/tunnelto/blob/06428f13c638180dd349a4c42a17b569ab51a25f/tunnelto_server/src/config.rs

it doesn't look like they currently support storing the API Key in an environment variable. I look forward to seeing what they come back to you with!

[AndyTitu]

What is blocking us here? Last time I checked it was about neither the file provisioning or env var provisioning working. There are some changes which need to be done in their Rust codebase for either to work right?

[arunsathiya]

@AndyTitu, the current implementation is using a file provisioner. We are trying to move to environment variables-based provisioner so that we can use host, port, scheme and subdomain as well. I talked to Alex (author of TunnelTo) and they pointed out that while they don't support envvars, they do accept a command like argument "--key".

So, I started working on a arguments provisioner last night. It's a work in progress (pushing what I have working so far) but has a couple of things pending:

• If an user supplies arguments as well, those values should be prioritised instead of the 1Password-provisioned argument values.
• Write tests for the arguments provisioner.
• Support only "http" and "https" as accepted values for the scheme field name. So, I am trying to implement SpecificString []string as a composition charset that accepts specific strings.

It may be a while before I wrap up all of this, so I'll be sure to update the PR status to "ready to review" when it's done!