--base-url option?

[vergenzt]

I'm trying to set myself up with shell script helper functions that are essentially wrappers around curl with some pre-specified options. (E.g. --oauth2-bearer, --cookie, etc.)

It'd be nice to be able to specify a base-url that main URL argument ends up being concatenated to!

curl/lib/urlapi.c

Lines 231 to 240 in 4ab601d

	/*
	* Concatenate a relative URL to a base URL making it absolute.
	* URL-encodes any spaces.
	* The returned pointer must be freed by the caller unless NULL
	* (returns NULL on out of memory).
	*
	* Note that this function destroys the 'base' string.
	*/
	static char *concat_url(char *base, const char *relurl)
	{

This would allow me to call e.g. my Gmail API helper as gmail /threads\?labelIds=STARRED to get all my starred Gmail threads instead of having to source the base URL from somewhere else (e.g. from memory, a separate helper function, etc.).

When specified, a main URL argument would still be required, but it would be passed to concat_url before executing the request. Successive --base-url options could either be concatenated together or last one could win - not clear to me which is better.

Only previous mention I've found of this idea is in a mailing list email from 2006.

Thoughts?

[dfandrich]

This sounds related to #9691

[vergenzt]

Related to, but definitely different 🙂

[bagder]

I don't think I understand why this is useful. Can you make up some examples of how a few use cases would look like?

[vergenzt]

Yes! It's useful for initializing an alias with a pre-defined set of options that includes the base URL, so the base URL prefix doesn't have to be repeated by the caller when invoking the alias. For example:

alias gmail='curl --base-url=https://gmail.googleapis.com/gmail/v1/users/me -H Authorization:"Bearer $GMAIL_TOKEN"`

Then, assuming $GMAIL_TOKEN is in the environment it can be used like:

gmail /messages --url-query labelIds=STARRED

rather than having to spell out:

gmail https://gmail.googleapis.com/gmail/v1/users/me/message --url-query labelIds=STARRED

I have helper aliases/scripts for lots of different APIs in my dotfiles, and currently I set up global global variables like $GMAIL_API_URL so that I can call it like gmail $GMAIL_API_URL/messages --url-query labelIds=STARRED, but it'd be nice to (a) not have to store the data in two different places, and (b) not have to have the stutter when the base URL for these requests is always going to be the same.

[bagder]

Here's a 'gmail' script:

#!/bin/sh

curl https://gmail.googleapis.com/gmail/v1/users/me$1 -H Authorization:"Bearer $GMAIL_TOKEN"` --url-query $2

Use it like this:

gmail /messages labelIds=STARRED

[vergenzt]

Yes - it's easy to make scripts like that. I'm proposing this option as a way to be able to encapsulate this (for me, common) behavior in a native curl option (e.g. that could be put into a config file) rather than having to create wrapper scripts.

[jay]

I think @bagder's script example is sufficient and an option in curl too niche. Also, you can put --oauth2-bearer in a curl configuration file if you don't want to show the bearer token on the command line, for example curl -K bearer.cfg. How are you getting the bearer token? I have some scripts that do it: https://github.com/jay/curl_google_oauth

[vergenzt]

Fair enough. 🤷 This suggestion was simply a "nice-to-have" of an option that would be helpful for me. No worries if you're not interested in adding it.

[mback2k]

I actually think something like --base-url would make sense if combined with multiple URLs to query or maybe even --next. This way one would not have to repeat the same base URL multiple times.

This would probably allow me to for example write the 4 curl commands in this README.md in a much nicer looking one-liner.

[bagder]

With variables you can do something very similar:

curl --variable base=https://example.com \
  --expand-url '{{base}}/path1' \
  --expand-url '{{base}}/path2' \
  --expand-url '{{base}}/path3'

[mback2k]

Thank you very much, @bagder!

[vergenzt]

With variables you can do something very similar:

curl --variable base=https://example.com \

  --expand-url '{{base}}/path1' \

  --expand-url '{{base}}/path2' \

  --expand-url '{{base}}/path3'

Woah! TIL about this feature. This may actually solve my use case pretty well!

[bagder]

Possibly best documented here: https://everything.curl.dev/cmdline/variables