New package: RestClient v1.0.0

[JuliaRegistrator]

• Registering package: RestClient
• Repository: https://github.com/tecosaur/RestClient.jl
• Created by: @tecosaur
• Version: v1.0.0
• Commit: e91bce0a35f3850f1998432e1027f2c0d8ccb2ca
• Git reference: HEAD
• Description: Mirror of https://code.tecosaur.net/tec/RestClient.jl

[github-actions]

Hello, I am an automated registration bot. I help manage the registration process by checking your registration against a set of AutoMerge guidelines. If all these guidelines are met, this pull request will be merged automatically, completing your registration. It is strongly recommended to follow the guidelines, since otherwise the pull request needs to be manually reviewed and merged by a human.

1. New package registration

Please make sure that you have read the package naming guidelines.

2. AutoMerge Guidelines are all met! ✅

Your new package registration met all of the guidelines for auto-merging and is scheduled to be merged when the mandatory waiting period (3 days) has elapsed.

3. To pause or stop registration

If you want to prevent this pull request from being auto-merged, simply leave a comment. If you want to post a comment without blocking auto-merging, you must include the text [noblock] in your comment.

Tip: You can edit blocking comments to add [noblock] in order to unblock auto-merging.

[stemann]

What is the reasoning behind the name? I would expect to get a RestClient from something like this package - except I would I expect it to be called RestClients (plural).

It seems there is no client type in the package?

[noblock]

[goerz]

[noblock] Seems fine to me. The whole package is a REST client, and the well-documented API is exactly what you would want to talk to a REST interface. Maybe think of it as a Singleton. That makes the singular name absolutely appropriate.

[tecosaur]

[noblock] Thanks for the feedback both. RestClient isn't the only name considered, it started out as RestClientScaffold and a few other longer names were considered. I checked with a few folks what seemed clear to them, and the consensus was that the short name was the clearest.

I expect REST clients for individual services to be named after the services, and not include "REST" in the name at all. If say the package provided a RestClient type I'd agree with the pluralisation, or a collection of different kinds of template rest clients, but it doesn't.

[stemann]

I agree that REST clients should in general be named after the services, but they could include REST in their name to disambiguate among, e.g. a REST client and a SOAP client for the same service.

I am very much in the target audience for a REST client package, but only after digging down to the tutorial (1) do I kind of understand what RestClient tries to achieve.

From what I can grasp the intended usage is to create a module, e.g. DeckAPI, and use RestClient macros in DeckAPI to specify the Deck API...

This does not seem very natural. I would expect a REST Client package (claiming that name in the General registry) to provide functionality that would allow the user to create a deck_api_client value, which is a DeckAPIClient type (or a generic, underspecified, "RestClient" type...). I do not get the impression that RestClient provides something like this. It also seems does not seem to map to an OpenAPI.jl client...

(1): From the README and the Introduction, I was mostly left with claims ("Simplify", "concise", "capable", "absorb as much of the common complexity ... as possible", "sensible OOTB behaviour, "minimal setup") - and no explanation from what RestClient actually provides (macros for defining a REST API client).

[noblock]

[tecosaur]

[noblock] It sounds like you're expecting that a package that helps you write REST clients would do so in a structure that involves creating a type for a particular API/client? I must admit I don't see why that would necessarily be the case. A module/package seems much more natural to me.

It also seems does not seem to map to an OpenAPI.jl client...

Writing a command that turns an OpenAPI definition into a package using RestClient is something I've thought of doing, but that will have to come later when I've got more time spare (or somebody else steps up!).

(1): From the README and the Introduction [...] and no explanation from what RestClient actually provides

That's surprising to hear, I had hoped that these bits of the readme would clarify that:

• Debugging utilities, URI encoding, caching, and rate limiting.
• Straightforward interoperability with JSON3 and XML via @jsondef and @xmldef.

If there are specific details that you think it would have been helpful to see earlier, it would be great if you could highlight them so I could improve the readme/docs.

[stemann]

For the documentation, an example front and center showing actual use of an API via RestClient would help a lot: When I dug into the tutorial yesterday and accidentally skimmed past what "new" does in the Deck API, I was incredibly puzzled by:

julia> deck = DeckAPI.new()
┌ Debug:  GET  https://deckofcardsapi.com/api/deck/new?deck_count=1
└ @ RestClient
┌ Debug:  200  80 bytes (saved to /tmp/api-get.dump) from https://deckofcardsapi.com/api/deck/new?deck_count=1
└ @ RestClient
DeckAPI.Deck(id="01n3ezer3rly", remaining=52)

What I was thinking was

What?! Creating a DeckAPI (client) makes an API call?!

😊

This is of course ridiculous, and I should have read the tutorial more thoroughly, but I think a concrete usage example in the README would have avoided the confusion.

Regarding the RestClient API, one source of confusion is the naming of the RequestConfig type - I think every other HTTP client library I've seen, across languages/environments, would call such an entity ...Client.

Another source of confusion is macros - and, in particular, macros with opaque names/functionality - e.g. what does @globalconfig do? I can guess that @jsondef / @xmldef adapts structs to being JSON/XML serialisable, and I can read the documentation, but... a bit opaque. It makes more sense that @endpoint defines (a method for calling) an endpoint.

I think the macros are a bit too magical - I guess the @endpoint's are tied to the RequestConfig (the "client") by virtue of being defined in the same module...?

Regarding the type vs module discussion: With the premise that the client API is defined via macros, it makes sense, of course, to define a module. And vice versa, without this premise, I think the module would not be necessary.

I think it might be worthwhile to, at least superficially, explore how the majority of APIs are defined and consumed today, via OpenAPI definitions: It would be very nice with a way to go directly from a YAML/JSON OpenAPI definition file and to a Julia client API. And it might indeed make sense to define such an API via macros, e.g., something like

client = @openapi_client Meta.parse(read("deck.json", String))

As an added argument for the plural, RestClients, naming, it also would not block consumers from defining a RestClient type (whatever reason they might have for doing this).

[noblock]

[tecosaur]

[noblock] Thanks for the feedback @stemann!

For the documentation, an example front and center showing actual use of an API via RestClient would help a lot

I had hoped that the example in the readme might do this? It's complete and fully-functional.

Do you find it a bad example, or is there something else about it that stops it from being good enough? Or are you specifically suggesting that I put a little complete example like it at the start of a tutorial?

What?! Creating a DeckAPI (client) makes an API call?!

Thanks for mentioning this specific confusion, do you think it would help if I clarified that line with something like:

julia> deck = DeckAPI.new() # Create a new 'deck' using the API

Regarding the RestClient API, one source of confusion is the naming of the RequestConfig type - I think every other HTTP client library I've seen, across languages/environments, would call such an entity ...Client.

Hmm, I'm open to renaming that, but I don't think Client would be a good fit, it's much to incomplete. It's intended to just hold the domain/auth/etc. information needed in the background to make a request successfully.

Another source of confusion is macros - and, in particular, macros with opaque names/functionality [...] I think the macros are a bit too magical

It is fully possible to benefit from the library without using macros. I try to avoid making them seem to magical and actually show what they do by implementing the first endpoint in the tutorial without using the @endpoint macro (https://tecosaur.github.io/RestClient.jl/dev/tutorial/#The-new-deck-endpoint). Is that something you looked at, or did it not quite make sense? It would be great to clarify the documentation so it's generally clear what the macros are doing 🙂

Regarding the type vs module discussion: With the premise that the client API is defined via macros, it makes sense, of course, to define a module. And vice versa, without this premise, I think the module would not be necessary.

Even without macros, I still think it makes sense. What would the type-based approach involve? Doing something like call_api(::MyApiType, params...)? I can't say I see much of a point when specific APIs will have their own packages anyway.

It would be very nice with a way to go directly from a YAML/JSON OpenAPI definition file and to a Julia client API.

This largely consists of code generation, which https://github.com/JuliaComputing/OpenAPI.jl does, I'm just want a library that (1) works for non-OpenAPI REST APIs (2) includes OOTB support for rate limiting and caching (3) produces nice code to read.

It's not likely there will be an @openapi_client macro in the future, but there could be a command that generates a package from a spec file, similar to OpenAPI.jl just using RestClient.jl.

As an added argument for the plural, RestClients, naming, it also would not block consumers from defining a RestClient type

I suppose, I can't see this being a major motivation though. I'll poll Slack or somewhere again to see if I draw on a few more second opinions regarding the namee.

[stemann]

For the documentation, an example front and center showing actual use of an API via RestClient would help a lot

I had hoped that the example in the readme might do this? It's complete and fully-functional.

Do you find it a bad example, or is there something else about it that stops it from being good enough? Or are you specifically suggesting that I put a little complete example like it at the start of a tutorial?

What?! Creating a DeckAPI (client) makes an API call?!

Thanks for mentioning this specific confusion, do you think it would help if I clarified that line with something like:

julia> deck = DeckAPI.new() # Create a new 'deck' using the API

Just adding this one line would help (a lot).

Regarding the RestClient API, one source of confusion is the naming of the RequestConfig type - I think every other HTTP client library I've seen, across languages/environments, would call such an entity ...Client.

Hmm, I'm open to renaming that, but I don't think Client would be a good fit, it's much to incomplete. It's intended to just hold the domain/auth/etc. information needed in the background to make a request successfully.

Do you intend for the RequestConfig to remain incomplete wrt. calling it Client?

I'm glad you mention auth. as that is definitely a key ingredient in calling APIs. I did not see any mentions of authentication/authorization earlier.

How to inject an API key into a RestClient API (client)?

Another source of confusion is macros - and, in particular, macros with opaque names/functionality [...] I think the macros are a bit too magical

It is fully possible to benefit from the library without using macros. I try to avoid making them seem to magical and actually show what they do by implementing the first endpoint in the tutorial without using the @endpoint macro (https://tecosaur.github.io/RestClient.jl/dev/tutorial/#The-new-deck-endpoint). Is that something you looked at, or did it not quite make sense? It would be great to clarify the documentation so it's generally clear what the macros are doing 🙂

I skimmed past that part (too) :-) A bit out of at-keyboard time right now, but might check again later...

Regarding the type vs module discussion: With the premise that the client API is defined via macros, it makes sense, of course, to define a module. And vice versa, without this premise, I think the module would not be necessary.

Even without macros, I still think it makes sense. What would the type-based approach involve? Doing something like call_api(::MyApiType, params...)? I can't say I see much of a point when specific APIs will have their own packages anyway.

Exactly, call_api(::MyApiType, params...).

It might, in particular, make sense if you would like to talk to two services with the same API: That is a concrete use case from my past - migrating e.g. from GitLab.com to a self-hosted GitLab instance.

It would be very nice with a way to go directly from a YAML/JSON OpenAPI definition file and to a Julia client API.

This largely consists of code generation, which https://github.com/JuliaComputing/OpenAPI.jl does, I'm just want a library that (1) works for non-OpenAPI REST APIs (2) includes OOTB support for rate limiting and caching (3) produces nice code to read.

It's not likely there will be an @openapi_client macro in the future, but there could be a command that generates a package from a spec file, similar to OpenAPI.jl just using RestClient.jl.

I don't quite get what you mean? The suggested @openapi_client was exactly a future way of generating a client module/type from an OpenAPI spec. file using RestClient.

The benefit wrt. OpenAPI.jl being to have the code generation done by an in-language Julia macro instead of Java-generated code (including nicer stack traces when stuff breaks).

Just a summary at this point: Wrt. to this PR, my suggestion is to go for

• either plural RestClients and consider setting the version to < 1.0 - in particular if the aim of the package is to fulfill the end-all-be-all REST client package (including authentication, multiple services with same API, and some way of supporting OpenAPI generation - possibly in another package),
• or go for e.g. RestClientMacros or something like that (as macros seem to be the prime way of defining a RestClient API).

[noblock]

[tecosaur]

[noblock]

Just adding this one line would help (a lot).

Thanks for this feedback, I'll be making this change.

Do you intend for the RequestConfig to remain incomplete wrt. calling it Client?

It's not intended to provide all of the information needed to call an API, just centralise some of the common configuration (base URL, API key, etc.)

I'm glad you mention auth. as that is definitely a key ingredient in calling APIs. I did not see any mentions of authentication/authorization earlier.

That's not explicitly covered in the docs (yet), you have to read between the lines a little bit.

It might, in particular, make sense if you would like to talk to two services with the same API: That is a concrete use case from my past - migrating e.g. from GitLab.com to a self-hosted GitLab instance.

For cases like this, you can just create a RequestConfig that changes the base URL and pass that as the first argument to an endpoint function.

I don't quite get what you mean?

I think this sort of code-generation (involving reading the filesystem at compile-time) goes beyond what's sensible for a macro.

---

Regarding the package name, I don't think RestClientMacros is a particularly good name. The macros are just a convenient layer on top of the capabilities the package provides.

Regarding RestClients, I'm still not swayed by the argument for pluralisation here, and initial poll results indicate that it's not surprising to at least a few other people too.

Lastly, regarding setting the version to < 1.0, sem-ver v1 doesn't mean "finished/feature-complete", it just means no API breakage till the next major release, and I think that's appropriate for the package at the moment.

[stemann]

Do you intend for the RequestConfig to remain incomplete wrt. calling it Client?

It's not intended to provide all of the information needed to call an API, just centralise some of the common configuration (base URL, API key, etc.)

That sounds like pretty much all of the information to me... what are you thinking is missing?

I'm glad you mention auth. as that is definitely a key ingredient in calling APIs. I did not see any mentions of authentication/authorization earlier.

That's not explicitly covered in the docs (yet), you have to read between the lines a little bit.

OK (though that sounds definitely pre-1.0 - not having auth).

It might, in particular, make sense if you would like to talk to two services with the same API: That is a concrete use case from my past - migrating e.g. from GitLab.com to a self-hosted GitLab instance.

For cases like this, you can just create a RequestConfig that changes the base URL and pass that as the first argument to an endpoint function.

How would this look? Wouldn't I have to define the API twice?

module GitLabFoo1API

using RestClient

@globalconfig RequestConfig("https://gitlab.foo1.com")

@endpoint bar ...

end

module GitLabFoo2API

using RestClient

@globalconfig RequestConfig("https://gitlab.foo2.com")

@endpoint bar ...

end

?

I don't quite get what you mean?

I think this sort of code-generation (involving reading the filesystem at compile-time) goes beyond what's sensible for a macro.

True - you could of course read the file separately and then feed something to the macros. I'm thinking that the content being fed to @endpoint is quite similar to an OpenAPI endpoint definition.

Regarding the package name, I don't think RestClientMacros is a particularly good name. The macros are just a convenient layer on top of the capabilities the package provides.

My thought here was that if you strip away the macros, there's not that much left in addition to JSON3, StructTypes, HTTP etc.

Regarding RestClients, I'm still not swayed by the argument for pluralisation here, and initial poll results indicate that it's not surprising to at least a few other people too.

In any case, please give it some time (measured in days at the very least).

Lastly, regarding setting the version to < 1.0, sem-ver v1 doesn't mean "finished/feature-complete", it just means no API breakage till the next major release, and I think that's appropriate for the package at the moment.

Sure, but so does v0.1 (at least in the Julia/Pkg interpretation of sem. ver.). I know there are vocal opponents to this, but I'm not on-board with the "everything has to start at v1.0" thing. I think v1.0 should at least be "I've considered the proposals in the design phase and consider this to be complete wrt. basic requirements" - this is of course entirely up to you, but my two cents would require authentication and being able to connect to two services with the same API (non-exhaustive list).

[noblock]

[tecosaur]

That sounds like pretty much all of the information to me... what are you thinking is missing?

The API itself 😛

OK (though that sounds definitely pre-1.0 - not having auth).

I'm personally comfortable with better docs/examples coming in minor releases.

How would this look? Wouldn't I have to define the API twice?

No, you'd do this

module MyFancyAPI

@globalconfig RequestConfig("https://original.domain")

@endpoint foo(...) ...

end

and then to use it somewhere else

newconfig = RequestConfing("https://other.domain")

MyFancyAPI.foo(newconfig, ...)

My thought here was that if you strip away the macros, there's not that much left in addition to JSON3, StructTypes, HTTP etc.

src/requests.jl and src/caching.jl may not be long, but they're certainly not trivial.

In any case, please give it some time (measured in days at the very least).

Package registration takes a minimum of three days, and unless you add [noblock] to all your comments it will delay registration longer.

I know there are vocal opponents to this, but I'm not on-board with the "everything has to start at v1.0" thing

Neither am I, but to quote myself from earlier "I think [v1.0] is appropriate for the package at the moment".

[noblock]

[stemann]

That sounds like pretty much all of the information to me... what are you thinking is missing?

The API itself 😛

Right, but I mean, all the information to call the entity a client seems to be there... from my perspective.

OK (though that sounds definitely pre-1.0 - not having auth).

I'm personally comfortable with better docs/examples coming in minor releases.

Definitely. I just got the impression that auth. is not even supported at this point?

How would this look? Wouldn't I have to define the API twice?

No, you'd do this

module MyFancyAPI

@globalconfig RequestConfig("https://original.domain")

@endpoint foo(...) ...

end

and then to use it somewhere else

newconfig = RequestConfig("https://other.domain")

MyFancyAPI.foo(newconfig, ...)

OK! That's a bit surprising. I've tried looking at the documentation again - trying to understand what a config is and where it is used (and passed to endpoints), and I really have a hard time with the API.

I might even say I'm having close to the opposite experience of what @goerz phrased: "the ... API is exactly what you would want to talk to a REST interface".

Edit: ... Though I do really appreciate that you are tackling this challenge, and I appreciate the clever work that went into this. ... I just have a hard time grasping the API design...

My thought here was that if you strip away the macros, there's not that much left in addition to JSON3, StructTypes, HTTP etc.

src/requests.jl and src/caching.jl may not be long, but they're certainly not trivial.

That does indeed look non-trivial.

My interpretation might've been pre-judicial (just based on a glance at the @jsondef documentation).

I know there are vocal opponents to this, but I'm not on-board with the "everything has to start at v1.0" thing

Neither am I,

Nice 😊

[noblock]

[tecosaur]

I just got the impression that auth. is not even supported at this point?

There's a dedicated slot for auth to be stored in RequestConfig, I'll push a docs update to make that clearer. Actually providing the auth key to an API has to be implemented by endpoints, because I don't want to think of all the different ways that auth is handled by various APIs. Helper functions for common standards like bearer auth may be provided in future releases if there is sufficient interest.

OK! That's a bit surprising. I've tried looking at the documentation again - trying to understand what a config is and where it is used (and passed to endpoints)

The config is exactly five things:

• The base URL of the API
• A lock used to implement rate-limiting
• An optional secret key
• The timeout of requests
• Whether responses should be cached

This is not clear in the current docstring, and is one of the docstrings I'll be improving.

I really have a hard time with the API.

That is a pity to hear. Hopefully, the docs improvements that I'm making in response to your comments will help. I imagine examples might also be useful, and in time I could point to a few within the package docs/readme.

[noblock]

Thanks, I'll edit my comments in a bit to unblock this registration seeing as the poll is sticking around ~75% in favour of the singular form, and I'm still not keen for longer forms like RestClientScaffold (I feel like multiple non-specific terms don't clarify meaning enough to be worth it here, e.g. another discarded candidate name was RestClientSkeletonFramework).

[goerz]

Thanks, I'll edit my comments in a bit to unblock this registration

I can also apply an override to “ignore blocking comments”. Let me know if you want me to do that.

[noblock]

[tecosaur]

I can also apply an override to “ignore blocking comments”. Let me know if you want me to do that.

Thanks for the offer Michael! @stemann has communicated with me over Slack that he wants to leave his first comment as blocking for another week though (which I'm not thrilled about but also not too fussed about), so that would probably be premature.

[noblock]

[stemann]

I added the multiple choice poll that I had requested - I had to add it as an edit to one of my messages, as I had hit a max-three-consecutive-messages limit: https://discourse.julialang.org/t/pre-ann-restclient-jl/123828/27

[noblock]

[stemann]

I've opened a couple of issues in RestClient.jl to split out the API discussion.

[noblock]

[tecosaur]

Friendly poke @stemann 🙂 [noblock]

[stemann]

Thanks for the ping, and sorry for the delay - been away from the keyboard for some days.

Just reached out to the web channel on slack (as I had hoped to get input from the JuliaWeb / JuliaServices communities) - if there’s no significantly altering feedback/discussion within 48 hours from now, I will unblock my comment.

[noblock]

[stemann]

Unblocking as there has been no further input 😊

[noblock]