Support limited form of HTTP service definition

[coder543]

Just throwing some ideas out here, but I believe that it would be possible to support a large subset of HTTP endpoints without too much added complexity.

type Cat {
  name: string;
  breed: string;
}

type AddResponse {
  success: bool;
  error: string;
}

endpoint GET "https://example.com/cats/list" list() -> [Cat];
endpoint GET url-encoded "https://example.com/cats/list" listByBreed(breed: string) -> [Cat];
endpoint GET url-encoded "https://example.com/cats/list" listByName(name: string) -> [Cat];
endpoint POST json-body "https://example.com/cats/add" add(cat: Cat) -> AddResponse;

The syntax could probably use some work, but I hope the intention is clear. Each defined endpoint would have a URL, an HTTP method, and a choice between sending the data in the HTTP body as JSON or url-encoding the parameters. If there are no parameters, then url-encoded and json-body would have no effect, so they could be optionally included or left out, but it should be mandatory to specify that when there are parameters.

With the generated Rust code, you would simply call listByBreed("siamese") and it would return a Result<Vec<Cat>, Error>, by retrieving and parsing the response from GETting https://example.com/cats/list?breed=siamese.

For web services where the authentication is part of the request body, this should cover almost any common case. Adding support for Basic Auth could be useful for legacy web services, but I don't think it's a huge concern.

[coder543]

Another style for writing the same endpoints above might be like this:

endpoint GET "https://example.com/cats/list" {
  list() -> [Cat];
  listByBreed(breed: string) -> [Cat];
  listByName(name: string) -> [Cat];
}

endpoint POST json-body "https://example.com/cats/add" add(cat: Cat) -> AddResponse;

This way if multiple methods are based on the same endpoint, but use different parameters or return different responses, they could all be grouped in a more DRY fashion.

[udoprog]

Hey, Thanks for giving this some thought!

There are two high-level approaches that can be taken here:

• First-class, syntactic support for HTTP endpoints.
• Metadata (options, attributes) associated with existing structures capable of adapting them for a new purpose.

I'd just like to outline what the second approach would look like for one of your examples:

Using options, an existing language concept:

service MyService {
  option base_url = "https://example.com";

  list_by_breed(breed: string) {
    // detect if a non-argument symbol, or one which cannot be marshalled as a query parameter is specified and error.
    option http.query_params = [breed];
    option http.method = "GET";
    option http.path = "/cats/list";
  }
}

Using rust-like attributes, a general purpose syntax addition for carrying metadata:

#[http(base_url = "https://example.com")]
service MyService {
  #[http(query_params = [breed], method = "GET", path = "/cats/list")]
  list_by_breed(breed: string) -> [Cat];
}

Attribute have the benefit of being fairly recognizable for many languages (annotations in Java, attributes in Rust, data annotations in c#, ...) and being compact.

Introducing a general purpose way of carrying metadata also means more "bang for the buck".
More capabilities for less syntactical changes.
Learning attributes as a concept once also means that they are instantly recognizable in other contexts.

First-class syntax is a bit scarier.

They typically occupy a number of keywords which then can't be used for field names, unless we change how the parser works.
I'd need a strong motivation to introduce more specialization in the parser.
I'm already thinking that using a dedicated stream keyword in the existing service declaration might be a bad idea, attributes could supersede this.

[coder543]

Both of the examples you provide seem very nice to me. It's true that supporting every possible HTTP endpoint is difficult, but supporting a useful subset somehow seems immensely worthwhile to me. It would remove a ton of boilerplate.

For example, with a weather.reproto file, you could write precise enough definitions to generate code to get weather data using only 3 or 4 lines of code in any given supported programming language, which would be awesome. When only the structures and serialization are handled, it's up to the user of the reproto file to handle all the boilerplate of accessing the endpoint.

I'm just voicing my support for this feature. Even in the absence of it, reproto still seems to be awesome.

[coder543]

Considering that any implementation of this concept of services would be highly opinionated, if it gets implemented, it might be worthwhile to include an option in reproto to not emit any code for services, since some users of a particular reproto file might only be interested in the structures themselves.

[udoprog]

Agreed on all counts. Putting it behind a feature flag is a good way to introduce it through experimentation. The existing module system would be suitable for that.

[udoprog]

We now have type-checked attributes:
https://github.com/reproto/reproto/blob/master/examples/src/io/reproto/toystore.reproto#L84

I've also started putting together client-side codegen for okhttp (Java) here:
https://github.com/reproto/reproto/blob/master/lib/backend-java/src/module/okhttp.rs

[coder543]

This looks awesome! I'm glad to see progress happening here!

[udoprog]

A number of test cases have been added which is working towards getting HTTP support through various frameworks. The code at least in Java and Python has been sufficiently cleaned up with codegen to better support this.

See: https://github.com/reproto/reproto/tree/master/it