Schema generation & client libraries.

[tomchristie]

Hi Tim,

First up, congratulations - really great work! 😄

I'm really liking the approach here, lovely and simple, and very much promoting interface rather than request-response.

I've got some thoughts on how you could approach schema generation and client library support for hug, that's driven by work I'm doing right now. Feel free to throw any of this clean away if you're not interested in this kind of take, as it's very clearly motivated by my personal opinions on how we should be exposing API interfaces and interacting with them.

Parsers, Renderers & Conneg

Rather than a single input_formatter/output_formatter, provide an interface for content negotiating each from a list of available alternatives. This would allow the user to:

• Present endpoints that can accept input in any one of multiple available encodings.
• Present endpoints that can return output in any one of multiple available encodings.

Support both Accept: application/json header style and URL suffix .json style.

The tablib library could be neat thing to use here if you wanted to have nice built-in textual formats, eg given this code:

@hug.get():
def user_list():
    return [
        {'id': ..., 'name': ..., 'email': ...},
        {'id': ..., 'name': ..., 'email': ...},
        {'id': ..., 'name': ..., 'email': ...}
   ]

@hug.get():
def user_detail(user_id):
    return {'id': ..., 'name': ..., 'email': ...}

Generate the following:

GET http://www.example.org/users.json
[
    {'id': 123, 'name': 'tim', 'email': 'tim@example.org'},
    {'id': 456, 'name': 'tom', 'email': 'tom@example.org'},
    {'id': 789, 'name': 'tam', 'email': 'tam@example.org'},
]

GET http://www.example.org/users.txt
id |name|email
123|tim |tim@example.org
456|tom |tom@example.org
789|tam |tam@example.org

GET http://www.example.org/users.csv
id,name,email
123,tim,tim@example.org
456,tom,tom@example.org
789,tam,tam@example.org

GET http://www.example.org/users/123.json
{
    'id': 123,
    'name': 'tim',
    'email': 'tim@example.org'
}

GET http://www.example.org/users/123.txt
id   |123
name |tim
email|tim@example.org

GET http://www.example.org/users/123.csv
id,123
name,tim
email,tim@example.org

Control of parameter location

It's not clear to me how hug determines if a parameter should be:

• A query parameter. - http://example.com/users/?user_id=123
• A templated URL string - http://example.com/users/123
• A value from the parsed request data. {"user_id": 123}
• The request body itself. 123

Being able to either explicitly control that, or at least documenting which gets used might make the resulting API style that hug generates more clear to the end-user?

Schemas & Documentation

You could use coreapi for hug's schema endpoint. Rather than return a rendered schema in a given format, instead return a Document and allow it to be rendered into one of multiple different schema formats. This approach would allow you to support multiple different types of schema, and leverage their respective documentation tools, client libraries or whatever else.

Together with the above you could eg, support:

GET http://www.example.org/schema.openapi
GET http://www.example.org/schema.raml
GET http://www.example.org/schema.apiblueprint

And have all of the above auto-generated.

The function docstrings could be pulled through to provide the schema with the endpoint description and parameter descriptions.

You could also choose one of the available docs tools, and automatically serve that.

(Right now Core API has support for Open API and for JSON HyperSchema, but the others will be coming too in due course.)

This approach could also give you a nice way of providing hypermedia support in hug. Allow a function to optionally return a Document rather than plain data, and in that case render to HAL or any other hypermedia format supported by Core API.

Client libraries

Exposing a schema endpoint would allow hug to take advantage of dynamic client libraries, allowing users to more easily interact with their deployed APIs. Taking an opinionated stance on this could help you lead users in a smart direction here.

For example, if you were exposing a schema endpoint in a format supported by Core API, then you'd be able to interact with it using the dynamic client library and command line tool.

Let's take your "happy birthday" example.

@hug.get('/happy_birthday')
def happy_birthday(name, age:hug.types.number=1):
    """Says happy birthday to a user"""
    return "Happy {age} Birthday {name}!".format(**locals())

Interacting with the service via the command line

$ coreapi get http://127.0.0.1:8000/
<Hug API - "http://127.0.0.1:8000/">
    happy_birthday(name, [age])
$ coreapi action happy_birthday --param name tom --param age 123
Happy 123 Birthday tom!

(Of course if this is being run locally, then you can just use hug's cli, but the above allows anyone to interact with a deployed service)

Using the python client library:

>>> client = coreapi.Client()
>>> api = client.get('http://127.0.0.1:8000/')
>>> print client.action(api, ['happy_birthday'], params={'name': 'tom', 'age': 123}
Happy 123 Birthday tom!

Presenting hug APIs as having client libraries available in various languages straight out-of-the-box looks like a big win from my POV. Right now Core API has a command line tool and a python client library, but I'm working flat-out on this project and there will be be support in other languages too soon enough - with JavaScript support coming next. Core API is an open standard, so other folks can build dynamic client libraries in their own favored language too. I'll likely also be tying this in with REST framework in the near future, so I'm pretty much all-in on it.

Apologies for lumping this all together - hopefully it comes across as coherent whole.
Be very interested to know your thoughts on any of this!

Cheers!

Tom

Refs: #163, #198

[timothycrosley]

Hi Tom,

Thank you! I'm honored that you spent the time to look at the project, generally seem to like the concept, and put together such a well thought out set of potential improvements. I think the direction you listed here is spot on with the direction I'm trying to take the project, while some of the implementation might be somewhat different. For instance in the development branch I've added a clean way to support outputting different output formats depending on the content-type or URL suffix, I've added hug.use.HTTP to enable using any hug API externally in a way that feels like local usage, and allows cleanly switching back to local usage, @BrandonHoffman is working on an ongoing project to add the ability to switch documentation outputs cleanly, and I've secured hug.rest as a website to host as much documentation for the framework as I and the community can put together.

One of the things I would really like to do when 2.0.0 is finished at the end of this month, is put together a roadmap for 3.0.0 - At that time I'll reflect on what recommendations in here, quite possibly all, that we can and should integrate into the project and add them to the road map.

Also, if you ever have any random time to contribute to the project https://github.com/timothycrosley/hug/blob/develop/CONTRIBUTING.md you could totally get a free t-shirt :D

Again thanks! And I'll make sure to further investigate all of these as well as look into leveraging the projects you are working on

~Timothy