Schemas & client libraries. #4179

Merged
merged 32 commits into from Jul 4, 2016

Projects

None yet

7 participants

@tomchristie
Owner
tomchristie commented Jun 8, 2016 edited

Support for schema generation & dynamic client libraries.

See the new tutorial section, schema documentation, and api client documentation.

Needed to complete tutorial 7:

  • Graceful behavior with or without Core API installed.
  • Add --auth basic support to Core API.
  • Check permissions on schema generation.
  • Include pagination parameters in automatic schemas.

Other work as part of this, prior to 3.4.0 release.

  • Include filter parameters in automatic schemas.
  • Schema documentation.
  • Handle Document, Error and Link in JSON renderer?
  • Handle ListSerializer and other non-form.
  • Handle multipart vs json and/or other encodings.
  • Tests.
  • Handle text downloads.
  • Client library documentation.
  • Add coreapi to list of optional packages on homepage.

Moved to separate tickets:

  • Easier overrides on SchemaGenerator
  • Document using OpenAPI and other schema formats.
  • Move alternate formats in coreapi into separate packages. (HTML, OpenAPI, HyperSchema, HAL...)

Deferred:

  • Handle binary downloads. Not strictly required for 3.4
  • Use id not pk in schema representations or change serializer representations? Not strictly required for 3.4
  • Add to quickstart Not strictly required for 3.4
@tomchristie Initial pass at schema support
bc836aa
@tomchristie tomchristie added this to the 3.4.0 Release milestone Jun 8, 2016
added some commits Jun 9, 2016
@tomchristie Add coreapi to optional requirements.
b64340b
@tomchristie Clean up test failures
c890ad4
@tomchristie Add missing newline 2d28390
@tomchristie Minor docs update
744dba4
@tomchristie Version bump for coreapi in requirements
56ece73
@tomchristie Catch SyntaxError when importing coreapi with python 3.2
99adbf1
@tomchristie Add --diff to isort
80c595e
@tomchristie Import coreapi from compat
47c7765
@tomchristie Fail gracefully if attempting to use schemas without coreapi being in…
…stalled.
29e228d
@tomchristie Tutorial updates
eeffca4
@tomchristie Docs update
6c60f58
@tomchristie Initial schema generation & first tutorial 7 draft
b7fcdd2
@tomchristie Spelling
2e60f41
@tomchristie Remove unused variable 2ffa145
@tomchristie Docs tweak
b709dd4
added some commits Jun 14, 2016
@tomchristie Merge branch 'master' into schema-support 474a23e
@tomchristie Added SchemaGenerator class
4822896
@kevin-brown kevin-brown and 1 other commented on an outdated diff Jun 15, 2016
rest_framework/routers.py
def get(self, request, *args, **kwargs):
+ if request.accepted_renderer.format == 'corejson':
@kevin-brown
kevin-brown Jun 15, 2016 Collaborator

Is there any reason why this couldn't be brought into the CoreJSONRenderer?

@tomchristie
tomchristie Jun 15, 2016 edited Owner

The problem is less what to do with CoreJSONRenderer and more about making sure that we preserve the behavior of that view for anything that isn't a schema renderer.

(Also I'd consider that part still slightly in flux - clearly needs refinement)

@kevin-brown
kevin-brown Jun 15, 2016 Collaborator

making sure that we preserve the behavior of that view for anything that isn't a schema renderer

Right, my main concern with that line (which I'm sure will probably change another few times before this is merged) is that it's hard-coding corejson as the only renderer with schema support.

Also I'd consider that part still slightly in flux - clearly needs refinement

Definitely understandable.

@kevin-brown kevin-brown commented on an outdated diff Jun 15, 2016
rest_framework/routers.py
@@ -277,10 +285,24 @@ def get_api_root_view(self):
for prefix, viewset, basename in self.registry:
api_root_dict[prefix] = list_name.format(basename=basename)
+ view_renderers = list(api_settings.DEFAULT_RENDERER_CLASSES)
+
+ if schema_urls and self.schema_title:
+ assert coreapi, '`coreapi` must be installed for schema support.'
+ view_renderers += [renderers.CoreJSONRenderer]
@kevin-brown
kevin-brown Jun 15, 2016 Collaborator

Wouldn't it make more sense to expect users to include the CoreJSONRenderer in their default renderers, and raise an error if it is missing from the defaults?

@tomchristie Fail gracefully if coreapi is not installed and SchemaGenerator is used
1f76cca
@kevin-brown kevin-brown commented on the diff Jun 15, 2016
rest_framework/renderers.py
@@ -790,3 +791,17 @@ def render(self, data, accepted_media_type=None, renderer_context=None):
"test case." % key
)
return encode_multipart(self.BOUNDARY, data)
+
+
+class CoreJSONRenderer(BaseRenderer):
@kevin-brown
kevin-brown Jun 15, 2016 Collaborator

There was a move in 3.1 to remove features which had third-party dependencies from the core and put them into third-party packages that were still highly visible. See django-rest-framework-xml, django-rest-framework-yaml, and django-rest-framework-oauth for examples.

Is that still something we are pushing for?

@tomchristie
tomchristie Jun 15, 2016 edited Owner

Kinda. I see coreapi as a foundational thing here, so it's a bit different.

The various types of schema and docs that you can use it to generate will be third party, yup.
So eg we can have third party packages for schema formats: Swagger / API Blueprint / JSON Hyperschema, for docs templates driven by a coreapi.Document object, and for various hypermedia styles.

Using coreapi ensures that we're able to provide a common interface for all of those, so I don't have any great issues with pulling it in. If it's in core, then we're making the promise that it's an interface that is available to third-party devs, which should help drive folks making schema renderers / hypermedia renderers and docs renderers. Having said that, we could push for it to be third-party, if we wanted to, so I might be open to discussion.

@tomchristie tomchristie referenced this pull request in marcgibbons/django-rest-swagger Jun 17, 2016
Closed

Swagger 2.0 roadmap #443

added some commits Jun 21, 2016
@tomchristie Schema docs, pagination controls, filter controls
cad24b1
@tomchristie Resolve NameError
8fb2602
@tomchristie Add 'view' argument to 'get_fields()'
b438281
@tomchristie Remove extranous blank line
8519b4e
@codecov-io
codecov-io commented Jun 21, 2016 edited

Current coverage is 91.39%

Merging #4179 into master will decrease coverage by 0.07%

@@             master      #4179   diff @@
==========================================
  Files            51         52     +1   
  Lines          5528       6186   +658   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits           5057       5654   +597   
- Misses          471        532    +61   
  Partials          0          0          

Powered by Codecov. Last updated by 1633a0a...12be5b3

added some commits Jun 22, 2016
@tomchristie Add integration tests for schema generation
2f5c974
@tomchristie Only set 'encoding' if a 'form' or 'body' field exists
e78753d
added some commits Jun 24, 2016
@tomchristie Do not use schmea in tests if coreapi is not installed
84bb5ea
@tomchristie Inital pass at API client docs
63e8467
@tomchristie Inital pass at API client docs
bdbcb33
@tomchristie More work towards client documentation
7236af3
@foresmac foresmac and 1 other commented on an outdated diff Jul 1, 2016
docs/api-guide/schemas.md
+for REST framework.
+
+ pip install coreapi
+
+REST framework includes functionality for auto-generating a schema,
+or allows you to specify one explicitly. There are a few different ways to
+add a schema to your API, depending on exactly what you need.
+
+## Using DefaultRouter
+
+If you're using `DefaultRouter` then you can include an auto-generated schema,
+simply by adding a `schema_title` argument to the router.
+
+ router = DefaultRouter(schema_title='Server Monitoring API')
+
+The schema will be included in by the root URL, `/`, and presented to clients
@foresmac
foresmac Jul 1, 2016

"included in by" seems confusing. Maybe take out the "by"?

@tomchristie
tomchristie Jul 1, 2016 Owner

Good catch, that's a typo.

@foresmac foresmac commented on the diff Jul 1, 2016
docs/api-guide/schemas.md
+ {
+ "_meta": {
+ "title": "Server Monitoring API"
+ },
+ "_type": "document",
+ ...
+ }
+
+This is a great zero-configuration option for when you want to get up and
+running really quickly. If you want a little more flexibility over the
+schema output then you'll need to consider using `SchemaGenerator` instead.
+
+## Using SchemaGenerator
+
+The most common way to add a schema to your API is to use the `SchemaGenerator`
+class to auto-generate the `Document` instance, and to return that from a view.
@foresmac
foresmac Jul 1, 2016

Does SchemaGenerator work with any DRF APIView?

@foresmac
foresmac Jul 1, 2016

This is awesome. 💯

added some commits Jul 4, 2016
@tomchristie Add coreapi to optional packages list
89540ab
@tomchristie Clean up API clients docs
e3ced75
@tomchristie Resolve typo
12be5b3
@tomchristie tomchristie referenced this pull request Jul 4, 2016
Closed

Schema & Hypermedia follow ups #4241

3 of 7 tasks complete
@tomchristie tomchristie merged commit 6ff9840 into master Jul 4, 2016

3 checks passed

codecov/project 91.39% (target 90.00%)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@tomchristie tomchristie deleted the schema-support branch Jul 4, 2016
@tomchristie
Owner

Remaining issues have now been split out into individual tickets.
Plan is to prepare up a 3.4.0 release, then resolve as many remaining tickets in the time remaining before Thursday 14th July, with a release planned on or before that date.

(Ensure it will be released prior to DjangoCon US, and give at least one clear working day to resolve any ciritical issues, post-release)

@marcgibbons
Contributor

💯

@tomchristie
Owner

Yeah. Indeed.

@kevin-brown kevin-brown commented on the diff Jul 5, 2016
rest_framework/filters.py
@@ -127,6 +130,17 @@ def to_html(self, request, queryset, view):
template = loader.get_template(self.template)
return template_render(template, context)
+ def get_fields(self, view):
+ filter_class = getattr(view, 'filter_class', None)
@kevin-brown
kevin-brown Jul 5, 2016 Collaborator

Wouldn't it make more sense to use self.get_filter_class here?

@tomchristie
tomchristie Sep 22, 2016 Owner

Possibly, tho we don't actually have a queryset/model available to us at this point, so couldn't dynamically generate the filter class if none was set explicitly.

@rpkilby
rpkilby Sep 22, 2016 Contributor

What are the problems with just calling view.get_queryset()?

@tomchristie
tomchristie Sep 22, 2016 Owner

Sure, that'd probably work okay.

@nealtodd nealtodd commented on the diff Jul 12, 2016
docs/topics/api-clients.md
+
+Because this documentation is driven by the API schema it will always be fully
+up to date with the most recently deployed version of the service.
+
+---
+
+# Command line client
+
+The command line client allows you to inspect and interact with any API that
+exposes a supported schema format.
+
+## Getting started
+
+To install the Core API command line client, use `pip`.
+
+ $ pip install coreapi
@nealtodd
nealtodd Jul 12, 2016

For the command line client this should be pip install coreapi-cli

@tomchristie
tomchristie Jul 12, 2016 Owner

Thanks, yup!

@tomchristie
tomchristie Jul 12, 2016 Owner

Resolved in master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment