New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add configurable serializers in each renderer #21496

Open
wants to merge 26 commits into
base: master
from

Conversation

Projects
None yet
9 participants
@bf4
Contributor

bf4 commented Sep 4, 2015

Controller Examples

class TheController < ApplicationController
  serializing json: ->(json, options) do
    return json if json.is_a?(String)
    json = json.as_json(options) if json.respond_to?(:as_json)

    JSON.pretty_generate(json, options)
  end
end
class UserController < ApplicationController
  serializing json: ->(json, options) { UserSerializer.new(json, options).as_json(options) }
end

Most relevant core contributor comments

Comment 2017-09-05

Not sure if it can use JBuilder. Per rails/jbuilder#321

class UserController < ApplicationController
  serializing json: ->(json, options) { 
    template = JbuilderTemplate.new(lookup_context)
    result = JbuilderHandler.call(template)
    JSON.pretty_generate(result)
  }
end

Update 2017-08-30

  • Merged in Rails master
  • Added relevant core contributor feedback before updates
  • Note MissingRenderer #21496 (comment)

Update 2016-09-25

Waiting for further feedback from Rails core. Test failures aren't related.

Update 2016-05-22

there's been a lot of churn in this PR over the time it's been worked on. Below is the original description. Current contents is best understood by reading the diff and comments on it.

A good summary can be obtained by reading #21496 (comment) and #21496 (comment)

Original description 2015-09-04

per https://groups.google.com/forum/#!topic/rubyonrails-core/K8t4-DZ_DkQ/discussion

Since 2011, work in EmberJS and on the JSON API has resulted in some changing naming conventions

  • Model: class that defines the properties and behavior of the data that you present to the user.
  • Record: A record is an instance of a model that contains data loaded from a server. Your application can also create new records and save them back to the server.
  • Adapter: knows how to talk to your server. knows how to translate requests from the client into requests on your server. object that translates requests from AMS/Rails/Grape etc (such as "find the user with an ID of 123") into a requests to a server. let you completely change how your API is implemented without impacting your application code.
  • Serializer: maps keys and values to desired format, see https://github.com/orbitjs/orbit.js/blob/master/lib/orbit-common/jsonapi-serializer.js
  • Store: the central repository of records in your application, use the store to retrieve records, as well to create new ones. The store will automatically cache records for you.
@rails-bot

This comment has been minimized.

Show comment
Hide comment
@rails-bot

rails-bot Sep 4, 2015

r? @carlosantoniodasilva

(@rails-bot has picked a reviewer for you, use r? to override)

rails-bot commented Sep 4, 2015

r? @carlosantoniodasilva

(@rails-bot has picked a reviewer for you, use r? to override)

@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Sep 4, 2015

Contributor

Existing tests should ensure no regression. If we move forward with this design or other feedback, I'll add tests and documentation for the new features as well.

Contributor

bf4 commented Sep 4, 2015

Existing tests should ensure no regression. If we move forward with this design or other feedback, I'll add tests and documentation for the new features as well.

@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Sep 4, 2015

Contributor
Contributor

bf4 commented Sep 4, 2015

@bf4

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Sep 9, 2015

Contributor
Contributor

bf4 commented Sep 9, 2015

@rafaelfranca

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@rafaelfranca

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Sep 9, 2015

Member

API looks weird but implementation is good. Maybe it would be better if we don't have the Serializers module and only the methods inside the Renderers module.

Member

rafaelfranca commented Sep 9, 2015

API looks weird but implementation is good. Maybe it would be better if we don't have the Serializers module and only the methods inside the Renderers module.

@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Sep 9, 2015

Contributor

Thanks @rafaelfranca

only the methods inside the Renderers module.

That works for me, and probably makes more sense and is simpler to understand.

I also apologize that I don't seem to have noted that this is a proposed solution that doesn't break anything (per tests), and that if we go with it, I'll write tests and some docs. So, I especially thank your for treatment of a PR like this without tests :)

Contributor

bf4 commented Sep 9, 2015

Thanks @rafaelfranca

only the methods inside the Renderers module.

That works for me, and probably makes more sense and is simpler to understand.

I also apologize that I don't seem to have noted that this is a proposed solution that doesn't break anything (per tests), and that if we go with it, I'll write tests and some docs. So, I especially thank your for treatment of a PR like this without tests :)

@bf4

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@bf4

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Sep 10, 2015

Contributor

@rafaelfranca Updated

cc

I figure tests might go in (?)

and doc changes would go in

Contributor

bf4 commented Sep 10, 2015

@rafaelfranca Updated

cc

I figure tests might go in (?)

and doc changes would go in

@bf4

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Sep 21, 2015

Contributor

Is there anything I can do to help move this along?

Contributor

bf4 commented Sep 21, 2015

Is there anything I can do to help move this along?

@rafaelfranca

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@rafaelfranca

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@rafaelfranca

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@rafaelfranca

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@rafaelfranca

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@rafaelfranca

View changes

Show outdated Hide outdated actionpack/lib/action_controller/metal/renderers.rb
@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Sep 26, 2015

Member

figure tests might go in (?)

https://github.com/rails/rails/blob/master/actionpack/test/controller/renderer_test.rb
https://github.com/rails/rails/blob/master/actionpack/test/controller/render_xml_test.rb
https://github.com/rails/rails/blob/master/actionpack/test/controller/mime/respond_to_test.rb#L139
https://github.com/rails/rails/blob/master/actionpack/test/controller/render_json_test.rb#L90
it looks like some tests may have been removed in ee77770#diff-28658021f35e96c7499fdf41908629f0L644 ?

def test_uses_renderer_if_an_api_behavior
ActionController::Renderers.add :csv do |obj, options|
send_data obj.to_csv, type: Mime::CSV
end
@controller = CsvRespondWithController.new
get :index, format: 'csv'
assert_equal Mime::CSV, @response.content_type
assert_equal "c,s,v", @response.body
ensure
ActionController::Renderers.remove :csv
end
def test_raises_missing_renderer_if_an_api_behavior_with_no_renderer
@controller = CsvRespondWithController.new
assert_raise ActionController::MissingRenderer do
get :index, format: 'csv'
end
end
def test_removing_renderers
ActionController::Renderers.add :csv do |obj, options|
send_data obj.to_csv, type: Mime::CSV
end
@controller = CsvRespondWithController.new
@request.accept = "text/csv"
get :index, format: 'csv'
assert_equal Mime::CSV, @response.content_type
ActionController::Renderers.remove :csv
assert_raise ActionController::MissingRenderer do
get :index, format: 'csv'
end
ensure
ActionController::Renderers.remove :csv
end

👍

Member

rafaelfranca commented Sep 26, 2015

figure tests might go in (?)

https://github.com/rails/rails/blob/master/actionpack/test/controller/renderer_test.rb
https://github.com/rails/rails/blob/master/actionpack/test/controller/render_xml_test.rb
https://github.com/rails/rails/blob/master/actionpack/test/controller/mime/respond_to_test.rb#L139
https://github.com/rails/rails/blob/master/actionpack/test/controller/render_json_test.rb#L90
it looks like some tests may have been removed in ee77770#diff-28658021f35e96c7499fdf41908629f0L644 ?

def test_uses_renderer_if_an_api_behavior
ActionController::Renderers.add :csv do |obj, options|
send_data obj.to_csv, type: Mime::CSV
end
@controller = CsvRespondWithController.new
get :index, format: 'csv'
assert_equal Mime::CSV, @response.content_type
assert_equal "c,s,v", @response.body
ensure
ActionController::Renderers.remove :csv
end
def test_raises_missing_renderer_if_an_api_behavior_with_no_renderer
@controller = CsvRespondWithController.new
assert_raise ActionController::MissingRenderer do
get :index, format: 'csv'
end
end
def test_removing_renderers
ActionController::Renderers.add :csv do |obj, options|
send_data obj.to_csv, type: Mime::CSV
end
@controller = CsvRespondWithController.new
@request.accept = "text/csv"
get :index, format: 'csv'
assert_equal Mime::CSV, @response.content_type
ActionController::Renderers.remove :csv
assert_raise ActionController::MissingRenderer do
get :index, format: 'csv'
end
ensure
ActionController::Renderers.remove :csv
end

👍

@joaomdmoura

This comment has been minimized.

Show comment
Hide comment
@joaomdmoura

joaomdmoura commented Oct 10, 2015

👍

@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4
Contributor

bf4 commented Nov 16, 2015

@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Jun 20, 2016

Contributor

@rafaelfranca any chance this will be in 5.0.0?

Contributor

bf4 commented Jun 20, 2016

@rafaelfranca any chance this will be in 5.0.0?

@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Sep 6, 2016

Contributor

Failure not related https://travis-ci.org/rails/rails/jobs/157761951

  log writing failed. PG::TRDeadlockDetected: ERROR:  deadlock detected
DETAIL:  Process 7399 waits for ShareLock on transaction 10049; blocked by process 7397.
Process 7397 waits for ShareLock on transaction 10048; blocked by process 7399.
HINT:  See server log for query details.
CONTEXT:  while updating tuple (0,2) in relation "samples"
: UPDATE "samples" SET "value" = $1 WHERE "samples"."id" = $2
.

No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.

The build has been terminated
Contributor

bf4 commented Sep 6, 2016

Failure not related https://travis-ci.org/rails/rails/jobs/157761951

  log writing failed. PG::TRDeadlockDetected: ERROR:  deadlock detected
DETAIL:  Process 7399 waits for ShareLock on transaction 10049; blocked by process 7397.
Process 7397 waits for ShareLock on transaction 10048; blocked by process 7399.
HINT:  See server log for query details.
CONTEXT:  while updating tuple (0,2) in relation "samples"
: UPDATE "samples" SET "value" = $1 WHERE "samples"."id" = $2
.

No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.

The build has been terminated

bf4 added some commits Aug 31, 2017

Merge branch 'master' into serializer_for_renderer
Conflicts:
	actionpack/CHANGELOG.md
	actionpack/lib/action_controller/metal/renderers.rb
	actionpack/test/controller/render_json_test.rb
	actionpack/test/controller/renderer_test.rb
	actionpack/test/controller/renderers_test.rb

bf4 referenced this pull request Aug 31, 2017

Do not default to any Serializer
We will wait until 5.1 to make a decision
@aCandidMind

This comment has been minimized.

Show comment
Hide comment
@aCandidMind

aCandidMind Sep 4, 2017

This really seems to be the right direction and is looking like a good addition to make Rails a solid API choice and driving correct usage of the MIME type for json:api. It's only missing documentation & changelog approval so it seems to be upcoming.

aCandidMind commented Sep 4, 2017

This really seems to be the right direction and is looking like a good addition to make Rails a solid API choice and driving correct usage of the MIME type for json:api. It's only missing documentation & changelog approval so it seems to be upcoming.

#
# json = json.as_json(options) if json.respond_to?(:as_json)
# JSON.pretty_generate(json, options)
# end

This comment has been minimized.

@bf4

bf4 Sep 5, 2017

Contributor

👀 Example

@bf4

bf4 Sep 5, 2017

Contributor

👀 Example

bf4 added some commits Sep 7, 2017

Merge branch 'master' into serializer_for_renderer
Conflicts:
	actionpack/CHANGELOG.md
~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1726:in `current_s…
…erver': DRb::DRbServerNotFound (DRb::DRbConnError)

```
rails/actionpack

📝 $ bundle exec ruby -Ilib:test test/controller/renderers_test.rb --seed 41487
Run options: --seed 41487

Running:

.....
~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1726:in `current_server': DRb::DRbServerNotFound (DRb::DRbConnError)
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1795:in `to_id'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1103:in `initialize'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:651:in `new'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:651:in `make_proxy'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:568:in `rescue in dump'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:565:in `dump'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:612:in `block in send_request'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:611:in `each'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:611:in `send_request'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:926:in `send_request'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1253:in `send_message'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1142:in `block (2 levels) in method_missing'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1229:in `open'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1141:in `block in method_missing'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1160:in `with_friend'
        from ~/.rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/drb/drb.rb:1140:in `method_missing'
        from rails/actionpack/test/abstract_unit.rb:404:in `block (2 levels) in shutdown'
        from rails/actionpack/test/abstract_unit.rb:393:in `fork'
        from rails/actionpack/test/abstract_unit.rb:393:in `block in shutdown'
        from rails/actionpack/test/abstract_unit.rb:392:in `times'
        from rails/actionpack/test/abstract_unit.rb:392:in `each'
        from rails/actionpack/test/abstract_unit.rb:392:in `map'
        from rails/actionpack/test/abstract_unit.rb:392:in `shutdown'
        from rails/bundle/ruby/2.4.0/gems/minitest-5.10.3/lib/minitest.rb:140:in `run'
        from rails/bundle/ruby/2.4.0/gems/minitest-5.10.3/lib/minitest.rb:63:in `block in autorun'
.

Finished in 0.056913s, 105.4235 runs/s, 245.9882 assertions/s.
6 runs, 14 assertions, 0 failures, 0 errors, 0 skips
```
Add Controller.serializing mime: ->(obj, options) { }
Isolate JSON serializing tests from class_attribute effects
@bf4

This comment has been minimized.

Show comment
Hide comment
@bf4

bf4 Sep 13, 2017

Contributor

@rafaelfranca below are the relevant commits to review. I made a kind of clean slate within the PR. You can see some of the commits are refactors that aren't strictly necessary.

Message ref
Merge branch 'master' into serializer_for_renderer 4e7fdf0
Make even with master to start impl over c848569
Add actionpack/CHANGELOG to document use_renderers 1f06aa7
Make test description/render format more explicit 596179a
Test explicit undefined renderer raises MissingTemplate e6091fc
Remove unnecessary self.response_body = d305814
Implement minimal ActionController.add/remove_serializer a158880
Raise MissingSerializer when only Renderer defined c124ffb
Assert exception messages 1e5b552
Add Controller.serializing mime: ->(obj, options) { } … 70b2f85
Refactor _render_to_body_with_renderer cd00431
Improve naming in _render_to_body_with_renderer 44a68f0
Add changelog 4093649

Things that I think need attention

  • Any refactors or changes to remove?

  • That all the comments and documentation make sense, have no typos, are useful

  • Consider adding a rescue_response to actionpack/lib/action_dispatch/middleware/exception_wrapper.rb for MissngSerializer and MissingRenderer

  • Would be good to allow specifying the serializer as an argument to render, e.g.

-        serialized_value = _serializers[renderer_format].call(value_to_render, options)
+        serializer_name = (options.delete(:serializer_name) || renderer_format).to_sym
+        serialized_value = _serializers[serializer_name].call(value_to_render, options)

and

--- a/actionpack/CHANGELOG.md
+++ b/actionpack/CHANGELOG.md
@@ -16,11 +16,16 @@
     Renderers now only handle non-serialization concerns, such as setting the
     mime-type, how the data is returned, and handling callbacks.
 
-    The `serializer_name` is the name of the format being rendered, e.g. `json`.
+    The `serializer_name` is either from the params `serializer_name`, when given, or
+    the name of the format being rendered, e.g. `json`.
 
     The serializer is a callable object that accepts the same arguments as the renderer,
-    i.e. the object and the rendering options.  For example,
-    `render json: model` will serialize `model` as `_serializers[:json].call(model, options)`.
+    i.e. the object and the rendering options.
+        render json: model`
+        # => `_serializers[:json].call(model, options)
+
+        render json: model, serializer_name: :json_api
+        # =>  `_serializers[:json_api].call(model,options)
 
     The controller with raise an `ActionController::MissingSerializer` if no serializer is found
     for the format being rendered.
  • Confirm there are no thread-safety issues of runtime errors. Tests would probably go in actionpack/test/controller/metal/renderers_test.rb

  • Review interface and recommended usage, ActionController.add_renderer vs. ActionController::Renderers.add, etc.

  • Compare _renderers/use_renderer to _serializers/serializing that they make sense to exist as implemented.

  • Consider having the renderer redefine the method, rather than define it.

  • Should the default _renderers be RENDERERS.dup.freeze instead of Set.new.freeze? If not, should the default for _serializers still be SERIALIZERS.dup. Should it be frozen?

  • Consider raising MissingRenderer in Rails itself, not just in the responders gem.

--- a/actionpack/lib/action_controller/metal/renderers.rb
+++ b/actionpack/lib/action_controller/metal/renderers.rb
@@ -240,7 +240,9 @@ def serializing(options)
     # If no renderer is found, +super+ returns control to
     # <tt>ActionView::Rendering.render_to_body</tt>, if present.
     def render_to_body(options)
-      _render_to_body_with_renderer(options) || super
+      _render_to_body_with_renderer(options) \
+        || super \
+        || fail(ActionController::MissingRenderer.new(request.format))
     end
Contributor

bf4 commented Sep 13, 2017

@rafaelfranca below are the relevant commits to review. I made a kind of clean slate within the PR. You can see some of the commits are refactors that aren't strictly necessary.

Message ref
Merge branch 'master' into serializer_for_renderer 4e7fdf0
Make even with master to start impl over c848569
Add actionpack/CHANGELOG to document use_renderers 1f06aa7
Make test description/render format more explicit 596179a
Test explicit undefined renderer raises MissingTemplate e6091fc
Remove unnecessary self.response_body = d305814
Implement minimal ActionController.add/remove_serializer a158880
Raise MissingSerializer when only Renderer defined c124ffb
Assert exception messages 1e5b552
Add Controller.serializing mime: ->(obj, options) { } … 70b2f85
Refactor _render_to_body_with_renderer cd00431
Improve naming in _render_to_body_with_renderer 44a68f0
Add changelog 4093649

Things that I think need attention

  • Any refactors or changes to remove?

  • That all the comments and documentation make sense, have no typos, are useful

  • Consider adding a rescue_response to actionpack/lib/action_dispatch/middleware/exception_wrapper.rb for MissngSerializer and MissingRenderer

  • Would be good to allow specifying the serializer as an argument to render, e.g.

-        serialized_value = _serializers[renderer_format].call(value_to_render, options)
+        serializer_name = (options.delete(:serializer_name) || renderer_format).to_sym
+        serialized_value = _serializers[serializer_name].call(value_to_render, options)

and

--- a/actionpack/CHANGELOG.md
+++ b/actionpack/CHANGELOG.md
@@ -16,11 +16,16 @@
     Renderers now only handle non-serialization concerns, such as setting the
     mime-type, how the data is returned, and handling callbacks.
 
-    The `serializer_name` is the name of the format being rendered, e.g. `json`.
+    The `serializer_name` is either from the params `serializer_name`, when given, or
+    the name of the format being rendered, e.g. `json`.
 
     The serializer is a callable object that accepts the same arguments as the renderer,
-    i.e. the object and the rendering options.  For example,
-    `render json: model` will serialize `model` as `_serializers[:json].call(model, options)`.
+    i.e. the object and the rendering options.
+        render json: model`
+        # => `_serializers[:json].call(model, options)
+
+        render json: model, serializer_name: :json_api
+        # =>  `_serializers[:json_api].call(model,options)
 
     The controller with raise an `ActionController::MissingSerializer` if no serializer is found
     for the format being rendered.
  • Confirm there are no thread-safety issues of runtime errors. Tests would probably go in actionpack/test/controller/metal/renderers_test.rb

  • Review interface and recommended usage, ActionController.add_renderer vs. ActionController::Renderers.add, etc.

  • Compare _renderers/use_renderer to _serializers/serializing that they make sense to exist as implemented.

  • Consider having the renderer redefine the method, rather than define it.

  • Should the default _renderers be RENDERERS.dup.freeze instead of Set.new.freeze? If not, should the default for _serializers still be SERIALIZERS.dup. Should it be frozen?

  • Consider raising MissingRenderer in Rails itself, not just in the responders gem.

--- a/actionpack/lib/action_controller/metal/renderers.rb
+++ b/actionpack/lib/action_controller/metal/renderers.rb
@@ -240,7 +240,9 @@ def serializing(options)
     # If no renderer is found, +super+ returns control to
     # <tt>ActionView::Rendering.render_to_body</tt>, if present.
     def render_to_body(options)
-      _render_to_body_with_renderer(options) || super
+      _render_to_body_with_renderer(options) \
+        || super \
+        || fail(ActionController::MissingRenderer.new(request.format))
     end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment