Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Oj Serializers 2.0 #9

Merged
merged 30 commits into from Mar 27, 2023
Merged

feat: Oj Serializers 2.0 #9

merged 30 commits into from Mar 27, 2023

Conversation

ElMassimo
Copy link
Owner

@ElMassimo ElMassimo commented Mar 17, 2023

Description πŸ“–

This pull request introduces a rework of the internals, after experimenting with ways to reduce the indirection to obtain the values for each attribute.

It has more features than v1, while increasing performance even further πŸš€

Features ✨

  • Added render_as_hash to efficiently build a Hash from the serializer
  • render shortcut, unifying one and many
  • transform_keys :camelize: a built-in setting to convert keys, in a way that does not affect runtime performance
  • sort_keys_by :name: allows to sort the response alphabetically, without affecting runtime performance
  • serialize as an easier approach to define serializer attributes

Breaking Changes

Since returning a Hash is more convenient than returning a Oj::StringWriter, and performance is comparable, I'm considering making default_format :hash the default.

The previous APIs will still be available as one_as_json and many_as_json, as well as a default_format :json setting to make the library work exactly the same as in version 1.

Performance πŸš€

Additional optimizations have made JSON rendering 20% to 40% faster than the previous version, so it's now 3x to 7x faster than active_model_serializers or blueprinter.

These are some preliminary results from running the benchmarks running on a MacBook Pro (16-inch, 2021, M1 Pro):

serializing options

        oj_serializers (v2):     9257.4 i/s
   oj_serializers (v2 hash):     9256.2 i/s
                      panko:     8902.1 i/s - 1.20x slower
        oj_serializers (v1):     6579.9 i/s - 1.40x slower
                       alba:     2330.7 i/s - 4.58x slower
                blueprinter:     1741.4 i/s - 5.32x slower
   active_model_serializers:     1308.1 i/s - 7.08x slower

serializing a model

   oj_serializers (v2 hash):    23819.3 i/s
        oj_serializers (v2):    22655.6 i/s - 1.05x slower
        oj_serializers (v1):    19870.2 i/s - 1.20x slower
                      panko:    13756.3 i/s - 1.72x slower
                blueprinter:     6635.5 i/s - 3.59x slower
   active_model_serializers:     6298.1 i/s - 3.78x slower
                       alba:     4197.5 i/s - 5.64x slower

serializing a collection

     oj_serializers as_hash:      243.6 i/s
             oj_serializers:      239.4 i/s - 1.02x slower
                      panko:      169.8 i/s - 1.43x slower
                blueprinter:       68.5 i/s - 3.56x slower
   active_model_serializers:       64.3 i/s - 3.79x slower
                       alba:       44.7 i/s - 5.44x slower

serializing a large collection

             oj_serializers:       23.1 i/s
     oj_serializers as_hash:       23.1 i/s - 1.00x slower
                      panko:       16.6 i/s - 1.40x slower
                blueprinter:        6.6 i/s - 3.53x slower
   active_model_serializers:        6.1 i/s - 3.79x slower
                       alba:        4.1 i/s - 5.64x slower

memory when serializing an object

                         oj:       5809 allocated
                    oj_hash:       6809 allocated - 1.17x more
                blueprinter:      17465 allocated - 3.01x more
                        ams:      26704 allocated - 4.60x more

memory when serializing a collection

                         oj:    5521290 allocated
                    oj_hash:    6777082 allocated - 1.23x more
                blueprinter:   17217298 allocated - 3.12x more
                        ams:   26672082 allocated - 4.83x more

@ElMassimo ElMassimo added the enhancement New feature or request label Mar 17, 2023
# options - list of external options to pass to the sub class (available in `item.options`)
#
# Returns a Hash, with the attributes specified in the serializer.
def one_as_hash(item, options = nil)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ElMassimo Can you make options default to {}? Our serializer specs were raising undefined method [] for nil errors since @object.options was nil.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds unusual, as options would never return nil, because it falls back to {}.

If you provide a small reproduction I can take a look.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, did not realize there was an options method directly in the serializer. Our serializer is calling @object.options directly, which is where the nil error comes from.

@usman-anwar-mindoula
Copy link

@ElMassimo Will this include changes to allow ControllerSerializer to use the new render method?

@ElMassimo
Copy link
Owner Author

ElMassimo commented Mar 18, 2023

Will this include changes to allow ControllerSerializer to use the new render method?

Correct, the controller will use whatever format is specified in the serializer. For example, if you use default_format :hash in your base serializer, then the controller will use one_as_hash and many_as_hash internally.

@usman-anwar-mindoula
Copy link

Also, can we add the benchmark results in the Readme?

@usman-anwar-mindoula
Copy link

Looks good!

@usman-anwar-mindoula
Copy link

Also, might be a good idea to add Panko to the benchmarks?

@usman-anwar-mindoula
Copy link

Can you add the belongs_to alias again?

@ElMassimo
Copy link
Owner Author

ElMassimo commented Mar 27, 2023

Added Alba and Panko to the benchmarks.

Following results from running on MacBook Pro (16-inch, 2021, M1 Pro):

serializing a model

          oj_serializers 22.667k (Β± 0.7%) i/s - 113.730k in 5.017690s
  oj_serializers as_hash 23.684k (Β± 0.4%) i/s - 120.615k in 5.092785s
                   panko 13.756k (Β± 0.4%) i/s -  69.050k in 5.019598s
             blueprinter  6.613k (Β± 0.3%) i/s -  33.660k in 5.090191s
active_model_serializers  6.199k (Β± 0.7%) i/s -  31.000k in 5.000678s
                    alba  4.197k (Β± 0.5%) i/s -  21.100k in 5.026957s

Comparison:

  oj_serializers as_hash:    23683.9 i/s
          oj_serializers:    22667.0 i/s - 1.04x slower
                   panko:    13756.3 i/s - 1.72x slower
             blueprinter:     6612.8 i/s - 3.58x slower
active_model_serializers:     6199.5 i/s - 3.82x slower
                    alba:     4197.5 i/s - 5.64x slower

serializing a collection

          oj_serializers    239.447  (Β± 0.4%) i/s -   1.219k in 5.090952s
  oj_serializers as_hash    243.253  (Β± 0.4%) i/s -   1.224k in 5.031894s
                   panko    169.772  (Β± 0.6%) i/s - 864.000  in 5.089264s
             blueprinter     68.523  (Β± 0.0%) i/s - 348.000  in 5.078683s
active_model_serializers     63.380  (Β± 1.6%) i/s - 318.000  in 5.018310s
                alba         44.679  (Β± 0.0%) i/s - 224.000  in 5.013989s

Comparison:

  oj_serializers as_hash:      243.3 i/s
          oj_serializers:      239.4 i/s - 1.02x slower
                   panko:      169.8 i/s - 1.43x slower
             blueprinter:       68.5 i/s - 3.55x slower
active_model_serializers:       63.4 i/s - 3.84x slower
                    alba:       44.7 i/s - 5.44x slower

serializing a large collection

          oj_serializers  23.279 i/s - 118.000 in 5.069285s
  oj_serializers as_hash  23.222 i/s - 118.000 in 5.081735s
                   panko  16.570 i/s -  83.000 in 5.009535s
             blueprinter   6.553 i/s -  33.000 in 5.036259s
active_model_serializers   6.108 i/s -  31.000 in 5.075949s
                    alba   4.127 i/s -  21.000 in 5.114307s

Comparison:
          oj_serializers: 23.3 i/s
  oj_serializers as_hash: 23.2 i/s - 1.00x slower
                   panko: 16.6 i/s - 1.40x slower
             blueprinter:  6.6 i/s - 3.55x slower
active_model_serializers:  6.1 i/s - 3.81x slower
                    alba:  4.1 i/s - 5.64x slower

I think the only reason Panko performs worse in this benchmark is that it's specialized for ActiveRecord, but the benchmark uses a normal object, and the fact that the CPU is an M1 Pro instead of an Intel chip.

Otherwise I'd expect Panko to be 2x faster in simple cases (although it requires a C extension, it's less compatible with AMS, has less features, and is not as flexible when combining with other libraries as it requires using Panko::Response).

@ElMassimo ElMassimo force-pushed the perf/render-in-hash branch 2 times, most recently from 2faeccd to 7b1bfd1 Compare March 27, 2023 14:38
@usman-anwar-mindoula
Copy link

When are you expecting to release to master?

@ElMassimo ElMassimo changed the title Oj Serializers 2.0 feat: Oj Serializers 2.0 Mar 27, 2023
@ElMassimo ElMassimo merged commit 5c38f8f into main Mar 27, 2023
8 checks passed
@ElMassimo ElMassimo deleted the perf/render-in-hash branch March 27, 2023 19:19
@usman-anwar-mindoula
Copy link

Nice work, congrats!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants