save field results in to_s version all the time

[tillcarlos]

Hey guys

Awesome that you put this together! It comes at the right time :)

When I edit the form in correct json (text/tree), and then save the field, it always results in the to string version "{\"key\": ...

This is my schema:

    create_table :grammatical_forms do |t|
      t.json :translations
      t.string :identifier, index: true
      t.timestamps
    end

I use it in the dashboard as described

  ATTRIBUTE_TYPES = {
    meanings: Field::HasMany,
    id: Field::Number,
    identifier: Field::String.with_options(searchable: false),
    translations: Field::JSONB,
    created_at: Field::DateTime,
    updated_at: Field::DateTime,
  }.freeze

Any ideas right away? I haven't debugged yet, but I might do in the next days.

Thanks again

[tillcarlos]

I guess that's what you intended the transform option for? But how would you pass JSON.parse in there? Apart from that, it feels wrong. I'm doing something differently than your requirements, I guess.

[volkov-sergey]

Hello, @tillcarlos. Thanks for the feedback. I got what problem is, let me check, I'll try to fix in a couple of hours.

[volkov-sergey]

@tillcarlos could I ask you to describe more, what you trying to solve and what the expected behavior? Thanks.

Note: we are using transform option to transform data before displaying in a form / show. We have additional serialization with Dry::Struct, so we need to do to_h before sending object to form / show.

[tillcarlos]

Easy: I have a json (not jsonb) field in my postgres, and I want to save a ruby hash in there. I don't have any additional serialization code on my model.

With administrate I want to edit that hash and save it. Not sure what other hints you might need?

Do you maybe have an example repo somewhere for me to compare what I might be doing wrong?

[tillcarlos]

Oh, and I tried it with a jsonb field - no difference (as expected)

[volkov-sergey]

@tillcarlos as a temporary workaround, could you try to add this to your model? Thanks.

def translations=(value)
  self[:translations] = value.is_a(String) ? JSON.parse(value) : value
end

[tillcarlos]

Yes, that works just as intended!

thanks!

[volkov-sergey]

@tillcarlos ok, in this case, I'll close the issue and think about how it should work and update the gem if found a good solution.

[sedubois]

@volkov-sergey I encountered the same issue, the workaround also worked however it's not very practical to define this serialization method for every JSON field. Would you have an idea for a "cleaner" fix?

[volkov-sergey]

@sedubois you can try to DRY

create the initializer with this code:

module DataSerializer

  extend ActiveSupport::Concern

  class_methods do

    def data_serialize(fields)

      fields.each do |field|

        define_method("#{field}=") do |value|
          self[field] = value.is_a?(String) ? JSON.parse(value) : value
        end

      end

    end

  end

end

ActiveSupport.on_load(:active_record) do
  ActiveRecord::Base.send(:include, DataSerializer)
end

and then at your model:

data_serialize %i[field_a field_b field_c]

I didn't check the code, please try it and update this issue if it was helpful, so others can use it also. Thanks.

[sedubois]

OK the workaround @volkov-sergey suggested works:

# config/initializers/administrate.rb

# https://github.com/codica2/administrate-field-jsonb/issues/1#issuecomment-545825725
module JsonSerializer
  extend ActiveSupport::Concern
  class_methods do
    def serialize_json(fields)
      fields.each do |field|
        define_method("#{field}=") do |value|
          self[field] = value.is_a?(String) ? JSON.parse(value) : value
        end
      end
    end
  end
end

ActiveSupport.on_load(:active_record) do
  ActiveRecord::Base.send(:include, JsonSerializer)
end

# app/models/my_model.rb
class MyModel < ApplicationRecord
  serialize_json %i[my_prop]
  ...
end

# app/dashboards/my_model_dashboard.rb
require "administrate/base_dashboard"
class MyModelDashboard < Administrate::BaseDashboard
  ATTRIBUTE_TYPES = {
    ...
    my_prop: Field::JSONB,
    ...
  }.freeze
  ...
end

[volkov-sergey]

@sedubois thanks for update mate, I'll add this initializer in the next gem versions or add this code to README

[sedubois]

@volkov-sergey naively I would think that there should be a way to serialize things properly without this workaround:

• Rails/ActiveRecord manages to do it automatically
• the serialize_json method above needs to be added to all the models in the entire app which have JSON fields, even though this issue only concerns admin dashboards (i.e. only admin code should be impacted)
• the syntax is "DSL-y" which goes contrary to the goals of Administrate

As a user of this library I would expect to just specify Field::JSONB, that's normally all the information needed.

So maybe a bit more reflection could be useful, before committing such a new (hopefully unnecessary) API to the lib.

[volkov-sergey]

@sedubois as far as I remember Rails 4 and Rails 5 have different behavior for jsonb field.

JSONeditor library (that we use) sends String by default and it's a valid JSON primitive.

At the Rails 4.2.2:

item = OrderItem.new(options: "{\"key\":\"value\"}")
 => #<OrderItem id: nil, options: {"key"=>"value"}>

item.options
 => {"key"=>"value"} 

item.options.is_a?(String)
 => false 

item.options.is_a?(Hash)
 => true

Rails 5.0.2:

item = OrderItem.new(options: "{\"key\":\"value\"}")
 => #<OrderItem id: nil, options: "{\"key\":\"value\"}">

item.options
 => "{\"key\":\"value\"}"

item.options.is_a?(String)
 => true 

item.options.is_a?(Hash)
 => false

Since the rails team fixed this bug at Rails 5+, serialization the string before saving to DB should be managed on your side. What to use Attributes API either overwrite setter - your choice.

I agree that adding this initializer is overhead, so I will not do this. I don't know what the best solution for this now. What do you think? Thanks.

Related issues:

• Postgres JSONB column is returned as string instead of hash in Rails 5 RC2 rails/rails#25594
• Rails 4.2.2 vs Rails 5.0.2 JSON Storage with PG rails/rails#28292

[sedubois]

@volkov-sergey What I see with Rails 5.2.3 is:

# db/schema.rb
  create_table "post_translations", force: :cascade do |t|
    ...
    t.jsonb "tags", default: [], null: false
  end

# Rails console

translation = Post::Translation.last

translation.update(tags: {key: "value"})
translation.tags
=> {"key"=>"value"}
translation.tags.class
=> Hash
translation.update(tags: '{"key":"value"}')
translation.tags
=> {"key"=>"value"}
translation.tags.class
=> Hash

translation.update(tags: ["value"])
translation.tags
=> ["value"]
translation.tags.class
=> Array
translation.update(tags: '["value"]')
translation.tags
=> ["value"]
translation.tags.class
=> Array

translation.update(tags: '"value"')
translation.tags
=> "value"
translation.tags.class
=> String

So ActiveRecord works like this:

• when assigning, the parameter can be a hash, an array or a string, either in Ruby syntax or string form
• the value stored in DB is a ::jsonb and shows up in DB inspector as {"key": "value"}
• when getting the value, it's just a Ruby object

This is how administrate-field-jsonb should also work, however it's not respecting this contract. For some reason when I enter {"key":"value"} in the JSON editor, what gets stored in DB is "{\"key\":\"value\"}" with escape characters.

When submitting this form:

it's sending the value with escaped characters instead of JSON:

Processing by Admin::Post::TranslationsController#update as HTML
  Parameters: {..., "post_translation"=>{"tags"=>"{\"key\":\"value\"}", ...

Something wrong in the Javascript code? Where is the code handling the form upload?

[sedubois]

https://github.com/XPBytes/administrate-serialized_fields by @SleeplessByte offers another semi-working implementation for the serialization workaround. This works:

# Gemfile
gem 'administrate'
gem 'administrate-serialized_fields'
gem 'administrate-field-jsonb'

# db/schema.rb
  create_table "administrators", id: :serial, force: :cascade do |t|
    t.jsonb "roles", default: []
    ...
  end

# app/controllers/admin/application_controller.rb
module Admin
  class ApplicationController < Administrate::ApplicationController
    include Administrate::SerializedFields

    protected

    def resource_params
      permitted = params.require(resource_class.model_name.param_key)
                    .permit(dashboard.permitted_attributes)
                    .to_h
      permitted.each_with_object(permitted) do |(k, v), result|
        result[k] = read_param(k, v)
      end
    end

    def read_param(_, data)
      if data.is_a?(ActionController::Parameters) && data[:type]
        return read_param_value(data)
      end
      data
    end
end

# app/dashboards/administrator_dashboard.rb
class AdministratorDashboard < Administrate::BaseDashboard
  ATTRIBUTE_TYPES = {
    ...
    roles: Field::JSONB,
  }.freeze
end

# app/controllers/admin/administrators_controller.rb
module Admin
  class AdministratorsController < Admin::ApplicationController
    deserialize_json_fields :roles
    ...
  end
end

However it does not seem to work for nested fields, so it can't completely replace the proposed workaround of this thread (see this issue).

But anyway I'm hoping that a cleaner fix could exist, by ensuring that the form receives actual JSON data instead of the serialized form to begin with (see previous comment).

[SleeplessByte]

But anyway I'm hoping that a cleaner fix could exist, by ensuring that the form receives actual JSON data instead of the serialized form to begin with (see previous comment).

Perhaps I can help with that too, but the problem is that "serialized" json is valid JSON... because it's a string, which is a valid JSON value (not valid JSON document).

[sedubois]

My point is that in the payload received by the endpoint, the JSON data in the Field::JSONB field isn't serialized the same way as the rest of the payload (payload which itself as a whole is a JSON structure):

Processing by Admin::PostsController#update as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"WWW+XXX+YYY/ZZZ==", "post"=>{"translations_attributes"=>{"0"=>{"locale"=>"en", "title"=>"My Title", "tags"=>"[\"my tag\"]", "content"=>"My first markdown paragraph ...", "published"=>"1", "published_at"=>"2019-10-25", ... }

This is how the payload is when entering a plain string in the JSONeditor (which must be wrapped with quotes for the editor to consider the input valid):

Processing by Admin::PostsController#update as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"WWW+XXX+YYY/ZZZ==", "post"=>{"translations_attributes"=>{"0"=>{"locale"=>"en", "title"=>"My Title", "tags"=>"\"plain string\"", "content"=>"My first markdown paragraph ...", "published"=>"1", "published_at"=>"2019-10-25", ... }

AFAIK the proper fix is to ensure that actual JSON is passed in this field value, not serialized JSON:

Processing by Admin::PostsController#update as HTML
  Parameters: { ... "tags"=>"["my tag"]", ... }

If someone wants to send an actual string, it can be passed as a "plain string" instead of a "\"quoted string\"":

Processing by Admin::PostsController#update as HTML
  Parameters: { ... "tags"=>"plain string", ... }

[garrettd714]

Sounds like I was/am having a similar issue (Rails 6.0.3). At the end of the day, my value is a JSON parsable String.
e.g. "{\"key\":\"value\"}" resulting in:

Here's my solution:

1. Create app/fields/jsonp_field.rb

# frozen_string_literal: true

require 'administrate/field/jsonb'

class JsonpField < Administrate::Field::JSONB
  def transform
    @data = JSON.parse(data) if data.is_a?(String)

    super
  end
end

2. Change Dashboard Attribute

ATTRIBUTE_TYPES = {
    ...
    details: JsonpField,
    ...
}.freeze

Result

Note:
Have not tested out with other transformations or edit/create, as I just need a clean show

[SleeplessByte]

I've been using administrate-serialized_fields in conjunction with
administrate-base_controller for a few years. Your solution only solves the case of the data being a JSON parsable string, but doesn't actually store the data as JSON.

Don't know if it's still required in Administrate 0.16, but it does still work with 0.16.

[garrettd714]

@SleeplessByte

Your solution only solves the case of the data being a JSON parsable string, but doesn't actually store the data as JSON.

As I stated, I'm not trying to solve any larger problems. I pretty much use as view-only. I'm just sharing my solution for this particular use case which I found myself in (on a single attribute, the rest of my jsonb columns work with Field::JSONB, out of the box (for viewing)). Cheers

[benoror]

@garrettd714 FWIW adding built-in :parse-json will also help on saving JSONB correctly with your solution:

ATTRIBUTE_TYPES = {
    ...
-    details: JsonpField,
+    details: JsonpField.with_options(transform: [:parse_json]),
    ...
}.freeze

---

⚠️ Update: It's still persisted as a String in the DB tho :(

---

⚠️ Update 2: Dismiss this. Following this recipe makes this redundant and it just "works": https://github.com/codica2/administrate-field-jsonb/blob/5cbe658b271e3666df984bbcabf39e8345270185/README.md#recipes