New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Idea: pluggable translation backends #500

Open
tomash opened this Issue Apr 18, 2016 · 12 comments

Comments

Projects
None yet
4 participants
@tomash
Contributor

tomash commented Apr 18, 2016

Two things have happened lately.

One: I've been browsing Globalize's issues and PRs, realizing many of them (if not most) are related to how we integrate with ActiveRecord associated translation tables. Which is not a surprise because Globalize is basically a nice DSL for managing these.
Two: Postgres, Rails community database of choice, is improving its complex-data storage mechanisms, mostly JSON ones.

I'm thinking about putting an abstraction layer between Globalize's logic (reading and saving translated attributes) and ActiveRecord-translation-associations, allowing the latter to be seamlessly replaced with a different approach for storing actual translations. Like, for example, a "transations" complex column (e.g. JSON type) storing translated attributes in the same row.

This will be not unlike a Repository pattern. And this way we could isolate and separate layers of Globalize's logic. Trasto shows a nice approach for starting with this:

    def define_localized_attribute(column)
      define_method(column) do
        read_localized_value(column)
      end

      define_method("#{column}=") do |value|
        write_localized_value(column, value)
      end
    end

@shioyama shioyama added the feature label Oct 1, 2016

@shioyama

This comment has been minimized.

Show comment
Hide comment
@shioyama

shioyama Oct 1, 2016

Contributor

@tomash I've been thinking about the same thing lately, also browsing how backends are implemented in svenfuchs/i18n.

However, to be perfectly honest, I think it might be more productive to start anew on something like this (i.e. a new gem). This is what I'm thinking of doing when I have some free time.

My thought is to have three backends to start:

  1. Using columns on model table (like traco et al). For most applications, this is probably sufficient and is very transparent.
  2. Using the hstore feature of postgres, similar to multilang-hstore, hstore_translate, trasto, etc.
  3. Using table translations, similar to globalize, but different.

For 3., I don't want to actually reproduce how globalize does it with one translation table per model, because I simply don't want to deal with the pain of migrations (the one part of globalize's code that I almost never look at). Instead, I'm thinking of implementing it using one shared table (a bit like i18n-active_record does it) and a polymorphic relationship. This should be possible since joins would be possible from model -> translations, just not the other way around, which anyway you never really need.

So that would be the idea: three interchangeable AR-based backends, and then some common DSL which then could be built upon to do more interesting stuff than we do in Globalize, since here we're always stuck debugging deep AR issues 😄

How does that sound to you?

Contributor

shioyama commented Oct 1, 2016

@tomash I've been thinking about the same thing lately, also browsing how backends are implemented in svenfuchs/i18n.

However, to be perfectly honest, I think it might be more productive to start anew on something like this (i.e. a new gem). This is what I'm thinking of doing when I have some free time.

My thought is to have three backends to start:

  1. Using columns on model table (like traco et al). For most applications, this is probably sufficient and is very transparent.
  2. Using the hstore feature of postgres, similar to multilang-hstore, hstore_translate, trasto, etc.
  3. Using table translations, similar to globalize, but different.

For 3., I don't want to actually reproduce how globalize does it with one translation table per model, because I simply don't want to deal with the pain of migrations (the one part of globalize's code that I almost never look at). Instead, I'm thinking of implementing it using one shared table (a bit like i18n-active_record does it) and a polymorphic relationship. This should be possible since joins would be possible from model -> translations, just not the other way around, which anyway you never really need.

So that would be the idea: three interchangeable AR-based backends, and then some common DSL which then could be built upon to do more interesting stuff than we do in Globalize, since here we're always stuck debugging deep AR issues 😄

How does that sound to you?

@shioyama

This comment has been minimized.

Show comment
Hide comment
@shioyama

shioyama Mar 2, 2017

Contributor

So @tomash I've done this!

I created a gem called Mobility:

And I posted a blog post explaining some background and what I'm trying to do with the gem.

Mobility implements every translation strategy I've seen so far, including model translation tables like Globalize, but also translatable columns, serialized columns, as well as Jsonb/Hstore and the solution I described in the comment above using a polymorphic relation with a shared translations table. As a bonus, it also implements them for Sequel, and has no AR dependency so the core part of the gem is actually just plain Ruby.

The API is very simple, so you can just specify which backend to use when you call translates:

class Post < ActiveRecord::Base
  include Mobility
  translates :title, backend: :key_value
  translates :content, backend: :table
end

This would create a translated attribute title which uses the "key-value" backend (the one I described in my last comment), and another translated attribute content which uses the "table" backend, which is basically equivalent to Globalize. The accessors, setup code, etc are executed in total isolation, so you can define attributes with different backends on the same class.

It also supports querying like Globalize, but for all strategies, so you can e.g. query on Jsonb columns if you use the :jsonb backend, etc. and it will convert to the appropriate SQL.

There's more details in the links above, and also in the API documentation, which is pretty thorough, please have a look when you have a chance. I'd be really interested to hear your thoughts.

cc @globalize/collaborators

Contributor

shioyama commented Mar 2, 2017

So @tomash I've done this!

I created a gem called Mobility:

And I posted a blog post explaining some background and what I'm trying to do with the gem.

Mobility implements every translation strategy I've seen so far, including model translation tables like Globalize, but also translatable columns, serialized columns, as well as Jsonb/Hstore and the solution I described in the comment above using a polymorphic relation with a shared translations table. As a bonus, it also implements them for Sequel, and has no AR dependency so the core part of the gem is actually just plain Ruby.

The API is very simple, so you can just specify which backend to use when you call translates:

class Post < ActiveRecord::Base
  include Mobility
  translates :title, backend: :key_value
  translates :content, backend: :table
end

This would create a translated attribute title which uses the "key-value" backend (the one I described in my last comment), and another translated attribute content which uses the "table" backend, which is basically equivalent to Globalize. The accessors, setup code, etc are executed in total isolation, so you can define attributes with different backends on the same class.

It also supports querying like Globalize, but for all strategies, so you can e.g. query on Jsonb columns if you use the :jsonb backend, etc. and it will convert to the appropriate SQL.

There's more details in the links above, and also in the API documentation, which is pretty thorough, please have a look when you have a chance. I'd be really interested to hear your thoughts.

cc @globalize/collaborators

@parndt

This comment has been minimized.

Show comment
Hide comment
@parndt

parndt Mar 2, 2017

Member

@shioyama wow! I'm so happy to see that you've done this. I've noticed recently how many gems are coupled to Rails and can't be used with exciting techs like dry-rb or hanami.

I have one question - can globalize depend on mobility (and thus enable us to delete a lot of our code)?

Member

parndt commented Mar 2, 2017

@shioyama wow! I'm so happy to see that you've done this. I've noticed recently how many gems are coupled to Rails and can't be used with exciting techs like dry-rb or hanami.

I have one question - can globalize depend on mobility (and thus enable us to delete a lot of our code)?

@shioyama

This comment has been minimized.

Show comment
Hide comment
@shioyama

shioyama Mar 2, 2017

Contributor

@parndt Yes actually, that should be possible. Mobility leaves out many of the instance/class methods that Globalize has (or namespaces them to e.g. mobility_translates_for, etc.) but I was thinking of having a gem which would try to reproduce Globalize behavior. But having Globalize itself depend on Mobility would certainly also be an option, and should make a lot of code redundant.

Contributor

shioyama commented Mar 2, 2017

@parndt Yes actually, that should be possible. Mobility leaves out many of the instance/class methods that Globalize has (or namespaces them to e.g. mobility_translates_for, etc.) but I was thinking of having a gem which would try to reproduce Globalize behavior. But having Globalize itself depend on Mobility would certainly also be an option, and should make a lot of code redundant.

@parndt

This comment has been minimized.

Show comment
Hide comment
@parndt

parndt Mar 2, 2017

Member

@shioyama I'm all for that.. then globalize can act as the activerecord gem for mobility?

Member

parndt commented Mar 2, 2017

@shioyama I'm all for that.. then globalize can act as the activerecord gem for mobility?

@parndt

This comment has been minimized.

Show comment
Hide comment
@parndt

parndt Mar 2, 2017

Member

Since becoming far more aware of http://dry-rb.org I am firmly of the opinion that we should reuse functionality amongst different gems as much as possible.

Member

parndt commented Mar 2, 2017

Since becoming far more aware of http://dry-rb.org I am firmly of the opinion that we should reuse functionality amongst different gems as much as possible.

@shioyama

This comment has been minimized.

Show comment
Hide comment
@shioyama

shioyama Mar 2, 2017

Contributor

@parndt Well, Mobility already does ActiveRecord, it would more be that Globalize would act as the Globalize-compatible gem (adding methods that Mobility doesn't have, etc.)

I already basically implemented the Globalize "pattern" (model translation tables) in the "table" backend which you can see here and here (and a little bit in the parent Mobility::Backend::Table class here). It's implemented a bit different from Globalize, but it's basically the same kind of thing.

Contributor

shioyama commented Mar 2, 2017

@parndt Well, Mobility already does ActiveRecord, it would more be that Globalize would act as the Globalize-compatible gem (adding methods that Mobility doesn't have, etc.)

I already basically implemented the Globalize "pattern" (model translation tables) in the "table" backend which you can see here and here (and a little bit in the parent Mobility::Backend::Table class here). It's implemented a bit different from Globalize, but it's basically the same kind of thing.

@shioyama

This comment has been minimized.

Show comment
Hide comment
@shioyama

shioyama Mar 2, 2017

Contributor

Also, I should clarify: Mobility doesn't depend on AR, but it does try to load it (same for Sequel). There is no such thing as an "optional" dependency in ruby so that's basically the only way to do it. So it has lots of AR specific stuff (namespaced under Mobility::ActiveRecord etc) but as long as you don't load it, you don't need to depend on AR/activesupport, etc.

Contributor

shioyama commented Mar 2, 2017

Also, I should clarify: Mobility doesn't depend on AR, but it does try to load it (same for Sequel). There is no such thing as an "optional" dependency in ruby so that's basically the only way to do it. So it has lots of AR specific stuff (namespaced under Mobility::ActiveRecord etc) but as long as you don't load it, you don't need to depend on AR/activesupport, etc.

@parndt

This comment has been minimized.

Show comment
Hide comment
@parndt

parndt Mar 3, 2017

Member

Fair enough. I'll have a think about it, but please alert me if I forget to respond 😄

Member

parndt commented Mar 3, 2017

Fair enough. I'll have a think about it, but please alert me if I forget to respond 😄

@kevin-jj

This comment has been minimized.

Show comment
Hide comment
@kevin-jj

kevin-jj Mar 23, 2018

Contributor

Is there a chance to combine globalize with mibility in the furture? I prefer to use key-value or hstore as a storage strategy. @parndt

Contributor

kevin-jj commented Mar 23, 2018

Is there a chance to combine globalize with mibility in the furture? I prefer to use key-value or hstore as a storage strategy. @parndt

@parndt

This comment has been minimized.

Show comment
Hide comment
@parndt

parndt Mar 23, 2018

Member

I think @shioyama is best placed to answer this question. 😄

Member

parndt commented Mar 23, 2018

I think @shioyama is best placed to answer this question. 😄

@shioyama

This comment has been minimized.

Show comment
Hide comment
@shioyama

shioyama Mar 23, 2018

Contributor

No sorry, not possible.

Contributor

shioyama commented Mar 23, 2018

No sorry, not possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment