'rake tolk:sync' seems slow #23

Open
MarkDBlackwell opened this Issue Jan 9, 2013 · 1 comment

Projects

None yet

2 participants

@MarkDBlackwell

The 'rake tolk:sync' command, which I do frequently, seems slow, even after loading Rails.

Could it be that tolk (so far) has used a quick-and-dirty algorithm for this syncing? Now that tolk is working well, could there be easy, low-hanging fruit to speed up the syncing?

For subtrees of YAML language keys, I guess it could:

  • Select bigger batches of rows (rowsets?) from the database;
  • Update rows in bigger batches, letting the database software return an error if someone else has changed a row, which usually wouldn't happen on development boxes.

Perhaps:

  1. From the database, request (just) the ID's for all the main language-file YAML keys, in one big response, to find those (that are) gone from YAML;

  2. Request from the database (just) the main language strings for all the ID's (that should still exist) in one big response, to see what's changed;

  3. Then, for each language's database table:

    a. Delete the unwanted rows in one big request;
    b. For the main language keys whose strings have changed:

    1. Update those rows of the main language's database in one big request;

    2. Then, for each other language:

      a. Blank (by updating) those rows in that language's database in one big request.

I'm using tolk 1.3.4 with Rails 3.2.9 on Ruby 1.9.3, and I have, in config/locales, two languages of YAML totaling 52,691 bytes.

@thomasdarde
Member

Hi Mark,

I'm really sorry to answer as late. Tolk has been taking me quite a lot of time recently, because of the YAML security issues. I think we are ok know.

This strategy seems ok to me, but I won't be able to implement before months, feel free to send a PR !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment