Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User Migration Meta bug #407

Closed
jrconlin opened this issue Jan 9, 2020 · 2 comments
Closed

User Migration Meta bug #407

jrconlin opened this issue Jan 9, 2020 · 2 comments
Assignees
Labels
5 Estimate - l - Moderately complex, will require some effort but clearly defined. epic A collection of smaller tasks

Comments

@jrconlin
Copy link
Member

jrconlin commented Jan 9, 2020

A Meta Bug to cross track all the User Migration issues and PRs.

mozilla-services/tokenserver#159
#401
mozilla-services/server-syncstorage#125
mozilla-services/services-engineering#31
mozilla-services/services-engineering#6

@jrconlin
Copy link
Member Author

jrconlin commented Feb 3, 2020

Additional info from sync meetup.

  • Need to add a second table to syncstorage containing the fxa_uids of migrating users. If id is present, kick back to tokenserver to eventually route to spanner
  • alter the migration table to indicate state of the user migration
  • check that user node hasn't changed since start and finish of migration.
  • ensure failed user migration ids are logged
  • verify that there are no additional issues around mobile clients.
  • generate fake m/g record for migrated user (before "done") that has new SyncIDs.

No longer using the external table. Instead dumping tokenserver data to a local csv.

@jrconlin jrconlin added the 5 Estimate - l - Moderately complex, will require some effort but clearly defined. label Mar 16, 2020
jrconlin added a commit that referenced this issue Apr 2, 2020
The `--user_percent` option will divvy up the users into blocks and
move the specified block. It takes an option formatted as
"block#:percentage". Block numbers are 1 based. For example,
--user_percent=2:33 will divide the total distinct users into
non-overlapping blocks of approximately 33%, and then move the second
block (e.g. the 33-65th users in the list). Extra users that may not be
evenly divided into percentage blocks will be appended to the last
block. (e.g. for `--user_percent=3:33`, users 66-99 would be copied
over, a total of 34 users)

Issue #407
jrconlin added a commit that referenced this issue Apr 21, 2020
jrconlin added a commit that referenced this issue Apr 29, 2020
* bug: Fix typos in tick, string replacements
* f multi-tread gen_bso_users
* added `--start_bso`, `--end_bso` to `gen_bso_users.py`
* added `bso_num` arg (same as `--start_bso=# --end_bso=#`) to `migrate_node.py`
* `gen_bso_users.py` takes same `bso_users_file` template as `migrate_node.py`
* f remove default value for BSO_Users.run bso_num
* f fix lock issue in gen_bso_users, trap for `` states in gen_fxa_users
* f make threading optional.
 There's a locking issue that appears to be inside of the mysql.
 Turning threading off for now (can be run in parallel)
* f fix tick, threading flag
* f rename confusing args in gen_bso and gen_fxa
 gen_bso_users:
  `--bso_users_file` => `--output_file`
 gen_fxa_users:
  `--fxa_file` => `--users_file`
  `--fxa_users_file` => `--output_file`
* f more tick fixes
* f don't use threading on Report if threading isn't available.
* f make `--bso_users_file` / `--fxa_users_file` consistent
* `--bso_user_file` is now `--bso_users_file`

Issue #407
@tublitzed
Copy link
Contributor

Related bugzilla link for migrating the first node. Going to close this out as the migration script portion of this work is done. Will open + link any further refinements needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 Estimate - l - Moderately complex, will require some effort but clearly defined. epic A collection of smaller tasks
Projects
None yet
Development

No branches or pull requests

2 participants