Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

User Migration Meta bug #407

Closed
jrconlin opened this issue Jan 9, 2020 · 2 comments
Closed

User Migration Meta bug #407

jrconlin opened this issue Jan 9, 2020 · 2 comments
Assignees
@jrconlin
Labels
5 Estimate - l - Moderately complex, will require some effort but clearly defined. epic A collection of smaller tasks
Projects
Services Engineering

Comments

@jrconlin
Copy link
Member

jrconlin commented Jan 9, 2020 •
edited by tublitzed

A Meta Bug to cross track all the User Migration issues and PRs.

mozilla-services/tokenserver#159
#401
mozilla-services/server-syncstorage#125
mozilla-services/services-engineering#31
mozilla-services/services-engineering#6
@jrconlin jrconlin self-assigned this Jan 9, 2020
@jrconlin jrconlin added this to Backlog: Misc in Services Engineering via automation Jan 9, 2020
@tublitzed tublitzed moved this from Backlog: Misc to Prioritized in Services Engineering Jan 10, 2020
@pjenvey pjenvey mentioned this issue Jan 15, 2020
Closed
2 tasks
@tublitzed tublitzed added the epic A collection of smaller tasks label Jan 15, 2020
@jrconlin
Copy link
Member Author

jrconlin commented Feb 3, 2020 •
edited

Additional info from sync meetup.

  • Need to add a second table to syncstorage containing the fxa_uids of migrating users. If id is present, kick back to tokenserver to eventually route to spanner
  • alter the migration table to indicate state of the user migration
  • check that user node hasn't changed since start and finish of migration.
  • ensure failed user migration ids are logged
  • verify that there are no additional issues around mobile clients.
  • generate fake m/g record for migrated user (before "done") that has new SyncIDs.

No longer using the external table. Instead dumping tokenserver data to a local csv.

@jrconlin jrconlin mentioned this issue Feb 26, 2020
Merged
@jrconlin jrconlin moved this from Prioritized to In Progress in Services Engineering Mar 16, 2020
@jrconlin jrconlin added the 5 Estimate - l - Moderately complex, will require some effort but clearly defined. label Mar 16, 2020
@jrconlin jrconlin mentioned this issue Mar 16, 2020
Merged
@jrconlin jrconlin mentioned this issue Apr 1, 2020
Merged
jrconlin added a commit that referenced this issue Apr 2, 2020
@jrconlin
feat: add --user_percent option
08a646a 
The `--user_percent` option will divvy up the users into blocks and
move the specified block. It takes an option formatted as
"block#:percentage". Block numbers are 1 based. For example,
--user_percent=2:33 will divide the total distinct users into
non-overlapping blocks of approximately 33%, and then move the second
block (e.g. the 33-65th users in the list). Extra users that may not be
evenly divided into percentage blocks will be appended to the last
block. (e.g. for `--user_percent=3:33`, users 66-99 would be copied
over, a total of 34 users)

Issue #407
jrconlin added a commit that referenced this issue Apr 21, 2020
@jrconlin
bug: Fix typos in tick, string replacements
9a5618c 
Issue #407
jrconlin added a commit that referenced this issue Apr 29, 2020
@jrconlin
User migration5 (#601)
8aaa449 
* bug: Fix typos in tick, string replacements
* f multi-tread gen_bso_users
* added `--start_bso`, `--end_bso` to `gen_bso_users.py`
* added `bso_num` arg (same as `--start_bso=# --end_bso=#`) to `migrate_node.py`
* `gen_bso_users.py` takes same `bso_users_file` template as `migrate_node.py`
* f remove default value for BSO_Users.run bso_num
* f fix lock issue in gen_bso_users, trap for `` states in gen_fxa_users
* f make threading optional.
 There's a locking issue that appears to be inside of the mysql.
 Turning threading off for now (can be run in parallel)
* f fix tick, threading flag
* f rename confusing args in gen_bso and gen_fxa
 gen_bso_users:
  `--bso_users_file` => `--output_file`
 gen_fxa_users:
  `--fxa_file` => `--users_file`
  `--fxa_users_file` => `--output_file`
* f more tick fixes
* f don't use threading on Report if threading isn't available.
* f make `--bso_users_file` / `--fxa_users_file` consistent
* `--bso_user_file` is now `--bso_users_file`

Issue #407
@tublitzed tublitzed moved this from In Progress to In Review in Services Engineering May 4, 2020
@tublitzed
Copy link
Contributor

Related bugzilla link for migrating the first node. Going to close this out as the migration script portion of this work is done. Will open + link any further refinements needed.

Services Engineering automation moved this from In Review to Done May 20, 2020
@tublitzed tublitzed moved this from Done to Archived in Services Engineering Jun 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jrconlin jrconlin

Labels
5 Estimate - l - Moderately complex, will require some effort but clearly defined. epic A collection of smaller tasks
Projects
Services Engineering
  
Archived
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jrconlin @tublitzed