UI Failed to load from CDN #19
Comments
|
@ibuildthecloud does the backend cache the static/index.html? It seems after a failure like this that it keeps serving up failure instead of getting the content from CDN until restart. |
|
@vincent99 It does cache the content but its supposed to only cache if on success and if it has never loaded the UI it will continue to retry @cloudnautique Did you see this error once on startup or repeatedly every time you hit the page. You should see that on each request it will try to load again. I'll test with a bad URL and see what behavior I see locally. |
|
@ibuildthecloud the error occurs with every page load. |
|
It appears that you had a networking issue. I couldn't reproduce this locally. I have tested various failure scenarios I and I believe they are all covered. Please reopen if you encounter this again. |
JeffersonBledsoe
pushed a commit
to JeffersonBledsoe/rancher-cli
that referenced
this issue
Apr 28, 2022
Enable using flags for default actions
anupama2501
pushed a commit
to anupama2501/rancher
that referenced
this issue
May 18, 2023
# This is the 1st commit message: restricted admin additional tests # This is the commit message rancher#2: Stop hosted clusters from deleting before tests run # This is the commit message rancher#3: User GUID for the PrincipalID for Active Directory # This is the commit message rancher#4: Migrate Active Directory users to use objectGUID as the principalId # This is the commit message rancher#5: Add defaults package to extensions. Update v2 tests and extensions to use the timeout value from the new defaults package # This is the commit message rancher#6: Fix atoi call with empty string in azure auth provider # This is the commit message rancher#7: Bump csp adapter to 2.0.2-rc2 # This is the commit message rancher#8: Add retries to kubeapi requests in integration tests The downstream cluster sometimes has random disconnects that interrupt the test setup, try to make the suite more resilient when connecting to the downstream. # This is the commit message rancher#9: Add retries to K3D cluster setup About 1/10 times the integrationsetup script fails with message Failed Cluster Preparation: Failed Network Preparation: failed to create cluster network: docker failed to create new network 'k3d-auto-k3d-cluster-xtrfk': Error response from daemon: Failed to program FILTER chain: iptables failed: iptables --wait -I FORWARD -o br-c722011a5900 -j DOCKER: iptables: Resource temporarily unavailable.\n (exit status 4) Since the failure is noted to be temporary, add retries to try to avoid having ithe whole job fail. # This is the commit message rancher#10: Add timeout to integration import cluster The integration test setup has a wait both in the ImportCluster routine and after it. If a networking error on the test node causes the import job to never run, the pipeline waits undefinitely until the drone timeout, and it is never clear why the step hung. This change adds a timeout to the internal import cluster step so that the wait is shorter and the problem is more clearly logged. Also add more logging so it is clear which step is getting stuck. # This is the commit message rancher#11: Use a random ID for integration test labels 5f78652 introduced labels to resources in the steveapi integration tests so that assertions could exclude non-test-generated resources. However, since the label is deterministic, if the tests are run multiiple times on the same cluster and if the resources weren't properly cleaned up after the last test run due to an unexpected failure, the subsequent test runs would include the old resources in their results. To prevent this, use a unique ID for the resource label in the steveapi integration tests. # This is the commit message rancher#12: Update steve for new project filtering feature # This is the commit message rancher#13: Add steve API tests for filtering by projects Add integration tests for the new `projectsornamespaces` query parameter in steve. # This is the commit message rancher#14: Improving default for PSP options Improves the default for global.cattle.psp.enabled to not require manual user override on k8s 1.25 # This is the commit message rancher#15: Rebasing helm unittests to use upstrem plugin Previously, chart unit tests used a fork of helm-unittests to run. This commit commit changes the unit tests to use the upstream plugin instead, which requires small changes to the tests and omitting the tests phase on the s390x architecture. # This is the commit message rancher#16: Tests for Improving default for PSP options # This is the commit message rancher#17: Bump Rancher-Webhook to v0.3.5-rc5 # This is the commit message rancher#18: Create a CRTB for a restricted admin when a GRB gets created for it # This is the commit message rancher#19: Enqueue restricted admin's GRB if CRTB is deleted from remote cluster # This is the commit message rancher#20: Stop creating unnecessary RBAC resources for restricted admins # This is the commit message rancher#21: Updated GRB handler for resetricted-admin. # This is the commit message rancher#22: Restructure restricted-admin rule reconciliation. related-resource logic for re-enqueuing GRBs was moved from `pkg/controllers/management/authprovisioningv2` to `pkg/controllers/managementuser/rbac` `pkg/controllers/management/restrictedadminrbac/register.go` no longer creates cluster and project handlers for giving the restricted-admin rules in the local cluster namespace. This also caused the removal of unused member variables from the handler GRB handler code now ensures a CRTB for the GRB subject to the cluster-owner roleTemplate if the GRB is for a restricted-admin. If not then the handler will bind the GRB subject to the cluster-admin role if the GRB is an admin GRB. This change also caused the removal of unused member variables from the handler. # This is the commit message rancher#23: Moves restricted-admin CRTB to management context. Restricted admin now gets their CRTB for cluster-owner to downstream cluster through controllers in the management context. # This is the commit message rancher#24: Adds unit tests for restrictedadminrbac controller # This is the commit message rancher#25: Fixes admin sync error and adds unit tests. # This is the commit message rancher#26: [CAPR] Enhance new provisioning tests for etcd snapshot creation/restore, encryption key rotation, and certificate rotation (rancher#41459) * Add new operations tests and refactor v2prov test framework, refactor test frameworks to prevent repeating the same code in two places, add more etcd snapshot related tests and additional conditional checks around secret conflicts, selectively check cluster readiness when scaling, check objectstore health to prevent race condition on startup and ensure snapshot file is not failed * Add unit test for condition manipulation check for managesystemagentplan * Fix rkebootstrap controller handling of etcd node safe removal annotation * Add RKE2 manifest removal instructions to encryption key rotation and certificate rotation to help ensure system components are restarted on major operations * Add additional etcd restore stage to clean up system pods, don't generate capr cluster tokens if the cluster has plans delivered, and don't short circuit plan delivery logic if planAppliedButWaitingForProbes * Bump rancher-machine version to v0.15.0-rancher100 * Fix S3 endpoint CA rendering and prefer snapshot S3 files and arguments * Bump system-agent to v0.3.3-rc3 * Consolidate etcd machine cleanup and force remove machines on etcd restore shutdown phase * Don't autoset join URL if annotation is set * Clean up non-matching nodes on restore * Fix unnecessarily noisy certificate rotation pausing Signed-off-by: Chris Kim <oats87g@gmail.com> # This is the commit message rancher#27: Add hostname truncation validation test # This is the commit message rancher#28: feat: Allows configuration of the 'type' used in Service * Defaults to the standard ClusterIP * Allows user to override with NodePort or LoadBalancer * Allows user to customise service with provided annotations * Chart docs have been updated * This allows smooth running on GKE clusters using static IP addresses and Google managed certificates Fixes issue: rancher#16061 # This is the commit message rancher#29: Adds tests for the new service type attribute # This is the commit message rancher#30: fix: Fixed silly issue with tests # This is the commit message rancher#31: feat: Allows service annotations to be configured # This is the commit message rancher#32: fix: Added missing annotations key. Doh. # This is the commit message rancher#33: fix: Add missing empty trailing new line. # This is the commit message rancher#34: Adds a path to the Ingress rule in the Rancher chart to make it compatible with ingress controllers that require a path to be present. Fixes rancher#39638 Signed-off-by: Bastian Hofmann <mail@bastianhofmann.de> # This is the commit message rancher#35: Fix ingress path unit test. # This is the commit message rancher#36: Add multi-environment support for AKS Issue: rancher/aks-operator#98 # This is the commit message rancher#37: Updating to Fleet v0.7.0-rc.3 # This is the commit message rancher#38: Keep all nodes during etcd restore that either match the machine UID label selector or have a corresponding node ref (rancher#41564) Signed-off-by: Chris Kim <oats87g@gmail.com> # This is the commit message rancher#39: Rework errNotConfigured into a type Using a type that implements Error, we can use that type in tests without needing to know about its underlying implementation. This keeps the underlying value opaque. Should be no change in behavior. # This is the commit message rancher#40: Initial round of unit tests for Okta+LDAP These are specifically intended to test the behavior in PR rancher#41269 so they are intentionally quite limited in scope. Mostly the goal is to ensure that when an ldapProvider is configured on a SAML provider, it is actually used when a principal search is performed. This would be fairly trivial to expand to the shibboleth provider, and in the future I'd like to include a group search suite. # This is the commit message rancher#41: Add doc comments to IsNotConfigured and ErrNotConfigured # This is the commit message rancher#42: bump the SUC version in the Dockerfile # This is the commit message rancher#43: Add cluster agent tests # This is the commit message rancher#44: run constructFilesSecret both when creating and deleting a node (rancher#41003) Some NodeDrivers need to have access to the same secrets they used when creating the node. For example, the Openstack node driver needs access to the cacert file that is used to connect to Openstack. # This is the commit message rancher#45: Fix run script to check for args # This is the commit message rancher#46: Pin the rancher-webhook chart to an exact version # This is the commit message rancher#47: Bring back the old version-comparing behavior and cover it with tests # This is the commit message rancher#48: Add new logic with exact version and cover it with tests # This is the commit message rancher#49: Adjust remaining behavior for the deprecated env var # This is the commit message rancher#50: Ensure the new and old Helm values are merged # This is the commit message rancher#51: Allow downgrades only when using exact version explicitly # This is the commit message rancher#52: Add test for agent customization in fleetcluster # This is the commit message rancher#53: Do not export RestConfig of test Client; configure a RestGetter instead # This is the commit message rancher#54: Additional restricted admin tests
crobby
pushed a commit
to crobby/rancher
that referenced
this issue
Aug 14, 2023
Removing unused error type check
crobby
pushed a commit
to crobby/rancher
that referenced
this issue
Aug 16, 2023
Squashed commit of the following: commit c2bb101 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:13:12 2023 -0400 Add a generic failure status, defer restoring logins on failure states commit f9c0398 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 13:21:29 2023 -0400 Permit retries (with backoff) when opening the LDAP connection Previously we were considering a failure during open (initial or otherwise) to be a hard, script-ending, permanent failure. That's frankly a bit silly, networks can be tempermental, so this fixes that somewhat. Notably, I can't seem to find any way to check the status of the connection on the lConn object, so we're tracking that manually using a tiny little state object. If there's a cleaner way to inspect this state I am all ears, but I don't think it's a majorly big deal. (Elsewhere in Rancher we don't try to share the ldap connection generally, but here it is a big performance boost, so it is worth the extra trouble.) commit b293d62 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:54:43 2023 -0400 Rework token logic to mirror *RTBs This both collects and processes tokens that the old logic would have missed, and is also considerably more efficient, now needing to scan the list of workunits and the list of tokens just once. commit fcd2b34 Merge: 005f102 3bdea12 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:12:36 2023 -0400 Merge pull request rancher#24 from crobby/migrationreview17 Fixing names to make ci happy commit 3bdea12 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 12:09:22 2023 -0400 Fixing names to make ci happy commit 005f102 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:01:31 2023 -0400 Missing users are Infof, not Errorf commit 540e494 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:10:27 2023 -0400 Don't create/update the configmap object in dry run mode What part of "dry run" did we forget, hrm? commit 9ced565 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:00:51 2023 -0400 If the config map is not found, it's fine. (Panic otherwise.) commit 80ea848 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 10:53:30 2023 -0400 Add logic to migrate list of allowed users commit c12dcef Merge: 33f494a ce1feb4 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:25:53 2023 -0400 Merge pull request rancher#23 from crobby/migrationreview14 Another round of updates commit 33f494a Merge: b897e47 e944b57 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:13:15 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit b897e47 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:12:51 2023 -0400 Rework CRTB,PRTB collection, add GRB migration logic commit ce1feb4 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 07:15:24 2023 -0400 Echoing the set options at the end of the banner commit 089412c Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:44:43 2023 -0400 Adding additional information to README commit a7c9484 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:38:19 2023 -0400 Include agent image location in banner commit 8854263 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 16:31:44 2023 -0400 Mirror script status to authconfig commit 5bc29d5 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 12:50:13 2023 -0400 Update script status codes commit e944b57 Merge: 14c5f72 80e928b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:58 2023 -0400 Merge pull request rancher#22 from crobby/migrationreview13 More updates commit 14c5f72 Merge: a3e85de 516bdeb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:03 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit a3e85de Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:35:46 2023 -0400 Break out migration logic into a bunch of smaller files commit 80e928b Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:51:39 2023 -0400 Use configmap cache instead of client commit 516bdeb Merge: a899779 f8369c8 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:13:56 2023 -0400 Merge pull request rancher#21 from crobby/migrationreview12 Display banner before doing version check commit f8369c8 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:12:31 2023 -0400 Display banner before doing version check commit a899779 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:08:24 2023 -0400 Update cleanup/ad-guid-README.md Co-authored-by: Michael Bolot <michael.bolot@suse.com> commit 4d09212 Merge: c110ae9 92483fa Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 09:58:56 2023 -0400 Merge pull request rancher#19 from crobby/migrationreview9 Removing unused error type check commit 92483fa Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 09:51:18 2023 -0400 Removing unused error type check commit c110ae9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:51:16 2023 -0400 goimports the things commit 7691146 Merge: 44d2375 6453484 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:39 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 6453484 Merge: baf84bf 50286a2 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:32 2023 -0400 Merge pull request rancher#18 from crobby/migrationreview7 Fixing error checking commit 44d2375 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:13:58 2023 -0400 Use wait's exponential backoff primitive instead of manual sleeps commit 50286a2 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 16:27:48 2023 -0400 Fixing error checking commit baf84bf Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:39:13 2023 -0400 Only yell if the user is doing a non-dry-run on v2.7.5 commit eed1416 Merge: 9a71e38 ad00983 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:53 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 9a71e38 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:08 2023 -0400 Cleanup timeout messaging, lower job start timeout to 5 minutes I misunderstood the bash logic when I first extended that to one hour. 5 minutes for an agent download is somewhat more sensible. commit ad00983 Merge: 4e18baa 344a05d Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:34:29 2023 -0400 Merge pull request rancher#17 from crobby/migrationreview6 Additional changes after review commit 344a05d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 14:16:55 2023 -0400 Adding version check for v2.7.5 before doing anything commit 682444d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 13:50:05 2023 -0400 Fix-up README for updated usage commit 4e18baa Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:54:15 2023 -0400 Spawn relevant resources in the cattle-system namespace commit f96eb3a Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:12:33 2023 -0400 Move the YAML configuration file into the bash script This dodges the whole "fetch it from a weird URL" thing, and also makes the script a self-contained single file, which is much nicer for support to deal with. commit 275f42b Merge: 4c98764 b99cab4 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 11:16:41 2023 -0400 Merge pull request rancher#16 from crobby/migrationreview5 More post review updates commit b99cab4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 09:53:57 2023 -0400 Fixing up handling of command line options and args commit 4f6da40 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:49:20 2023 -0400 Fixing up LdapFoundDuplicateGUID name commit 9f577f6 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:31:20 2023 -0400 Adding percentage done indicator to status config map commit 43f19e4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:06:02 2023 -0400 Adding lists of special status users to configmap commit fa9979e Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 06:33:46 2023 -0400 Adding rancher-cleanup label to all cleanup objects commit 4c98764 Merge: 2d59ac6 c301303 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:38:29 2023 -0400 Merge pull request rancher#15 from crobby/migrationreview4 Post review updates commit c301303 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:33:39 2023 -0400 Updated isGUID function commit 2d59ac6 Merge: c0cdc07 86330c6 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:14:48 2023 -0400 Merge pull request rancher#14 from crobby/migrationreview3 Migration review updates 3 commit c0cdc07 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:12:22 2023 -0400 Log if we need to skip a CRTB/PRTB due to the user not existing This feels like the safer option versus applying permissions that none of the users we've collected actually have, even with the GUID/DN matching. This situation should be relatively uncommon, as Rancher usually cleans these up when users are deleted, but with the GUID duplicate bug I'm not sure how successful that will have been in practice. Best to be safe (and noisy) commit 86330c6 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:09:05 2023 -0400 Updating SA permissions for nonResourceURLs commit 4ae2d58 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 12:12:19 2023 -0400 Seeding README, adding script banner commit f8c941b Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:20:10 2023 -0400 Token collection checking userID and now setting userID and label for token updates commit e742102 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:03:04 2023 -0400 Adding additional dry-run logging information commit dc46114 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 16:57:02 2023 -0400 Rework CRTB/PRTB collection to check usernames, run through list once There are still nested for loops in here, but they are a bit more hidden :P commit ad32ccd Merge: ccb0b84 cb98c12 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:52:25 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ccb0b84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:50:27 2023 -0400 Break out the user modification flow into separate functions This mostly cleans up the main loop, but it also separates concerns and makes the smaller bits of logic easier to find and follow. commit aa41893 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:19:08 2023 -0400 Move user principal printing into its respective utility function commit ef909ab Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:12:05 2023 -0400 Respect the adConfig's UserObjectClass when performing a GUID lookup This is for parity with the auth provider; most AD configurations shouldn't have changed this from the default. commit 3963205 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:44:10 2023 -0400 Consider multiple users with the same GUID as a hard error This shouldn't be possible in practice, so it almost certainly indicates either a configuration error, or something wrong on the AD side of things. Either way we will refuse to process any user that trips this logic, and complain about it quite loudly. commit 0cebb89 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:27:24 2023 -0400 We don't need the scope, so simplify -> getExternalId commit da7ef22 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:11:41 2023 -0400 Start the scaledContext. Don't give it managers it doesn't need commit a60b144 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:34:25 2023 -0400 Remove the ratelimiting exception. Prefer safety over speed We need to check the performance ramifications of this during testing, but considering that we will almost certainly be iterating over hundreds of users, we should probably let k8s itself rate limit us so we don't overwhelm whatever is running the control plane. That might otherwise be a nasty situation, especially for stuff like AKS and GKE. commit 16715df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:32:57 2023 -0400 For bonus safety, redundantly check for dryRun here The logic up top should make this check unnecessary, but we want to be extra certain that in dryRun mode no changes are made, so we'll explicitly guard on it every time. This protects the code less from itself, and more from future modifications. commit cb98c12 Merge: e17d56f 4d2f735 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:20:06 2023 -0400 Merge pull request rancher#13 from crobby/migrationreview2 More updates based on review comments commit 4d2f735 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 8 10:17:38 2023 -0400 More updates based on review comments commit e17d56f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:38:59 2023 -0400 EscapeUUID -> escapeUUID commit 139ce3c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:37:34 2023 -0400 Relocate environment variable use to the agent-specific code path commit 795c94b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:33:13 2023 -0400 Remove unnecessary namespace from cluster role definitions commit 01ea868 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:30:53 2023 -0400 One minute is *awfully optimistic.* Let's be more realistic commit b9d4487 Merge: 17250da 0efbb02 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:21:42 2023 -0400 Merge pull request rancher#12 from crobby/migrationreview Update based on review comments commit 0efbb02 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 7 15:55:46 2023 -0400 Update based on review comments commit 17250da Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 10:29:05 2023 -0400 Don't hide the migration script from windows agents ... which in hindsight are probably somewhat likely to be using the Active Directory auth provider. commit cadf021 Merge: 9b8fd58 3926f7b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 08:18:10 2023 -0400 Merge pull request rancher#11 from crobby/migrateimports Fixing imports commit 3926f7b Author: Chad Roberts <chad.roberts@suse.com> Date: Sat Aug 5 07:45:25 2023 -0400 Fixing imports commit 9b8fd58 Merge: de38ffe 26dd505 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 17:10:43 2023 -0400 Merge pull request rancher#10 from crobby/dntokens Fix tokens going to local principal commit 26dd505 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 17:08:20 2023 -0400 Fix tokens going to local principal commit de38ffe Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 15:36:12 2023 -0400 Cleanup debug/info logs somewhat commit 1581b5d Merge: 5dfcda0 29c87eb Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:56:22 2023 -0400 Merge pull request rancher#9 from crobby/linter2 More cleaning up lint commit 29c87eb Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:54:40 2023 -0400 More cleaning up lint commit 5dfcda0 Merge: a119663 d37ef2f Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:49:55 2023 -0400 Merge pull request rancher#8 from crobby/linter Cleaning up lint commit d37ef2f Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:47:44 2023 -0400 Cleaning up lint commit a119663 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:38:46 2023 -0400 Add an option to automatically delete missing-guid users This is only available when running the standalone script. At Rancher startup this option is set to false, so missing users will be logged instead and require manual intervention. commit 60f31f8 Merge: 7e620d5 9d82578 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 13:22:56 2023 -0400 Merge pull request rancher#7 from crobby/0805-migration Update migration start logic so an automated run will only happen if another run has not completed commit 9d82578 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 12:12:56 2023 -0400 Update migration start logic so an automated run will only happen if another run has not completed commit 7e620d5 Merge: 30c9f64 6c352a5 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:26:52 2023 -0400 Merge pull request rancher#4 from crobby/migrateatstart Add guid migration to rancher startup commit 30c9f64 Merge: b9aa392 72895b4 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:10:58 2023 -0400 Merge pull request rancher#5 from crobby/0803-migration Make sure annotations/labels are not nil commit 72895b4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 16:58:56 2023 -0400 Make sure annotations/labels are not nil commit b9aa392 Merge: 79762cb 7546cdf Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 10:43:30 2023 -0400 Merge pull request rancher#6 from crobby/0804-migration Fix crtb, prtb collection and add token collection/migration commit 7546cdf Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 08:59:54 2023 -0400 Fix crtb, prtb collection and add token collection/migration commit 79762cb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 3 18:00:53 2023 -0400 Collect CRTBs and PRTBs in a single pass commit b6b6085 Merge: 3de5aa3 b3acab9 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 3 11:44:13 2023 -0400 Merge pull request rancher#3 from crobby/0802-2migration Adding annotation/labels for migrated objects also blocking login while migration is active commit b3acab9 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 11:37:16 2023 -0400 Update role for SA commit 673e765 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 09:33:45 2023 -0400 Blocking login while migration is running commit 6c352a5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 13:42:33 2023 -0400 Add guid migration to rancher startup commit 840c5a7 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 12:20:41 2023 -0400 Adding annotation/labels for migrated objects commit 3de5aa3 Merge: 5dc7bd7 04ea1ce Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 2 09:57:48 2023 -0400 Merge pull request rancher#2 from crobby/0802migration Fix status function and use user copies in workUnit slices commit 04ea1ce Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 1 18:02:19 2023 -0400 Fixing status function and using copies of users in workUnit slices commit 5dc7bd7 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:29:15 2023 -0400 Skip over configmap updates for now, just to get the script running commit ac3afe6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:19:52 2023 -0400 Massively overhaul main loop, check for and handle duplicate users This is largely untested because I'm having some trouble with the configmaps code, but I wanted to get this committed before I start troubleshooting commit 5295f8f Merge: 29f9332 552e73f Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 1 08:58:41 2023 -0400 Merge pull request rancher#1 from crobby/tokenunmigrate Additional unmigration functionality commit 552e73f Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Jul 31 13:22:26 2023 -0400 Additional unmigration functionality commit 29f9332 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 17:30:10 2023 -0400 Actually perform the GUID -> DN migration on the happy path And it works too! Thank goodness. Now we mostly need to clean up the logic and handle a few dozen edge cases. commit 62a6747 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 12:53:43 2023 -0400 Cleanup the logs a bit, flatten the central logic with early exits commit ac20a2c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 09:58:54 2023 -0400 Switch to using the scaledContext for everything Since it can do all the lookups we need, it seems silly to setup and use two different interfaces to the same underlying datastore. The UnstructuredClient is the only way we can read AD configuration right now, and we need that info, so let's stick to that method. commit 18b39d3 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Jul 28 17:38:27 2023 -0400 First pass at migration scaffolding, enough to do GUID -> DN lookups There is still much work to do, but at the very least we can read the relevant auth configuration details from k8s and use those details to make LDAP queries, and that's nearly all of what we need to perform the migration.
deniseschannon
pushed a commit
that referenced
this issue
Aug 17, 2023
Squashed commit of the following: commit 5b32df6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:59:35 2023 -0400 Turns out the token.userPrincipal.UID is not normally set commit 064526f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:12:17 2023 -0400 Pull token fields from the ldap attributes instead of the old user commit e33bba9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 10:11:57 2023 -0400 Outdent returns to make drone happy commit 6c084df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 09:01:45 2023 -0400 Squashed commit of the following: commit 3db22eb Merge: 8039207 552fb84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:57:01 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 8039207 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:56:53 2023 -0400 tiny, tiny fix to logging commit 552fb84 Merge: ea68517 99a1814 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 17 07:39:00 2023 -0400 Merge pull request #30 from crobby/migrationreview31 Outdent else blocks to make lint happy commit 99a1814 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 17 05:00:47 2023 -0400 Outdent else blocks to make lint happy commit ea68517 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 20:28:14 2023 -0400 Apply exponential retry logic to GRB and Token migrations Also, like *RTBs, these are considered non-fatal if a permanent error of some sort occurs. We continue to migrate the user anyway. commit 4a2ae0b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 19:24:42 2023 -0400 For CRTB/PRTBs, rework error handling to gracefully retry In particular, this treats internal errors (usually related to webhook timeouts) as transient, and retries them with a little bit of exponential backoff. Furthermore, after reviewing some scenarios with Michael, we've decided to consider non-internal errors from the webhook as non-fatal in terms of continuing to process the individual user. There are a few situations where old bindings to disabled templates would otherwise block users from migrating, and this permits those to have a better chance of overall success. commit 35d647c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:58:50 2023 -0400 When merging user tokens, copy over all relevant principal fields These aren't used for anything that I'm aware of, so this is really more just for consistency, since we want the two to be fully paired. commit f3e8094 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:52:15 2023 -0400 Cleanup error handling, consider AD retrieval to be a harder error commit 90f2ec1 Merge: ffcec58 b56138b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:28 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ffcec58 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:10 2023 -0400 ... once. Add the DN-based principal once. commit b56138b Merge: 78a66e0 bfb7176 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:45 2023 -0400 Merge pull request #29 from crobby/migrationreview25 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit 78a66e0 Merge: edf3535 df507b5 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:24 2023 -0400 Merge pull request #28 from crobby/migrationreview24 Remove unnecessary json marshal/unmarshal commit edf3535 Merge: b93e6d0 12020af Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:10 2023 -0400 Merge pull request #27 from crobby/migrationreview23 Give the job pod a chance to come up before tailing the log commit b93e6d0 Merge: a2c2acb 58a0a1d Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:52 2023 -0400 Merge pull request #26 from crobby/migrationreview22 Now using AuthConfig annotation as source of truth to block login during migration commit a2c2acb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:06 2023 -0400 Rework allowed user migration to handle duplicates and missing users commit bfb7176 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 14:38:22 2023 -0400 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit df507b5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:38:39 2023 -0400 Remove unnecessary json marshal/unmarshal commit 12020af Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:01:18 2023 -0400 Give the job pod a chance to come up before tailing the log commit 58a0a1d Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 12:50:57 2023 -0400 Now using AuthConfig annotation as source of truth to block login during migration commit 3ef3fb0 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:27:23 2023 -0400 Wait to do the AuthConfig principals until after updating users This kicks off some rancher-side tasks based on the updated list, and we'd really like to make sure that those user changes have been made in advance just for sanity purposes. commit b29bfb8 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:25:30 2023 -0400 When collecting duplicates, we need to track the workunit index commit df0307e Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:23:47 2023 -0400 Have the dry run guard writing new principal IDs This is mostly just to make the code clearer and more obvious. The safety is redundant, as the dry run also blocks making changes to the user object later. commit 59bafdf Merge: 2dd5250 2473062 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:12:08 2023 -0400 Merge pull request #25 from crobby/migrationreview21 Append copy of user rather than pointer to duplicate list commit 2473062 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 08:00:41 2023 -0400 append copy of user rather than pointer to duplicate list commit 2dd5250 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:48:34 2023 -0400 Explicitly check to see if AD is disabled, and exit success in this case commit 4a3aa80 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:00:25 2023 -0400 Actually *use* the final migration status commit 255ef68 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:36:19 2023 -0400 Add uuid-unmigration script, prevent AD logins during execution Squashed commit of the following: commit c2bb101 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:13:12 2023 -0400 Add a generic failure status, defer restoring logins on failure states commit f9c0398 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 13:21:29 2023 -0400 Permit retries (with backoff) when opening the LDAP connection Previously we were considering a failure during open (initial or otherwise) to be a hard, script-ending, permanent failure. That's frankly a bit silly, networks can be tempermental, so this fixes that somewhat. Notably, I can't seem to find any way to check the status of the connection on the lConn object, so we're tracking that manually using a tiny little state object. If there's a cleaner way to inspect this state I am all ears, but I don't think it's a majorly big deal. (Elsewhere in Rancher we don't try to share the ldap connection generally, but here it is a big performance boost, so it is worth the extra trouble.) commit b293d62 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:54:43 2023 -0400 Rework token logic to mirror *RTBs This both collects and processes tokens that the old logic would have missed, and is also considerably more efficient, now needing to scan the list of workunits and the list of tokens just once. commit fcd2b34 Merge: 005f102 3bdea12 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:12:36 2023 -0400 Merge pull request #24 from crobby/migrationreview17 Fixing names to make ci happy commit 3bdea12 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 12:09:22 2023 -0400 Fixing names to make ci happy commit 005f102 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:01:31 2023 -0400 Missing users are Infof, not Errorf commit 540e494 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:10:27 2023 -0400 Don't create/update the configmap object in dry run mode What part of "dry run" did we forget, hrm? commit 9ced565 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:00:51 2023 -0400 If the config map is not found, it's fine. (Panic otherwise.) commit 80ea848 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 10:53:30 2023 -0400 Add logic to migrate list of allowed users commit c12dcef Merge: 33f494a ce1feb4 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:25:53 2023 -0400 Merge pull request #23 from crobby/migrationreview14 Another round of updates commit 33f494a Merge: b897e47 e944b57 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:13:15 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit b897e47 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:12:51 2023 -0400 Rework CRTB,PRTB collection, add GRB migration logic commit ce1feb4 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 07:15:24 2023 -0400 Echoing the set options at the end of the banner commit 089412c Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:44:43 2023 -0400 Adding additional information to README commit a7c9484 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:38:19 2023 -0400 Include agent image location in banner commit 8854263 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 16:31:44 2023 -0400 Mirror script status to authconfig commit 5bc29d5 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 12:50:13 2023 -0400 Update script status codes commit e944b57 Merge: 14c5f72 80e928b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:58 2023 -0400 Merge pull request #22 from crobby/migrationreview13 More updates commit 14c5f72 Merge: a3e85de 516bdeb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:03 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit a3e85de Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:35:46 2023 -0400 Break out migration logic into a bunch of smaller files commit 80e928b Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:51:39 2023 -0400 Use configmap cache instead of client commit 516bdeb Merge: a899779 f8369c8 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:13:56 2023 -0400 Merge pull request #21 from crobby/migrationreview12 Display banner before doing version check commit f8369c8 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:12:31 2023 -0400 Display banner before doing version check commit a899779 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:08:24 2023 -0400 Update cleanup/ad-guid-README.md Co-authored-by: Michael Bolot <michael.bolot@suse.com> commit 4d09212 Merge: c110ae9 92483fa Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 09:58:56 2023 -0400 Merge pull request #19 from crobby/migrationreview9 Removing unused error type check commit 92483fa Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 09:51:18 2023 -0400 Removing unused error type check commit c110ae9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:51:16 2023 -0400 goimports the things commit 7691146 Merge: 44d2375 6453484 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:39 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 6453484 Merge: baf84bf 50286a2 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:32 2023 -0400 Merge pull request #18 from crobby/migrationreview7 Fixing error checking commit 44d2375 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:13:58 2023 -0400 Use wait's exponential backoff primitive instead of manual sleeps commit 50286a2 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 16:27:48 2023 -0400 Fixing error checking commit baf84bf Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:39:13 2023 -0400 Only yell if the user is doing a non-dry-run on v2.7.5 commit eed1416 Merge: 9a71e38 ad00983 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:53 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 9a71e38 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:08 2023 -0400 Cleanup timeout messaging, lower job start timeout to 5 minutes I misunderstood the bash logic when I first extended that to one hour. 5 minutes for an agent download is somewhat more sensible. commit ad00983 Merge: 4e18baa 344a05d Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:34:29 2023 -0400 Merge pull request #17 from crobby/migrationreview6 Additional changes after review commit 344a05d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 14:16:55 2023 -0400 Adding version check for v2.7.5 before doing anything commit 682444d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 13:50:05 2023 -0400 Fix-up README for updated usage commit 4e18baa Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:54:15 2023 -0400 Spawn relevant resources in the cattle-system namespace commit f96eb3a Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:12:33 2023 -0400 Move the YAML configuration file into the bash script This dodges the whole "fetch it from a weird URL" thing, and also makes the script a self-contained single file, which is much nicer for support to deal with. commit 275f42b Merge: 4c98764 b99cab4 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 11:16:41 2023 -0400 Merge pull request #16 from crobby/migrationreview5 More post review updates commit b99cab4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 09:53:57 2023 -0400 Fixing up handling of command line options and args commit 4f6da40 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:49:20 2023 -0400 Fixing up LdapFoundDuplicateGUID name commit 9f577f6 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:31:20 2023 -0400 Adding percentage done indicator to status config map commit 43f19e4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:06:02 2023 -0400 Adding lists of special status users to configmap commit fa9979e Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 06:33:46 2023 -0400 Adding rancher-cleanup label to all cleanup objects commit 4c98764 Merge: 2d59ac6 c301303 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:38:29 2023 -0400 Merge pull request #15 from crobby/migrationreview4 Post review updates commit c301303 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:33:39 2023 -0400 Updated isGUID function commit 2d59ac6 Merge: c0cdc07 86330c6 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:14:48 2023 -0400 Merge pull request #14 from crobby/migrationreview3 Migration review updates 3 commit c0cdc07 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:12:22 2023 -0400 Log if we need to skip a CRTB/PRTB due to the user not existing This feels like the safer option versus applying permissions that none of the users we've collected actually have, even with the GUID/DN matching. This situation should be relatively uncommon, as Rancher usually cleans these up when users are deleted, but with the GUID duplicate bug I'm not sure how successful that will have been in practice. Best to be safe (and noisy) commit 86330c6 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:09:05 2023 -0400 Updating SA permissions for nonResourceURLs commit 4ae2d58 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 12:12:19 2023 -0400 Seeding README, adding script banner commit f8c941b Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:20:10 2023 -0400 Token collection checking userID and now setting userID and label for token updates commit e742102 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:03:04 2023 -0400 Adding additional dry-run logging information commit dc46114 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 16:57:02 2023 -0400 Rework CRTB/PRTB collection to check usernames, run through list once There are still nested for loops in here, but they are a bit more hidden :P commit ad32ccd Merge: ccb0b84 cb98c12 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:52:25 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ccb0b84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:50:27 2023 -0400 Break out the user modification flow into separate functions This mostly cleans up the main loop, but it also separates concerns and makes the smaller bits of logic easier to find and follow. commit aa41893 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:19:08 2023 -0400 Move user principal printing into its respective utility function commit ef909ab Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:12:05 2023 -0400 Respect the adConfig's UserObjectClass when performing a GUID lookup This is for parity with the auth provider; most AD configurations shouldn't have changed this from the default. commit 3963205 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:44:10 2023 -0400 Consider multiple users with the same GUID as a hard error This shouldn't be possible in practice, so it almost certainly indicates either a configuration error, or something wrong on the AD side of things. Either way we will refuse to process any user that trips this logic, and complain about it quite loudly. commit 0cebb89 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:27:24 2023 -0400 We don't need the scope, so simplify -> getExternalId commit da7ef22 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:11:41 2023 -0400 Start the scaledContext. Don't give it managers it doesn't need commit a60b144 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:34:25 2023 -0400 Remove the ratelimiting exception. Prefer safety over speed We need to check the performance ramifications of this during testing, but considering that we will almost certainly be iterating over hundreds of users, we should probably let k8s itself rate limit us so we don't overwhelm whatever is running the control plane. That might otherwise be a nasty situation, especially for stuff like AKS and GKE. commit 16715df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:32:57 2023 -0400 For bonus safety, redundantly check for dryRun here The logic up top should make this check unnecessary, but we want to be extra certain that in dryRun mode no changes are made, so we'll explicitly guard on it every time. This protects the code less from itself, and more from future modifications. commit cb98c12 Merge: e17d56f 4d2f735 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:20:06 2023 -0400 Merge pull request #13 from crobby/migrationreview2 More updates based on review comments commit 4d2f735 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 8 10:17:38 2023 -0400 More updates based on review comments commit e17d56f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:38:59 2023 -0400 EscapeUUID -> escapeUUID commit 139ce3c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:37:34 2023 -0400 Relocate environment variable use to the agent-specific code path commit 795c94b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:33:13 2023 -0400 Remove unnecessary namespace from cluster role definitions commit 01ea868 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:30:53 2023 -0400 One minute is *awfully optimistic.* Let's be more realistic commit b9d4487 Merge: 17250da 0efbb02 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:21:42 2023 -0400 Merge pull request #12 from crobby/migrationreview Update based on review comments commit 0efbb02 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 7 15:55:46 2023 -0400 Update based on review comments commit 17250da Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 10:29:05 2023 -0400 Don't hide the migration script from windows agents ... which in hindsight are probably somewhat likely to be using the Active Directory auth provider. commit cadf021 Merge: 9b8fd58 3926f7b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 08:18:10 2023 -0400 Merge pull request #11 from crobby/migrateimports Fixing imports commit 3926f7b Author: Chad Roberts <chad.roberts@suse.com> Date: Sat Aug 5 07:45:25 2023 -0400 Fixing imports commit 9b8fd58 Merge: de38ffe 26dd505 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 17:10:43 2023 -0400 Merge pull request #10 from crobby/dntokens Fix tokens going to local principal commit 26dd505 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 17:08:20 2023 -0400 Fix tokens going to local principal commit de38ffe Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 15:36:12 2023 -0400 Cleanup debug/info logs somewhat commit 1581b5d Merge: 5dfcda0 29c87eb Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:56:22 2023 -0400 Merge pull request #9 from crobby/linter2 More cleaning up lint commit 29c87eb Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:54:40 2023 -0400 More cleaning up lint commit 5dfcda0 Merge: a119663 d37ef2f Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:49:55 2023 -0400 Merge pull request #8 from crobby/linter Cleaning up lint commit d37ef2f Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:47:44 2023 -0400 Cleaning up lint commit a119663 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:38:46 2023 -0400 Add an option to automatically delete missing-guid users This is only available when running the standalone script. At Rancher startup this option is set to false, so missing users will be logged instead and require manual intervention. commit 60f31f8 Merge: 7e620d5 9d82578 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 13:22:56 2023 -0400 Merge pull request #7 from crobby/0805-migration Update migration start logic so an automated run will only happen if another run has not completed commit 9d82578 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 12:12:56 2023 -0400 Update migration start logic so an automated run will only happen if another run has not completed commit 7e620d5 Merge: 30c9f64 6c352a5 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:26:52 2023 -0400 Merge pull request #4 from crobby/migrateatstart Add guid migration to rancher startup commit 30c9f64 Merge: b9aa392 72895b4 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:10:58 2023 -0400 Merge pull request #5 from crobby/0803-migration Make sure annotations/labels are not nil commit 72895b4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 16:58:56 2023 -0400 Make sure annotations/labels are not nil commit b9aa392 Merge: 79762cb 7546cdf Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 10:43:30 2023 -0400 Merge pull request #6 from crobby/0804-migration Fix crtb, prtb collection and add token collection/migration commit 7546cdf Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 08:59:54 2023 -0400 Fix crtb, prtb collection and add token collection/migration commit 79762cb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 3 18:00:53 2023 -0400 Collect CRTBs and PRTBs in a single pass commit b6b6085 Merge: 3de5aa3 b3acab9 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 3 11:44:13 2023 -0400 Merge pull request #3 from crobby/0802-2migration Adding annotation/labels for migrated objects also blocking login while migration is active commit b3acab9 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 11:37:16 2023 -0400 Update role for SA commit 673e765 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 09:33:45 2023 -0400 Blocking login while migration is running commit 6c352a5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 13:42:33 2023 -0400 Add guid migration to rancher startup commit 840c5a7 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 12:20:41 2023 -0400 Adding annotation/labels for migrated objects commit 3de5aa3 Merge: 5dc7bd7 04ea1ce Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 2 09:57:48 2023 -0400 Merge pull request #2 from crobby/0802migration Fix status function and use user copies in workUnit slices commit 04ea1ce Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 1 18:02:19 2023 -0400 Fixing status function and using copies of users in workUnit slices commit 5dc7bd7 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:29:15 2023 -0400 Skip over configmap updates for now, just to get the script running commit ac3afe6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:19:52 2023 -0400 Massively overhaul main loop, check for and handle duplicate users This is largely untested because I'm having some trouble with the configmaps code, but I wanted to get this committed before I start troubleshooting commit 5295f8f Merge: 29f9332 552e73f Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 1 08:58:41 2023 -0400 Merge pull request #1 from crobby/tokenunmigrate Additional unmigration functionality commit 552e73f Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Jul 31 13:22:26 2023 -0400 Additional unmigration functionality commit 29f9332 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 17:30:10 2023 -0400 Actually perform the GUID -> DN migration on the happy path And it works too! Thank goodness. Now we mostly need to clean up the logic and handle a few dozen edge cases. commit 62a6747 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 12:53:43 2023 -0400 Cleanup the logs a bit, flatten the central logic with early exits commit ac20a2c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 09:58:54 2023 -0400 Switch to using the scaledContext for everything Since it can do all the lookups we need, it seems silly to setup and use two different interfaces to the same underlying datastore. The UnstructuredClient is the only way we can read AD configuration right now, and we need that info, so let's stick to that method. commit 18b39d3 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Jul 28 17:38:27 2023 -0400 First pass at migration scaffolding, enough to do GUID -> DN lookups There is still much work to do, but at the very least we can read the relevant auth configuration details from k8s and use those details to make LDAP queries, and that's nearly all of what we need to perform the migration.
crobby
pushed a commit
to crobby/rancher
that referenced
this issue
Aug 25, 2023
Squashed commit of the following: commit 5b32df6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:59:35 2023 -0400 Turns out the token.userPrincipal.UID is not normally set commit 064526f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:12:17 2023 -0400 Pull token fields from the ldap attributes instead of the old user commit e33bba9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 10:11:57 2023 -0400 Outdent returns to make drone happy commit 6c084df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 09:01:45 2023 -0400 Squashed commit of the following: commit 3db22eb Merge: 8039207 552fb84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:57:01 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 8039207 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:56:53 2023 -0400 tiny, tiny fix to logging commit 552fb84 Merge: ea68517 99a1814 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 17 07:39:00 2023 -0400 Merge pull request rancher#30 from crobby/migrationreview31 Outdent else blocks to make lint happy commit 99a1814 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 17 05:00:47 2023 -0400 Outdent else blocks to make lint happy commit ea68517 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 20:28:14 2023 -0400 Apply exponential retry logic to GRB and Token migrations Also, like *RTBs, these are considered non-fatal if a permanent error of some sort occurs. We continue to migrate the user anyway. commit 4a2ae0b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 19:24:42 2023 -0400 For CRTB/PRTBs, rework error handling to gracefully retry In particular, this treats internal errors (usually related to webhook timeouts) as transient, and retries them with a little bit of exponential backoff. Furthermore, after reviewing some scenarios with Michael, we've decided to consider non-internal errors from the webhook as non-fatal in terms of continuing to process the individual user. There are a few situations where old bindings to disabled templates would otherwise block users from migrating, and this permits those to have a better chance of overall success. commit 35d647c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:58:50 2023 -0400 When merging user tokens, copy over all relevant principal fields These aren't used for anything that I'm aware of, so this is really more just for consistency, since we want the two to be fully paired. commit f3e8094 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:52:15 2023 -0400 Cleanup error handling, consider AD retrieval to be a harder error commit 90f2ec1 Merge: ffcec58 b56138b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:28 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ffcec58 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:10 2023 -0400 ... once. Add the DN-based principal once. commit b56138b Merge: 78a66e0 bfb7176 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:45 2023 -0400 Merge pull request rancher#29 from crobby/migrationreview25 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit 78a66e0 Merge: edf3535 df507b5 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:24 2023 -0400 Merge pull request rancher#28 from crobby/migrationreview24 Remove unnecessary json marshal/unmarshal commit edf3535 Merge: b93e6d0 12020af Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:10 2023 -0400 Merge pull request rancher#27 from crobby/migrationreview23 Give the job pod a chance to come up before tailing the log commit b93e6d0 Merge: a2c2acb 58a0a1d Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:52 2023 -0400 Merge pull request rancher#26 from crobby/migrationreview22 Now using AuthConfig annotation as source of truth to block login during migration commit a2c2acb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:06 2023 -0400 Rework allowed user migration to handle duplicates and missing users commit bfb7176 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 14:38:22 2023 -0400 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit df507b5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:38:39 2023 -0400 Remove unnecessary json marshal/unmarshal commit 12020af Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:01:18 2023 -0400 Give the job pod a chance to come up before tailing the log commit 58a0a1d Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 12:50:57 2023 -0400 Now using AuthConfig annotation as source of truth to block login during migration commit 3ef3fb0 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:27:23 2023 -0400 Wait to do the AuthConfig principals until after updating users This kicks off some rancher-side tasks based on the updated list, and we'd really like to make sure that those user changes have been made in advance just for sanity purposes. commit b29bfb8 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:25:30 2023 -0400 When collecting duplicates, we need to track the workunit index commit df0307e Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:23:47 2023 -0400 Have the dry run guard writing new principal IDs This is mostly just to make the code clearer and more obvious. The safety is redundant, as the dry run also blocks making changes to the user object later. commit 59bafdf Merge: 2dd5250 2473062 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:12:08 2023 -0400 Merge pull request rancher#25 from crobby/migrationreview21 Append copy of user rather than pointer to duplicate list commit 2473062 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 08:00:41 2023 -0400 append copy of user rather than pointer to duplicate list commit 2dd5250 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:48:34 2023 -0400 Explicitly check to see if AD is disabled, and exit success in this case commit 4a3aa80 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:00:25 2023 -0400 Actually *use* the final migration status commit 255ef68 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:36:19 2023 -0400 Add uuid-unmigration script, prevent AD logins during execution Squashed commit of the following: commit c2bb101 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:13:12 2023 -0400 Add a generic failure status, defer restoring logins on failure states commit f9c0398 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 13:21:29 2023 -0400 Permit retries (with backoff) when opening the LDAP connection Previously we were considering a failure during open (initial or otherwise) to be a hard, script-ending, permanent failure. That's frankly a bit silly, networks can be tempermental, so this fixes that somewhat. Notably, I can't seem to find any way to check the status of the connection on the lConn object, so we're tracking that manually using a tiny little state object. If there's a cleaner way to inspect this state I am all ears, but I don't think it's a majorly big deal. (Elsewhere in Rancher we don't try to share the ldap connection generally, but here it is a big performance boost, so it is worth the extra trouble.) commit b293d62 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:54:43 2023 -0400 Rework token logic to mirror *RTBs This both collects and processes tokens that the old logic would have missed, and is also considerably more efficient, now needing to scan the list of workunits and the list of tokens just once. commit fcd2b34 Merge: 005f102 3bdea12 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:12:36 2023 -0400 Merge pull request rancher#24 from crobby/migrationreview17 Fixing names to make ci happy commit 3bdea12 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 12:09:22 2023 -0400 Fixing names to make ci happy commit 005f102 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:01:31 2023 -0400 Missing users are Infof, not Errorf commit 540e494 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:10:27 2023 -0400 Don't create/update the configmap object in dry run mode What part of "dry run" did we forget, hrm? commit 9ced565 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:00:51 2023 -0400 If the config map is not found, it's fine. (Panic otherwise.) commit 80ea848 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 10:53:30 2023 -0400 Add logic to migrate list of allowed users commit c12dcef Merge: 33f494a ce1feb4 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:25:53 2023 -0400 Merge pull request rancher#23 from crobby/migrationreview14 Another round of updates commit 33f494a Merge: b897e47 e944b57 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:13:15 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit b897e47 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:12:51 2023 -0400 Rework CRTB,PRTB collection, add GRB migration logic commit ce1feb4 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 07:15:24 2023 -0400 Echoing the set options at the end of the banner commit 089412c Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:44:43 2023 -0400 Adding additional information to README commit a7c9484 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:38:19 2023 -0400 Include agent image location in banner commit 8854263 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 16:31:44 2023 -0400 Mirror script status to authconfig commit 5bc29d5 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 12:50:13 2023 -0400 Update script status codes commit e944b57 Merge: 14c5f72 80e928b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:58 2023 -0400 Merge pull request rancher#22 from crobby/migrationreview13 More updates commit 14c5f72 Merge: a3e85de 516bdeb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:03 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit a3e85de Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:35:46 2023 -0400 Break out migration logic into a bunch of smaller files commit 80e928b Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:51:39 2023 -0400 Use configmap cache instead of client commit 516bdeb Merge: a899779 f8369c8 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:13:56 2023 -0400 Merge pull request rancher#21 from crobby/migrationreview12 Display banner before doing version check commit f8369c8 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:12:31 2023 -0400 Display banner before doing version check commit a899779 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:08:24 2023 -0400 Update cleanup/ad-guid-README.md Co-authored-by: Michael Bolot <michael.bolot@suse.com> commit 4d09212 Merge: c110ae9 92483fa Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 09:58:56 2023 -0400 Merge pull request rancher#19 from crobby/migrationreview9 Removing unused error type check commit 92483fa Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 09:51:18 2023 -0400 Removing unused error type check commit c110ae9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:51:16 2023 -0400 goimports the things commit 7691146 Merge: 44d2375 6453484 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:39 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 6453484 Merge: baf84bf 50286a2 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:32 2023 -0400 Merge pull request rancher#18 from crobby/migrationreview7 Fixing error checking commit 44d2375 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:13:58 2023 -0400 Use wait's exponential backoff primitive instead of manual sleeps commit 50286a2 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 16:27:48 2023 -0400 Fixing error checking commit baf84bf Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:39:13 2023 -0400 Only yell if the user is doing a non-dry-run on v2.7.5 commit eed1416 Merge: 9a71e38 ad00983 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:53 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 9a71e38 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:08 2023 -0400 Cleanup timeout messaging, lower job start timeout to 5 minutes I misunderstood the bash logic when I first extended that to one hour. 5 minutes for an agent download is somewhat more sensible. commit ad00983 Merge: 4e18baa 344a05d Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:34:29 2023 -0400 Merge pull request rancher#17 from crobby/migrationreview6 Additional changes after review commit 344a05d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 14:16:55 2023 -0400 Adding version check for v2.7.5 before doing anything commit 682444d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 13:50:05 2023 -0400 Fix-up README for updated usage commit 4e18baa Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:54:15 2023 -0400 Spawn relevant resources in the cattle-system namespace commit f96eb3a Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:12:33 2023 -0400 Move the YAML configuration file into the bash script This dodges the whole "fetch it from a weird URL" thing, and also makes the script a self-contained single file, which is much nicer for support to deal with. commit 275f42b Merge: 4c98764 b99cab4 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 11:16:41 2023 -0400 Merge pull request rancher#16 from crobby/migrationreview5 More post review updates commit b99cab4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 09:53:57 2023 -0400 Fixing up handling of command line options and args commit 4f6da40 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:49:20 2023 -0400 Fixing up LdapFoundDuplicateGUID name commit 9f577f6 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:31:20 2023 -0400 Adding percentage done indicator to status config map commit 43f19e4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:06:02 2023 -0400 Adding lists of special status users to configmap commit fa9979e Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 06:33:46 2023 -0400 Adding rancher-cleanup label to all cleanup objects commit 4c98764 Merge: 2d59ac6 c301303 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:38:29 2023 -0400 Merge pull request rancher#15 from crobby/migrationreview4 Post review updates commit c301303 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:33:39 2023 -0400 Updated isGUID function commit 2d59ac6 Merge: c0cdc07 86330c6 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:14:48 2023 -0400 Merge pull request rancher#14 from crobby/migrationreview3 Migration review updates 3 commit c0cdc07 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:12:22 2023 -0400 Log if we need to skip a CRTB/PRTB due to the user not existing This feels like the safer option versus applying permissions that none of the users we've collected actually have, even with the GUID/DN matching. This situation should be relatively uncommon, as Rancher usually cleans these up when users are deleted, but with the GUID duplicate bug I'm not sure how successful that will have been in practice. Best to be safe (and noisy) commit 86330c6 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:09:05 2023 -0400 Updating SA permissions for nonResourceURLs commit 4ae2d58 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 12:12:19 2023 -0400 Seeding README, adding script banner commit f8c941b Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:20:10 2023 -0400 Token collection checking userID and now setting userID and label for token updates commit e742102 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:03:04 2023 -0400 Adding additional dry-run logging information commit dc46114 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 16:57:02 2023 -0400 Rework CRTB/PRTB collection to check usernames, run through list once There are still nested for loops in here, but they are a bit more hidden :P commit ad32ccd Merge: ccb0b84 cb98c12 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:52:25 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ccb0b84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:50:27 2023 -0400 Break out the user modification flow into separate functions This mostly cleans up the main loop, but it also separates concerns and makes the smaller bits of logic easier to find and follow. commit aa41893 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:19:08 2023 -0400 Move user principal printing into its respective utility function commit ef909ab Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:12:05 2023 -0400 Respect the adConfig's UserObjectClass when performing a GUID lookup This is for parity with the auth provider; most AD configurations shouldn't have changed this from the default. commit 3963205 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:44:10 2023 -0400 Consider multiple users with the same GUID as a hard error This shouldn't be possible in practice, so it almost certainly indicates either a configuration error, or something wrong on the AD side of things. Either way we will refuse to process any user that trips this logic, and complain about it quite loudly. commit 0cebb89 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:27:24 2023 -0400 We don't need the scope, so simplify -> getExternalId commit da7ef22 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:11:41 2023 -0400 Start the scaledContext. Don't give it managers it doesn't need commit a60b144 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:34:25 2023 -0400 Remove the ratelimiting exception. Prefer safety over speed We need to check the performance ramifications of this during testing, but considering that we will almost certainly be iterating over hundreds of users, we should probably let k8s itself rate limit us so we don't overwhelm whatever is running the control plane. That might otherwise be a nasty situation, especially for stuff like AKS and GKE. commit 16715df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:32:57 2023 -0400 For bonus safety, redundantly check for dryRun here The logic up top should make this check unnecessary, but we want to be extra certain that in dryRun mode no changes are made, so we'll explicitly guard on it every time. This protects the code less from itself, and more from future modifications. commit cb98c12 Merge: e17d56f 4d2f735 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:20:06 2023 -0400 Merge pull request rancher#13 from crobby/migrationreview2 More updates based on review comments commit 4d2f735 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 8 10:17:38 2023 -0400 More updates based on review comments commit e17d56f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:38:59 2023 -0400 EscapeUUID -> escapeUUID commit 139ce3c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:37:34 2023 -0400 Relocate environment variable use to the agent-specific code path commit 795c94b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:33:13 2023 -0400 Remove unnecessary namespace from cluster role definitions commit 01ea868 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:30:53 2023 -0400 One minute is *awfully optimistic.* Let's be more realistic commit b9d4487 Merge: 17250da 0efbb02 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:21:42 2023 -0400 Merge pull request rancher#12 from crobby/migrationreview Update based on review comments commit 0efbb02 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 7 15:55:46 2023 -0400 Update based on review comments commit 17250da Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 10:29:05 2023 -0400 Don't hide the migration script from windows agents ... which in hindsight are probably somewhat likely to be using the Active Directory auth provider. commit cadf021 Merge: 9b8fd58 3926f7b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 08:18:10 2023 -0400 Merge pull request rancher#11 from crobby/migrateimports Fixing imports commit 3926f7b Author: Chad Roberts <chad.roberts@suse.com> Date: Sat Aug 5 07:45:25 2023 -0400 Fixing imports commit 9b8fd58 Merge: de38ffe 26dd505 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 17:10:43 2023 -0400 Merge pull request rancher#10 from crobby/dntokens Fix tokens going to local principal commit 26dd505 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 17:08:20 2023 -0400 Fix tokens going to local principal commit de38ffe Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 15:36:12 2023 -0400 Cleanup debug/info logs somewhat commit 1581b5d Merge: 5dfcda0 29c87eb Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:56:22 2023 -0400 Merge pull request rancher#9 from crobby/linter2 More cleaning up lint commit 29c87eb Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:54:40 2023 -0400 More cleaning up lint commit 5dfcda0 Merge: a119663 d37ef2f Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:49:55 2023 -0400 Merge pull request rancher#8 from crobby/linter Cleaning up lint commit d37ef2f Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:47:44 2023 -0400 Cleaning up lint commit a119663 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:38:46 2023 -0400 Add an option to automatically delete missing-guid users This is only available when running the standalone script. At Rancher startup this option is set to false, so missing users will be logged instead and require manual intervention. commit 60f31f8 Merge: 7e620d5 9d82578 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 13:22:56 2023 -0400 Merge pull request rancher#7 from crobby/0805-migration Update migration start logic so an automated run will only happen if another run has not completed commit 9d82578 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 12:12:56 2023 -0400 Update migration start logic so an automated run will only happen if another run has not completed commit 7e620d5 Merge: 30c9f64 6c352a5 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:26:52 2023 -0400 Merge pull request rancher#4 from crobby/migrateatstart Add guid migration to rancher startup commit 30c9f64 Merge: b9aa392 72895b4 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:10:58 2023 -0400 Merge pull request rancher#5 from crobby/0803-migration Make sure annotations/labels are not nil commit 72895b4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 16:58:56 2023 -0400 Make sure annotations/labels are not nil commit b9aa392 Merge: 79762cb 7546cdf Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 10:43:30 2023 -0400 Merge pull request rancher#6 from crobby/0804-migration Fix crtb, prtb collection and add token collection/migration commit 7546cdf Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 08:59:54 2023 -0400 Fix crtb, prtb collection and add token collection/migration commit 79762cb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 3 18:00:53 2023 -0400 Collect CRTBs and PRTBs in a single pass commit b6b6085 Merge: 3de5aa3 b3acab9 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 3 11:44:13 2023 -0400 Merge pull request rancher#3 from crobby/0802-2migration Adding annotation/labels for migrated objects also blocking login while migration is active commit b3acab9 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 11:37:16 2023 -0400 Update role for SA commit 673e765 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 09:33:45 2023 -0400 Blocking login while migration is running commit 6c352a5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 13:42:33 2023 -0400 Add guid migration to rancher startup commit 840c5a7 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 12:20:41 2023 -0400 Adding annotation/labels for migrated objects commit 3de5aa3 Merge: 5dc7bd7 04ea1ce Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 2 09:57:48 2023 -0400 Merge pull request rancher#2 from crobby/0802migration Fix status function and use user copies in workUnit slices commit 04ea1ce Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 1 18:02:19 2023 -0400 Fixing status function and using copies of users in workUnit slices commit 5dc7bd7 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:29:15 2023 -0400 Skip over configmap updates for now, just to get the script running commit ac3afe6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:19:52 2023 -0400 Massively overhaul main loop, check for and handle duplicate users This is largely untested because I'm having some trouble with the configmaps code, but I wanted to get this committed before I start troubleshooting commit 5295f8f Merge: 29f9332 552e73f Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 1 08:58:41 2023 -0400 Merge pull request rancher#1 from crobby/tokenunmigrate Additional unmigration functionality commit 552e73f Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Jul 31 13:22:26 2023 -0400 Additional unmigration functionality commit 29f9332 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 17:30:10 2023 -0400 Actually perform the GUID -> DN migration on the happy path And it works too! Thank goodness. Now we mostly need to clean up the logic and handle a few dozen edge cases. commit 62a6747 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 12:53:43 2023 -0400 Cleanup the logs a bit, flatten the central logic with early exits commit ac20a2c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 09:58:54 2023 -0400 Switch to using the scaledContext for everything Since it can do all the lookups we need, it seems silly to setup and use two different interfaces to the same underlying datastore. The UnstructuredClient is the only way we can read AD configuration right now, and we need that info, so let's stick to that method. commit 18b39d3 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Jul 28 17:38:27 2023 -0400 First pass at migration scaffolding, enough to do GUID -> DN lookups There is still much work to do, but at the very least we can read the relevant auth configuration details from k8s and use those details to make LDAP queries, and that's nearly all of what we need to perform the migration.
crobby
pushed a commit
to crobby/rancher
that referenced
this issue
Aug 25, 2023
Squashed commit of the following: commit 5b32df6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:59:35 2023 -0400 Turns out the token.userPrincipal.UID is not normally set commit 064526f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:12:17 2023 -0400 Pull token fields from the ldap attributes instead of the old user commit e33bba9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 10:11:57 2023 -0400 Outdent returns to make drone happy commit 6c084df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 09:01:45 2023 -0400 Squashed commit of the following: commit 3db22eb Merge: 8039207 552fb84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:57:01 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 8039207 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:56:53 2023 -0400 tiny, tiny fix to logging commit 552fb84 Merge: ea68517 99a1814 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 17 07:39:00 2023 -0400 Merge pull request rancher#30 from crobby/migrationreview31 Outdent else blocks to make lint happy commit 99a1814 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 17 05:00:47 2023 -0400 Outdent else blocks to make lint happy commit ea68517 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 20:28:14 2023 -0400 Apply exponential retry logic to GRB and Token migrations Also, like *RTBs, these are considered non-fatal if a permanent error of some sort occurs. We continue to migrate the user anyway. commit 4a2ae0b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 19:24:42 2023 -0400 For CRTB/PRTBs, rework error handling to gracefully retry In particular, this treats internal errors (usually related to webhook timeouts) as transient, and retries them with a little bit of exponential backoff. Furthermore, after reviewing some scenarios with Michael, we've decided to consider non-internal errors from the webhook as non-fatal in terms of continuing to process the individual user. There are a few situations where old bindings to disabled templates would otherwise block users from migrating, and this permits those to have a better chance of overall success. commit 35d647c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:58:50 2023 -0400 When merging user tokens, copy over all relevant principal fields These aren't used for anything that I'm aware of, so this is really more just for consistency, since we want the two to be fully paired. commit f3e8094 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:52:15 2023 -0400 Cleanup error handling, consider AD retrieval to be a harder error commit 90f2ec1 Merge: ffcec58 b56138b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:28 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ffcec58 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:10 2023 -0400 ... once. Add the DN-based principal once. commit b56138b Merge: 78a66e0 bfb7176 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:45 2023 -0400 Merge pull request rancher#29 from crobby/migrationreview25 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit 78a66e0 Merge: edf3535 df507b5 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:24 2023 -0400 Merge pull request rancher#28 from crobby/migrationreview24 Remove unnecessary json marshal/unmarshal commit edf3535 Merge: b93e6d0 12020af Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:10 2023 -0400 Merge pull request rancher#27 from crobby/migrationreview23 Give the job pod a chance to come up before tailing the log commit b93e6d0 Merge: a2c2acb 58a0a1d Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:52 2023 -0400 Merge pull request rancher#26 from crobby/migrationreview22 Now using AuthConfig annotation as source of truth to block login during migration commit a2c2acb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:06 2023 -0400 Rework allowed user migration to handle duplicates and missing users commit bfb7176 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 14:38:22 2023 -0400 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit df507b5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:38:39 2023 -0400 Remove unnecessary json marshal/unmarshal commit 12020af Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:01:18 2023 -0400 Give the job pod a chance to come up before tailing the log commit 58a0a1d Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 12:50:57 2023 -0400 Now using AuthConfig annotation as source of truth to block login during migration commit 3ef3fb0 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:27:23 2023 -0400 Wait to do the AuthConfig principals until after updating users This kicks off some rancher-side tasks based on the updated list, and we'd really like to make sure that those user changes have been made in advance just for sanity purposes. commit b29bfb8 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:25:30 2023 -0400 When collecting duplicates, we need to track the workunit index commit df0307e Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:23:47 2023 -0400 Have the dry run guard writing new principal IDs This is mostly just to make the code clearer and more obvious. The safety is redundant, as the dry run also blocks making changes to the user object later. commit 59bafdf Merge: 2dd5250 2473062 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:12:08 2023 -0400 Merge pull request rancher#25 from crobby/migrationreview21 Append copy of user rather than pointer to duplicate list commit 2473062 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 08:00:41 2023 -0400 append copy of user rather than pointer to duplicate list commit 2dd5250 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:48:34 2023 -0400 Explicitly check to see if AD is disabled, and exit success in this case commit 4a3aa80 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:00:25 2023 -0400 Actually *use* the final migration status commit 255ef68 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:36:19 2023 -0400 Add uuid-unmigration script, prevent AD logins during execution Squashed commit of the following: commit c2bb101 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:13:12 2023 -0400 Add a generic failure status, defer restoring logins on failure states commit f9c0398 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 13:21:29 2023 -0400 Permit retries (with backoff) when opening the LDAP connection Previously we were considering a failure during open (initial or otherwise) to be a hard, script-ending, permanent failure. That's frankly a bit silly, networks can be tempermental, so this fixes that somewhat. Notably, I can't seem to find any way to check the status of the connection on the lConn object, so we're tracking that manually using a tiny little state object. If there's a cleaner way to inspect this state I am all ears, but I don't think it's a majorly big deal. (Elsewhere in Rancher we don't try to share the ldap connection generally, but here it is a big performance boost, so it is worth the extra trouble.) commit b293d62 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:54:43 2023 -0400 Rework token logic to mirror *RTBs This both collects and processes tokens that the old logic would have missed, and is also considerably more efficient, now needing to scan the list of workunits and the list of tokens just once. commit fcd2b34 Merge: 005f102 3bdea12 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:12:36 2023 -0400 Merge pull request rancher#24 from crobby/migrationreview17 Fixing names to make ci happy commit 3bdea12 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 12:09:22 2023 -0400 Fixing names to make ci happy commit 005f102 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:01:31 2023 -0400 Missing users are Infof, not Errorf commit 540e494 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:10:27 2023 -0400 Don't create/update the configmap object in dry run mode What part of "dry run" did we forget, hrm? commit 9ced565 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:00:51 2023 -0400 If the config map is not found, it's fine. (Panic otherwise.) commit 80ea848 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 10:53:30 2023 -0400 Add logic to migrate list of allowed users commit c12dcef Merge: 33f494a ce1feb4 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:25:53 2023 -0400 Merge pull request rancher#23 from crobby/migrationreview14 Another round of updates commit 33f494a Merge: b897e47 e944b57 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:13:15 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit b897e47 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:12:51 2023 -0400 Rework CRTB,PRTB collection, add GRB migration logic commit ce1feb4 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 07:15:24 2023 -0400 Echoing the set options at the end of the banner commit 089412c Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:44:43 2023 -0400 Adding additional information to README commit a7c9484 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:38:19 2023 -0400 Include agent image location in banner commit 8854263 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 16:31:44 2023 -0400 Mirror script status to authconfig commit 5bc29d5 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 12:50:13 2023 -0400 Update script status codes commit e944b57 Merge: 14c5f72 80e928b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:58 2023 -0400 Merge pull request rancher#22 from crobby/migrationreview13 More updates commit 14c5f72 Merge: a3e85de 516bdeb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:03 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit a3e85de Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:35:46 2023 -0400 Break out migration logic into a bunch of smaller files commit 80e928b Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:51:39 2023 -0400 Use configmap cache instead of client commit 516bdeb Merge: a899779 f8369c8 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:13:56 2023 -0400 Merge pull request rancher#21 from crobby/migrationreview12 Display banner before doing version check commit f8369c8 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:12:31 2023 -0400 Display banner before doing version check commit a899779 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:08:24 2023 -0400 Update cleanup/ad-guid-README.md Co-authored-by: Michael Bolot <michael.bolot@suse.com> commit 4d09212 Merge: c110ae9 92483fa Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 09:58:56 2023 -0400 Merge pull request rancher#19 from crobby/migrationreview9 Removing unused error type check commit 92483fa Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 09:51:18 2023 -0400 Removing unused error type check commit c110ae9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:51:16 2023 -0400 goimports the things commit 7691146 Merge: 44d2375 6453484 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:39 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 6453484 Merge: baf84bf 50286a2 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:32 2023 -0400 Merge pull request rancher#18 from crobby/migrationreview7 Fixing error checking commit 44d2375 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:13:58 2023 -0400 Use wait's exponential backoff primitive instead of manual sleeps commit 50286a2 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 16:27:48 2023 -0400 Fixing error checking commit baf84bf Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:39:13 2023 -0400 Only yell if the user is doing a non-dry-run on v2.7.5 commit eed1416 Merge: 9a71e38 ad00983 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:53 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 9a71e38 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:08 2023 -0400 Cleanup timeout messaging, lower job start timeout to 5 minutes I misunderstood the bash logic when I first extended that to one hour. 5 minutes for an agent download is somewhat more sensible. commit ad00983 Merge: 4e18baa 344a05d Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:34:29 2023 -0400 Merge pull request rancher#17 from crobby/migrationreview6 Additional changes after review commit 344a05d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 14:16:55 2023 -0400 Adding version check for v2.7.5 before doing anything commit 682444d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 13:50:05 2023 -0400 Fix-up README for updated usage commit 4e18baa Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:54:15 2023 -0400 Spawn relevant resources in the cattle-system namespace commit f96eb3a Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:12:33 2023 -0400 Move the YAML configuration file into the bash script This dodges the whole "fetch it from a weird URL" thing, and also makes the script a self-contained single file, which is much nicer for support to deal with. commit 275f42b Merge: 4c98764 b99cab4 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 11:16:41 2023 -0400 Merge pull request rancher#16 from crobby/migrationreview5 More post review updates commit b99cab4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 09:53:57 2023 -0400 Fixing up handling of command line options and args commit 4f6da40 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:49:20 2023 -0400 Fixing up LdapFoundDuplicateGUID name commit 9f577f6 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:31:20 2023 -0400 Adding percentage done indicator to status config map commit 43f19e4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:06:02 2023 -0400 Adding lists of special status users to configmap commit fa9979e Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 06:33:46 2023 -0400 Adding rancher-cleanup label to all cleanup objects commit 4c98764 Merge: 2d59ac6 c301303 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:38:29 2023 -0400 Merge pull request rancher#15 from crobby/migrationreview4 Post review updates commit c301303 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:33:39 2023 -0400 Updated isGUID function commit 2d59ac6 Merge: c0cdc07 86330c6 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:14:48 2023 -0400 Merge pull request rancher#14 from crobby/migrationreview3 Migration review updates 3 commit c0cdc07 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:12:22 2023 -0400 Log if we need to skip a CRTB/PRTB due to the user not existing This feels like the safer option versus applying permissions that none of the users we've collected actually have, even with the GUID/DN matching. This situation should be relatively uncommon, as Rancher usually cleans these up when users are deleted, but with the GUID duplicate bug I'm not sure how successful that will have been in practice. Best to be safe (and noisy) commit 86330c6 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:09:05 2023 -0400 Updating SA permissions for nonResourceURLs commit 4ae2d58 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 12:12:19 2023 -0400 Seeding README, adding script banner commit f8c941b Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:20:10 2023 -0400 Token collection checking userID and now setting userID and label for token updates commit e742102 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:03:04 2023 -0400 Adding additional dry-run logging information commit dc46114 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 16:57:02 2023 -0400 Rework CRTB/PRTB collection to check usernames, run through list once There are still nested for loops in here, but they are a bit more hidden :P commit ad32ccd Merge: ccb0b84 cb98c12 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:52:25 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ccb0b84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:50:27 2023 -0400 Break out the user modification flow into separate functions This mostly cleans up the main loop, but it also separates concerns and makes the smaller bits of logic easier to find and follow. commit aa41893 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:19:08 2023 -0400 Move user principal printing into its respective utility function commit ef909ab Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:12:05 2023 -0400 Respect the adConfig's UserObjectClass when performing a GUID lookup This is for parity with the auth provider; most AD configurations shouldn't have changed this from the default. commit 3963205 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:44:10 2023 -0400 Consider multiple users with the same GUID as a hard error This shouldn't be possible in practice, so it almost certainly indicates either a configuration error, or something wrong on the AD side of things. Either way we will refuse to process any user that trips this logic, and complain about it quite loudly. commit 0cebb89 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:27:24 2023 -0400 We don't need the scope, so simplify -> getExternalId commit da7ef22 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:11:41 2023 -0400 Start the scaledContext. Don't give it managers it doesn't need commit a60b144 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:34:25 2023 -0400 Remove the ratelimiting exception. Prefer safety over speed We need to check the performance ramifications of this during testing, but considering that we will almost certainly be iterating over hundreds of users, we should probably let k8s itself rate limit us so we don't overwhelm whatever is running the control plane. That might otherwise be a nasty situation, especially for stuff like AKS and GKE. commit 16715df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:32:57 2023 -0400 For bonus safety, redundantly check for dryRun here The logic up top should make this check unnecessary, but we want to be extra certain that in dryRun mode no changes are made, so we'll explicitly guard on it every time. This protects the code less from itself, and more from future modifications. commit cb98c12 Merge: e17d56f 4d2f735 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:20:06 2023 -0400 Merge pull request rancher#13 from crobby/migrationreview2 More updates based on review comments commit 4d2f735 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 8 10:17:38 2023 -0400 More updates based on review comments commit e17d56f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:38:59 2023 -0400 EscapeUUID -> escapeUUID commit 139ce3c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:37:34 2023 -0400 Relocate environment variable use to the agent-specific code path commit 795c94b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:33:13 2023 -0400 Remove unnecessary namespace from cluster role definitions commit 01ea868 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:30:53 2023 -0400 One minute is *awfully optimistic.* Let's be more realistic commit b9d4487 Merge: 17250da 0efbb02 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:21:42 2023 -0400 Merge pull request rancher#12 from crobby/migrationreview Update based on review comments commit 0efbb02 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 7 15:55:46 2023 -0400 Update based on review comments commit 17250da Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 10:29:05 2023 -0400 Don't hide the migration script from windows agents ... which in hindsight are probably somewhat likely to be using the Active Directory auth provider. commit cadf021 Merge: 9b8fd58 3926f7b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 08:18:10 2023 -0400 Merge pull request rancher#11 from crobby/migrateimports Fixing imports commit 3926f7b Author: Chad Roberts <chad.roberts@suse.com> Date: Sat Aug 5 07:45:25 2023 -0400 Fixing imports commit 9b8fd58 Merge: de38ffe 26dd505 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 17:10:43 2023 -0400 Merge pull request rancher#10 from crobby/dntokens Fix tokens going to local principal commit 26dd505 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 17:08:20 2023 -0400 Fix tokens going to local principal commit de38ffe Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 15:36:12 2023 -0400 Cleanup debug/info logs somewhat commit 1581b5d Merge: 5dfcda0 29c87eb Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:56:22 2023 -0400 Merge pull request rancher#9 from crobby/linter2 More cleaning up lint commit 29c87eb Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:54:40 2023 -0400 More cleaning up lint commit 5dfcda0 Merge: a119663 d37ef2f Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:49:55 2023 -0400 Merge pull request rancher#8 from crobby/linter Cleaning up lint commit d37ef2f Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:47:44 2023 -0400 Cleaning up lint commit a119663 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:38:46 2023 -0400 Add an option to automatically delete missing-guid users This is only available when running the standalone script. At Rancher startup this option is set to false, so missing users will be logged instead and require manual intervention. commit 60f31f8 Merge: 7e620d5 9d82578 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 13:22:56 2023 -0400 Merge pull request rancher#7 from crobby/0805-migration Update migration start logic so an automated run will only happen if another run has not completed commit 9d82578 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 12:12:56 2023 -0400 Update migration start logic so an automated run will only happen if another run has not completed commit 7e620d5 Merge: 30c9f64 6c352a5 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:26:52 2023 -0400 Merge pull request rancher#4 from crobby/migrateatstart Add guid migration to rancher startup commit 30c9f64 Merge: b9aa392 72895b4 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:10:58 2023 -0400 Merge pull request rancher#5 from crobby/0803-migration Make sure annotations/labels are not nil commit 72895b4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 16:58:56 2023 -0400 Make sure annotations/labels are not nil commit b9aa392 Merge: 79762cb 7546cdf Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 10:43:30 2023 -0400 Merge pull request rancher#6 from crobby/0804-migration Fix crtb, prtb collection and add token collection/migration commit 7546cdf Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 08:59:54 2023 -0400 Fix crtb, prtb collection and add token collection/migration commit 79762cb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 3 18:00:53 2023 -0400 Collect CRTBs and PRTBs in a single pass commit b6b6085 Merge: 3de5aa3 b3acab9 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 3 11:44:13 2023 -0400 Merge pull request rancher#3 from crobby/0802-2migration Adding annotation/labels for migrated objects also blocking login while migration is active commit b3acab9 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 11:37:16 2023 -0400 Update role for SA commit 673e765 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 09:33:45 2023 -0400 Blocking login while migration is running commit 6c352a5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 13:42:33 2023 -0400 Add guid migration to rancher startup commit 840c5a7 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 12:20:41 2023 -0400 Adding annotation/labels for migrated objects commit 3de5aa3 Merge: 5dc7bd7 04ea1ce Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 2 09:57:48 2023 -0400 Merge pull request rancher#2 from crobby/0802migration Fix status function and use user copies in workUnit slices commit 04ea1ce Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 1 18:02:19 2023 -0400 Fixing status function and using copies of users in workUnit slices commit 5dc7bd7 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:29:15 2023 -0400 Skip over configmap updates for now, just to get the script running commit ac3afe6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:19:52 2023 -0400 Massively overhaul main loop, check for and handle duplicate users This is largely untested because I'm having some trouble with the configmaps code, but I wanted to get this committed before I start troubleshooting commit 5295f8f Merge: 29f9332 552e73f Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 1 08:58:41 2023 -0400 Merge pull request rancher#1 from crobby/tokenunmigrate Additional unmigration functionality commit 552e73f Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Jul 31 13:22:26 2023 -0400 Additional unmigration functionality commit 29f9332 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 17:30:10 2023 -0400 Actually perform the GUID -> DN migration on the happy path And it works too! Thank goodness. Now we mostly need to clean up the logic and handle a few dozen edge cases. commit 62a6747 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 12:53:43 2023 -0400 Cleanup the logs a bit, flatten the central logic with early exits commit ac20a2c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 09:58:54 2023 -0400 Switch to using the scaledContext for everything Since it can do all the lookups we need, it seems silly to setup and use two different interfaces to the same underlying datastore. The UnstructuredClient is the only way we can read AD configuration right now, and we need that info, so let's stick to that method. commit 18b39d3 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Jul 28 17:38:27 2023 -0400 First pass at migration scaffolding, enough to do GUID -> DN lookups There is still much work to do, but at the very least we can read the relevant auth configuration details from k8s and use those details to make LDAP queries, and that's nearly all of what we need to perform the migration.
crobby
pushed a commit
to crobby/rancher
that referenced
this issue
Aug 25, 2023
Squashed commit of the following: commit 5b32df6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:59:35 2023 -0400 Turns out the token.userPrincipal.UID is not normally set commit 064526f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:12:17 2023 -0400 Pull token fields from the ldap attributes instead of the old user commit e33bba9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 10:11:57 2023 -0400 Outdent returns to make drone happy commit 6c084df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 09:01:45 2023 -0400 Squashed commit of the following: commit 3db22eb Merge: 8039207 552fb84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:57:01 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 8039207 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:56:53 2023 -0400 tiny, tiny fix to logging commit 552fb84 Merge: ea68517 99a1814 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 17 07:39:00 2023 -0400 Merge pull request rancher#30 from crobby/migrationreview31 Outdent else blocks to make lint happy commit 99a1814 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 17 05:00:47 2023 -0400 Outdent else blocks to make lint happy commit ea68517 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 20:28:14 2023 -0400 Apply exponential retry logic to GRB and Token migrations Also, like *RTBs, these are considered non-fatal if a permanent error of some sort occurs. We continue to migrate the user anyway. commit 4a2ae0b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 19:24:42 2023 -0400 For CRTB/PRTBs, rework error handling to gracefully retry In particular, this treats internal errors (usually related to webhook timeouts) as transient, and retries them with a little bit of exponential backoff. Furthermore, after reviewing some scenarios with Michael, we've decided to consider non-internal errors from the webhook as non-fatal in terms of continuing to process the individual user. There are a few situations where old bindings to disabled templates would otherwise block users from migrating, and this permits those to have a better chance of overall success. commit 35d647c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:58:50 2023 -0400 When merging user tokens, copy over all relevant principal fields These aren't used for anything that I'm aware of, so this is really more just for consistency, since we want the two to be fully paired. commit f3e8094 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:52:15 2023 -0400 Cleanup error handling, consider AD retrieval to be a harder error commit 90f2ec1 Merge: ffcec58 b56138b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:28 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ffcec58 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:10 2023 -0400 ... once. Add the DN-based principal once. commit b56138b Merge: 78a66e0 bfb7176 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:45 2023 -0400 Merge pull request rancher#29 from crobby/migrationreview25 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit 78a66e0 Merge: edf3535 df507b5 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:24 2023 -0400 Merge pull request rancher#28 from crobby/migrationreview24 Remove unnecessary json marshal/unmarshal commit edf3535 Merge: b93e6d0 12020af Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:10 2023 -0400 Merge pull request rancher#27 from crobby/migrationreview23 Give the job pod a chance to come up before tailing the log commit b93e6d0 Merge: a2c2acb 58a0a1d Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:52 2023 -0400 Merge pull request rancher#26 from crobby/migrationreview22 Now using AuthConfig annotation as source of truth to block login during migration commit a2c2acb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:06 2023 -0400 Rework allowed user migration to handle duplicates and missing users commit bfb7176 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 14:38:22 2023 -0400 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit df507b5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:38:39 2023 -0400 Remove unnecessary json marshal/unmarshal commit 12020af Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:01:18 2023 -0400 Give the job pod a chance to come up before tailing the log commit 58a0a1d Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 12:50:57 2023 -0400 Now using AuthConfig annotation as source of truth to block login during migration commit 3ef3fb0 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:27:23 2023 -0400 Wait to do the AuthConfig principals until after updating users This kicks off some rancher-side tasks based on the updated list, and we'd really like to make sure that those user changes have been made in advance just for sanity purposes. commit b29bfb8 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:25:30 2023 -0400 When collecting duplicates, we need to track the workunit index commit df0307e Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:23:47 2023 -0400 Have the dry run guard writing new principal IDs This is mostly just to make the code clearer and more obvious. The safety is redundant, as the dry run also blocks making changes to the user object later. commit 59bafdf Merge: 2dd5250 2473062 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:12:08 2023 -0400 Merge pull request rancher#25 from crobby/migrationreview21 Append copy of user rather than pointer to duplicate list commit 2473062 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 08:00:41 2023 -0400 append copy of user rather than pointer to duplicate list commit 2dd5250 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:48:34 2023 -0400 Explicitly check to see if AD is disabled, and exit success in this case commit 4a3aa80 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:00:25 2023 -0400 Actually *use* the final migration status commit 255ef68 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:36:19 2023 -0400 Add uuid-unmigration script, prevent AD logins during execution Squashed commit of the following: commit c2bb101 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:13:12 2023 -0400 Add a generic failure status, defer restoring logins on failure states commit f9c0398 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 13:21:29 2023 -0400 Permit retries (with backoff) when opening the LDAP connection Previously we were considering a failure during open (initial or otherwise) to be a hard, script-ending, permanent failure. That's frankly a bit silly, networks can be tempermental, so this fixes that somewhat. Notably, I can't seem to find any way to check the status of the connection on the lConn object, so we're tracking that manually using a tiny little state object. If there's a cleaner way to inspect this state I am all ears, but I don't think it's a majorly big deal. (Elsewhere in Rancher we don't try to share the ldap connection generally, but here it is a big performance boost, so it is worth the extra trouble.) commit b293d62 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:54:43 2023 -0400 Rework token logic to mirror *RTBs This both collects and processes tokens that the old logic would have missed, and is also considerably more efficient, now needing to scan the list of workunits and the list of tokens just once. commit fcd2b34 Merge: 005f102 3bdea12 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:12:36 2023 -0400 Merge pull request rancher#24 from crobby/migrationreview17 Fixing names to make ci happy commit 3bdea12 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 12:09:22 2023 -0400 Fixing names to make ci happy commit 005f102 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:01:31 2023 -0400 Missing users are Infof, not Errorf commit 540e494 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:10:27 2023 -0400 Don't create/update the configmap object in dry run mode What part of "dry run" did we forget, hrm? commit 9ced565 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:00:51 2023 -0400 If the config map is not found, it's fine. (Panic otherwise.) commit 80ea848 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 10:53:30 2023 -0400 Add logic to migrate list of allowed users commit c12dcef Merge: 33f494a ce1feb4 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:25:53 2023 -0400 Merge pull request rancher#23 from crobby/migrationreview14 Another round of updates commit 33f494a Merge: b897e47 e944b57 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:13:15 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit b897e47 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:12:51 2023 -0400 Rework CRTB,PRTB collection, add GRB migration logic commit ce1feb4 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 07:15:24 2023 -0400 Echoing the set options at the end of the banner commit 089412c Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:44:43 2023 -0400 Adding additional information to README commit a7c9484 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:38:19 2023 -0400 Include agent image location in banner commit 8854263 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 16:31:44 2023 -0400 Mirror script status to authconfig commit 5bc29d5 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 12:50:13 2023 -0400 Update script status codes commit e944b57 Merge: 14c5f72 80e928b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:58 2023 -0400 Merge pull request rancher#22 from crobby/migrationreview13 More updates commit 14c5f72 Merge: a3e85de 516bdeb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:03 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit a3e85de Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:35:46 2023 -0400 Break out migration logic into a bunch of smaller files commit 80e928b Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:51:39 2023 -0400 Use configmap cache instead of client commit 516bdeb Merge: a899779 f8369c8 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:13:56 2023 -0400 Merge pull request rancher#21 from crobby/migrationreview12 Display banner before doing version check commit f8369c8 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:12:31 2023 -0400 Display banner before doing version check commit a899779 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:08:24 2023 -0400 Update cleanup/ad-guid-README.md Co-authored-by: Michael Bolot <michael.bolot@suse.com> commit 4d09212 Merge: c110ae9 92483fa Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 09:58:56 2023 -0400 Merge pull request rancher#19 from crobby/migrationreview9 Removing unused error type check commit 92483fa Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 09:51:18 2023 -0400 Removing unused error type check commit c110ae9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:51:16 2023 -0400 goimports the things commit 7691146 Merge: 44d2375 6453484 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:39 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 6453484 Merge: baf84bf 50286a2 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:32 2023 -0400 Merge pull request rancher#18 from crobby/migrationreview7 Fixing error checking commit 44d2375 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:13:58 2023 -0400 Use wait's exponential backoff primitive instead of manual sleeps commit 50286a2 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 16:27:48 2023 -0400 Fixing error checking commit baf84bf Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:39:13 2023 -0400 Only yell if the user is doing a non-dry-run on v2.7.5 commit eed1416 Merge: 9a71e38 ad00983 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:53 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 9a71e38 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:08 2023 -0400 Cleanup timeout messaging, lower job start timeout to 5 minutes I misunderstood the bash logic when I first extended that to one hour. 5 minutes for an agent download is somewhat more sensible. commit ad00983 Merge: 4e18baa 344a05d Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:34:29 2023 -0400 Merge pull request rancher#17 from crobby/migrationreview6 Additional changes after review commit 344a05d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 14:16:55 2023 -0400 Adding version check for v2.7.5 before doing anything commit 682444d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 13:50:05 2023 -0400 Fix-up README for updated usage commit 4e18baa Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:54:15 2023 -0400 Spawn relevant resources in the cattle-system namespace commit f96eb3a Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:12:33 2023 -0400 Move the YAML configuration file into the bash script This dodges the whole "fetch it from a weird URL" thing, and also makes the script a self-contained single file, which is much nicer for support to deal with. commit 275f42b Merge: 4c98764 b99cab4 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 11:16:41 2023 -0400 Merge pull request rancher#16 from crobby/migrationreview5 More post review updates commit b99cab4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 09:53:57 2023 -0400 Fixing up handling of command line options and args commit 4f6da40 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:49:20 2023 -0400 Fixing up LdapFoundDuplicateGUID name commit 9f577f6 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:31:20 2023 -0400 Adding percentage done indicator to status config map commit 43f19e4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:06:02 2023 -0400 Adding lists of special status users to configmap commit fa9979e Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 06:33:46 2023 -0400 Adding rancher-cleanup label to all cleanup objects commit 4c98764 Merge: 2d59ac6 c301303 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:38:29 2023 -0400 Merge pull request rancher#15 from crobby/migrationreview4 Post review updates commit c301303 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:33:39 2023 -0400 Updated isGUID function commit 2d59ac6 Merge: c0cdc07 86330c6 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:14:48 2023 -0400 Merge pull request rancher#14 from crobby/migrationreview3 Migration review updates 3 commit c0cdc07 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:12:22 2023 -0400 Log if we need to skip a CRTB/PRTB due to the user not existing This feels like the safer option versus applying permissions that none of the users we've collected actually have, even with the GUID/DN matching. This situation should be relatively uncommon, as Rancher usually cleans these up when users are deleted, but with the GUID duplicate bug I'm not sure how successful that will have been in practice. Best to be safe (and noisy) commit 86330c6 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:09:05 2023 -0400 Updating SA permissions for nonResourceURLs commit 4ae2d58 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 12:12:19 2023 -0400 Seeding README, adding script banner commit f8c941b Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:20:10 2023 -0400 Token collection checking userID and now setting userID and label for token updates commit e742102 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:03:04 2023 -0400 Adding additional dry-run logging information commit dc46114 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 16:57:02 2023 -0400 Rework CRTB/PRTB collection to check usernames, run through list once There are still nested for loops in here, but they are a bit more hidden :P commit ad32ccd Merge: ccb0b84 cb98c12 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:52:25 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ccb0b84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:50:27 2023 -0400 Break out the user modification flow into separate functions This mostly cleans up the main loop, but it also separates concerns and makes the smaller bits of logic easier to find and follow. commit aa41893 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:19:08 2023 -0400 Move user principal printing into its respective utility function commit ef909ab Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:12:05 2023 -0400 Respect the adConfig's UserObjectClass when performing a GUID lookup This is for parity with the auth provider; most AD configurations shouldn't have changed this from the default. commit 3963205 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:44:10 2023 -0400 Consider multiple users with the same GUID as a hard error This shouldn't be possible in practice, so it almost certainly indicates either a configuration error, or something wrong on the AD side of things. Either way we will refuse to process any user that trips this logic, and complain about it quite loudly. commit 0cebb89 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:27:24 2023 -0400 We don't need the scope, so simplify -> getExternalId commit da7ef22 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:11:41 2023 -0400 Start the scaledContext. Don't give it managers it doesn't need commit a60b144 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:34:25 2023 -0400 Remove the ratelimiting exception. Prefer safety over speed We need to check the performance ramifications of this during testing, but considering that we will almost certainly be iterating over hundreds of users, we should probably let k8s itself rate limit us so we don't overwhelm whatever is running the control plane. That might otherwise be a nasty situation, especially for stuff like AKS and GKE. commit 16715df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:32:57 2023 -0400 For bonus safety, redundantly check for dryRun here The logic up top should make this check unnecessary, but we want to be extra certain that in dryRun mode no changes are made, so we'll explicitly guard on it every time. This protects the code less from itself, and more from future modifications. commit cb98c12 Merge: e17d56f 4d2f735 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:20:06 2023 -0400 Merge pull request rancher#13 from crobby/migrationreview2 More updates based on review comments commit 4d2f735 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 8 10:17:38 2023 -0400 More updates based on review comments commit e17d56f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:38:59 2023 -0400 EscapeUUID -> escapeUUID commit 139ce3c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:37:34 2023 -0400 Relocate environment variable use to the agent-specific code path commit 795c94b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:33:13 2023 -0400 Remove unnecessary namespace from cluster role definitions commit 01ea868 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:30:53 2023 -0400 One minute is *awfully optimistic.* Let's be more realistic commit b9d4487 Merge: 17250da 0efbb02 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:21:42 2023 -0400 Merge pull request rancher#12 from crobby/migrationreview Update based on review comments commit 0efbb02 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 7 15:55:46 2023 -0400 Update based on review comments commit 17250da Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 10:29:05 2023 -0400 Don't hide the migration script from windows agents ... which in hindsight are probably somewhat likely to be using the Active Directory auth provider. commit cadf021 Merge: 9b8fd58 3926f7b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 08:18:10 2023 -0400 Merge pull request rancher#11 from crobby/migrateimports Fixing imports commit 3926f7b Author: Chad Roberts <chad.roberts@suse.com> Date: Sat Aug 5 07:45:25 2023 -0400 Fixing imports commit 9b8fd58 Merge: de38ffe 26dd505 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 17:10:43 2023 -0400 Merge pull request rancher#10 from crobby/dntokens Fix tokens going to local principal commit 26dd505 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 17:08:20 2023 -0400 Fix tokens going to local principal commit de38ffe Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 15:36:12 2023 -0400 Cleanup debug/info logs somewhat commit 1581b5d Merge: 5dfcda0 29c87eb Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:56:22 2023 -0400 Merge pull request rancher#9 from crobby/linter2 More cleaning up lint commit 29c87eb Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:54:40 2023 -0400 More cleaning up lint commit 5dfcda0 Merge: a119663 d37ef2f Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:49:55 2023 -0400 Merge pull request rancher#8 from crobby/linter Cleaning up lint commit d37ef2f Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:47:44 2023 -0400 Cleaning up lint commit a119663 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:38:46 2023 -0400 Add an option to automatically delete missing-guid users This is only available when running the standalone script. At Rancher startup this option is set to false, so missing users will be logged instead and require manual intervention. commit 60f31f8 Merge: 7e620d5 9d82578 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 13:22:56 2023 -0400 Merge pull request rancher#7 from crobby/0805-migration Update migration start logic so an automated run will only happen if another run has not completed commit 9d82578 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 12:12:56 2023 -0400 Update migration start logic so an automated run will only happen if another run has not completed commit 7e620d5 Merge: 30c9f64 6c352a5 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:26:52 2023 -0400 Merge pull request rancher#4 from crobby/migrateatstart Add guid migration to rancher startup commit 30c9f64 Merge: b9aa392 72895b4 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:10:58 2023 -0400 Merge pull request rancher#5 from crobby/0803-migration Make sure annotations/labels are not nil commit 72895b4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 16:58:56 2023 -0400 Make sure annotations/labels are not nil commit b9aa392 Merge: 79762cb 7546cdf Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 10:43:30 2023 -0400 Merge pull request rancher#6 from crobby/0804-migration Fix crtb, prtb collection and add token collection/migration commit 7546cdf Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 08:59:54 2023 -0400 Fix crtb, prtb collection and add token collection/migration commit 79762cb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 3 18:00:53 2023 -0400 Collect CRTBs and PRTBs in a single pass commit b6b6085 Merge: 3de5aa3 b3acab9 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 3 11:44:13 2023 -0400 Merge pull request rancher#3 from crobby/0802-2migration Adding annotation/labels for migrated objects also blocking login while migration is active commit b3acab9 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 11:37:16 2023 -0400 Update role for SA commit 673e765 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 09:33:45 2023 -0400 Blocking login while migration is running commit 6c352a5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 13:42:33 2023 -0400 Add guid migration to rancher startup commit 840c5a7 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 12:20:41 2023 -0400 Adding annotation/labels for migrated objects commit 3de5aa3 Merge: 5dc7bd7 04ea1ce Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 2 09:57:48 2023 -0400 Merge pull request rancher#2 from crobby/0802migration Fix status function and use user copies in workUnit slices commit 04ea1ce Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 1 18:02:19 2023 -0400 Fixing status function and using copies of users in workUnit slices commit 5dc7bd7 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:29:15 2023 -0400 Skip over configmap updates for now, just to get the script running commit ac3afe6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:19:52 2023 -0400 Massively overhaul main loop, check for and handle duplicate users This is largely untested because I'm having some trouble with the configmaps code, but I wanted to get this committed before I start troubleshooting commit 5295f8f Merge: 29f9332 552e73f Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 1 08:58:41 2023 -0400 Merge pull request rancher#1 from crobby/tokenunmigrate Additional unmigration functionality commit 552e73f Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Jul 31 13:22:26 2023 -0400 Additional unmigration functionality commit 29f9332 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 17:30:10 2023 -0400 Actually perform the GUID -> DN migration on the happy path And it works too! Thank goodness. Now we mostly need to clean up the logic and handle a few dozen edge cases. commit 62a6747 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 12:53:43 2023 -0400 Cleanup the logs a bit, flatten the central logic with early exits commit ac20a2c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 09:58:54 2023 -0400 Switch to using the scaledContext for everything Since it can do all the lookups we need, it seems silly to setup and use two different interfaces to the same underlying datastore. The UnstructuredClient is the only way we can read AD configuration right now, and we need that info, so let's stick to that method. commit 18b39d3 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Jul 28 17:38:27 2023 -0400 First pass at migration scaffolding, enough to do GUID -> DN lookups There is still much work to do, but at the very least we can read the relevant auth configuration details from k8s and use those details to make LDAP queries, and that's nearly all of what we need to perform the migration.
crobby
pushed a commit
to crobby/rancher
that referenced
this issue
Aug 25, 2023
Squashed commit of the following: commit 5b32df6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:59:35 2023 -0400 Turns out the token.userPrincipal.UID is not normally set commit 064526f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:12:17 2023 -0400 Pull token fields from the ldap attributes instead of the old user commit e33bba9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 10:11:57 2023 -0400 Outdent returns to make drone happy commit 6c084df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 09:01:45 2023 -0400 Squashed commit of the following: commit 3db22eb Merge: 8039207 552fb84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:57:01 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 8039207 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:56:53 2023 -0400 tiny, tiny fix to logging commit 552fb84 Merge: ea68517 99a1814 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 17 07:39:00 2023 -0400 Merge pull request rancher#30 from crobby/migrationreview31 Outdent else blocks to make lint happy commit 99a1814 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 17 05:00:47 2023 -0400 Outdent else blocks to make lint happy commit ea68517 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 20:28:14 2023 -0400 Apply exponential retry logic to GRB and Token migrations Also, like *RTBs, these are considered non-fatal if a permanent error of some sort occurs. We continue to migrate the user anyway. commit 4a2ae0b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 19:24:42 2023 -0400 For CRTB/PRTBs, rework error handling to gracefully retry In particular, this treats internal errors (usually related to webhook timeouts) as transient, and retries them with a little bit of exponential backoff. Furthermore, after reviewing some scenarios with Michael, we've decided to consider non-internal errors from the webhook as non-fatal in terms of continuing to process the individual user. There are a few situations where old bindings to disabled templates would otherwise block users from migrating, and this permits those to have a better chance of overall success. commit 35d647c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:58:50 2023 -0400 When merging user tokens, copy over all relevant principal fields These aren't used for anything that I'm aware of, so this is really more just for consistency, since we want the two to be fully paired. commit f3e8094 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:52:15 2023 -0400 Cleanup error handling, consider AD retrieval to be a harder error commit 90f2ec1 Merge: ffcec58 b56138b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:28 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ffcec58 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:10 2023 -0400 ... once. Add the DN-based principal once. commit b56138b Merge: 78a66e0 bfb7176 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:45 2023 -0400 Merge pull request rancher#29 from crobby/migrationreview25 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit 78a66e0 Merge: edf3535 df507b5 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:24 2023 -0400 Merge pull request rancher#28 from crobby/migrationreview24 Remove unnecessary json marshal/unmarshal commit edf3535 Merge: b93e6d0 12020af Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:10 2023 -0400 Merge pull request rancher#27 from crobby/migrationreview23 Give the job pod a chance to come up before tailing the log commit b93e6d0 Merge: a2c2acb 58a0a1d Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:52 2023 -0400 Merge pull request rancher#26 from crobby/migrationreview22 Now using AuthConfig annotation as source of truth to block login during migration commit a2c2acb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:06 2023 -0400 Rework allowed user migration to handle duplicates and missing users commit bfb7176 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 14:38:22 2023 -0400 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit df507b5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:38:39 2023 -0400 Remove unnecessary json marshal/unmarshal commit 12020af Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:01:18 2023 -0400 Give the job pod a chance to come up before tailing the log commit 58a0a1d Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 12:50:57 2023 -0400 Now using AuthConfig annotation as source of truth to block login during migration commit 3ef3fb0 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:27:23 2023 -0400 Wait to do the AuthConfig principals until after updating users This kicks off some rancher-side tasks based on the updated list, and we'd really like to make sure that those user changes have been made in advance just for sanity purposes. commit b29bfb8 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:25:30 2023 -0400 When collecting duplicates, we need to track the workunit index commit df0307e Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:23:47 2023 -0400 Have the dry run guard writing new principal IDs This is mostly just to make the code clearer and more obvious. The safety is redundant, as the dry run also blocks making changes to the user object later. commit 59bafdf Merge: 2dd5250 2473062 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:12:08 2023 -0400 Merge pull request rancher#25 from crobby/migrationreview21 Append copy of user rather than pointer to duplicate list commit 2473062 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 08:00:41 2023 -0400 append copy of user rather than pointer to duplicate list commit 2dd5250 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:48:34 2023 -0400 Explicitly check to see if AD is disabled, and exit success in this case commit 4a3aa80 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:00:25 2023 -0400 Actually *use* the final migration status commit 255ef68 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:36:19 2023 -0400 Add uuid-unmigration script, prevent AD logins during execution Squashed commit of the following: commit c2bb101 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:13:12 2023 -0400 Add a generic failure status, defer restoring logins on failure states commit f9c0398 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 13:21:29 2023 -0400 Permit retries (with backoff) when opening the LDAP connection Previously we were considering a failure during open (initial or otherwise) to be a hard, script-ending, permanent failure. That's frankly a bit silly, networks can be tempermental, so this fixes that somewhat. Notably, I can't seem to find any way to check the status of the connection on the lConn object, so we're tracking that manually using a tiny little state object. If there's a cleaner way to inspect this state I am all ears, but I don't think it's a majorly big deal. (Elsewhere in Rancher we don't try to share the ldap connection generally, but here it is a big performance boost, so it is worth the extra trouble.) commit b293d62 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:54:43 2023 -0400 Rework token logic to mirror *RTBs This both collects and processes tokens that the old logic would have missed, and is also considerably more efficient, now needing to scan the list of workunits and the list of tokens just once. commit fcd2b34 Merge: 005f102 3bdea12 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:12:36 2023 -0400 Merge pull request rancher#24 from crobby/migrationreview17 Fixing names to make ci happy commit 3bdea12 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 12:09:22 2023 -0400 Fixing names to make ci happy commit 005f102 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:01:31 2023 -0400 Missing users are Infof, not Errorf commit 540e494 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:10:27 2023 -0400 Don't create/update the configmap object in dry run mode What part of "dry run" did we forget, hrm? commit 9ced565 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:00:51 2023 -0400 If the config map is not found, it's fine. (Panic otherwise.) commit 80ea848 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 10:53:30 2023 -0400 Add logic to migrate list of allowed users commit c12dcef Merge: 33f494a ce1feb4 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:25:53 2023 -0400 Merge pull request rancher#23 from crobby/migrationreview14 Another round of updates commit 33f494a Merge: b897e47 e944b57 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:13:15 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit b897e47 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:12:51 2023 -0400 Rework CRTB,PRTB collection, add GRB migration logic commit ce1feb4 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 07:15:24 2023 -0400 Echoing the set options at the end of the banner commit 089412c Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:44:43 2023 -0400 Adding additional information to README commit a7c9484 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:38:19 2023 -0400 Include agent image location in banner commit 8854263 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 16:31:44 2023 -0400 Mirror script status to authconfig commit 5bc29d5 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 12:50:13 2023 -0400 Update script status codes commit e944b57 Merge: 14c5f72 80e928b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:58 2023 -0400 Merge pull request rancher#22 from crobby/migrationreview13 More updates commit 14c5f72 Merge: a3e85de 516bdeb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:03 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit a3e85de Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:35:46 2023 -0400 Break out migration logic into a bunch of smaller files commit 80e928b Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:51:39 2023 -0400 Use configmap cache instead of client commit 516bdeb Merge: a899779 f8369c8 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:13:56 2023 -0400 Merge pull request rancher#21 from crobby/migrationreview12 Display banner before doing version check commit f8369c8 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:12:31 2023 -0400 Display banner before doing version check commit a899779 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:08:24 2023 -0400 Update cleanup/ad-guid-README.md Co-authored-by: Michael Bolot <michael.bolot@suse.com> commit 4d09212 Merge: c110ae9 92483fa Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 09:58:56 2023 -0400 Merge pull request rancher#19 from crobby/migrationreview9 Removing unused error type check commit 92483fa Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 09:51:18 2023 -0400 Removing unused error type check commit c110ae9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:51:16 2023 -0400 goimports the things commit 7691146 Merge: 44d2375 6453484 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:39 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 6453484 Merge: baf84bf 50286a2 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:32 2023 -0400 Merge pull request rancher#18 from crobby/migrationreview7 Fixing error checking commit 44d2375 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:13:58 2023 -0400 Use wait's exponential backoff primitive instead of manual sleeps commit 50286a2 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 16:27:48 2023 -0400 Fixing error checking commit baf84bf Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:39:13 2023 -0400 Only yell if the user is doing a non-dry-run on v2.7.5 commit eed1416 Merge: 9a71e38 ad00983 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:53 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 9a71e38 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:08 2023 -0400 Cleanup timeout messaging, lower job start timeout to 5 minutes I misunderstood the bash logic when I first extended that to one hour. 5 minutes for an agent download is somewhat more sensible. commit ad00983 Merge: 4e18baa 344a05d Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:34:29 2023 -0400 Merge pull request rancher#17 from crobby/migrationreview6 Additional changes after review commit 344a05d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 14:16:55 2023 -0400 Adding version check for v2.7.5 before doing anything commit 682444d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 13:50:05 2023 -0400 Fix-up README for updated usage commit 4e18baa Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:54:15 2023 -0400 Spawn relevant resources in the cattle-system namespace commit f96eb3a Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:12:33 2023 -0400 Move the YAML configuration file into the bash script This dodges the whole "fetch it from a weird URL" thing, and also makes the script a self-contained single file, which is much nicer for support to deal with. commit 275f42b Merge: 4c98764 b99cab4 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 11:16:41 2023 -0400 Merge pull request rancher#16 from crobby/migrationreview5 More post review updates commit b99cab4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 09:53:57 2023 -0400 Fixing up handling of command line options and args commit 4f6da40 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:49:20 2023 -0400 Fixing up LdapFoundDuplicateGUID name commit 9f577f6 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:31:20 2023 -0400 Adding percentage done indicator to status config map commit 43f19e4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:06:02 2023 -0400 Adding lists of special status users to configmap commit fa9979e Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 06:33:46 2023 -0400 Adding rancher-cleanup label to all cleanup objects commit 4c98764 Merge: 2d59ac6 c301303 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:38:29 2023 -0400 Merge pull request rancher#15 from crobby/migrationreview4 Post review updates commit c301303 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:33:39 2023 -0400 Updated isGUID function commit 2d59ac6 Merge: c0cdc07 86330c6 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:14:48 2023 -0400 Merge pull request rancher#14 from crobby/migrationreview3 Migration review updates 3 commit c0cdc07 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:12:22 2023 -0400 Log if we need to skip a CRTB/PRTB due to the user not existing This feels like the safer option versus applying permissions that none of the users we've collected actually have, even with the GUID/DN matching. This situation should be relatively uncommon, as Rancher usually cleans these up when users are deleted, but with the GUID duplicate bug I'm not sure how successful that will have been in practice. Best to be safe (and noisy) commit 86330c6 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:09:05 2023 -0400 Updating SA permissions for nonResourceURLs commit 4ae2d58 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 12:12:19 2023 -0400 Seeding README, adding script banner commit f8c941b Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:20:10 2023 -0400 Token collection checking userID and now setting userID and label for token updates commit e742102 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:03:04 2023 -0400 Adding additional dry-run logging information commit dc46114 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 16:57:02 2023 -0400 Rework CRTB/PRTB collection to check usernames, run through list once There are still nested for loops in here, but they are a bit more hidden :P commit ad32ccd Merge: ccb0b84 cb98c12 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:52:25 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ccb0b84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:50:27 2023 -0400 Break out the user modification flow into separate functions This mostly cleans up the main loop, but it also separates concerns and makes the smaller bits of logic easier to find and follow. commit aa41893 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:19:08 2023 -0400 Move user principal printing into its respective utility function commit ef909ab Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:12:05 2023 -0400 Respect the adConfig's UserObjectClass when performing a GUID lookup This is for parity with the auth provider; most AD configurations shouldn't have changed this from the default. commit 3963205 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:44:10 2023 -0400 Consider multiple users with the same GUID as a hard error This shouldn't be possible in practice, so it almost certainly indicates either a configuration error, or something wrong on the AD side of things. Either way we will refuse to process any user that trips this logic, and complain about it quite loudly. commit 0cebb89 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:27:24 2023 -0400 We don't need the scope, so simplify -> getExternalId commit da7ef22 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:11:41 2023 -0400 Start the scaledContext. Don't give it managers it doesn't need commit a60b144 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:34:25 2023 -0400 Remove the ratelimiting exception. Prefer safety over speed We need to check the performance ramifications of this during testing, but considering that we will almost certainly be iterating over hundreds of users, we should probably let k8s itself rate limit us so we don't overwhelm whatever is running the control plane. That might otherwise be a nasty situation, especially for stuff like AKS and GKE. commit 16715df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:32:57 2023 -0400 For bonus safety, redundantly check for dryRun here The logic up top should make this check unnecessary, but we want to be extra certain that in dryRun mode no changes are made, so we'll explicitly guard on it every time. This protects the code less from itself, and more from future modifications. commit cb98c12 Merge: e17d56f 4d2f735 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:20:06 2023 -0400 Merge pull request rancher#13 from crobby/migrationreview2 More updates based on review comments commit 4d2f735 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 8 10:17:38 2023 -0400 More updates based on review comments commit e17d56f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:38:59 2023 -0400 EscapeUUID -> escapeUUID commit 139ce3c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:37:34 2023 -0400 Relocate environment variable use to the agent-specific code path commit 795c94b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:33:13 2023 -0400 Remove unnecessary namespace from cluster role definitions commit 01ea868 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:30:53 2023 -0400 One minute is *awfully optimistic.* Let's be more realistic commit b9d4487 Merge: 17250da 0efbb02 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:21:42 2023 -0400 Merge pull request rancher#12 from crobby/migrationreview Update based on review comments commit 0efbb02 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 7 15:55:46 2023 -0400 Update based on review comments commit 17250da Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 10:29:05 2023 -0400 Don't hide the migration script from windows agents ... which in hindsight are probably somewhat likely to be using the Active Directory auth provider. commit cadf021 Merge: 9b8fd58 3926f7b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 08:18:10 2023 -0400 Merge pull request rancher#11 from crobby/migrateimports Fixing imports commit 3926f7b Author: Chad Roberts <chad.roberts@suse.com> Date: Sat Aug 5 07:45:25 2023 -0400 Fixing imports commit 9b8fd58 Merge: de38ffe 26dd505 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 17:10:43 2023 -0400 Merge pull request rancher#10 from crobby/dntokens Fix tokens going to local principal commit 26dd505 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 17:08:20 2023 -0400 Fix tokens going to local principal commit de38ffe Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 15:36:12 2023 -0400 Cleanup debug/info logs somewhat commit 1581b5d Merge: 5dfcda0 29c87eb Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:56:22 2023 -0400 Merge pull request rancher#9 from crobby/linter2 More cleaning up lint commit 29c87eb Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:54:40 2023 -0400 More cleaning up lint commit 5dfcda0 Merge: a119663 d37ef2f Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:49:55 2023 -0400 Merge pull request rancher#8 from crobby/linter Cleaning up lint commit d37ef2f Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:47:44 2023 -0400 Cleaning up lint commit a119663 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:38:46 2023 -0400 Add an option to automatically delete missing-guid users This is only available when running the standalone script. At Rancher startup this option is set to false, so missing users will be logged instead and require manual intervention. commit 60f31f8 Merge: 7e620d5 9d82578 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 13:22:56 2023 -0400 Merge pull request rancher#7 from crobby/0805-migration Update migration start logic so an automated run will only happen if another run has not completed commit 9d82578 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 12:12:56 2023 -0400 Update migration start logic so an automated run will only happen if another run has not completed commit 7e620d5 Merge: 30c9f64 6c352a5 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:26:52 2023 -0400 Merge pull request rancher#4 from crobby/migrateatstart Add guid migration to rancher startup commit 30c9f64 Merge: b9aa392 72895b4 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:10:58 2023 -0400 Merge pull request rancher#5 from crobby/0803-migration Make sure annotations/labels are not nil commit 72895b4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 16:58:56 2023 -0400 Make sure annotations/labels are not nil commit b9aa392 Merge: 79762cb 7546cdf Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 10:43:30 2023 -0400 Merge pull request rancher#6 from crobby/0804-migration Fix crtb, prtb collection and add token collection/migration commit 7546cdf Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 08:59:54 2023 -0400 Fix crtb, prtb collection and add token collection/migration commit 79762cb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 3 18:00:53 2023 -0400 Collect CRTBs and PRTBs in a single pass commit b6b6085 Merge: 3de5aa3 b3acab9 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 3 11:44:13 2023 -0400 Merge pull request rancher#3 from crobby/0802-2migration Adding annotation/labels for migrated objects also blocking login while migration is active commit b3acab9 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 11:37:16 2023 -0400 Update role for SA commit 673e765 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 09:33:45 2023 -0400 Blocking login while migration is running commit 6c352a5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 13:42:33 2023 -0400 Add guid migration to rancher startup commit 840c5a7 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 12:20:41 2023 -0400 Adding annotation/labels for migrated objects commit 3de5aa3 Merge: 5dc7bd7 04ea1ce Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 2 09:57:48 2023 -0400 Merge pull request rancher#2 from crobby/0802migration Fix status function and use user copies in workUnit slices commit 04ea1ce Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 1 18:02:19 2023 -0400 Fixing status function and using copies of users in workUnit slices commit 5dc7bd7 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:29:15 2023 -0400 Skip over configmap updates for now, just to get the script running commit ac3afe6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:19:52 2023 -0400 Massively overhaul main loop, check for and handle duplicate users This is largely untested because I'm having some trouble with the configmaps code, but I wanted to get this committed before I start troubleshooting commit 5295f8f Merge: 29f9332 552e73f Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 1 08:58:41 2023 -0400 Merge pull request rancher#1 from crobby/tokenunmigrate Additional unmigration functionality commit 552e73f Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Jul 31 13:22:26 2023 -0400 Additional unmigration functionality commit 29f9332 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 17:30:10 2023 -0400 Actually perform the GUID -> DN migration on the happy path And it works too! Thank goodness. Now we mostly need to clean up the logic and handle a few dozen edge cases. commit 62a6747 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 12:53:43 2023 -0400 Cleanup the logs a bit, flatten the central logic with early exits commit ac20a2c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 09:58:54 2023 -0400 Switch to using the scaledContext for everything Since it can do all the lookups we need, it seems silly to setup and use two different interfaces to the same underlying datastore. The UnstructuredClient is the only way we can read AD configuration right now, and we need that info, so let's stick to that method. commit 18b39d3 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Jul 28 17:38:27 2023 -0400 First pass at migration scaffolding, enough to do GUID -> DN lookups There is still much work to do, but at the very least we can read the relevant auth configuration details from k8s and use those details to make LDAP queries, and that's nearly all of what we need to perform the migration.
crobby
pushed a commit
to crobby/rancher
that referenced
this issue
Aug 25, 2023
Squashed commit of the following: commit 5b32df6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:59:35 2023 -0400 Turns out the token.userPrincipal.UID is not normally set commit 064526f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:12:17 2023 -0400 Pull token fields from the ldap attributes instead of the old user commit e33bba9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 10:11:57 2023 -0400 Outdent returns to make drone happy commit 6c084df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 09:01:45 2023 -0400 Squashed commit of the following: commit 3db22eb Merge: 8039207 552fb84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:57:01 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 8039207 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:56:53 2023 -0400 tiny, tiny fix to logging commit 552fb84 Merge: ea68517 99a1814 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 17 07:39:00 2023 -0400 Merge pull request rancher#30 from crobby/migrationreview31 Outdent else blocks to make lint happy commit 99a1814 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 17 05:00:47 2023 -0400 Outdent else blocks to make lint happy commit ea68517 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 20:28:14 2023 -0400 Apply exponential retry logic to GRB and Token migrations Also, like *RTBs, these are considered non-fatal if a permanent error of some sort occurs. We continue to migrate the user anyway. commit 4a2ae0b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 19:24:42 2023 -0400 For CRTB/PRTBs, rework error handling to gracefully retry In particular, this treats internal errors (usually related to webhook timeouts) as transient, and retries them with a little bit of exponential backoff. Furthermore, after reviewing some scenarios with Michael, we've decided to consider non-internal errors from the webhook as non-fatal in terms of continuing to process the individual user. There are a few situations where old bindings to disabled templates would otherwise block users from migrating, and this permits those to have a better chance of overall success. commit 35d647c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:58:50 2023 -0400 When merging user tokens, copy over all relevant principal fields These aren't used for anything that I'm aware of, so this is really more just for consistency, since we want the two to be fully paired. commit f3e8094 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:52:15 2023 -0400 Cleanup error handling, consider AD retrieval to be a harder error commit 90f2ec1 Merge: ffcec58 b56138b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:28 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ffcec58 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:10 2023 -0400 ... once. Add the DN-based principal once. commit b56138b Merge: 78a66e0 bfb7176 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:45 2023 -0400 Merge pull request rancher#29 from crobby/migrationreview25 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit 78a66e0 Merge: edf3535 df507b5 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:24 2023 -0400 Merge pull request rancher#28 from crobby/migrationreview24 Remove unnecessary json marshal/unmarshal commit edf3535 Merge: b93e6d0 12020af Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:10 2023 -0400 Merge pull request rancher#27 from crobby/migrationreview23 Give the job pod a chance to come up before tailing the log commit b93e6d0 Merge: a2c2acb 58a0a1d Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:52 2023 -0400 Merge pull request rancher#26 from crobby/migrationreview22 Now using AuthConfig annotation as source of truth to block login during migration commit a2c2acb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:06 2023 -0400 Rework allowed user migration to handle duplicates and missing users commit bfb7176 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 14:38:22 2023 -0400 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit df507b5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:38:39 2023 -0400 Remove unnecessary json marshal/unmarshal commit 12020af Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:01:18 2023 -0400 Give the job pod a chance to come up before tailing the log commit 58a0a1d Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 12:50:57 2023 -0400 Now using AuthConfig annotation as source of truth to block login during migration commit 3ef3fb0 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:27:23 2023 -0400 Wait to do the AuthConfig principals until after updating users This kicks off some rancher-side tasks based on the updated list, and we'd really like to make sure that those user changes have been made in advance just for sanity purposes. commit b29bfb8 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:25:30 2023 -0400 When collecting duplicates, we need to track the workunit index commit df0307e Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:23:47 2023 -0400 Have the dry run guard writing new principal IDs This is mostly just to make the code clearer and more obvious. The safety is redundant, as the dry run also blocks making changes to the user object later. commit 59bafdf Merge: 2dd5250 2473062 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:12:08 2023 -0400 Merge pull request rancher#25 from crobby/migrationreview21 Append copy of user rather than pointer to duplicate list commit 2473062 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 08:00:41 2023 -0400 append copy of user rather than pointer to duplicate list commit 2dd5250 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:48:34 2023 -0400 Explicitly check to see if AD is disabled, and exit success in this case commit 4a3aa80 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:00:25 2023 -0400 Actually *use* the final migration status commit 255ef68 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:36:19 2023 -0400 Add uuid-unmigration script, prevent AD logins during execution Squashed commit of the following: commit c2bb101 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:13:12 2023 -0400 Add a generic failure status, defer restoring logins on failure states commit f9c0398 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 13:21:29 2023 -0400 Permit retries (with backoff) when opening the LDAP connection Previously we were considering a failure during open (initial or otherwise) to be a hard, script-ending, permanent failure. That's frankly a bit silly, networks can be tempermental, so this fixes that somewhat. Notably, I can't seem to find any way to check the status of the connection on the lConn object, so we're tracking that manually using a tiny little state object. If there's a cleaner way to inspect this state I am all ears, but I don't think it's a majorly big deal. (Elsewhere in Rancher we don't try to share the ldap connection generally, but here it is a big performance boost, so it is worth the extra trouble.) commit b293d62 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:54:43 2023 -0400 Rework token logic to mirror *RTBs This both collects and processes tokens that the old logic would have missed, and is also considerably more efficient, now needing to scan the list of workunits and the list of tokens just once. commit fcd2b34 Merge: 005f102 3bdea12 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:12:36 2023 -0400 Merge pull request rancher#24 from crobby/migrationreview17 Fixing names to make ci happy commit 3bdea12 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 12:09:22 2023 -0400 Fixing names to make ci happy commit 005f102 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:01:31 2023 -0400 Missing users are Infof, not Errorf commit 540e494 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:10:27 2023 -0400 Don't create/update the configmap object in dry run mode What part of "dry run" did we forget, hrm? commit 9ced565 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:00:51 2023 -0400 If the config map is not found, it's fine. (Panic otherwise.) commit 80ea848 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 10:53:30 2023 -0400 Add logic to migrate list of allowed users commit c12dcef Merge: 33f494a ce1feb4 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:25:53 2023 -0400 Merge pull request rancher#23 from crobby/migrationreview14 Another round of updates commit 33f494a Merge: b897e47 e944b57 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:13:15 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit b897e47 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:12:51 2023 -0400 Rework CRTB,PRTB collection, add GRB migration logic commit ce1feb4 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 07:15:24 2023 -0400 Echoing the set options at the end of the banner commit 089412c Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:44:43 2023 -0400 Adding additional information to README commit a7c9484 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:38:19 2023 -0400 Include agent image location in banner commit 8854263 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 16:31:44 2023 -0400 Mirror script status to authconfig commit 5bc29d5 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 12:50:13 2023 -0400 Update script status codes commit e944b57 Merge: 14c5f72 80e928b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:58 2023 -0400 Merge pull request rancher#22 from crobby/migrationreview13 More updates commit 14c5f72 Merge: a3e85de 516bdeb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:03 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit a3e85de Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:35:46 2023 -0400 Break out migration logic into a bunch of smaller files commit 80e928b Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:51:39 2023 -0400 Use configmap cache instead of client commit 516bdeb Merge: a899779 f8369c8 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:13:56 2023 -0400 Merge pull request rancher#21 from crobby/migrationreview12 Display banner before doing version check commit f8369c8 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:12:31 2023 -0400 Display banner before doing version check commit a899779 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:08:24 2023 -0400 Update cleanup/ad-guid-README.md Co-authored-by: Michael Bolot <michael.bolot@suse.com> commit 4d09212 Merge: c110ae9 92483fa Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 09:58:56 2023 -0400 Merge pull request rancher#19 from crobby/migrationreview9 Removing unused error type check commit 92483fa Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 09:51:18 2023 -0400 Removing unused error type check commit c110ae9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:51:16 2023 -0400 goimports the things commit 7691146 Merge: 44d2375 6453484 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:39 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 6453484 Merge: baf84bf 50286a2 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:32 2023 -0400 Merge pull request rancher#18 from crobby/migrationreview7 Fixing error checking commit 44d2375 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:13:58 2023 -0400 Use wait's exponential backoff primitive instead of manual sleeps commit 50286a2 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 16:27:48 2023 -0400 Fixing error checking commit baf84bf Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:39:13 2023 -0400 Only yell if the user is doing a non-dry-run on v2.7.5 commit eed1416 Merge: 9a71e38 ad00983 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:53 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 9a71e38 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:08 2023 -0400 Cleanup timeout messaging, lower job start timeout to 5 minutes I misunderstood the bash logic when I first extended that to one hour. 5 minutes for an agent download is somewhat more sensible. commit ad00983 Merge: 4e18baa 344a05d Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:34:29 2023 -0400 Merge pull request rancher#17 from crobby/migrationreview6 Additional changes after review commit 344a05d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 14:16:55 2023 -0400 Adding version check for v2.7.5 before doing anything commit 682444d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 13:50:05 2023 -0400 Fix-up README for updated usage commit 4e18baa Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:54:15 2023 -0400 Spawn relevant resources in the cattle-system namespace commit f96eb3a Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:12:33 2023 -0400 Move the YAML configuration file into the bash script This dodges the whole "fetch it from a weird URL" thing, and also makes the script a self-contained single file, which is much nicer for support to deal with. commit 275f42b Merge: 4c98764 b99cab4 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 11:16:41 2023 -0400 Merge pull request rancher#16 from crobby/migrationreview5 More post review updates commit b99cab4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 09:53:57 2023 -0400 Fixing up handling of command line options and args commit 4f6da40 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:49:20 2023 -0400 Fixing up LdapFoundDuplicateGUID name commit 9f577f6 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:31:20 2023 -0400 Adding percentage done indicator to status config map commit 43f19e4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:06:02 2023 -0400 Adding lists of special status users to configmap commit fa9979e Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 06:33:46 2023 -0400 Adding rancher-cleanup label to all cleanup objects commit 4c98764 Merge: 2d59ac6 c301303 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:38:29 2023 -0400 Merge pull request rancher#15 from crobby/migrationreview4 Post review updates commit c301303 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:33:39 2023 -0400 Updated isGUID function commit 2d59ac6 Merge: c0cdc07 86330c6 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:14:48 2023 -0400 Merge pull request rancher#14 from crobby/migrationreview3 Migration review updates 3 commit c0cdc07 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:12:22 2023 -0400 Log if we need to skip a CRTB/PRTB due to the user not existing This feels like the safer option versus applying permissions that none of the users we've collected actually have, even with the GUID/DN matching. This situation should be relatively uncommon, as Rancher usually cleans these up when users are deleted, but with the GUID duplicate bug I'm not sure how successful that will have been in practice. Best to be safe (and noisy) commit 86330c6 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:09:05 2023 -0400 Updating SA permissions for nonResourceURLs commit 4ae2d58 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 12:12:19 2023 -0400 Seeding README, adding script banner commit f8c941b Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:20:10 2023 -0400 Token collection checking userID and now setting userID and label for token updates commit e742102 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:03:04 2023 -0400 Adding additional dry-run logging information commit dc46114 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 16:57:02 2023 -0400 Rework CRTB/PRTB collection to check usernames, run through list once There are still nested for loops in here, but they are a bit more hidden :P commit ad32ccd Merge: ccb0b84 cb98c12 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:52:25 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ccb0b84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:50:27 2023 -0400 Break out the user modification flow into separate functions This mostly cleans up the main loop, but it also separates concerns and makes the smaller bits of logic easier to find and follow. commit aa41893 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:19:08 2023 -0400 Move user principal printing into its respective utility function commit ef909ab Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:12:05 2023 -0400 Respect the adConfig's UserObjectClass when performing a GUID lookup This is for parity with the auth provider; most AD configurations shouldn't have changed this from the default. commit 3963205 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:44:10 2023 -0400 Consider multiple users with the same GUID as a hard error This shouldn't be possible in practice, so it almost certainly indicates either a configuration error, or something wrong on the AD side of things. Either way we will refuse to process any user that trips this logic, and complain about it quite loudly. commit 0cebb89 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:27:24 2023 -0400 We don't need the scope, so simplify -> getExternalId commit da7ef22 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:11:41 2023 -0400 Start the scaledContext. Don't give it managers it doesn't need commit a60b144 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:34:25 2023 -0400 Remove the ratelimiting exception. Prefer safety over speed We need to check the performance ramifications of this during testing, but considering that we will almost certainly be iterating over hundreds of users, we should probably let k8s itself rate limit us so we don't overwhelm whatever is running the control plane. That might otherwise be a nasty situation, especially for stuff like AKS and GKE. commit 16715df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:32:57 2023 -0400 For bonus safety, redundantly check for dryRun here The logic up top should make this check unnecessary, but we want to be extra certain that in dryRun mode no changes are made, so we'll explicitly guard on it every time. This protects the code less from itself, and more from future modifications. commit cb98c12 Merge: e17d56f 4d2f735 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:20:06 2023 -0400 Merge pull request rancher#13 from crobby/migrationreview2 More updates based on review comments commit 4d2f735 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 8 10:17:38 2023 -0400 More updates based on review comments commit e17d56f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:38:59 2023 -0400 EscapeUUID -> escapeUUID commit 139ce3c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:37:34 2023 -0400 Relocate environment variable use to the agent-specific code path commit 795c94b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:33:13 2023 -0400 Remove unnecessary namespace from cluster role definitions commit 01ea868 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:30:53 2023 -0400 One minute is *awfully optimistic.* Let's be more realistic commit b9d4487 Merge: 17250da 0efbb02 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:21:42 2023 -0400 Merge pull request rancher#12 from crobby/migrationreview Update based on review comments commit 0efbb02 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 7 15:55:46 2023 -0400 Update based on review comments commit 17250da Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 10:29:05 2023 -0400 Don't hide the migration script from windows agents ... which in hindsight are probably somewhat likely to be using the Active Directory auth provider. commit cadf021 Merge: 9b8fd58 3926f7b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 08:18:10 2023 -0400 Merge pull request rancher#11 from crobby/migrateimports Fixing imports commit 3926f7b Author: Chad Roberts <chad.roberts@suse.com> Date: Sat Aug 5 07:45:25 2023 -0400 Fixing imports commit 9b8fd58 Merge: de38ffe 26dd505 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 17:10:43 2023 -0400 Merge pull request rancher#10 from crobby/dntokens Fix tokens going to local principal commit 26dd505 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 17:08:20 2023 -0400 Fix tokens going to local principal commit de38ffe Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 15:36:12 2023 -0400 Cleanup debug/info logs somewhat commit 1581b5d Merge: 5dfcda0 29c87eb Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:56:22 2023 -0400 Merge pull request rancher#9 from crobby/linter2 More cleaning up lint commit 29c87eb Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:54:40 2023 -0400 More cleaning up lint commit 5dfcda0 Merge: a119663 d37ef2f Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:49:55 2023 -0400 Merge pull request rancher#8 from crobby/linter Cleaning up lint commit d37ef2f Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:47:44 2023 -0400 Cleaning up lint commit a119663 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:38:46 2023 -0400 Add an option to automatically delete missing-guid users This is only available when running the standalone script. At Rancher startup this option is set to false, so missing users will be logged instead and require manual intervention. commit 60f31f8 Merge: 7e620d5 9d82578 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 13:22:56 2023 -0400 Merge pull request rancher#7 from crobby/0805-migration Update migration start logic so an automated run will only happen if another run has not completed commit 9d82578 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 12:12:56 2023 -0400 Update migration start logic so an automated run will only happen if another run has not completed commit 7e620d5 Merge: 30c9f64 6c352a5 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:26:52 2023 -0400 Merge pull request rancher#4 from crobby/migrateatstart Add guid migration to rancher startup commit 30c9f64 Merge: b9aa392 72895b4 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:10:58 2023 -0400 Merge pull request rancher#5 from crobby/0803-migration Make sure annotations/labels are not nil commit 72895b4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 16:58:56 2023 -0400 Make sure annotations/labels are not nil commit b9aa392 Merge: 79762cb 7546cdf Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 10:43:30 2023 -0400 Merge pull request rancher#6 from crobby/0804-migration Fix crtb, prtb collection and add token collection/migration commit 7546cdf Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 08:59:54 2023 -0400 Fix crtb, prtb collection and add token collection/migration commit 79762cb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 3 18:00:53 2023 -0400 Collect CRTBs and PRTBs in a single pass commit b6b6085 Merge: 3de5aa3 b3acab9 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 3 11:44:13 2023 -0400 Merge pull request rancher#3 from crobby/0802-2migration Adding annotation/labels for migrated objects also blocking login while migration is active commit b3acab9 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 11:37:16 2023 -0400 Update role for SA commit 673e765 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 09:33:45 2023 -0400 Blocking login while migration is running commit 6c352a5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 13:42:33 2023 -0400 Add guid migration to rancher startup commit 840c5a7 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 12:20:41 2023 -0400 Adding annotation/labels for migrated objects commit 3de5aa3 Merge: 5dc7bd7 04ea1ce Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 2 09:57:48 2023 -0400 Merge pull request rancher#2 from crobby/0802migration Fix status function and use user copies in workUnit slices commit 04ea1ce Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 1 18:02:19 2023 -0400 Fixing status function and using copies of users in workUnit slices commit 5dc7bd7 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:29:15 2023 -0400 Skip over configmap updates for now, just to get the script running commit ac3afe6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:19:52 2023 -0400 Massively overhaul main loop, check for and handle duplicate users This is largely untested because I'm having some trouble with the configmaps code, but I wanted to get this committed before I start troubleshooting commit 5295f8f Merge: 29f9332 552e73f Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 1 08:58:41 2023 -0400 Merge pull request rancher#1 from crobby/tokenunmigrate Additional unmigration functionality commit 552e73f Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Jul 31 13:22:26 2023 -0400 Additional unmigration functionality commit 29f9332 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 17:30:10 2023 -0400 Actually perform the GUID -> DN migration on the happy path And it works too! Thank goodness. Now we mostly need to clean up the logic and handle a few dozen edge cases. commit 62a6747 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 12:53:43 2023 -0400 Cleanup the logs a bit, flatten the central logic with early exits commit ac20a2c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 09:58:54 2023 -0400 Switch to using the scaledContext for everything Since it can do all the lookups we need, it seems silly to setup and use two different interfaces to the same underlying datastore. The UnstructuredClient is the only way we can read AD configuration right now, and we need that info, so let's stick to that method. commit 18b39d3 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Jul 28 17:38:27 2023 -0400 First pass at migration scaffolding, enough to do GUID -> DN lookups There is still much work to do, but at the very least we can read the relevant auth configuration details from k8s and use those details to make LDAP queries, and that's nearly all of what we need to perform the migration.
crobby
pushed a commit
to crobby/rancher
that referenced
this issue
Aug 25, 2023
Squashed commit of the following: commit 5b32df6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:59:35 2023 -0400 Turns out the token.userPrincipal.UID is not normally set commit 064526f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:12:17 2023 -0400 Pull token fields from the ldap attributes instead of the old user commit e33bba9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 10:11:57 2023 -0400 Outdent returns to make drone happy commit 6c084df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 09:01:45 2023 -0400 Squashed commit of the following: commit 3db22eb Merge: 8039207 552fb84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:57:01 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 8039207 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:56:53 2023 -0400 tiny, tiny fix to logging commit 552fb84 Merge: ea68517 99a1814 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 17 07:39:00 2023 -0400 Merge pull request rancher#30 from crobby/migrationreview31 Outdent else blocks to make lint happy commit 99a1814 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 17 05:00:47 2023 -0400 Outdent else blocks to make lint happy commit ea68517 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 20:28:14 2023 -0400 Apply exponential retry logic to GRB and Token migrations Also, like *RTBs, these are considered non-fatal if a permanent error of some sort occurs. We continue to migrate the user anyway. commit 4a2ae0b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 19:24:42 2023 -0400 For CRTB/PRTBs, rework error handling to gracefully retry In particular, this treats internal errors (usually related to webhook timeouts) as transient, and retries them with a little bit of exponential backoff. Furthermore, after reviewing some scenarios with Michael, we've decided to consider non-internal errors from the webhook as non-fatal in terms of continuing to process the individual user. There are a few situations where old bindings to disabled templates would otherwise block users from migrating, and this permits those to have a better chance of overall success. commit 35d647c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:58:50 2023 -0400 When merging user tokens, copy over all relevant principal fields These aren't used for anything that I'm aware of, so this is really more just for consistency, since we want the two to be fully paired. commit f3e8094 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:52:15 2023 -0400 Cleanup error handling, consider AD retrieval to be a harder error commit 90f2ec1 Merge: ffcec58 b56138b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:28 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ffcec58 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:10 2023 -0400 ... once. Add the DN-based principal once. commit b56138b Merge: 78a66e0 bfb7176 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:45 2023 -0400 Merge pull request rancher#29 from crobby/migrationreview25 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit 78a66e0 Merge: edf3535 df507b5 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:24 2023 -0400 Merge pull request rancher#28 from crobby/migrationreview24 Remove unnecessary json marshal/unmarshal commit edf3535 Merge: b93e6d0 12020af Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:10 2023 -0400 Merge pull request rancher#27 from crobby/migrationreview23 Give the job pod a chance to come up before tailing the log commit b93e6d0 Merge: a2c2acb 58a0a1d Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:52 2023 -0400 Merge pull request rancher#26 from crobby/migrationreview22 Now using AuthConfig annotation as source of truth to block login during migration commit a2c2acb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:06 2023 -0400 Rework allowed user migration to handle duplicates and missing users commit bfb7176 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 14:38:22 2023 -0400 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit df507b5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:38:39 2023 -0400 Remove unnecessary json marshal/unmarshal commit 12020af Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:01:18 2023 -0400 Give the job pod a chance to come up before tailing the log commit 58a0a1d Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 12:50:57 2023 -0400 Now using AuthConfig annotation as source of truth to block login during migration commit 3ef3fb0 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:27:23 2023 -0400 Wait to do the AuthConfig principals until after updating users This kicks off some rancher-side tasks based on the updated list, and we'd really like to make sure that those user changes have been made in advance just for sanity purposes. commit b29bfb8 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:25:30 2023 -0400 When collecting duplicates, we need to track the workunit index commit df0307e Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:23:47 2023 -0400 Have the dry run guard writing new principal IDs This is mostly just to make the code clearer and more obvious. The safety is redundant, as the dry run also blocks making changes to the user object later. commit 59bafdf Merge: 2dd5250 2473062 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:12:08 2023 -0400 Merge pull request rancher#25 from crobby/migrationreview21 Append copy of user rather than pointer to duplicate list commit 2473062 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 08:00:41 2023 -0400 append copy of user rather than pointer to duplicate list commit 2dd5250 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:48:34 2023 -0400 Explicitly check to see if AD is disabled, and exit success in this case commit 4a3aa80 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:00:25 2023 -0400 Actually *use* the final migration status commit 255ef68 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:36:19 2023 -0400 Add uuid-unmigration script, prevent AD logins during execution Squashed commit of the following: commit c2bb101 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:13:12 2023 -0400 Add a generic failure status, defer restoring logins on failure states commit f9c0398 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 13:21:29 2023 -0400 Permit retries (with backoff) when opening the LDAP connection Previously we were considering a failure during open (initial or otherwise) to be a hard, script-ending, permanent failure. That's frankly a bit silly, networks can be tempermental, so this fixes that somewhat. Notably, I can't seem to find any way to check the status of the connection on the lConn object, so we're tracking that manually using a tiny little state object. If there's a cleaner way to inspect this state I am all ears, but I don't think it's a majorly big deal. (Elsewhere in Rancher we don't try to share the ldap connection generally, but here it is a big performance boost, so it is worth the extra trouble.) commit b293d62 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:54:43 2023 -0400 Rework token logic to mirror *RTBs This both collects and processes tokens that the old logic would have missed, and is also considerably more efficient, now needing to scan the list of workunits and the list of tokens just once. commit fcd2b34 Merge: 005f102 3bdea12 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:12:36 2023 -0400 Merge pull request rancher#24 from crobby/migrationreview17 Fixing names to make ci happy commit 3bdea12 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 12:09:22 2023 -0400 Fixing names to make ci happy commit 005f102 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:01:31 2023 -0400 Missing users are Infof, not Errorf commit 540e494 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:10:27 2023 -0400 Don't create/update the configmap object in dry run mode What part of "dry run" did we forget, hrm? commit 9ced565 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:00:51 2023 -0400 If the config map is not found, it's fine. (Panic otherwise.) commit 80ea848 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 10:53:30 2023 -0400 Add logic to migrate list of allowed users commit c12dcef Merge: 33f494a ce1feb4 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:25:53 2023 -0400 Merge pull request rancher#23 from crobby/migrationreview14 Another round of updates commit 33f494a Merge: b897e47 e944b57 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:13:15 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit b897e47 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:12:51 2023 -0400 Rework CRTB,PRTB collection, add GRB migration logic commit ce1feb4 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 07:15:24 2023 -0400 Echoing the set options at the end of the banner commit 089412c Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:44:43 2023 -0400 Adding additional information to README commit a7c9484 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:38:19 2023 -0400 Include agent image location in banner commit 8854263 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 16:31:44 2023 -0400 Mirror script status to authconfig commit 5bc29d5 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 12:50:13 2023 -0400 Update script status codes commit e944b57 Merge: 14c5f72 80e928b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:58 2023 -0400 Merge pull request rancher#22 from crobby/migrationreview13 More updates commit 14c5f72 Merge: a3e85de 516bdeb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:03 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit a3e85de Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:35:46 2023 -0400 Break out migration logic into a bunch of smaller files commit 80e928b Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:51:39 2023 -0400 Use configmap cache instead of client commit 516bdeb Merge: a899779 f8369c8 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:13:56 2023 -0400 Merge pull request rancher#21 from crobby/migrationreview12 Display banner before doing version check commit f8369c8 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:12:31 2023 -0400 Display banner before doing version check commit a899779 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:08:24 2023 -0400 Update cleanup/ad-guid-README.md Co-authored-by: Michael Bolot <michael.bolot@suse.com> commit 4d09212 Merge: c110ae9 92483fa Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 09:58:56 2023 -0400 Merge pull request rancher#19 from crobby/migrationreview9 Removing unused error type check commit 92483fa Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 09:51:18 2023 -0400 Removing unused error type check commit c110ae9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:51:16 2023 -0400 goimports the things commit 7691146 Merge: 44d2375 6453484 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:39 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 6453484 Merge: baf84bf 50286a2 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:32 2023 -0400 Merge pull request rancher#18 from crobby/migrationreview7 Fixing error checking commit 44d2375 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:13:58 2023 -0400 Use wait's exponential backoff primitive instead of manual sleeps commit 50286a2 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 16:27:48 2023 -0400 Fixing error checking commit baf84bf Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:39:13 2023 -0400 Only yell if the user is doing a non-dry-run on v2.7.5 commit eed1416 Merge: 9a71e38 ad00983 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:53 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 9a71e38 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:08 2023 -0400 Cleanup timeout messaging, lower job start timeout to 5 minutes I misunderstood the bash logic when I first extended that to one hour. 5 minutes for an agent download is somewhat more sensible. commit ad00983 Merge: 4e18baa 344a05d Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:34:29 2023 -0400 Merge pull request rancher#17 from crobby/migrationreview6 Additional changes after review commit 344a05d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 14:16:55 2023 -0400 Adding version check for v2.7.5 before doing anything commit 682444d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 13:50:05 2023 -0400 Fix-up README for updated usage commit 4e18baa Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:54:15 2023 -0400 Spawn relevant resources in the cattle-system namespace commit f96eb3a Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:12:33 2023 -0400 Move the YAML configuration file into the bash script This dodges the whole "fetch it from a weird URL" thing, and also makes the script a self-contained single file, which is much nicer for support to deal with. commit 275f42b Merge: 4c98764 b99cab4 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 11:16:41 2023 -0400 Merge pull request rancher#16 from crobby/migrationreview5 More post review updates commit b99cab4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 09:53:57 2023 -0400 Fixing up handling of command line options and args commit 4f6da40 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:49:20 2023 -0400 Fixing up LdapFoundDuplicateGUID name commit 9f577f6 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:31:20 2023 -0400 Adding percentage done indicator to status config map commit 43f19e4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:06:02 2023 -0400 Adding lists of special status users to configmap commit fa9979e Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 06:33:46 2023 -0400 Adding rancher-cleanup label to all cleanup objects commit 4c98764 Merge: 2d59ac6 c301303 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:38:29 2023 -0400 Merge pull request rancher#15 from crobby/migrationreview4 Post review updates commit c301303 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:33:39 2023 -0400 Updated isGUID function commit 2d59ac6 Merge: c0cdc07 86330c6 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:14:48 2023 -0400 Merge pull request rancher#14 from crobby/migrationreview3 Migration review updates 3 commit c0cdc07 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:12:22 2023 -0400 Log if we need to skip a CRTB/PRTB due to the user not existing This feels like the safer option versus applying permissions that none of the users we've collected actually have, even with the GUID/DN matching. This situation should be relatively uncommon, as Rancher usually cleans these up when users are deleted, but with the GUID duplicate bug I'm not sure how successful that will have been in practice. Best to be safe (and noisy) commit 86330c6 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:09:05 2023 -0400 Updating SA permissions for nonResourceURLs commit 4ae2d58 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 12:12:19 2023 -0400 Seeding README, adding script banner commit f8c941b Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:20:10 2023 -0400 Token collection checking userID and now setting userID and label for token updates commit e742102 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:03:04 2023 -0400 Adding additional dry-run logging information commit dc46114 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 16:57:02 2023 -0400 Rework CRTB/PRTB collection to check usernames, run through list once There are still nested for loops in here, but they are a bit more hidden :P commit ad32ccd Merge: ccb0b84 cb98c12 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:52:25 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ccb0b84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:50:27 2023 -0400 Break out the user modification flow into separate functions This mostly cleans up the main loop, but it also separates concerns and makes the smaller bits of logic easier to find and follow. commit aa41893 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:19:08 2023 -0400 Move user principal printing into its respective utility function commit ef909ab Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:12:05 2023 -0400 Respect the adConfig's UserObjectClass when performing a GUID lookup This is for parity with the auth provider; most AD configurations shouldn't have changed this from the default. commit 3963205 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:44:10 2023 -0400 Consider multiple users with the same GUID as a hard error This shouldn't be possible in practice, so it almost certainly indicates either a configuration error, or something wrong on the AD side of things. Either way we will refuse to process any user that trips this logic, and complain about it quite loudly. commit 0cebb89 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:27:24 2023 -0400 We don't need the scope, so simplify -> getExternalId commit da7ef22 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:11:41 2023 -0400 Start the scaledContext. Don't give it managers it doesn't need commit a60b144 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:34:25 2023 -0400 Remove the ratelimiting exception. Prefer safety over speed We need to check the performance ramifications of this during testing, but considering that we will almost certainly be iterating over hundreds of users, we should probably let k8s itself rate limit us so we don't overwhelm whatever is running the control plane. That might otherwise be a nasty situation, especially for stuff like AKS and GKE. commit 16715df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:32:57 2023 -0400 For bonus safety, redundantly check for dryRun here The logic up top should make this check unnecessary, but we want to be extra certain that in dryRun mode no changes are made, so we'll explicitly guard on it every time. This protects the code less from itself, and more from future modifications. commit cb98c12 Merge: e17d56f 4d2f735 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:20:06 2023 -0400 Merge pull request rancher#13 from crobby/migrationreview2 More updates based on review comments commit 4d2f735 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 8 10:17:38 2023 -0400 More updates based on review comments commit e17d56f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:38:59 2023 -0400 EscapeUUID -> escapeUUID commit 139ce3c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:37:34 2023 -0400 Relocate environment variable use to the agent-specific code path commit 795c94b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:33:13 2023 -0400 Remove unnecessary namespace from cluster role definitions commit 01ea868 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:30:53 2023 -0400 One minute is *awfully optimistic.* Let's be more realistic commit b9d4487 Merge: 17250da 0efbb02 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:21:42 2023 -0400 Merge pull request rancher#12 from crobby/migrationreview Update based on review comments commit 0efbb02 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 7 15:55:46 2023 -0400 Update based on review comments commit 17250da Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 10:29:05 2023 -0400 Don't hide the migration script from windows agents ... which in hindsight are probably somewhat likely to be using the Active Directory auth provider. commit cadf021 Merge: 9b8fd58 3926f7b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 08:18:10 2023 -0400 Merge pull request rancher#11 from crobby/migrateimports Fixing imports commit 3926f7b Author: Chad Roberts <chad.roberts@suse.com> Date: Sat Aug 5 07:45:25 2023 -0400 Fixing imports commit 9b8fd58 Merge: de38ffe 26dd505 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 17:10:43 2023 -0400 Merge pull request rancher#10 from crobby/dntokens Fix tokens going to local principal commit 26dd505 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 17:08:20 2023 -0400 Fix tokens going to local principal commit de38ffe Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 15:36:12 2023 -0400 Cleanup debug/info logs somewhat commit 1581b5d Merge: 5dfcda0 29c87eb Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:56:22 2023 -0400 Merge pull request rancher#9 from crobby/linter2 More cleaning up lint commit 29c87eb Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:54:40 2023 -0400 More cleaning up lint commit 5dfcda0 Merge: a119663 d37ef2f Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:49:55 2023 -0400 Merge pull request rancher#8 from crobby/linter Cleaning up lint commit d37ef2f Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:47:44 2023 -0400 Cleaning up lint commit a119663 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:38:46 2023 -0400 Add an option to automatically delete missing-guid users This is only available when running the standalone script. At Rancher startup this option is set to false, so missing users will be logged instead and require manual intervention. commit 60f31f8 Merge: 7e620d5 9d82578 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 13:22:56 2023 -0400 Merge pull request rancher#7 from crobby/0805-migration Update migration start logic so an automated run will only happen if another run has not completed commit 9d82578 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 12:12:56 2023 -0400 Update migration start logic so an automated run will only happen if another run has not completed commit 7e620d5 Merge: 30c9f64 6c352a5 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:26:52 2023 -0400 Merge pull request rancher#4 from crobby/migrateatstart Add guid migration to rancher startup commit 30c9f64 Merge: b9aa392 72895b4 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:10:58 2023 -0400 Merge pull request rancher#5 from crobby/0803-migration Make sure annotations/labels are not nil commit 72895b4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 16:58:56 2023 -0400 Make sure annotations/labels are not nil commit b9aa392 Merge: 79762cb 7546cdf Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 10:43:30 2023 -0400 Merge pull request rancher#6 from crobby/0804-migration Fix crtb, prtb collection and add token collection/migration commit 7546cdf Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 08:59:54 2023 -0400 Fix crtb, prtb collection and add token collection/migration commit 79762cb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 3 18:00:53 2023 -0400 Collect CRTBs and PRTBs in a single pass commit b6b6085 Merge: 3de5aa3 b3acab9 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 3 11:44:13 2023 -0400 Merge pull request rancher#3 from crobby/0802-2migration Adding annotation/labels for migrated objects also blocking login while migration is active commit b3acab9 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 11:37:16 2023 -0400 Update role for SA commit 673e765 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 09:33:45 2023 -0400 Blocking login while migration is running commit 6c352a5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 13:42:33 2023 -0400 Add guid migration to rancher startup commit 840c5a7 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 12:20:41 2023 -0400 Adding annotation/labels for migrated objects commit 3de5aa3 Merge: 5dc7bd7 04ea1ce Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 2 09:57:48 2023 -0400 Merge pull request rancher#2 from crobby/0802migration Fix status function and use user copies in workUnit slices commit 04ea1ce Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 1 18:02:19 2023 -0400 Fixing status function and using copies of users in workUnit slices commit 5dc7bd7 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:29:15 2023 -0400 Skip over configmap updates for now, just to get the script running commit ac3afe6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:19:52 2023 -0400 Massively overhaul main loop, check for and handle duplicate users This is largely untested because I'm having some trouble with the configmaps code, but I wanted to get this committed before I start troubleshooting commit 5295f8f Merge: 29f9332 552e73f Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 1 08:58:41 2023 -0400 Merge pull request rancher#1 from crobby/tokenunmigrate Additional unmigration functionality commit 552e73f Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Jul 31 13:22:26 2023 -0400 Additional unmigration functionality commit 29f9332 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 17:30:10 2023 -0400 Actually perform the GUID -> DN migration on the happy path And it works too! Thank goodness. Now we mostly need to clean up the logic and handle a few dozen edge cases. commit 62a6747 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 12:53:43 2023 -0400 Cleanup the logs a bit, flatten the central logic with early exits commit ac20a2c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 09:58:54 2023 -0400 Switch to using the scaledContext for everything Since it can do all the lookups we need, it seems silly to setup and use two different interfaces to the same underlying datastore. The UnstructuredClient is the only way we can read AD configuration right now, and we need that info, so let's stick to that method. commit 18b39d3 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Jul 28 17:38:27 2023 -0400 First pass at migration scaffolding, enough to do GUID -> DN lookups There is still much work to do, but at the very least we can read the relevant auth configuration details from k8s and use those details to make LDAP queries, and that's nearly all of what we need to perform the migration.
crobby
pushed a commit
to crobby/rancher
that referenced
this issue
Aug 29, 2023
Squashed commit of the following: commit 5b32df6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:59:35 2023 -0400 Turns out the token.userPrincipal.UID is not normally set commit 064526f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 11:12:17 2023 -0400 Pull token fields from the ldap attributes instead of the old user commit e33bba9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 10:11:57 2023 -0400 Outdent returns to make drone happy commit 6c084df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 09:01:45 2023 -0400 Squashed commit of the following: commit 3db22eb Merge: 8039207 552fb84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:57:01 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 8039207 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 17 08:56:53 2023 -0400 tiny, tiny fix to logging commit 552fb84 Merge: ea68517 99a1814 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 17 07:39:00 2023 -0400 Merge pull request rancher#30 from crobby/migrationreview31 Outdent else blocks to make lint happy commit 99a1814 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 17 05:00:47 2023 -0400 Outdent else blocks to make lint happy commit ea68517 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 20:28:14 2023 -0400 Apply exponential retry logic to GRB and Token migrations Also, like *RTBs, these are considered non-fatal if a permanent error of some sort occurs. We continue to migrate the user anyway. commit 4a2ae0b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 19:24:42 2023 -0400 For CRTB/PRTBs, rework error handling to gracefully retry In particular, this treats internal errors (usually related to webhook timeouts) as transient, and retries them with a little bit of exponential backoff. Furthermore, after reviewing some scenarios with Michael, we've decided to consider non-internal errors from the webhook as non-fatal in terms of continuing to process the individual user. There are a few situations where old bindings to disabled templates would otherwise block users from migrating, and this permits those to have a better chance of overall success. commit 35d647c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:58:50 2023 -0400 When merging user tokens, copy over all relevant principal fields These aren't used for anything that I'm aware of, so this is really more just for consistency, since we want the two to be fully paired. commit f3e8094 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:52:15 2023 -0400 Cleanup error handling, consider AD retrieval to be a harder error commit 90f2ec1 Merge: ffcec58 b56138b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:28 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ffcec58 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 16:13:10 2023 -0400 ... once. Add the DN-based principal once. commit b56138b Merge: 78a66e0 bfb7176 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:45 2023 -0400 Merge pull request rancher#29 from crobby/migrationreview25 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit 78a66e0 Merge: edf3535 df507b5 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:24 2023 -0400 Merge pull request rancher#28 from crobby/migrationreview24 Remove unnecessary json marshal/unmarshal commit edf3535 Merge: b93e6d0 12020af Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:47:10 2023 -0400 Merge pull request rancher#27 from crobby/migrationreview23 Give the job pod a chance to come up before tailing the log commit b93e6d0 Merge: a2c2acb 58a0a1d Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:52 2023 -0400 Merge pull request rancher#26 from crobby/migrationreview22 Now using AuthConfig annotation as source of truth to block login during migration commit a2c2acb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 15:46:06 2023 -0400 Rework allowed user migration to handle duplicates and missing users commit bfb7176 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 14:38:22 2023 -0400 Store skipped/missing user count in configmap and do not store the actual list on the authconfig object commit df507b5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:38:39 2023 -0400 Remove unnecessary json marshal/unmarshal commit 12020af Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 13:01:18 2023 -0400 Give the job pod a chance to come up before tailing the log commit 58a0a1d Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 12:50:57 2023 -0400 Now using AuthConfig annotation as source of truth to block login during migration commit 3ef3fb0 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:27:23 2023 -0400 Wait to do the AuthConfig principals until after updating users This kicks off some rancher-side tasks based on the updated list, and we'd really like to make sure that those user changes have been made in advance just for sanity purposes. commit b29bfb8 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 12:25:30 2023 -0400 When collecting duplicates, we need to track the workunit index commit df0307e Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:23:47 2023 -0400 Have the dry run guard writing new principal IDs This is mostly just to make the code clearer and more obvious. The safety is redundant, as the dry run also blocks making changes to the user object later. commit 59bafdf Merge: 2dd5250 2473062 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 16 09:12:08 2023 -0400 Merge pull request rancher#25 from crobby/migrationreview21 Append copy of user rather than pointer to duplicate list commit 2473062 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 16 08:00:41 2023 -0400 append copy of user rather than pointer to duplicate list commit 2dd5250 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:48:34 2023 -0400 Explicitly check to see if AD is disabled, and exit success in this case commit 4a3aa80 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 16:00:25 2023 -0400 Actually *use* the final migration status commit 255ef68 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:36:19 2023 -0400 Add uuid-unmigration script, prevent AD logins during execution Squashed commit of the following: commit c2bb101 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 15:13:12 2023 -0400 Add a generic failure status, defer restoring logins on failure states commit f9c0398 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 13:21:29 2023 -0400 Permit retries (with backoff) when opening the LDAP connection Previously we were considering a failure during open (initial or otherwise) to be a hard, script-ending, permanent failure. That's frankly a bit silly, networks can be tempermental, so this fixes that somewhat. Notably, I can't seem to find any way to check the status of the connection on the lConn object, so we're tracking that manually using a tiny little state object. If there's a cleaner way to inspect this state I am all ears, but I don't think it's a majorly big deal. (Elsewhere in Rancher we don't try to share the ldap connection generally, but here it is a big performance boost, so it is worth the extra trouble.) commit b293d62 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:54:43 2023 -0400 Rework token logic to mirror *RTBs This both collects and processes tokens that the old logic would have missed, and is also considerably more efficient, now needing to scan the list of workunits and the list of tokens just once. commit fcd2b34 Merge: 005f102 3bdea12 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:12:36 2023 -0400 Merge pull request rancher#24 from crobby/migrationreview17 Fixing names to make ci happy commit 3bdea12 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 12:09:22 2023 -0400 Fixing names to make ci happy commit 005f102 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 12:01:31 2023 -0400 Missing users are Infof, not Errorf commit 540e494 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:10:27 2023 -0400 Don't create/update the configmap object in dry run mode What part of "dry run" did we forget, hrm? commit 9ced565 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 11:00:51 2023 -0400 If the config map is not found, it's fine. (Panic otherwise.) commit 80ea848 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 10:53:30 2023 -0400 Add logic to migrate list of allowed users commit c12dcef Merge: 33f494a ce1feb4 Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:25:53 2023 -0400 Merge pull request rancher#23 from crobby/migrationreview14 Another round of updates commit 33f494a Merge: b897e47 e944b57 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:13:15 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit b897e47 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 15 09:12:51 2023 -0400 Rework CRTB,PRTB collection, add GRB migration logic commit ce1feb4 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 07:15:24 2023 -0400 Echoing the set options at the end of the banner commit 089412c Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:44:43 2023 -0400 Adding additional information to README commit a7c9484 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 15 06:38:19 2023 -0400 Include agent image location in banner commit 8854263 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 16:31:44 2023 -0400 Mirror script status to authconfig commit 5bc29d5 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 12:50:13 2023 -0400 Update script status codes commit e944b57 Merge: 14c5f72 80e928b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:58 2023 -0400 Merge pull request rancher#22 from crobby/migrationreview13 More updates commit 14c5f72 Merge: a3e85de 516bdeb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:36:03 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit a3e85de Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 14 11:35:46 2023 -0400 Break out migration logic into a bunch of smaller files commit 80e928b Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:51:39 2023 -0400 Use configmap cache instead of client commit 516bdeb Merge: a899779 f8369c8 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:13:56 2023 -0400 Merge pull request rancher#21 from crobby/migrationreview12 Display banner before doing version check commit f8369c8 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 10:12:31 2023 -0400 Display banner before doing version check commit a899779 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 10:08:24 2023 -0400 Update cleanup/ad-guid-README.md Co-authored-by: Michael Bolot <michael.bolot@suse.com> commit 4d09212 Merge: c110ae9 92483fa Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 14 09:58:56 2023 -0400 Merge pull request rancher#19 from crobby/migrationreview9 Removing unused error type check commit 92483fa Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 14 09:51:18 2023 -0400 Removing unused error type check commit c110ae9 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:51:16 2023 -0400 goimports the things commit 7691146 Merge: 44d2375 6453484 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:39 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 6453484 Merge: baf84bf 50286a2 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:19:32 2023 -0400 Merge pull request rancher#18 from crobby/migrationreview7 Fixing error checking commit 44d2375 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 19:13:58 2023 -0400 Use wait's exponential backoff primitive instead of manual sleeps commit 50286a2 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 16:27:48 2023 -0400 Fixing error checking commit baf84bf Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:39:13 2023 -0400 Only yell if the user is doing a non-dry-run on v2.7.5 commit eed1416 Merge: 9a71e38 ad00983 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:53 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit 9a71e38 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:36:08 2023 -0400 Cleanup timeout messaging, lower job start timeout to 5 minutes I misunderstood the bash logic when I first extended that to one hour. 5 minutes for an agent download is somewhat more sensible. commit ad00983 Merge: 4e18baa 344a05d Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 15:34:29 2023 -0400 Merge pull request rancher#17 from crobby/migrationreview6 Additional changes after review commit 344a05d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 14:16:55 2023 -0400 Adding version check for v2.7.5 before doing anything commit 682444d Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 13:50:05 2023 -0400 Fix-up README for updated usage commit 4e18baa Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:54:15 2023 -0400 Spawn relevant resources in the cattle-system namespace commit f96eb3a Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 10 14:12:33 2023 -0400 Move the YAML configuration file into the bash script This dodges the whole "fetch it from a weird URL" thing, and also makes the script a self-contained single file, which is much nicer for support to deal with. commit 275f42b Merge: 4c98764 b99cab4 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 10 11:16:41 2023 -0400 Merge pull request rancher#16 from crobby/migrationreview5 More post review updates commit b99cab4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 09:53:57 2023 -0400 Fixing up handling of command line options and args commit 4f6da40 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:49:20 2023 -0400 Fixing up LdapFoundDuplicateGUID name commit 9f577f6 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:31:20 2023 -0400 Adding percentage done indicator to status config map commit 43f19e4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 07:06:02 2023 -0400 Adding lists of special status users to configmap commit fa9979e Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 10 06:33:46 2023 -0400 Adding rancher-cleanup label to all cleanup objects commit 4c98764 Merge: 2d59ac6 c301303 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:38:29 2023 -0400 Merge pull request rancher#15 from crobby/migrationreview4 Post review updates commit c301303 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:33:39 2023 -0400 Updated isGUID function commit 2d59ac6 Merge: c0cdc07 86330c6 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:14:48 2023 -0400 Merge pull request rancher#14 from crobby/migrationreview3 Migration review updates 3 commit c0cdc07 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 17:12:22 2023 -0400 Log if we need to skip a CRTB/PRTB due to the user not existing This feels like the safer option versus applying permissions that none of the users we've collected actually have, even with the GUID/DN matching. This situation should be relatively uncommon, as Rancher usually cleans these up when users are deleted, but with the GUID duplicate bug I'm not sure how successful that will have been in practice. Best to be safe (and noisy) commit 86330c6 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 17:09:05 2023 -0400 Updating SA permissions for nonResourceURLs commit 4ae2d58 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 12:12:19 2023 -0400 Seeding README, adding script banner commit f8c941b Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:20:10 2023 -0400 Token collection checking userID and now setting userID and label for token updates commit e742102 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 9 11:03:04 2023 -0400 Adding additional dry-run logging information commit dc46114 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 16:57:02 2023 -0400 Rework CRTB/PRTB collection to check usernames, run through list once There are still nested for loops in here, but they are a bit more hidden :P commit ad32ccd Merge: ccb0b84 cb98c12 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:52:25 2023 -0400 Merge branch 'uuid-unmigration' of github.com:nflynt/rancher into uuid-unmigration commit ccb0b84 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:50:27 2023 -0400 Break out the user modification flow into separate functions This mostly cleans up the main loop, but it also separates concerns and makes the smaller bits of logic easier to find and follow. commit aa41893 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:19:08 2023 -0400 Move user principal printing into its respective utility function commit ef909ab Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 12:12:05 2023 -0400 Respect the adConfig's UserObjectClass when performing a GUID lookup This is for parity with the auth provider; most AD configurations shouldn't have changed this from the default. commit 3963205 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:44:10 2023 -0400 Consider multiple users with the same GUID as a hard error This shouldn't be possible in practice, so it almost certainly indicates either a configuration error, or something wrong on the AD side of things. Either way we will refuse to process any user that trips this logic, and complain about it quite loudly. commit 0cebb89 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:27:24 2023 -0400 We don't need the scope, so simplify -> getExternalId commit da7ef22 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 11:11:41 2023 -0400 Start the scaledContext. Don't give it managers it doesn't need commit a60b144 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:34:25 2023 -0400 Remove the ratelimiting exception. Prefer safety over speed We need to check the performance ramifications of this during testing, but considering that we will almost certainly be iterating over hundreds of users, we should probably let k8s itself rate limit us so we don't overwhelm whatever is running the control plane. That might otherwise be a nasty situation, especially for stuff like AKS and GKE. commit 16715df Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:32:57 2023 -0400 For bonus safety, redundantly check for dryRun here The logic up top should make this check unnecessary, but we want to be extra certain that in dryRun mode no changes are made, so we'll explicitly guard on it every time. This protects the code less from itself, and more from future modifications. commit cb98c12 Merge: e17d56f 4d2f735 Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 9 10:20:06 2023 -0400 Merge pull request rancher#13 from crobby/migrationreview2 More updates based on review comments commit 4d2f735 Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 8 10:17:38 2023 -0400 More updates based on review comments commit e17d56f Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:38:59 2023 -0400 EscapeUUID -> escapeUUID commit 139ce3c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:37:34 2023 -0400 Relocate environment variable use to the agent-specific code path commit 795c94b Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:33:13 2023 -0400 Remove unnecessary namespace from cluster role definitions commit 01ea868 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:30:53 2023 -0400 One minute is *awfully optimistic.* Let's be more realistic commit b9d4487 Merge: 17250da 0efbb02 Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 16:21:42 2023 -0400 Merge pull request rancher#12 from crobby/migrationreview Update based on review comments commit 0efbb02 Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Aug 7 15:55:46 2023 -0400 Update based on review comments commit 17250da Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Aug 7 10:29:05 2023 -0400 Don't hide the migration script from windows agents ... which in hindsight are probably somewhat likely to be using the Active Directory auth provider. commit cadf021 Merge: 9b8fd58 3926f7b Author: nflynt <nicholas.flynt@suse.com> Date: Mon Aug 7 08:18:10 2023 -0400 Merge pull request rancher#11 from crobby/migrateimports Fixing imports commit 3926f7b Author: Chad Roberts <chad.roberts@suse.com> Date: Sat Aug 5 07:45:25 2023 -0400 Fixing imports commit 9b8fd58 Merge: de38ffe 26dd505 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 17:10:43 2023 -0400 Merge pull request rancher#10 from crobby/dntokens Fix tokens going to local principal commit 26dd505 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 17:08:20 2023 -0400 Fix tokens going to local principal commit de38ffe Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 15:36:12 2023 -0400 Cleanup debug/info logs somewhat commit 1581b5d Merge: 5dfcda0 29c87eb Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:56:22 2023 -0400 Merge pull request rancher#9 from crobby/linter2 More cleaning up lint commit 29c87eb Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:54:40 2023 -0400 More cleaning up lint commit 5dfcda0 Merge: a119663 d37ef2f Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:49:55 2023 -0400 Merge pull request rancher#8 from crobby/linter Cleaning up lint commit d37ef2f Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 14:47:44 2023 -0400 Cleaning up lint commit a119663 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Aug 4 14:38:46 2023 -0400 Add an option to automatically delete missing-guid users This is only available when running the standalone script. At Rancher startup this option is set to false, so missing users will be logged instead and require manual intervention. commit 60f31f8 Merge: 7e620d5 9d82578 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 13:22:56 2023 -0400 Merge pull request rancher#7 from crobby/0805-migration Update migration start logic so an automated run will only happen if another run has not completed commit 9d82578 Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 12:12:56 2023 -0400 Update migration start logic so an automated run will only happen if another run has not completed commit 7e620d5 Merge: 30c9f64 6c352a5 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:26:52 2023 -0400 Merge pull request rancher#4 from crobby/migrateatstart Add guid migration to rancher startup commit 30c9f64 Merge: b9aa392 72895b4 Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 11:10:58 2023 -0400 Merge pull request rancher#5 from crobby/0803-migration Make sure annotations/labels are not nil commit 72895b4 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 16:58:56 2023 -0400 Make sure annotations/labels are not nil commit b9aa392 Merge: 79762cb 7546cdf Author: nflynt <nicholas.flynt@suse.com> Date: Fri Aug 4 10:43:30 2023 -0400 Merge pull request rancher#6 from crobby/0804-migration Fix crtb, prtb collection and add token collection/migration commit 7546cdf Author: Chad Roberts <chad.roberts@suse.com> Date: Fri Aug 4 08:59:54 2023 -0400 Fix crtb, prtb collection and add token collection/migration commit 79762cb Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Thu Aug 3 18:00:53 2023 -0400 Collect CRTBs and PRTBs in a single pass commit b6b6085 Merge: 3de5aa3 b3acab9 Author: nflynt <nicholas.flynt@suse.com> Date: Thu Aug 3 11:44:13 2023 -0400 Merge pull request rancher#3 from crobby/0802-2migration Adding annotation/labels for migrated objects also blocking login while migration is active commit b3acab9 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 11:37:16 2023 -0400 Update role for SA commit 673e765 Author: Chad Roberts <chad.roberts@suse.com> Date: Thu Aug 3 09:33:45 2023 -0400 Blocking login while migration is running commit 6c352a5 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 13:42:33 2023 -0400 Add guid migration to rancher startup commit 840c5a7 Author: Chad Roberts <chad.roberts@suse.com> Date: Wed Aug 2 12:20:41 2023 -0400 Adding annotation/labels for migrated objects commit 3de5aa3 Merge: 5dc7bd7 04ea1ce Author: nflynt <nicholas.flynt@suse.com> Date: Wed Aug 2 09:57:48 2023 -0400 Merge pull request rancher#2 from crobby/0802migration Fix status function and use user copies in workUnit slices commit 04ea1ce Author: Chad Roberts <chad.roberts@suse.com> Date: Tue Aug 1 18:02:19 2023 -0400 Fixing status function and using copies of users in workUnit slices commit 5dc7bd7 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:29:15 2023 -0400 Skip over configmap updates for now, just to get the script running commit ac3afe6 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Tue Aug 1 16:19:52 2023 -0400 Massively overhaul main loop, check for and handle duplicate users This is largely untested because I'm having some trouble with the configmaps code, but I wanted to get this committed before I start troubleshooting commit 5295f8f Merge: 29f9332 552e73f Author: nflynt <nicholas.flynt@suse.com> Date: Tue Aug 1 08:58:41 2023 -0400 Merge pull request rancher#1 from crobby/tokenunmigrate Additional unmigration functionality commit 552e73f Author: Chad Roberts <chad.roberts@suse.com> Date: Mon Jul 31 13:22:26 2023 -0400 Additional unmigration functionality commit 29f9332 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 17:30:10 2023 -0400 Actually perform the GUID -> DN migration on the happy path And it works too! Thank goodness. Now we mostly need to clean up the logic and handle a few dozen edge cases. commit 62a6747 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 12:53:43 2023 -0400 Cleanup the logs a bit, flatten the central logic with early exits commit ac20a2c Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Mon Jul 31 09:58:54 2023 -0400 Switch to using the scaledContext for everything Since it can do all the lookups we need, it seems silly to setup and use two different interfaces to the same underlying datastore. The UnstructuredClient is the only way we can read AD configuration right now, and we need that info, so let's stick to that method. commit 18b39d3 Author: Nicholas Flynt <nicholas.flynt@suse.com> Date: Fri Jul 28 17:38:27 2023 -0400 First pass at migration scaffolding, enough to do GUID -> DN lookups There is still much work to do, but at the very least we can read the relevant auth configuration details from k8s and use those details to make LDAP queries, and that's nearly all of what we need to perform the migration.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
On a new rancher server there was a failure to get the hosts from the CDN.
Server rancher server is latest.
The text was updated successfully, but these errors were encountered: