Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

too many service comparisons #4254

Open
ryanfkeepers opened this issue Sep 14, 2023 · 0 comments
Open

too many service comparisons #4254

ryanfkeepers opened this issue Sep 14, 2023 · 0 comments
Assignees
Labels
tech-debt Non-feature, non-bug improvements to the codebase.

Comments

@ryanfkeepers
Copy link
Contributor

ryanfkeepers commented Sep 14, 2023

Issue

One of the results of the repo's organic growth up to this point is the proliferation of service-specific switches and conditionals. While Corso only covered two services, this was largely a non issue. After three services we ignored the problem. Now in the process of adding Groups and Teams we're seeing just how many places in the code we need to acknowledge the existence of the service in order for the support to behave as expected.

This is a death-by-1000-papercuts maintenance issue. It requires multiple tiny changes to implement high level concepts, increases cognitive load and code familiarity requirements, and will scale poorly with future feature additions. Further, this problem creates a barrier for code contribution: the domain knowledge required for large-scale additions already races against contributor enthusiasm. Adding a hundred points of minutia to update will antagonize the experience.

The various implementations of this issue appear as three basic design smells:

  1. Domain-isolated data containers
  2. Reactive handling to generic structures
  3. One-off service conditionals

Across the solution suggestions for these three issues you'll see a couple common design principles:

  • For behavior and categorization, interfaces should be prioritized to ensure service implementations can self-describe their state.
  • For actual data, generic data structures should be prioritized over Getter interfaces.
  • Packages with clear knowledge and ownership of the utilization of common structures should also own the transformations required by common endpoints.
  • Proactive is better than reactive. New implementation should comply with expected, identified conditions to minimize the risk of missing a needed implementation.

Domain-isolated data containers

Example: Details ItemInfo

Domain isolation occurs when we juggle multiple structs to identify expected metadata, as opposed to a generic data structure (such as a map) and/or a single interface value.

In the example above, data is split acros five different DomainInfo entries (four for services, one for folders). In order to utilize an ItemInfo struct, all code must first check which of these structs is actually populated (if any), either by comparing each value or by utilizing some external identifier such as path service type. In addition, many of these types express overlapping data (ex: four of the five contain DriveID and DriveName values). The result of this is not only poor scaling and code duplication, but also potential gotchas around constraints like having multiple populated types or seeing different in property keys between structs for similar values.

An ideal solution would reduce the ItemInfo to a single DomainInfo interface. The interface could describe common data categorization checks (WhatCategory, WhatService, IsFolder, IsDriveItem, etc), and should be able to produce a common container of data with well-known property keys (ie: either map or single, standardized struct).

Reactive handling to generic structures

Example: Paths Transformers

Reactive handling occurs when transformations that could be owned by the caller get passed downstream into the generic function instead.

This issue can be fairly nuanced as there's a fine line between elegant generic code and this smell. The key issue here occurs when the caller is an opinionated package with clear knowledge about how it utilizes a generic structure, but still passes off those utilization details onto the centralized function. This punt distributes design ownership between the two packages, and requires maintainers to remember to update both pacakges for a successful change.

By identifying the places where opinionated packages expect generic functions to understand their design, we can pull those transformations back into the owning package to centralize that design specification within a single location. Future implementations should naturally recognize when they're being asked to own these transformations by the nature of the func parameters and comments. This priority also ensures a minimum of churn within generic packages, and thus the smallest likelihood of unintentional changes to other packages that share the code space.

One-off service conditionals

Example: m365 backup

One offs are the most insidious of the issues, and also the most difficult to tackle. By their nature, there's no large-scale solution. We've allowed them to proliferate because each occurrence seems benign at the time of its implementation. It's only now that we've gone back to recount all of the places we have to update a one-off (two-off, I suppose?) that the burden of maintenance becomes clear.

The best solution here is a change in accepted code standards: prevent the adoption of one-offs in the first place. It's always a case-by-case basis. Every case will be unique, and the contributor will need to ask themselves, "how do we avoid a conditional or switch here?". Any effort we put in up-front to avoid a one-off will ensure more stable code with less maintenance and contribution whiplash in the future.

Required Changes

These problems are too large and too distributed to address in a single triage. Changes will need to be divided into smaller chunks, each with their own ticket. After each one, we can revisit this parent issue to check whether the changes are sufficient to close it out.

@ryanfkeepers ryanfkeepers added triage-needed tech-debt Non-feature, non-bug improvements to the codebase. labels Sep 14, 2023
aviator-app bot pushed a commit that referenced this issue Oct 11, 2023
Do a couple of high-level things with drive handlers:
* pull out generic functionality into a base handler
* use handler-specific implementations of AugmentItemInfo
* remove old switch-based augmentItemInfo

Overall goal is to reduce places we use service type
comparisons to handle behavior differences

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Oct 11, 2023
Renames focus on OneDrive and Libraries handlers.
Files are also renamed to match

item -> userDrive
library -> site

If diff is large due to file renames view by
commit since file renames are done in a separate
commit

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
@ashmrtn ashmrtn self-assigned this Oct 13, 2023
aviator-app bot pushed a commit that referenced this issue Oct 26, 2023
Export doesn't pull meaningful stats from the controller. Just remove
the collection/usage of these stats since it's confusing. Substitute the
check for not doing anything from a check on stats to a check on the
number of collections returned by kopia.

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
@ashmrtn ashmrtn mentioned this issue Oct 27, 2023
12 tasks
aviator-app bot pushed a commit that referenced this issue Nov 1, 2023
First step in reducing the number of places we have to check the service type manually. Create a way to get a handle to a service specific handler and implement exports for those handlers

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Nov 13, 2023
Rename these handlers to be base<Service>Handler so that we get some segmentation of functionality. This will allow us a bit of compile-time safety when it comes to accessing graph because the base handler doesn't contain a client instance. Essentially it allows us local access to things for operations like restore.

Future additions to the handler that require a client should wrap this base handler to provide that functionality. Export functions shouldn't be updated to be part of the new handler wrapper but should stay as part of the base handler so they continue to not have access to graph.

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
ryanfkeepers pushed a commit that referenced this issue Nov 15, 2023
Rename these handlers to be base<Service>Handler so that we get some segmentation of functionality. This will allow us a bit of compile-time safety when it comes to accessing graph because the base handler doesn't contain a client instance. Essentially it allows us local access to things for operations like restore.

Future additions to the handler that require a client should wrap this base handler to provide that functionality. Export functions shouldn't be updated to be part of the new handler wrapper but should stay as part of the base handler so they continue to not have access to graph.

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Nov 16, 2023
)

This continues the push towards having service-level handlers that know
how to perform different operations. It adds the helper functions that
will be used during restore operations to the existing handler code

This logic is not currently used nor does this PR change the restore
call path

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Nov 16, 2023
Add service-level restore handlers for exchange

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

merge after:
* #4687
* #4651

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Nov 18, 2023
Minor cleanup that will also help reduce diff for future changes.

Instead of taking in a details builder and adding to it during
restore, just create a local details builder and return the built
details to the caller

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Nov 27, 2023
Rework restore return status so that later PRs will have a smaller diff

Instead of returning a status and then waiting on the message just return the restore stats directly. The status getter was only setup to wait for one item anyway and was setup to wait after the entire restore operation already completed (at end of
m365/restore.go:ConsumeRestoreCollections())

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
aviator-app bot pushed a commit that referenced this issue Nov 28, 2023
Switch restore code path to use service-level handlers. Does a few
things:
* switches existing service-level functions to be methods on
  service-level handlers
* update interfaces as necessary
* moves some logic from old controller-level restore function to either
  the new handlers or the operation-level function
* removes old code

May be easiest to review by commit

---

#### Does this PR need a docs update or release note?

- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tech-debt Non-feature, non-bug improvements to the codebase.
Projects
None yet
Development

No branches or pull requests

3 participants