Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove django-cacheops dependency/usage #1721

Closed
2 of 19 tasks
glennmatthews opened this issue May 2, 2022 · 11 comments · Fixed by #2415 or #3645
Closed
2 of 19 tasks

Remove django-cacheops dependency/usage #1721

glennmatthews opened this issue May 2, 2022 · 11 comments · Fixed by #2415 or #3645
Assignees
@glennmatthews
Labels
type: housekeeping Changes to the application which do not directly impact the end user
Milestone
v2.0.0

Comments

@glennmatthews
Copy link
Contributor

glennmatthews commented May 2, 2022 •
edited by bryanculver
Loading

As ...

Nelly - Network Engineer

I want ...

To have confidence that the Nautobot UI and API are accurately showing me data as it is in the database, not as it was in the past.

So that ...

I will not be shown outdated information that misleads me as to what the current state of the database is, such as can be seen in #1627.

I know this is done when...

  • Nautobot no longer uses django-cacheops
  • The invalidate mgmt command is removed
  • Nautobot overall performance is acceptably similar to where it was when cacheops was still in use
  • Issues such as Manufacturers page does not update correctly #1627 are no longer seen

Optional - Feature groups this request pertains to.

  • Automation
  • Circuits
  • DCIM
  • IPAM
  • Misc (including Data Sources)
  • Organization
  • Plugins (and other Extensibility)
  • Security (Secrets, etc)
  • Image Management
  • UI/UX
  • Documentation
  • Other (not directly a platform feature)

Database Changes

N/A

External Dependencies

Remove django-cacheops dependency; shouldn't add any new dependencies.

TODO
@glennmatthews glennmatthews added the type: feature Introduction of new or enhanced functionality to the application label May 2, 2022
@bryanculver
Copy link
Member

We need to be mindful of how to not regress on #1515.

We leveraged cacheops because if it's file-based caching for the problems it tried to solve there.

@jathanism
Copy link
Contributor

One option is that we could leave cacheops as a dependency for using @cacheops.file_cache.cached() but disable it for use in queryset caching

Another would be to switch to another library that supports disk caching. One of the newer ones DiskCache has built-in support for Django caching framework. A decorator pattern would still need to be considered.

@bryanculver
Copy link
Member

Let's also see if this resolves #1723, #1694

@bryanculver bryanculver added this to the v1.4.0 milestone May 3, 2022
@bryanculver
Copy link
Member

We will likely need to keep the dependency in the short term due to being leveraged by plugins, but ultimately rip out in the future.

@bryanculver bryanculver added the type: housekeeping Changes to the application which do not directly impact the end user label May 17, 2022
@bryanculver bryanculver modified the milestones: v1.4.0, v1.5.0 Jul 12, 2022
@jathanism jathanism self-assigned this Sep 8, 2022
@jathanism
Copy link
Contributor

jathanism commented Sep 8, 2022 •
edited
Loading

Last comment is that we also need to consider the monkey-mixin pattern, to carry forward into v2 when we ditch it. We could forklift the requisite code and ditch cacheops and of course provide attribution in the NOTICE file.

If we did this, then we could completely remove cacheops and replace it with another file-based caching solution.

@glennmatthews glennmatthews mentioned this issue Sep 12, 2022
Closed
@jathanism jathanism mentioned this issue Sep 13, 2022
Merged
6 tasks
@glennmatthews
Copy link
Contributor Author

Is there further work required for 1.5 or can we close this now that #2415 is merged?

@glennmatthews glennmatthews mentioned this issue Oct 24, 2022
Merged
4 tasks
@bryanculver bryanculver modified the milestones: v1.5.0, v2.0.0 Nov 2, 2022
@bryanculver
Copy link
Member

Transitioned #2415 to be the 1.5 milestone record and moving this to be the 2.0 work of removing cacheops entirely.

A majority of the work wasn't done in #2415

@jathanism jathanism mentioned this issue Nov 15, 2022
Closed
@jathanism
Copy link
Contributor

Remaining work to wrap this up, sorted by filename:

nautobot/circuits/api/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related (for tags it should stay)
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/circuits/models.py
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/circuits/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/core/api/authentication.py
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/core/api/views.py
- # v2 TODO(jathan): Revisit whether this is still valid post-cacheops. Re: prefetch_related vs.
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/core/authentication.py
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/core/constants.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/core/settings.py
- "cacheops",  # v2 TODO(jathan); Remove cacheops.
- # v2 TODO(jathan): Remove all cacheops settings.

nautobot/dcim/api/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (content_types should remain
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags because it is m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags because it is m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (extap tags because it is m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)

nautobot/dcim/forms.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/dcim/models/racks.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/dcim/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/extras/api/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related (except the
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/extras/homepage.py
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/extras/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/ipam/api/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tgs: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)
- # v2 TODO(jathan): Replace prefetch_related with select_related (except tags: m2m)

nautobot/ipam/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/tenancy/api/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related (it should stay for tags)

nautobot/tenancy/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/users/admin.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related (these # might be m2m)

nautobot/users/api/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_relatedn

nautobot/utilities/tables.py
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/virtualization/api/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/virtualization/forms.py
- # v2 TODO(jathan): Replace prefetch_related with select_related

nautobot/virtualization/views.py
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related although this one may be
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related
- # v2 TODO(jathan): Replace prefetch_related with select_related

@jathanism
Copy link
Contributor 

nautobot/core/tests/nautobot_config.py
- CACHEOPS_ENABLED = False  # 2.0 TODO(jathan): Remove me.

@bryanculver bryanculver mentioned this issue Jan 13, 2023
Merged
3 tasks
@bryanculver bryanculver removed the type: feature Introduction of new or enhanced functionality to the application label Feb 1, 2023
@bryanculver
Copy link
Member

Blocked by #1633

@bryanculver bryanculver added the status: blocked Another issue or external requirement is preventing implementation label Feb 6, 2023
@bryanculver bryanculver mentioned this issue Feb 14, 2023
Closed
@glennmatthews glennmatthews removed the status: blocked Another issue or external requirement is preventing implementation label Apr 24, 2023
@glennmatthews glennmatthews self-assigned this Apr 24, 2023
glennmatthews added a commit that referenced this issue Apr 24, 2023
@glennmatthews
Fix #1721 - remove django-cacheops
7a5f48c
@glennmatthews glennmatthews mentioned this issue Apr 24, 2023
Merged
6 tasks
@glennmatthews glennmatthews linked a pull request May 2, 2023 that will close this issue
Fix #1721 - remove django-cacheops #3645
Merged
6 tasks
bryanculver pushed a commit that referenced this issue May 5, 2023
@glennmatthews
Fix #1721 - remove django-cacheops (#3645)
73ef56e 
* Fix #1721 - remove django-cacheops
@bryanculver
Copy link
Member

Closed with #3645

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 4, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@glennmatthews glennmatthews

Labels
type: housekeeping Changes to the application which do not directly impact the end user
Projects
No open projects
Nautobot Core ⚙️
Archived in project
Milestone
v2.0.0
3 participants
@bryanculver @jathanism @glennmatthews