Allow to clear the cache before anything is cached

[maximlt]

Fixes #5968

[codecov]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (6690bd3) 84.36% compared to head (b33678d) 82.67%.

Files	Patch %	Lines
panel/io/cache.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5981      +/-   ##
==========================================
- Coverage   84.36%   82.67%   -1.70%     
==========================================
  Files         291      291              
  Lines       43452    43458       +6     
==========================================
- Hits        36660    35930     -730     
- Misses       6792     7528     +736     

Flag	Coverage Δ
ui-tests	38.37% <16.66%> (-2.44%)	⬇️
unitexamples-tests	72.26% <83.33%> (-0.20%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.