Overly aggressive caching for UWP #1096

Closed
bradwilson opened this Issue Aug 3, 2015 · 8 comments

Projects

None yet

6 participants

@bradwilson

I fell victim to overly aggressive caching twice today while working with UWP projects.

Package Update

I was attempting to upgrade packages using the NuGet UI in VS 2015. NuGet clearly showed a new version of the package; it successfully installed into all my projects EXCEPT the UWP project. This is what it showed:

uwp package update

You can clearly see that the NuGet UI is offering version 110, but NuGet refuses to install it into the UWP package because its cached data says it doesn't exist.

Package Restore

The second time I hit this was during package restore on a CI machine. All packages restored for non-UWP projects including the same versions of other packages; however, the UWP project refused to restore the package which was there (presumably again because of cache, though the NuGet 3.1 beta command line utility was not nearly as forthcoming as the VS UI was):

Unable to find version '2.1.0-beta4-build3106' of package 'xunit.runner.utility'

I logged into the CI machine and tried to run the solution-level restore by hand, using the -nocache flag, but even that wouldn't restore (it seems like the UWP restore ignores that flag). I had to manually install the package into the packages folder with a simple nuget install, which of course succeeded immediately; once the package was then in the local machine cache, the CI builds were allowed to succeed.

Conclusion

Whatever package caching mechanism is in play is clearly: (a) too aggressive, (b) not in line with other NuGet restoration policies (the fact that UI finds the package and then refuses to download it and install it is mind boggling), (c) unable to be overridden by any mechanism.

This was an amazingly poor user experience.

@neurospeech

Please try this, this is long procedure, but once setup, it ignores NuGet caching https://github.com/neurospeech/NuGetProxy

@yishaigalatzer

@neurospeech this is creating extra load on the server that is already overloaded, and does not resolve the issue (because caching happens before you even go out to the wire in this case).

Please be careful with sharing it.

@bradwilson this is the caching code from DNX, we do need to improve it. You can delete the cache manually (yes I know it sucks) and we do need to improve it as soon as possible

CC @davidfowl @damianedwards

@yishaigalatzer yishaigalatzer added this to the 3.1.2 milestone Aug 6, 2015
@neurospeech

@yishaigalatzer this is not creating extra load, in fact all it does is routes query to new version which is used in UI, package is already available on UI and on website. Problem is NuGet is breaking build, one developer adds new package and is not available to other developers. Either package should not be available on UI or if it is available, it should be available everywhere.

@yishaigalatzer

What happens with this proxy is that it routes from methods that hit the lucene index to methods that hit the already overburdened database. Which is a solution to when the lucene index is in bad state (and we have just deployed a major fix)/

The issue described here has to do with local caching for v3 queries that 1. Don't go through the odata server at all and hence rewrite wont help. 2. Are cached aggressively on the user machine and hence wouldn't hit the proxy.

@MysticEmpires MysticEmpires referenced this issue in Yortw/tweetmoasharp Aug 10, 2015
Closed

Issus in Nuget package? #13

@yishaigalatzer

We have a design for this. I will post it later today in our new wiki section

@emgarten
Contributor

I've fixed this for the 3.2.0 release. Installs and updates will only use cached entries newer than the start time of the package operation. If an older entry is cached from before the package was uploaded it will be cleared out.

NuGetArchive/NuGet.PackageManagement@d1d8b39
NuGetArchive/NuGet3@8d2d74f

@emgarten emgarten closed this Aug 26, 2015
@Yortw
Yortw commented Aug 26, 2015

Excellent, thanks everyone!

@emgarten emgarten was unassigned by yishaigalatzer Sep 1, 2015
@emgarten emgarten assigned deepakaravindr and unassigned emgarten Sep 1, 2015
@deepakaravindr
Member

Verified that the behavior is as desired. For Install-Package and Update-Package from powershell console, the cache is busted. Whereas with restore, we use the cache

@deepakaravindr deepakaravindr added 3 - Done and removed 2 - Working labels Sep 2, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment