New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overly aggressive caching for UWP #1096

Closed
bradwilson opened this Issue Aug 3, 2015 · 8 comments

Comments

Projects
None yet
6 participants
@bradwilson

bradwilson commented Aug 3, 2015

I fell victim to overly aggressive caching twice today while working with UWP projects.

Package Update

I was attempting to upgrade packages using the NuGet UI in VS 2015. NuGet clearly showed a new version of the package; it successfully installed into all my projects EXCEPT the UWP project. This is what it showed:

uwp package update

You can clearly see that the NuGet UI is offering version 110, but NuGet refuses to install it into the UWP package because its cached data says it doesn't exist.

Package Restore

The second time I hit this was during package restore on a CI machine. All packages restored for non-UWP projects including the same versions of other packages; however, the UWP project refused to restore the package which was there (presumably again because of cache, though the NuGet 3.1 beta command line utility was not nearly as forthcoming as the VS UI was):

Unable to find version '2.1.0-beta4-build3106' of package 'xunit.runner.utility'

I logged into the CI machine and tried to run the solution-level restore by hand, using the -nocache flag, but even that wouldn't restore (it seems like the UWP restore ignores that flag). I had to manually install the package into the packages folder with a simple nuget install, which of course succeeded immediately; once the package was then in the local machine cache, the CI builds were allowed to succeed.

Conclusion

Whatever package caching mechanism is in play is clearly: (a) too aggressive, (b) not in line with other NuGet restoration policies (the fact that UI finds the package and then refuses to download it and install it is mind boggling), (c) unable to be overridden by any mechanism.

This was an amazingly poor user experience.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Aug 6, 2015

Please try this, this is long procedure, but once setup, it ignores NuGet caching https://github.com/neurospeech/NuGetProxy

ghost commented Aug 6, 2015

Please try this, this is long procedure, but once setup, it ignores NuGet caching https://github.com/neurospeech/NuGetProxy

@yishaigalatzer

This comment has been minimized.

Show comment
Hide comment
@yishaigalatzer

yishaigalatzer Aug 6, 2015

@neurospeech this is creating extra load on the server that is already overloaded, and does not resolve the issue (because caching happens before you even go out to the wire in this case).

Please be careful with sharing it.

@bradwilson this is the caching code from DNX, we do need to improve it. You can delete the cache manually (yes I know it sucks) and we do need to improve it as soon as possible

CC @davidfowl @DamianEdwards

yishaigalatzer commented Aug 6, 2015

@neurospeech this is creating extra load on the server that is already overloaded, and does not resolve the issue (because caching happens before you even go out to the wire in this case).

Please be careful with sharing it.

@bradwilson this is the caching code from DNX, we do need to improve it. You can delete the cache manually (yes I know it sucks) and we do need to improve it as soon as possible

CC @davidfowl @DamianEdwards

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Aug 7, 2015

@yishaigalatzer this is not creating extra load, in fact all it does is routes query to new version which is used in UI, package is already available on UI and on website. Problem is NuGet is breaking build, one developer adds new package and is not available to other developers. Either package should not be available on UI or if it is available, it should be available everywhere.

ghost commented Aug 7, 2015

@yishaigalatzer this is not creating extra load, in fact all it does is routes query to new version which is used in UI, package is already available on UI and on website. Problem is NuGet is breaking build, one developer adds new package and is not available to other developers. Either package should not be available on UI or if it is available, it should be available everywhere.

@yishaigalatzer

This comment has been minimized.

Show comment
Hide comment
@yishaigalatzer

yishaigalatzer Aug 7, 2015

What happens with this proxy is that it routes from methods that hit the lucene index to methods that hit the already overburdened database. Which is a solution to when the lucene index is in bad state (and we have just deployed a major fix)/

The issue described here has to do with local caching for v3 queries that 1. Don't go through the odata server at all and hence rewrite wont help. 2. Are cached aggressively on the user machine and hence wouldn't hit the proxy.

yishaigalatzer commented Aug 7, 2015

What happens with this proxy is that it routes from methods that hit the lucene index to methods that hit the already overburdened database. Which is a solution to when the lucene index is in bad state (and we have just deployed a major fix)/

The issue described here has to do with local caching for v3 queries that 1. Don't go through the odata server at all and hence rewrite wont help. 2. Are cached aggressively on the user machine and hence wouldn't hit the proxy.

@yishaigalatzer

This comment has been minimized.

Show comment
Hide comment
@yishaigalatzer

yishaigalatzer Aug 19, 2015

We have a design for this. I will post it later today in our new wiki section

yishaigalatzer commented Aug 19, 2015

We have a design for this. I will post it later today in our new wiki section

@emgarten

This comment has been minimized.

Show comment
Hide comment
@emgarten

emgarten Aug 26, 2015

Collaborator

I've fixed this for the 3.2.0 release. Installs and updates will only use cached entries newer than the start time of the package operation. If an older entry is cached from before the package was uploaded it will be cleared out.

NuGet/NuGet.PAckageMAnagement@d1d8b391fe45c60c5e7382f8c1af82c851ba871c
NuGetArchive/NuGet3@8d2d74f

Collaborator

emgarten commented Aug 26, 2015

I've fixed this for the 3.2.0 release. Installs and updates will only use cached entries newer than the start time of the package operation. If an older entry is cached from before the package was uploaded it will be cleared out.

NuGet/NuGet.PAckageMAnagement@d1d8b391fe45c60c5e7382f8c1af82c851ba871c
NuGetArchive/NuGet3@8d2d74f

@emgarten emgarten closed this Aug 26, 2015

@Yortw

This comment has been minimized.

Show comment
Hide comment
@Yortw

Yortw Aug 26, 2015

Excellent, thanks everyone!

Yortw commented Aug 26, 2015

Excellent, thanks everyone!

@deepakaravindr

This comment has been minimized.

Show comment
Hide comment
@deepakaravindr

deepakaravindr Sep 2, 2015

Member

Verified that the behavior is as desired. For Install-Package and Update-Package from powershell console, the cache is busted. Whereas with restore, we use the cache

Member

deepakaravindr commented Sep 2, 2015

Verified that the behavior is as desired. For Install-Package and Update-Package from powershell console, the cache is busted. Whereas with restore, we use the cache

@deepakaravindr deepakaravindr added 3 - Done and removed 2 - Working labels Sep 2, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment