Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - optimizations break LaGet / NuGet Simple Server (PHP) / PHPNuget / Gemfury / Azure DevOps #1770

Closed
godofgrunts opened this issue Mar 18, 2019 · 15 comments
Assignees
Milestone

Comments

@godofgrunts
Copy link

godofgrunts commented Mar 18, 2019

What You Are Seeing?

After upgrading to 0.10.12 and 0.10.13, choco install [package] can no longer connect to our local LaGet server. --verbose --trace shows that we are receiving an error 500.

What is Expected?

choco install [package] should connect to our server, download the package, and complete.

How Did You Get This To Happen? (Steps to Reproduce)

  1. Start with choco version 0.10.11.
  2. Install a package in our local source
  3. Uninstall that package
  4. Upgrade choco to version 0.10.13
  5. Try to install package

Output Log

Please see this gist

I have included the full log, but I have also separated out the verbose logs for the 0.10.13 and 0.10.11 attempts.


NOTE: Added by admin

Caused by #1397 optimizations

Release Notes

In Chocolatey 0.10.14, you should run choco feature disable -n usePackageRepositoryOptimizations - this will set Chocolatey back to the older methods it used for package install/upgrade queries. Keep in mind those are horribly inefficient. It would be good to also let the folks who create that repo know they should implement fixes that work appropriately with filtering on Packages() url.

Known Affected Repositories

@godofgrunts
Copy link
Author

The issue seems to be in how packages are requested.

0.10.11 uses

System.Net Information: 0 : [3084] HttpWebRequest#31334997 - Request: GET /api/v2/FindPackagesById()?id='jre6' HTTP/1.1

While 0.10.13 does

System.Net Information: 0 : [5912] HttpWebRequest#6471942 - Request: GET /api/v2/Packages()?$filter=(tolower(Id)%20eq%20'jre6')%20and%20IsLatestVersion HTTP/1.1

@ferventcoder ferventcoder changed the title Chocolatey v0.10.13 is returning 500 internal server error when connecting to local repo. Chocolatey v0.10.13 is returning 500 internal server error when connecting to LaGet Mar 20, 2019
@ferventcoder
Copy link
Member

@godofgrunts thanks for logging this.

Can I ask what you get when you go to <your url>/api/v2/$metadata - It should look something like https://chocolatey.org/api/v2/$metadata (but obviously not exactly like that as it doesn't support data service packages)

@ferventcoder
Copy link
Member

I see this in the logs:

- Using 'http://[REDACTED]/api/v2'.
2019-03-18 12:54:26,010 3528 [DEBUG] - - Supports prereleases? 'True'.
2019-03-18 12:54:26,010 3528 [DEBUG] - - Is ServiceBased? 'True'.

@godofgrunts That second part, Is Service Based - True means it should support queries with the filter, otherwise it should continue to use the old method. Once you get me that $metadata output, we can see what the issue is.

@ferventcoder
Copy link
Member

If it doesn't have a $metadata resource hanging off of it, it should be determined as NOT service-based IIRC.

@godofgrunts
Copy link
Author

Sorry for the delay, here is the $metadata from my server.

@ferventcoder ferventcoder changed the title Chocolatey v0.10.13 is returning 500 internal server error when connecting to LaGet feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well Mar 22, 2019
@ferventcoder ferventcoder self-assigned this Mar 22, 2019
@ferventcoder
Copy link
Member

In working on #1771, we found a few things. For what you are experiencing, we've added a feature that you can disable so that Chocolatey queries in the old inefficient method it did in 0.10.11 and before. I'd recommend reading over #1397 (which caused this issue) to understand why we went this route in the first place.

In the meantime, this is fixed for Chocolatey 0.10.14, where you can simply turn off the feature with choco feature disable -n usePackageRepositoryOptimizations - this will set Chocolatey back to the older methods it used for package install/upgrade queries. There should be a beta available in a little while.

It would be good to also let the folks who work on LaGet know they might want to implement fixes that work appropriately with filtering on Packages() url.

@ferventcoder
Copy link
Member

BTW @godofgrunts the metadata looks pretty standard.

ferventcoder added a commit that referenced this issue Mar 22, 2019
Add a feature turned on by default named
`usePackageRepositoryOptimizations` that uses the new methods for
querying. Also allow turning off per install/upgrade command with
`--disable-repository-optimizations`. When turned off or disabled,
Chocolatey will query for available packages for
install/upgrade/outdated using the older methods. This allows
compatibility with repositories that error or do not support the
optimizations.
ferventcoder added a commit that referenced this issue Mar 22, 2019
* stable:
  (GH-1774) log on failing repo
  (GH-1770) Feature - usePackageRepositoryOptimizations
  (GH-1771) Download nupkg, extract nuspec
  (maint) formatting
  (maint) ensure config for removing nuget cache
  (doc) update copyright year
@godofgrunts
Copy link
Author

Thanks for the info. LaGet hasn't been updated in a long time so maybe it's time I find an alternative NuGet server.

I appreciate your time and providing a work around.

@ferventcoder ferventcoder changed the title feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - LaGet / NuGet Server Mar 22, 2019
@ferventcoder ferventcoder changed the title feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - LaGet / NuGet Server feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - LaGet / NuGet Simple Server (PHP) Mar 22, 2019
@ferventcoder ferventcoder changed the title feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - LaGet / NuGet Simple Server (PHP) feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - optimizations break LaGet / NuGet Simple Server (PHP) Mar 22, 2019
@ferventcoder
Copy link
Member

Related to #1775 - it appears Simple NuGet Server (PHP) is also affected by these optimizations.

@ferventcoder ferventcoder changed the title feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - optimizations break LaGet / NuGet Simple Server (PHP) feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - optimizations break LaGet / NuGet Simple Server (PHP) / PHPNuget Apr 4, 2019
@ferventcoder ferventcoder changed the title feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - optimizations break LaGet / NuGet Simple Server (PHP) / PHPNuget feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - optimizations break LaGet / NuGet Simple Server (PHP) / PHPNuget / Gemfury Apr 17, 2019
@ferventcoder
Copy link
Member

It's been noted that Gemfury also has issues with the repository optimizations.

@godofgrunts
Copy link
Author

Since this breaks compatibility, maybe you should add something to the metadata on the server end that will tell the client that it accepts the new optimized way of searching so this is seemless for end users.

@ferventcoder
Copy link
Member

@godofgrunts that's not a bad idea - like an opt-in kind of scenario.

@kendarorg
Copy link

Tried with PHPNuget (last git version/master) and choco 0.10.15, seems working!

C:\windows\system32>choco install putty.install --source http://localhost:9999/api/v2/
Chocolatey v0.10.15
Installing the following packages:
putty.install
By installing you accept licenses for the packages.

putty.install v0.71
putty.install package files install completed. Performing other installation steps.
The package putty.install wants to run 'chocolateyInstall.ps1'.
Note: If you don't run this script, the installation will fail.
Note: To confirm automatically next time, use '-y' or consider:
choco feature enable -n allowGlobalConfirmation
Do you want to run the script?([Y]es/[A]ll - yes to all/[N]o/[P]rint): A

Installing 64-bit putty.install...
putty.install has been installed.
putty.install may be able to be automatically uninstalled.
Environment Vars (like PATH) have changed. Close/reopen your shell to
see the changes (or in powershell/cmd.exe just type refreshenv).
The install of putty.install was successful.
Software installed as 'msi', install location is likely default.

Chocolatey installed 1/1 packages.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

@israelvaldez
Copy link

israelvaldez commented Jun 5, 2019

Same here (i.e. 0.10.15 works fine now), thanks guys!

@godofgrunts
Copy link
Author

We just updated to 0.10.15 and it's working fine (just to confirm with you). Thank you for the assistance!

@ferventcoder ferventcoder changed the title feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - optimizations break LaGet / NuGet Simple Server (PHP) / PHPNuget / Gemfury feature - usePackageRepositoryOptimizations to control install/upgrade for repositories that don't handle the newer optimizations well - optimizations break LaGet / NuGet Simple Server (PHP) / PHPNuget / Gemfury / Azure DevOps Jul 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants