Disable merging of chocolatey searches #14327
Conversation
jaymzjulian
force-pushed
the
jj/no-more-choco-search-merging
branch
from
c82da2f
to
306427e
Compare
|
The kitchen failure seems to be an issue with the test suite: I don't think i can force it to re-run myself |
|
@jaymzjulian what sets |
|
An implementation where the upstream nuget can support multi query properly
would set it. So currently that would be never, but I was avoiding
completely closing that off. Happy to remove that is desired.
…On Wed, Apr 10, 2024, 7:03 AM John McCrae ***@***.***> wrote:
@jaymzjulian <https://github.com/jaymzjulian> what sets walk_resource_tree:
true
—
Reply to this email directly, view it on GitHub
<#14327 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGAHF2BUFXJ4YJWYH4YOV3Y4VBBVAVCNFSM6AAAAABF76FDP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBXGYZTQNJTGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@jaymzjulian Hi, can i ask for a bit more documentation on the walk_resource_tree variable and when we might use it so we have a richer future context? |
There was a time when the following worked: ``` choco list -r firefox wireshark choco search -r firefox wireshark ``` However, it turns out this only worked by accident, not by spec - and this has stopped working now. It's known to not work, see chocolatey/choco#2116 . Additionally, the other potential workarounds are also not well supported. So instead, we tighten the caching a little to at least avoid multiple API requests for the same package, and drop back to single package at a time searches for now, since that's what upstream chocolatey demands. Once we can fix the issue in chocolatey itself, then we can bring this feature back. (if you have just teh right server versions, which I did when I was testing, it does work - I was able to reproduce it as a regression, but it turns out what I considered "working" was, in fact, the bug and not the correct behaviour) Note that this means now you see in debug: ``` [2024-04-10T00:36:54-07:00] INFO: Fetching chocolatey package list [2024-04-10T00:36:54-07:00] DEBUG: Choco List Command: ["search", "-r", "firefox"] [2024-04-10T00:36:55-07:00] DEBUG: Choco List Command: ["search", "-r", "wireshark"] [2024-04-10T00:36:58-07:00] INFO: chocolatey_package[wireshark, firefox] upgraded(allow_downgrade) ["wireshark"] to ["3.0.1"] ``` which is correct per chocolatey spec, sadly. Signed-off-by: Jaymz Julian <jaymzjulian@meta.com>
jaymzjulian
force-pushed
the
jj/no-more-choco-search-merging
branch
from
306427e
to
66138f9
Compare
|
yes you can - i've just pushed an updated comment around that
…On Wed, Apr 10, 2024 at 11:44 AM John McCrae ***@***.***> wrote:
@jaymzjulian <https://github.com/jaymzjulian> Hi, can i ask for a bit
more documentation on the walk_resource_tree variable and when we might use
it so we have a richer future context?
—
Reply to this email directly, view it on GitHub
<#14327 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGAHF4QC22LRG7NYHZCZJLY4WB7TAVCNFSM6AAAAABF76FDP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBYGIYTINRRGM>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
The 2 failures are related to a yum configuration issue and have nothing to do with the code at hand. |
|
@jaymzjulian I keep getting a failure when I try to add 4 packages. The first package is Babyftp. My recipe block is in the middle of this error. What am I doing wrong? ❯ bundle exec chef-apply .\choco_tester.rb
ffi-libarchive could not be loaded, libarchive is probably not installed on system, archive_file will not be available
Recipe: (chef-apply cookbook)::(chef-apply recipe)
* chocolatey_package[install all the things] action install
================================================================================
Error executing action `install` on resource 'chocolatey_package[install all the things]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0, 2], but received '1'
---- Begin output of ["C:\\ProgramData\\chocolatey/bin/choco.exe", "install", "-y", "babyftp", "firefox", "winscp", "wireshark"] ----
STDOUT: Chocolatey v2.2.2
Installing the following packages:
babyftp;firefox;winscp;wireshark
By installing, you accept licenses for the packages.
Progress: Downloading babyftp 1.24... 100%
babyftp v1.24.0 [Approved]
babyftp package files install completed. Performing other installation steps.
babyftp not installed. An error occurred during installation:
Item has already been added. Key in dictionary: 'PATH' Key being added: 'Path'
Chocolatey installed 0/0 packages.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
The process cannot access the file 'C:\ProgramData\chocolatey\lib\babyftp\.chocolateyPending' because it is being used by another process.
STDERR:
---- End output of ["C:\\ProgramData\\chocolatey/bin/choco.exe", "install", "-y", "babyftp", "firefox", "winscp", "wireshark"] ----
Ran ["C:\\ProgramData\\chocolatey/bin/choco.exe", "install", "-y", "babyftp", "firefox", "winscp", "wireshark"] returned 1
Resource Declaration:
---------------------
# In .\choco_tester.rb
1: chocolatey_package 'install all the things' do
2: package_name ['babyftp', 'firefox', 'winscp', 'wireshark']
3: action :install
4: end
Compiled Resource:
------------------
# Declared in .\choco_tester.rb:1:in `run_chef_recipe'
chocolatey_package("install all the things") do
package_name ["babyftp", "firefox", "winscp", "wireshark"]
action [:install]
default_guard_interpreter :default
declared_type :chocolatey_package
cookbook_name "(chef-apply cookbook)"
recipe_name "(chef-apply recipe)"
returns [0, 2]
end
System Info:
------------
chef_version=18.4.34
platform=windows
platform_version=10.0.22631
ruby=ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x64-mingw-ucrt]
program_name=C:/Users/JohnMcCrae/AppData/Local/chef/gem/ruby/3.1.0/bin/chef-apply
executable=C:/Users/JohnMcCrae/AppData/Local/chef/gem/ruby/3.1.0/bin/chef-apply
[2024-04-11T07:31:29-07:00] FATAL: Stacktrace dumped to C:/chef/cache/chef-stacktrace.out
[2024-04-11T07:31:29-07:00] FATAL: ---------------------------------------------------------------------------------------
[2024-04-11T07:31:29-07:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2024-04-11T07:31:29-07:00] FATAL: ---------------------------------------------------------------------------------------
[2024-04-11T07:31:29-07:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: chocolatey_package[install all the things] ((chef-apply cookbook)::(chef-apply recipe) line 1) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0, 2], but received '1'
---- Begin output of ["C:\\ProgramData\\chocolatey/bin/choco.exe", "install", "-y", "babyftp", "firefox", "winscp", "wireshark"] ----
STDOUT: Chocolatey v2.2.2
Installing the following packages:
babyftp;firefox;winscp;wireshark
By installing, you accept licenses for the packages.
Progress: Downloading babyftp 1.24... 100%
babyftp v1.24.0 [Approved]
babyftp package files install completed. Performing other installation steps.
babyftp not installed. An error occurred during installation:
Item has already been added. Key in dictionary: 'PATH' Key being added: 'Path'
Chocolatey installed 0/0 packages.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
The process cannot access the file 'C:\ProgramData\chocolatey\lib\babyftp\.chocolateyPending' because it is being used by another process.
STDERR:
---- End output of ["C:\\ProgramData\\chocolatey/bin/choco.exe", "install", "-y", "babyftp", "firefox", "winscp", "wireshark"] ----
Ran ["C:\\ProgramData\\chocolatey/bin/choco.exe", "install", "-y", "babyftp", "firefox", "winscp", "wireshark"] returned 1 |
[... omitted...]
Thisn is actually a chocolatey bug - you can see that it's doing the right command in your stderr (that's the log bit i left in), but chocolatey nis tripping over itself. Usually it's because the C:\ProgramData\chocolatey\lib\babyftp.chocolateyPending file is left over from an aborted process. as for the |
|
I guess I should have added, @jaymzjulian, that I get that babyftp error the first time I try to add it. Choco leaves garbage behind, I choco uninstall it, I nuke the babyftp directory from the choco cache and try it again and I get the same error., |
|
This works correctly from a powershell prompt: |
Oh that's really interesting - obviously the same command should yield the same result in and out of chef - particular if you can have it happen reliably! I'll see if I can reproduce this here - one question is if you leave out babyftp, does it work through chef then? I.e. is this babyftp metadata related |
results in: On a fresh windows box with chocolatey 2.2.2. So I do think the issue you're experiencing there is isolated to your system, at least somewhat :(. (which is to say: the initial issue is totally legit!) |
|
action :upgrade also seems to work for me in the same configuration |
There was a problem hiding this comment.
I'm going to try @johnmccrae's test a couple of places to double-check
|
@jaymzjulian Hi, can you please fix the DCO bits and take a look at the Windows functional tests. All the choco package stuff is failing there. |
Co-authored-by: Thomas Powell <104777878+tpowell-progress@users.noreply.github.com> Signed-off-by: Jaymz Julian <jaymzjulian@meta.com>
jaymzjulian
force-pushed
the
jj/no-more-choco-search-merging
branch
from
cb58ab3
to
1128e85
Compare
|
huh, TIL that accepting a correction in github breaks DCO :( looking at why windows functional tests are failing now - weirdly they weren't earlier in the week! |
Looks like every case passed after re-pushing with the DCO fix. THat said, I did notice something in the logs - i don't think that affects this, but.... just as an FYI as much as anything - this is from win2002 integration in buildkite: Other tests are succesfully upgrading to choco2 though, so it's sitll getting exercised at least. I don't know enough about the build kite stuff to easily fix the test :(. |
|
Just checking this isn't waiting on me anymore - is there something else needed here? |
|
@jaymzjulian the choco install errors are due to the image not having .NET 4.8, so not related to your changes. However, I am not a bit concerned with how well choco is being tested/covered. |
(Note from @tpowell-progress ... this maps back to CHEF-10654)
Description
There was a time when the following worked:
However, it turns out this only worked by accident, not by spec - and this has stopped working now. It's known to not work, see chocolatey/choco#2116 . Additionally, the other potential workarounds are also not well supported. So instead, we tighten the caching a little to at least avoid multiple API requests for the same package, and drop back to single package at a time searches for now, since that's what upstream chocolatey demands. Once we can fix the issue in chocolatey itself, then we can bring this feature back.
(if you have just teh right server versions, which I did when I was testing, it does work - I was able to reproduce it as a regression, but it turns out what I considered "working" was, in fact, the bug and not the correct behaviour)
Note that this means now you see in debug:
which is correct per chocolatey spec, sadly.
Related Issue
[This exists, but I don't have it myself - it was reported to me by @johnmccrae
Types of changes
Checklist:
Gemfile.lockhas changed, I have used--conservativeto do it and included the full output in the Description above.