[Bug]: Web Searches catalog empty

[rosano]

Before submitting your bug report, please confirm you have completed the following steps

• Searched the issue is not already listed in the Issue List (I've searched it)
• Searched in the Quicksilver Users' Group for questions relating to tech support.
• Downloaded the latest version of Quicksilver for your system and confirmed the issue still persists.

Bug description

Continuing from https://groups.google.com/g/blacktree-quicksilver/c/SAiNvKCukT8

It seems the Web Searches catalog is empty. When clicking 'Rescan resource' it seems to complete the scan without finding results:

Steps to reproduce

1. Enter 'hello' via dot mode
2. Select the 'Find with…' action
3. [there are no options here]

Expected behavior

The options from https://qs0.qsapp.com/plugin-data/web-search-list.php?type=simple should be visible.

MacOS Version

macOS 10.14

Quicksilver Version

2.0.3 (4031)

Crash Logs or Spindump

No response

Screenshots

No response

Additional info

Based on the thread, I wonder if it's a command-line tools issue as I recently removed Xcode from my system. But I'm not clear if it's asking for permission.

[pjrobertson]

It's unlikely to be related to the command-line tools, the Web Search plugin isn't connected. Can you please update the original post with a little more information:

1. What version of the Web Search Plugin are you using
2. In the Google Groups thread, you said it was hanging. If so, can you please provide a spin dump
3. If it's not hanging, please provide an output from Console.app. Filter by "process:quicksilver"

Thanks

[rosano]

@pjrobertson I'm using the Web Search Plugin version 2.9.1. Someone else said it was hanging actually, it's not hanging for me. Here's the output when pressing "Rescan source"

default	11:18:49.712943 +0200	Quicksilver	27366555: RECEIVED OUT-OF-SEQUENCE NOTIFICATION: 452 vs 4241, 512, <private>
default	11:18:53.162580 +0200	Quicksilver	LSExceptions shared instance invalidated for timeout.
default	11:19:04.629702 +0200	Quicksilver	Task <D2F98928-812D-4D07-9E79-75E35D8B18B0>.<0> {strength 0, tls 4, ct 0, sub 0, sig 1, ciphers 0, bundle 0, builtin 0}
default	11:19:04.630546 +0200	Quicksilver	TIC Enabling TLS [3:0x7faf20354390]
default	11:19:04.630639 +0200	Quicksilver	TIC TCP Conn Start [3:0x7faf20354390]
default	11:19:04.631208 +0200	Quicksilver	[C3 Hostname#21b81749:443 tcp, url hash: 983dd876, tls] start
default	11:19:04.632304 +0200	Quicksilver	nw_connection_report_state_with_handler_locked [C3] reporting state preparing
default	11:19:04.633635 +0200	Quicksilver	Task <D2F98928-812D-4D07-9E79-75E35D8B18B0>.<0> setting up Connection 3
default	11:19:04.762683 +0200	Quicksilver	nw_socket_handle_socket_event [C3.1:3] Socket received CONNECTED event
default	11:19:04.762885 +0200	Quicksilver	nw_endpoint_flow_protocol_connected [C3.1 IPv4#b3c2f4ab:443 in_progress socket-flow (satisfied)] Transport protocol connected
default	11:19:04.766361 +0200	Quicksilver	TIC TLS Event [3:0x7faf20354390]: 1, Pending(0)
default	11:19:04.870523 +0200	Quicksilver	TIC TLS Event [3:0x7faf20354390]: 11, Pending(0)
default	11:19:04.870642 +0200	Quicksilver	TIC TLS Event [3:0x7faf20354390]: 14, Pending(0)
default	11:19:05.027023 +0200	Quicksilver	TIC TLS Trust Result [3:0x7faf20354390]: 0
default	11:19:05.028112 +0200	Quicksilver	TIC TLS Event [3:0x7faf20354390]: 2, Pending(0)
default	11:19:05.130914 +0200	Quicksilver	nw_endpoint_flow_protocol_connected [C3.1 IPv4#b3c2f4ab:443 in_progress socket-flow (satisfied)] Output protocol connected
default	11:19:05.131108 +0200	Quicksilver	nw_connection_report_state_with_handler_locked [C3] reporting state ready
default	11:19:05.131730 +0200	Quicksilver	TIC TLS Event [3:0x7faf20354390]: 20, Pending(0)
default	11:19:05.131780 +0200	Quicksilver	TIC TCP Conn Connected [3:0x7faf20354390]: Err(16)
default	11:19:05.132081 +0200	Quicksilver	TIC TCP Conn Event [3:0x7faf20354390]: 1
default	11:19:05.132144 +0200	Quicksilver	TIC TCP Conn Event [3:0x7faf20354390]: 8
default	11:19:05.132206 +0200	Quicksilver	TIC TLS Handshake Complete [3:0x7faf20354390]
default	11:19:05.132495 +0200	Quicksilver	Task <D2F98928-812D-4D07-9E79-75E35D8B18B0>.<0> now using Connection 3
default	11:19:05.132717 +0200	Quicksilver	nw_endpoint_flow_protocol_connected [C3.1 IPv4#b3c2f4ab:443 ready socket-flow (satisfied)] Output protocol connected
default	11:19:05.132970 +0200	Quicksilver	Task <D2F98928-812D-4D07-9E79-75E35D8B18B0>.<0> sent request, body N
default	11:19:05.238680 +0200	Quicksilver	Task <D2F98928-812D-4D07-9E79-75E35D8B18B0>.<0> received response, status 200 content C
default	11:19:05.239047 +0200	Quicksilver	Task <D2F98928-812D-4D07-9E79-75E35D8B18B0>.<0> response ended
default	11:19:05.239145 +0200	Quicksilver	Task <D2F98928-812D-4D07-9E79-75E35D8B18B0>.<0> done using Connection 3
default	11:19:12.082443 +0200	Quicksilver	27366555: RECEIVED OUT-OF-SEQUENCE NOTIFICATION: 456 vs 8471, 512, <private>
default	11:19:15.998960 +0200	Quicksilver	27366555: RECEIVED OUT-OF-SEQUENCE NOTIFICATION: 100 vs 456, 640, <private>
default	11:19:17.302489 +0200	Quicksilver	CoreAnimation: warning, deleted thread with uncommitted CATransaction; set CA_DEBUG_TRANSACTIONS=1 in environment to log backtraces, or set CA_ASSERT_MAIN_THREAD_TRANSACTIONS=1 to abort when an implicit transaction isn't created on a main thread.
default	11:19:18.257212 +0200	Quicksilver	27366555: RECEIVED OUT-OF-SEQUENCE NOTIFICATION: 101 vs 8472, 640, <private>
default	11:19:21.571028 +0200	Quicksilver	27366555: RECEIVED OUT-OF-SEQUENCE NOTIFICATION: 0 vs 101, 513, <private>
default	11:19:21.846022 +0200	Quicksilver	LSExceptions shared instance invalidated for timeout.

[skurfer]

I’m seeing this now, so maybe I can debug if I get time tonight. I noticed a few days ago that the list of plug-ins wouldn’t update for me. It’s populated now, but the web searches have gone missing. I wonder if this is related to the switch to NSURLSession.

[pjrobertson]

I’ve seen this now with the release version of 2.0.3. But every local build I make is fine. The NSURLSession switch was just the Updater stuff so won’t affect this. The Web Searches source uses QSParser’s: - (NSArray *)objectsFromURL:(NSURL *)url withSettings:(NSDictionary *)settings; And that hasn’t been touched since 2012. However, a quick look through QSHTMLLinkParser.m shows that it’s launching a task to run QSURLExtractor.py Perhaps that’s what’s failing - possible that file hasn’t been code-signed so can’t run? (see QSHTMLLinkParser.m:L49) Seems to me it’d be much simpler to do away with the HTML formatted web searches, and use a JSON dict instead of launching a task to run a python script.

…

On 11 Apr 2022, at 22:30, Rob McBroom ***@***.***> wrote: I’m seeing this now, so maybe I can debug if I get time tonight. I noticed a few days ago that the list of plug-ins wouldn’t update for me. It’s populated now, but the web searches have gone missing. I wonder if this is related to the switch to NSURLSession. — Reply to this email directly, view it on GitHub <#2728 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEXH2LRXI2PKBFEOISRDDVEQZR5ANCNFSM5TAMFXIA>. You are receiving this because you were mentioned.

[n8henrie]

Uh oh, also working well for me, but I updated QSURLExtractor.py to python3 in b619167 as part of #2571.

@rosano it looks like you're on 10.14? You might not have python3 available (it didn't come in older versions of MacOS by default... on newer versions maybe prompts you to install command line tools if not there?). If you open a terminal prompt and run python3 --version (or in QS input the same and Run Command in Shell if you have the Terminal Plugin installed), what do you see?

[n8henrie]

I don't think continuing to support python2 is a great choice, as it's not even getting security patches anymore.

https://www.python.org/doc/sunset-python-2/

We are volunteers who make and take care of the Python programming language. We have decided that January 1, 2020, was the day that we sunset Python 2. That means that we will not improve it anymore after that day, even if someone finds a security problem in it. You should upgrade to Python 3 as soon as you can.

Also, it looks like MacOS no longer includes python2 by default as of 12.3. On my machine:

$ find /usr/bin /bin /usr/sbin -name 'python*' -executable 2>/dev/null
/usr/bin/python3
$

[skurfer]

A quick way to test if the URL parser is able to run: Enter python.org in text entry mode, hit ⎋ to turn it into a URL, then hit → or /. Do you see the links from the page? It works for me.

[skurfer]

🤦🏻‍♂️

If I enable those presets before I attempt to rescan, they load just fine. So I can’t reproduce it after all.

[pjrobertson]

A quick way to test if the URL parser is able to run: Enter python.org in text entry mode, hit ⎋ to turn it into a URL, then hit → or /. Do you see the links from the page? It works for me.

OK, that's a good test. I can confirm that using the Released build (2.0.3) I cannot → into python.org. I'm leaning towards thinking this is a codesigning issue. I will work on implementing QSURLParser.py into pure Obj-c using either this or this

[pjrobertson]

@rosano - can you please download the following version of Quicksilver, and let us know if it solves the problem:

https://github.com/quicksilver/Quicksilver/suites/6092242958/artifacts/211189389

Thanks!

[rosano]

@pjrobertson still with the same problem, no entries in 'Find with' and here's the log when clicking 're-scan source'

default	15:51:57.782345 +0200	Quicksilver	Task <4C80BF3D-1D8E-4571-BDF8-986E51685F44>.<0> {strength 0, tls 4, ct 0, sub 0, sig 1, ciphers 0, bundle 0, builtin 0}
default	15:51:57.782724 +0200	Quicksilver	TIC Enabling TLS [2:0x6000036d7000]
default	15:51:57.782808 +0200	Quicksilver	TIC TCP Conn Start [2:0x6000036d7000]
default	15:51:57.783147 +0200	Quicksilver	[C2 Hostname#e4a130b5:443 tcp, url hash: 584f4cfa, tls] start
default	15:51:57.783927 +0200	Quicksilver	nw_connection_report_state_with_handler_locked [C2] reporting state preparing
default	15:51:57.785031 +0200	Quicksilver	Task <4C80BF3D-1D8E-4571-BDF8-986E51685F44>.<0> setting up Connection 2
default	15:51:57.890131 +0200	Quicksilver	nw_socket_handle_socket_event [C2.1:3] Socket received CONNECTED event
default	15:51:57.890253 +0200	Quicksilver	nw_endpoint_flow_protocol_connected [C2.1 IPv4#88fb2c54:443 in_progress socket-flow (satisfied)] Transport protocol connected
default	15:51:57.891879 +0200	Quicksilver	TIC TLS Event [2:0x6000036d7000]: 1, Pending(0)
default	15:51:58.002607 +0200	Quicksilver	TIC TLS Event [2:0x6000036d7000]: 11, Pending(0)
default	15:51:58.002684 +0200	Quicksilver	TIC TLS Event [2:0x6000036d7000]: 14, Pending(0)
default	15:51:58.058090 +0200	Quicksilver	TIC TLS Trust Result [2:0x6000036d7000]: 0
default	15:51:58.059127 +0200	Quicksilver	TIC TLS Event [2:0x6000036d7000]: 2, Pending(0)
default	15:51:58.165626 +0200	Quicksilver	nw_endpoint_flow_protocol_connected [C2.1 IPv4#88fb2c54:443 in_progress socket-flow (satisfied)] Output protocol connected
default	15:51:58.166309 +0200	Quicksilver	nw_connection_report_state_with_handler_locked [C2] reporting state ready
default	15:51:58.167278 +0200	Quicksilver	TIC TLS Event [2:0x6000036d7000]: 20, Pending(0)
default	15:51:58.167474 +0200	Quicksilver	TIC TCP Conn Connected [2:0x6000036d7000]: Err(16)
default	15:51:58.168107 +0200	Quicksilver	TIC TCP Conn Event [2:0x6000036d7000]: 1
default	15:51:58.168240 +0200	Quicksilver	TIC TCP Conn Event [2:0x6000036d7000]: 8
default	15:51:58.168347 +0200	Quicksilver	TIC TLS Handshake Complete [2:0x6000036d7000]
default	15:51:58.168791 +0200	Quicksilver	Task <4C80BF3D-1D8E-4571-BDF8-986E51685F44>.<0> now using Connection 2
default	15:51:58.169104 +0200	Quicksilver	nw_endpoint_flow_protocol_connected [C2.1 IPv4#88fb2c54:443 ready socket-flow (satisfied)] Output protocol connected
default	15:51:58.169684 +0200	Quicksilver	Task <4C80BF3D-1D8E-4571-BDF8-986E51685F44>.<0> sent request, body N
default	15:51:58.273537 +0200	Quicksilver	Task <4C80BF3D-1D8E-4571-BDF8-986E51685F44>.<0> received response, status 200 content C
default	15:51:58.273853 +0200	Quicksilver	Task <4C80BF3D-1D8E-4571-BDF8-986E51685F44>.<0> response ended
default	15:51:58.273969 +0200	Quicksilver	Task <4C80BF3D-1D8E-4571-BDF8-986E51685F44>.<0> done using Connection 2
default	15:51:59.153150 +0200	Quicksilver	27366555: RECEIVED OUT-OF-SEQUENCE NOTIFICATION: 4791 vs 12554, 512, <private>

[pjrobertson]

Can you please try downloading this version: https://github.com/quicksilver/Quicksilver/suites/6092242473/artifacts/211268363

And confirm you are running the right version by checking the version number in the Quicksilver preferences:

Also, please try typing "qsapp.com" into the first pane in Quicksilver, then →. You should get a list of URLs like this:

[n8henrie]

OK, that's a good test. I can confirm that using the Released build (2.0.3) I cannot → into python.org. I'm leaning towards thinking this is a codesigning issue.

@pjrobertson I still can't reproduce on 2.0.3 release (from our publicly released DMG). It takes a second after I hit the arrow, but it works. Looks like @skurfer can't reproduce either.

[rosano]

Whoops, sorry! I've mistaken, it seems like the searches are there now in this first new version (https://github.com/quicksilver/Quicksilver/suites/6092242958/artifacts/211189389) but maybe after I re-scanned. Also still works after restarting Quicksilver, so it's persisting I guess. 👍🏽

[pjrobertson]

@pjrobertson I still can't reproduce on 2.0.3 release (from our publicly released DMG).

So it seems like a 10.14 problem then (the OP is also on 10.14)

it seems like the searches are there now in this first new version (

Great! As expected, #2729 fixes the problem :)