Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

search: remove remote searching #15209

Merged
merged 1 commit into from Apr 12, 2023
Merged

Conversation

Bo98
Copy link
Member

@Bo98 Bo98 commented Apr 11, 2023

Option A: Algolia
Option B (this PR): remove remote searching entirely

Given the use of our own JSON API, the need for remote searching has largely diminished as you can no longer "untap" Homebrew/cask.

However with both options we do lose the ability to search cask-versions/drivers/fonts when they are untapped, which was the reason the Algolia pull request was rejected so this PR is perhaps not enough of an improvement over the original proposal (though the current functionality is now broken without API tokens). If any ever get added to formulae.brew.sh (even if just as HTML with no JSON), then Algolia could cover them.

Option C is to use the GitHub files API which I could look at, though will require a number of API requests (per repository and perhaps pagination but I haven't checked).

Fixes #15208
Closes #15107

@Bo98 Bo98 force-pushed the search-no-remote branch 3 times, most recently from df23f10 to c7b893d Compare April 11, 2023 23:27
Copy link
Contributor

@apainintheneck apainintheneck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this make sense in the short-term because search seems to broken right now anyway. This should fix #15208.

Any of the other methods of adding the other cask taps to search can be handled in another PR. Either we can use #15107 for that or we can create a new issue to remind ourselves to follow up on that.

@@ -810,6 +810,7 @@ fi
AUTO_UPDATE_COMMANDS=(
install
outdated
search
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this necessary now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea was this is the replacement for remote searching. The code we're removing always worked based on the latest remote state of the taps so doing an auto-update would keep similar behaviour.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm conflicted on this. I see the benefit of this always being up-to-date. On the flip side, I've heard a bunch of people complain that "brew search is too slow" and I fear this may make it worse.

I'm tempted to say we merge without this and then be willing to add it in if we get any complaints or confusion over inconsistencies?

Copy link
Member Author

@Bo98 Bo98 Apr 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speed shouldn't really be any slower than brew outdated after this change, unless the slowness is in DidYouMean or something. Most of the slowness was likely the GitHub code search.

So not really convinced about that as a reason, but I am happy to drop it anyway.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped now.

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work here so far @Bo98. One tiny tweak and think this is good to go.

@@ -810,6 +810,7 @@ fi
AUTO_UPDATE_COMMANDS=(
install
outdated
search
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm conflicted on this. I see the benefit of this always being up-to-date. On the flip side, I've heard a bunch of people complain that "brew search is too slow" and I fear this may make it worse.

I'm tempted to say we merge without this and then be willing to add it in if we get any complaints or confusion over inconsistencies?

@@ -42,51 +42,6 @@ def search_descriptions(string_or_regex, args, search_type: :desc)
end
end

def search_taps(query, silent: false)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥

@MikeMcQuaid
Copy link
Member

Option B (this PR): remove remote searching entirely

I agree this is the best fit.

However with both options we do lose the ability to search cask-versions/drivers/fonts when they are untapped, which was the reason the Algolia pull request was rejected so this PR is perhaps not enough of an improvement over the original proposal (though the current functionality is now broken without API tokens).

IMO this is yet another good reason to combine these taps. I still think we should be combining cask-fonts too, at least for those that are clearly used through analytics CC @Homebrew/cask.

@Bo98
Copy link
Member Author

Bo98 commented Apr 12, 2023

I still think we should be combining cask-fonts too, at least for those that are clearly used through analytics CC @Homebrew/cask.

It's possible we could have formulae.brew.sh support for cask-fonts, I wouldn't say that's completely out of the question. Focus on the others first though.

@MikeMcQuaid
Copy link
Member

It's possible we could have formulae.brew.sh support for cask-fonts, I wouldn't say that's completely out of the question. Focus on the others first though.

I don't think it's worth it for a single tap. I think there's a host of issues with us having multiple official cask taps in the Homebrew organisation.

@Bo98
Copy link
Member Author

Bo98 commented Apr 12, 2023

We're talking about 2000 packages here though rather than a couple, which would increase the size of homebrew-cask by 50%. There's maintainability questions that would be best to discuss with the people who maintain it on a day-to-day basis.

If there are benefits, then it would absolutely be worth it - formulae.brew.sh integration isn't really that tough of a task. Though I'm not the person to ask about benefits of that tap since I don't maintain it.

@MikeMcQuaid
Copy link
Member

We're talking about 2000 packages here though rather than a couple, which would increase the size of homebrew-cask by 50%.

I think using analytics can slim this down a bit but, post-sharding: I think this is still worthwhile.

There's maintainability questions that would be best to discuss with the people who maintain it on a day-to-day basis.

I think maintainability is important but: so is user/contributor experience and the current level of discoverability makes both of those harder.

formulae.brew.sh integration isn't really that tough of a task

Yeh, probably but but: we've got a host of other API-related issues affecting users that seem much higher priority to me: https://github.com/Homebrew/brew/issues?q=is%3Aissue+is%3Aopen+label%3A%22install+from+api%22

These are blocking our ability to say in #14592 "hey, try enabling API support again".

@SMillerDev
Copy link
Member

There's maintainability questions that would be best to discuss with the people who maintain it on a day-to-day basis.

I only maintain the main cask repo because the others have increasingly hand-wavy rules so for me keeping them all in the same repo with the same rules would be a big improvement. Also note that Homebrew/homebrew-cask#112102 has existed for 1,5 years at this point. Clearly nobody objects significantly.

@Bo98
Copy link
Member Author

Bo98 commented Apr 12, 2023

Yeh, probably but but: we've got a host of other API-related issues affecting users that seem much higher priority to me: https://github.com/Homebrew/brew/issues?q=is%3Aissue+is%3Aopen+label%3A%22install+from+api%22

Let's keep that separate for now given we'd ideally look at those in the short term and cask-fonts in the medium term (sharding needs to happen before adding another 2000 casks, and versions & drivers should be tackled first).

I think maintainability is important but: so is user/contributor experience and the current level of discoverability makes both of those harder.

Users don't really care as much about taps so maintainability is always going to be an important factor. The user experience can definitely be improved regardless of the approach and I don't agree it's impossible to make a good user experience with separate taps - it's more that we've not committed to an approach yet so there's not been any point putting time in.

Also note that Homebrew/homebrew-cask#112102 has existed for 1,5 years at this point. Clearly nobody objects significantly.

Indeed there hasn't been any objections to versions (that linked issue) and drivers. There has been to fonts (as even mentioned in that issue) but that's going to be done last anyway so plenty of time for opinions to change and workflows to be discussed.

@MikeMcQuaid
Copy link
Member

Let's keep that separate for now given we'd ideally look at those in the short term and cask-fonts in the medium term (sharding needs to happen before adding another 2000 casks, and versions & drivers should be tackled first).

My point was intended to be: I think all of those issues are much more important than fixing search for taps.

Users don't really care as much about taps so maintainability is always going to be an important factor.

Users indirectly care about taps when they don't discover things because they don't know about non-default taps.

I don't agree it's impossible to make a good user experience with separate taps

I don't think this either. I just think it requires a bunch of work that would be unnecessary for maintainers, contributors and users when you subdivide things into taps. This is why we stopped doing this (very) long ago for formulae.

@Bo98
Copy link
Member Author

Bo98 commented Apr 12, 2023

I don't think this either. I just think it requires a bunch of work that would be unnecessary for maintainers, contributors and users when you subdivide things into taps. This is why we stopped doing this (very) long ago for formulae.

For JSON API, these things can even be in the same JSON while technically separate taps under the hood. AFAICT that requires very few changes to do. There's a lot more flexibility now than there was years ago.

My point is, I'd like to discuss the driving motivations from a maintainer POV first, and see what the initial preference on that is and whether any workflows can be improved, rather than drive through the migration and considering maintainability later.

My point was intended to be: I think all of those issues are much more important than fixing search for taps.

I agree with that: though I'd also say they're more important than migrating homebrew-cask-fonts.

@MikeMcQuaid
Copy link
Member

Thanks again @Bo98!

@MikeMcQuaid MikeMcQuaid merged commit 81b6c79 into Homebrew:master Apr 12, 2023
25 checks passed
@Bo98 Bo98 deleted the search-no-remote branch April 12, 2023 12:34
@github-actions github-actions bot added the outdated PR was locked due to age label May 13, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
outdated PR was locked due to age
Projects
None yet
4 participants