Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/pkgsite: support searching by <package>.<identifier> #44142

Closed
matttproud opened this issue Feb 7, 2021 · 157 comments
Closed

x/pkgsite: support searching by <package>.<identifier> #44142

matttproud opened this issue Feb 7, 2021 · 157 comments
Labels
FeatureRequest FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. pkgsite
Milestone
pkgsite/2021

Comments

@matttproud
Copy link
Contributor

matttproud commented Feb 7, 2021 •
edited
Loading

What is the URL of the page with the issue?

https://pkg.go.dev/search?q=sql.DB

What is your user agent?

Mozilla/5.0 (X11; CrOS x86_64 13597.66.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.109 Safari/537.36

Screenshot

Screenshot 2021-02-07 13 52 26

What did you do?

I wanted to view the documentation for sql.DB, which is part of the standard distribution. I entered the search query of sql.DB into the interface, and it gave me no good results.

What did you expect to see?

Basically this but on the new web site: https://golang.org/pkg/database/sql/#DB

What did you see instead?

I don't know how the web application is implemented, but I expect either standard distribution packages are not indexed, the search query format "<package name>.<Identifier>" is not supported, or the result ranking is somehow deficient. I had been using this on old Godoc with great success since time immemorial.
@gopherbot gopherbot added this to the Unreleased milestone Feb 7, 2021
@jamalc jamalc added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 8, 2021
@julieqiu julieqiu changed the title x/pkgsite: search by <package>.<identifier> no longer possible x/pkgsite: support searching by <package>.<identifier> Feb 9, 2021
@rsc rsc mentioned this issue Feb 18, 2021
Closed
@magical
Copy link
Contributor

magical commented Feb 20, 2021

I'd go further and allow searching for just <identifier>, at least within the standard library. I occasionally use golang.org's search to find, e.g., a function that i can't remember the package for https://golang.org/search?q=WriteFile#Functions, or all the types that implement some method https://golang.org/search?q=Sum#Global.

@ALTree
Copy link
Member

ALTree commented May 12, 2021 •
edited
Loading

To add another datapoint, this Go Tour page links to https://golang.org/search?q=Read#Global. The link is currently broken and I couldn't find a good pkgsite alternative to replace it.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/318651 mentions this issue: internal/postgres: refactor getPackageSymbols

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/318490 mentions this issue: internal/postgres: insert search_documents path ids

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/318489 mentions this issue: migrations: add search_documents package_path_id and module_path_id

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/318650 mentions this issue: migrations: add symbol_search_documents

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/318652 mentions this issue: internal/postgres: insert symbol search documents

gopherbot pushed a commit to golang/pkgsite that referenced this issue May 13, 2021
@julieqiu
migrations: add search_documents package_path_id and module_path_id
c31e996 
For golang/go#44142

Change-Id: I9d1908b07924a02153a249773d9440676a0e349a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318489
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue May 13, 2021
@julieqiu
internal/postgres: insert search_documents path ids
1255f84 
The search_documents columns package_path_id and module_path_id are now
populated.

For golang/go#44142

Change-Id: I1833bfc821f3a3ebe5b71a562e616d0c6b2dc20a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318490
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue May 13, 2021
@julieqiu
migrations: add symbol_search_documents
bdfe390 
A table is added to contain information for searching for symbols.

Build contexts are added as a text array instead of as individual rows,
since the only use case for that information is linking users to the
correct URL if the symbol is not present in the default build context.

For golang/go#44142

Change-Id: I759b22e0ced7fec547778aba11a1b5f641530fb8
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318650
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue May 13, 2021
@julieqiu
internal/postgres: refactor getPackageSymbols
dec93ab 
getPackageSymbols is refactored using squirrel, so that it can be
modified for us in other similar queries.

For golang/go#44142

Change-Id: I974c51155d30c59cf61ded0386c5d3b2702188c5
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318651
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/318871 mentions this issue: content,internal/frontend: add symbol search

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/318870 mentions this issue: internal/postgres: add symbol search

gopherbot pushed a commit to golang/pkgsite that referenced this issue May 17, 2021
@julieqiu
internal/postgres: insert symbol search documents
ba9e25d 
Data is now inserted into symbol_search_documents.

For golang/go#44142

Change-Id: Icf00cd7c5e57a91465673d1828f961334abc3a48
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318652
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue May 17, 2021
@julieqiu
internal/postgres: add symbol search
e9a096a 
An initial version of symbol search is added. This runs separately from
regular search and is not connected to the frontend yet.

For golang/go#44142

Change-Id: I34a21a4a743a07dcfcdfeb64b52a86bc2254f62a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318870
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue May 17, 2021
@julieqiu
content,internal/frontend: add symbol search
c2cd1f8 
The frontend now supports searching for symbols using the syntax
"identifier:<query>".

For golang/go#44142

Change-Id: Iae1db22091cf3921d47fbbdbdfbbf87e25792fd9
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318871
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/329497 mentions this issue: static: move symbol search template

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/329349 mentions this issue: internal/postgres: move insertSymbols to saveModule

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/329493 mentions this issue: internal/postgres: change logic for upsertSearchDocumentSymbols

gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 21, 2021
@julieqiu
internal/postgres: move insertSymbols to saveModule
fc9b57a 
Logic for insertSymbols is moved out of insertUnits to saveModule. This
will allows us to run that code after the check for isLatest, which is
necessary to insert symbols where the latest unit is not a
release version.

For golang/go#44142

Change-Id: Id917b6b74020957549ad1c6594f21d3f8d27e464
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/329349
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Jun 21, 2021
@julieqiu
internal/postgres: change logic for upsertSearchDocumentSymbols
de3c212 
upsertSearchDocumentSymbols is changed based on the new schema.

For golang/go#44142

Change-Id: I1857b04cceeb9b702e4c54fc3a8d39efa811a3ce
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/329493
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 9, 2021
@julieqiu
static/frontend/homepage: update examples
0293c8d 
The search examples on the homepage are updated to reflect the updated
search heuristics.

For golang/go#44142

Change-Id: I2154485580d6b3c43c87cf1169e72429d709c95a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348374
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/348814 mentions this issue: internal/frontend: add package tab query

gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 9, 2021
@julieqiu
internal/frontend: add package tab query
1800abc 
If a user searches for #DB, they will always be brought to the symbols
tab. However, if they click on the packages tab from that page, we want
them to see something useful.

As a result, the packages tab now has href set to:

/search?m=package&q=<query-stripped-of-filters.

For golang/go#44142

Change-Id: Iff7f3d8b3f8752a40855b816cc94354b0f7ea93f
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348814
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 13, 2021
@julieqiu
migrations: drop unused columns
70ec084 
For golang/go#44142

Change-Id: I79d57d96f033197298d9f2219f6acaeeabcc873a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/349609
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/349609 mentions this issue: migrations: drop unused columns

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/349896 mentions this issue: internal/postgres: update search_documents and symbol_search_documents simultaneously

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/349892 mentions this issue: internal: delete stale symbol_search_documents

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/349610 mentions this issue: migrations: add FK to search_documents.unit_id

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/350129 mentions this issue: internal/postgres/symbolsearch: rename package

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/350130 mentions this issue: internal/postgres/search: rename Query to SymbolQuery

gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 20, 2021
@julieqiu
internal: delete stale symbol_search_documents
0d3582b 
When the latest version of a module is inserted, delete
symbol_search_documents rows for symbols not in that version of the
package, so that stale rows are removed.

For golang/go#44142

Change-Id: I9a5c45eff4713cb765fd15b57530bf139385a3a8
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/349892
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 20, 2021
@julieqiu
internal/postgres/symbolsearch: rename package
1f178e2 
internal/postgres/symbolsearch is renamed to internal/postgres/search,
since the package search queries will also be moved in the next CL.

For golang/go#44142

Change-Id: Ieee78787406e969ab838859526855b8f892438b1
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/350129
Reviewed-by: Jamal Carvalho <jamal@golang.org>
Trust: Julie Qiu <julie@golang.org>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 20, 2021
@julieqiu
internal/postgres/search: rename Query to SymbolQuery
3b4c4ab 
Query is renamed to SymbolQuery, since the package was renamed from
symbolsearch to search.

For golang/go#44142

Change-Id: I02c2f04305d604a3b4efc60708154bd53e07fd11
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/350130
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/351092 mentions this issue: static/frontend/search-help: update for redesigned search

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/346169 mentions this issue: migrations: set symbol_search_documents.symbol_name not null

gopherbot pushed a commit to golang/pkgsite that referenced this issue Oct 1, 2021
@julieqiu
migrations: set symbol_search_documents.symbol_name not null
eaed5d2 
Now that symbol_search_documents.symbol_name is populated in all
environments, a NOT NULL constraint is added.

For golang/go#44142

Change-Id: I04a852084a7fe36ff9da2725bd0f8e6ecb9d5524
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/346169
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/355229 mentions this issue: internal/frontend: fix filter when not in first word

gopherbot pushed a commit to golang/pkgsite that referenced this issue Oct 14, 2021
@julieqiu
internal/frontend: fix filter when not in first word
391fe1a 
Previously, the search filter # only defaulted to symbol search mode
when it was on the first word of a search. It now works regardless of
which word has the filter.

For golang/go#44142

Change-Id: Id32010f9071c7f5db82127a490372fdf89548fe5
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/355229
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/358474 mentions this issue: migrations: set search_documents.ln_imported_by_count NOT NULL

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/358475 mentions this issue: internal/postgres/search: use search_documents.ln_imported_by_count

gopherbot pushed a commit to golang/pkgsite that referenced this issue Oct 25, 2021
@julieqiu
migrations: set search_documents.ln_imported_by_count NOT NULL
1d61b29 
For golang/go#44142

Change-Id: I053c36aa995be2cd87075b5a5938b1fc5728a97d
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/358474
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Oct 25, 2021
@julieqiu
internal/postgres/search: use search_documents.ln_imported_by_count
753199e 
Instead of using symbol_search_documents.ln_imported_by_count, use
search_documents.ln_imported_by_count.

For golang/go#44142

Change-Id: I16a91fd5eb674a27e1f3bd3703d1c4ff3b58deb5
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/358475
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Nov 9, 2021
@julieqiu
static/frontend/search-help: update for redesigned search
6174505 
The search help page is updated based on the new search redesign.

For golang/go#44142

Change-Id: Icdc75e2077ab38028e3a79967b2a104b776854ca
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/351092
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
@julieqiu
Copy link
Member

julieqiu commented Nov 9, 2021

This is live on pkg.go.dev! See https://go.dev/blog/pkgsite-search-redesign for details.

@julieqiu julieqiu closed this as completed Nov 9, 2021
@matttproud
Copy link
Contributor Author

matttproud commented Nov 9, 2021 via email

@golang golang locked and limited conversation to collaborators Nov 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FeatureRequest FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. pkgsite
Projects
None yet
Milestone
pkgsite/2021
Development

No branches or pull requests
6 participants
@matttproud @magical @ALTree @julieqiu @gopherbot @jamalc