x/pkgsite: support searching by <package>.<identifier> #44142

matttproud · 2021-02-07T12:56:32Z

What is the URL of the page with the issue?

https://pkg.go.dev/search?q=sql.DB

What is your user agent?

Mozilla/5.0 (X11; CrOS x86_64 13597.66.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.109 Safari/537.36

Screenshot

What did you do?

I wanted to view the documentation for sql.DB, which is part of the standard distribution. I entered the search query of sql.DB into the interface, and it gave me no good results.

What did you expect to see?

Basically this but on the new web site: https://golang.org/pkg/database/sql/#DB

What did you see instead?

I don't know how the web application is implemented, but I expect either standard distribution packages are not indexed, the search query format "<package name>.<Identifier>" is not supported, or the result ranking is somehow deficient. I had been using this on old Godoc with great success since time immemorial.

The text was updated successfully, but these errors were encountered:

magical · 2021-02-20T23:52:12Z

I'd go further and allow searching for just <identifier>, at least within the standard library. I occasionally use golang.org's search to find, e.g., a function that i can't remember the package for https://golang.org/search?q=WriteFile#Functions, or all the types that implement some method https://golang.org/search?q=Sum#Global.

ALTree · 2021-05-12T16:31:50Z

To add another datapoint, this Go Tour page links to https://golang.org/search?q=Read#Global. The link is currently broken and I couldn't find a good pkgsite alternative to replace it.

gopherbot · 2021-05-12T21:17:56Z

Change https://golang.org/cl/318651 mentions this issue: internal/postgres: refactor getPackageSymbols

gopherbot · 2021-05-12T21:17:56Z

Change https://golang.org/cl/318490 mentions this issue: internal/postgres: insert search_documents path ids

gopherbot · 2021-05-12T21:17:57Z

Change https://golang.org/cl/318489 mentions this issue: migrations: add search_documents package_path_id and module_path_id

gopherbot · 2021-05-12T21:17:58Z

Change https://golang.org/cl/318650 mentions this issue: migrations: add symbol_search_documents

gopherbot · 2021-05-12T21:17:58Z

Change https://golang.org/cl/318652 mentions this issue: internal/postgres: insert symbol search documents

For golang/go#44142 Change-Id: I9d1908b07924a02153a249773d9440676a0e349a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318489 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>

The search_documents columns package_path_id and module_path_id are now populated. For golang/go#44142 Change-Id: I1833bfc821f3a3ebe5b71a562e616d0c6b2dc20a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318490 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>

A table is added to contain information for searching for symbols. Build contexts are added as a text array instead of as individual rows, since the only use case for that information is linking users to the correct URL if the symbol is not present in the default build context. For golang/go#44142 Change-Id: I759b22e0ced7fec547778aba11a1b5f641530fb8 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318650 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>

getPackageSymbols is refactored using squirrel, so that it can be modified for us in other similar queries. For golang/go#44142 Change-Id: I974c51155d30c59cf61ded0386c5d3b2702188c5 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318651 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>

gopherbot · 2021-05-14T21:03:17Z

Change https://golang.org/cl/318871 mentions this issue: content,internal/frontend: add symbol search

gopherbot · 2021-05-14T21:03:18Z

Change https://golang.org/cl/318870 mentions this issue: internal/postgres: add symbol search

Data is now inserted into symbol_search_documents. For golang/go#44142 Change-Id: Icf00cd7c5e57a91465673d1828f961334abc3a48 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318652 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>

An initial version of symbol search is added. This runs separately from regular search and is not connected to the frontend yet. For golang/go#44142 Change-Id: I34a21a4a743a07dcfcdfeb64b52a86bc2254f62a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318870 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>

The frontend now supports searching for symbols using the syntax "identifier:<query>". For golang/go#44142 Change-Id: Iae1db22091cf3921d47fbbdbdfbbf87e25792fd9 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318871 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>

gopherbot · 2021-06-18T21:55:50Z

Change https://golang.org/cl/329497 mentions this issue: static: move symbol search template

gopherbot · 2021-06-18T21:55:51Z

Change https://golang.org/cl/329349 mentions this issue: internal/postgres: move insertSymbols to saveModule

gopherbot · 2021-06-18T21:55:51Z

Change https://golang.org/cl/329493 mentions this issue: internal/postgres: change logic for upsertSearchDocumentSymbols

Logic for insertSymbols is moved out of insertUnits to saveModule. This will allows us to run that code after the check for isLatest, which is necessary to insert symbols where the latest unit is not a release version. For golang/go#44142 Change-Id: Id917b6b74020957549ad1c6594f21d3f8d27e464 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/329349 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>

upsertSearchDocumentSymbols is changed based on the new schema. For golang/go#44142 Change-Id: I1857b04cceeb9b702e4c54fc3a8d39efa811a3ce Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/329493 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>

searchQueryAndMode now returns a string to indicate the mode, as opposed to a boolean indicating searchSymbols. For golang/go#44142 Change-Id: I2568902897ceb196ad8413e737a08e6372c89a90 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348130 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jamal Carvalho <jamal@golang.org>

gopherbot · 2021-09-08T02:13:50Z

Change https://golang.org/cl/348130 mentions this issue: internal/frontend: change searchSymbols to searchMode

gopherbot · 2021-09-08T14:44:46Z

Change https://golang.org/cl/348112 mentions this issue: internal/frontend: split searchQueryAndMode

gopherbot · 2021-09-08T14:49:35Z

Change https://golang.org/cl/348113 mentions this issue: internal/frontend: add symbol filters

searchQueryAndMode is split into searchQuery and searchMode. searchQuery now returns the raw query string, so that user can see the query that they typed in. For golang/go#44142 Change-Id: I696b8f65f732417c43a5290b43ba1887b0c1bc06 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348112 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>

gopherbot · 2021-09-08T16:02:58Z

Change https://golang.org/cl/348374 mentions this issue: static/frontend/homepage: update examples

The # symbol filter is updated so that it can be used with any word, instead of only as a prefix to indicate the user wants to search in symbol mode. For example, "#foo bar" was understood to mean, search for the query string 'foo bar', in symbol search mode. This is now understood to mean, search for the query string 'foo bar', in symbol search mode, where 'foo' MUST match a symbol name. This syntax allows users to specify which symbol to search for. postgres.Search will support filters in the next CL. For golang/go#44142 Change-Id: Ifb69f0d2745d68b5e0c28d734b48e58cc02c1819 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347610 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com>

gopherbot · 2021-09-09T13:38:45Z

Change https://golang.org/cl/348793 mentions this issue: tests/search: modify test script format

The script format in tests/search is modified such that: * The mode that is written in the test file indicates the "m=<mode>" query param. * The mode that is wanted after that request is determined by looking at the test cases. This can be different than the m=<mode> param if the user uses filters. For golang/go#44142 Change-Id: Idff108852599a7777d4814f1f46bba94d2c6c369 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348793 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>

postgres.Search now supports symbol and package filters. tests/search is also updated to match this change. For golang/go#44142 Change-Id: Icbd05b99ea3a74956d5726e03c8767ab6062d133 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347611 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>

The search examples on the homepage are updated to reflect the updated search heuristics. For golang/go#44142 Change-Id: I2154485580d6b3c43c87cf1169e72429d709c95a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348374 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>

gopherbot · 2021-09-09T16:03:26Z

Change https://golang.org/cl/348814 mentions this issue: internal/frontend: add package tab query

If a user searches for #DB, they will always be brought to the symbols tab. However, if they click on the packages tab from that page, we want them to see something useful. As a result, the packages tab now has href set to: /search?m=package&q=<query-stripped-of-filters. For golang/go#44142 Change-Id: Iff7f3d8b3f8752a40855b816cc94354b0f7ea93f Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348814 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>

For golang/go#44142 Change-Id: I79d57d96f033197298d9f2219f6acaeeabcc873a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/349609 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jamal Carvalho <jamal@golang.org>

gopherbot · 2021-09-13T20:25:21Z

Change https://golang.org/cl/349609 mentions this issue: migrations: drop unused columns

gopherbot · 2021-09-14T20:45:35Z

Change https://golang.org/cl/349896 mentions this issue: internal/postgres: update search_documents and symbol_search_documents simultaneously

gopherbot · 2021-09-14T20:45:36Z

Change https://golang.org/cl/349892 mentions this issue: internal: delete stale symbol_search_documents

gopherbot · 2021-09-14T21:58:42Z

Change https://golang.org/cl/349610 mentions this issue: migrations: add FK to search_documents.unit_id

gopherbot · 2021-09-17T13:29:19Z

Change https://golang.org/cl/350129 mentions this issue: internal/postgres/symbolsearch: rename package

gopherbot · 2021-09-17T13:29:20Z

Change https://golang.org/cl/350130 mentions this issue: internal/postgres/search: rename Query to SymbolQuery

When the latest version of a module is inserted, delete symbol_search_documents rows for symbols not in that version of the package, so that stale rows are removed. For golang/go#44142 Change-Id: I9a5c45eff4713cb765fd15b57530bf139385a3a8 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/349892 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>

internal/postgres/symbolsearch is renamed to internal/postgres/search, since the package search queries will also be moved in the next CL. For golang/go#44142 Change-Id: Ieee78787406e969ab838859526855b8f892438b1 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/350129 Reviewed-by: Jamal Carvalho <jamal@golang.org> Trust: Julie Qiu <julie@golang.org>

Query is renamed to SymbolQuery, since the package was renamed from symbolsearch to search. For golang/go#44142 Change-Id: I02c2f04305d604a3b4efc60708154bd53e07fd11 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/350130 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jamal Carvalho <jamal@golang.org>

gopherbot · 2021-09-20T20:19:04Z

Change https://golang.org/cl/351092 mentions this issue: static/frontend/search-help: update for redesigned search

gopherbot · 2021-10-01T15:09:03Z

Change https://golang.org/cl/346169 mentions this issue: migrations: set symbol_search_documents.symbol_name not null

Now that symbol_search_documents.symbol_name is populated in all environments, a NOT NULL constraint is added. For golang/go#44142 Change-Id: I04a852084a7fe36ff9da2725bd0f8e6ecb9d5524 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/346169 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>

gopherbot · 2021-10-11T23:00:59Z

Change https://golang.org/cl/355229 mentions this issue: internal/frontend: fix filter when not in first word

Previously, the search filter # only defaulted to symbol search mode when it was on the first word of a search. It now works regardless of which word has the filter. For golang/go#44142 Change-Id: Id32010f9071c7f5db82127a490372fdf89548fe5 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/355229 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>

gopherbot added the pkgsite label Feb 7, 2021

gopherbot added this to the Unreleased milestone Feb 7, 2021

jamalc added the NeedsInvestigation label Feb 8, 2021

jamalc removed this from the Unreleased milestone Feb 8, 2021

jamalc added this to the pkgsite/golang.org-features milestone Feb 8, 2021

julieqiu changed the title ~~x/pkgsite: search by <package>.<identifier> no longer possible~~ x/pkgsite: support searching by <package>.<identifier> Feb 9, 2021

julieqiu removed this from the pkgsite/golang.org-features milestone Feb 9, 2021

julieqiu added this to the pkgsite/search milestone Feb 9, 2021

rsc mentioned this issue Feb 18, 2021

x/website: remove search box from golang.org #44357

Closed

julieqiu removed this from the pkgsite/golang.org-features milestone Sep 13, 2021

julieqiu added this to the pkgsite/2021 milestone Sep 13, 2021

golang / go Public

x/pkgsite: support searching by <package>.<identifier> #44142

x/pkgsite: support searching by <package>.<identifier> #44142

matttproud commented Feb 7, 2021 •

edited

magical commented Feb 20, 2021

ALTree commented May 12, 2021 •

edited

gopherbot commented May 12, 2021

gopherbot commented May 12, 2021

gopherbot commented May 12, 2021

gopherbot commented May 12, 2021

gopherbot commented May 12, 2021

gopherbot commented May 14, 2021

gopherbot commented May 14, 2021

gopherbot commented Jun 18, 2021

gopherbot commented Jun 18, 2021

gopherbot commented Jun 18, 2021

gopherbot commented Sep 8, 2021

gopherbot commented Sep 8, 2021

gopherbot commented Sep 8, 2021

gopherbot commented Sep 8, 2021

gopherbot commented Sep 9, 2021

gopherbot commented Sep 9, 2021

gopherbot commented Sep 13, 2021

gopherbot commented Sep 14, 2021

gopherbot commented Sep 14, 2021

gopherbot commented Sep 14, 2021

gopherbot commented Sep 17, 2021

gopherbot commented Sep 17, 2021

gopherbot commented Sep 20, 2021

gopherbot commented Oct 1, 2021

gopherbot commented Oct 11, 2021

golang / go Public

x/pkgsite: support searching by <package>.<identifier> #44142

x/pkgsite: support searching by <package>.<identifier> #44142

Comments

matttproud commented Feb 7, 2021 • edited

What is the URL of the page with the issue?

What is your user agent?

Screenshot

What did you do?

What did you expect to see?

What did you see instead?

magical commented Feb 20, 2021

ALTree commented May 12, 2021 • edited

gopherbot commented May 12, 2021

gopherbot commented May 12, 2021

gopherbot commented May 12, 2021

gopherbot commented May 12, 2021

gopherbot commented May 12, 2021

gopherbot commented May 14, 2021

gopherbot commented May 14, 2021

gopherbot commented Jun 18, 2021

gopherbot commented Jun 18, 2021

gopherbot commented Jun 18, 2021

gopherbot commented Sep 8, 2021

gopherbot commented Sep 8, 2021

gopherbot commented Sep 8, 2021

gopherbot commented Sep 8, 2021

gopherbot commented Sep 9, 2021

gopherbot commented Sep 9, 2021

gopherbot commented Sep 13, 2021

gopherbot commented Sep 14, 2021

gopherbot commented Sep 14, 2021

gopherbot commented Sep 14, 2021

gopherbot commented Sep 17, 2021

gopherbot commented Sep 17, 2021

gopherbot commented Sep 20, 2021

gopherbot commented Oct 1, 2021

gopherbot commented Oct 11, 2021

matttproud commented Feb 7, 2021 •

edited

ALTree commented May 12, 2021 •

edited