Skip to content
Branch: master
Find file History
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
lenses Round the elapsed time to the nearest second. Mar 22, 2019
BUILD.bazel Merge pull request #11723 from Katharine/first-class-spyglass-links Mar 13, 2019
OWNERS
README.md
artifacts.go spyglass: show "links" in the top bar. Mar 13, 2019
gcsartifact.go Read the entire expected artifact from GCS. Mar 20, 2019
gcsartifact_fetcher.go
gcsartifact_fetcher_test.go clean up code style Oct 9, 2018
gcsartifact_test.go Read the entire expected artifact from GCS. Mar 20, 2019
podlogartifact.go Revert "Revert "Merge pull request #11000 from stevekuznetsov/skuznet… Mar 11, 2019
podlogartifact_fetcher.go spyglass: fetch pod log for /gcs/ endpoint Dec 6, 2018
podlogartifact_fetcher_test.go
podlogartifact_test.go Migrate to use new Prow API types Jan 30, 2019
spyglass-example.png Update the spyglass readme. Feb 20, 2019
spyglass.go
spyglass_test.go Merge pull request #11723 from Katharine/first-class-spyglass-links Mar 13, 2019
testgrid.go More unit tests. Feb 15, 2019
testgrid_test.go Add TestGrid unit tests. Feb 15, 2019

README.md

GoDoc Widget

Spyglass

A spyglass is a lensed monocular maritime instrument used to see things that may have been difficult to see otherwise.

Spyglass is a pluggable artifact viewer framework for Prow and a crude metaphor for the real object. It collects artifacts (usually files in a storage bucket) from various sources and distributes them to registered viewers, which are responsible for consuming them and rendering a view.

A typical Spyglass page might look something like this: I'm not a graphic designer I just make the backend

A general Spyglass query will proceed as follows:

  • User provides a job source in the query (usually a job name and build ID).
  • Spyglass finds all artifact names associated with the given job source.
  • Spyglass builds a cache of which artifacts match which lenses via configured regular expressions.
  • Lenses with matching artifacts are pre-rendered in order of descending priority.
  • Spyglass then sends render requests to each registered lens with its matching artifacts.
  • Each lens performs any necessary operations on the artifacts and produces a blob of HTML.
  • Views (HTML) are inserted asynchronously as viewers return.

Available views

Spyglass currently exposes the following views:

  • /job-history/<gcs-bucket-name>/pr-logs/directory/<job-name> to get the history of a job
  • /pr-history?org=<org>&repo=<repo>&pr=<pr number> to get the history of a PR
  • /view/gcs/<gcs-bucket-name>/pr-logs/pull/<repo-name>/<pull-number>/<job-name>/<build-id> to get the job result after it finished
  • /view/prowjob/<job-name>/<build-id> to check on the running job, this only works as long as the pod that runs the job still exists

Lenses

A lens is an set of functions that consume a list of artifacts and produces some HTML.

Lens names are unique, and must much the package name for the lens.

Built-in Viewers

Spyglass comes with some built-in viewers for commonly produced artifacts.

  • Prow Metadata
    Name: metadata
    Title: Metadata
    Match: finished.json|started.json
    Priority: 0
    
  • JUnit
    Name: junit
    Title: JUnit
    Matches: artifacts/junit.*\.xml
    Priority: 5
    
  • Logs
    Name: buildlog
    Title: Build Log
    Matches: build-log.txt|pod-log
    Priority: 10
    

Building your own viewer

Building a viewer consists of three main steps.

Write Boilerplate

First, create a package lensnamehere under prow/spyglass/lenses and import the lenses package.

Implement

Next, implement the necessary functions for a viewer. More specifically, implement the following interface (defined in lenses.go):

type Lens interface {
	// Config returns the name, title, priority, and other information about your lens.
	Config() LensConfig
	// Header is used to inject content into the lens's <head>. It will only ever be called once per load.
	Header(artifacts []Artifact, resourceDir string) string
	// Body is used to generate the contents of the lens's <body>. It will initially be called with empty data, but
	// the lens front-end code may choose to re-render itself with custom data.
	Body(artifacts []Artifact, resourceDir string, data string) string
	// Callback is used for the viewer to exchange arbitrary data with the frontend. It is called with lens-specified
	// data, and returns data to be passed to the lens. JSON encoding is recommended in both directions.
	Callback(artifacts []Artifact, resourceDir string, data string) string
}

In the init method, call lenses.RegisterLens() with an instance of your implementation of the interface. Spyglass should now be aware of your lens.

Additionally, some front-end TypeScript code can be provided. Configure your BUILD.bazel to build it, then emit a <script> tag with a relative reference to it in your Header() implementation. See buildlog/BUILD.bazel for an example.

In your typescript code, a global spyglass object will be available, providing the following interface:

export interface Spyglass {
  /**
   * Replaces the lens display with a new server-rendered page.
   * The returned promise will be resolved once the page has been updated.
   */
  updatePage(data: string): Promise<void>;
  /**
   * Requests that the server re-render the lens with the provided data, and
   * returns a promise that will resolve with that HTML as a string.
   *
   * This is equivalent to updatePage(), except that the displayed content is
   * not automatically changed.
   */
  requestPage(data: string): Promise<string>;
  /**
   * Sends a request to the server-side lens backend with the provided data, and
   * returns a promise that will resolve with the response as a string.
   */
  request(data: string): Promise<string>;
  /**
   * Inform Spyglass that the lens content has updated. This should be called whenever
   * the visible content changes, so Spyglass can ensure that all content is visible.
   */
  contentUpdated(): void;
}

Add to config

Finally, decide which artifacts you want your viewer to consume and create a regex that matches these artifacts. The JUnit viewer, for example, consumes all artifacts that match artifacts/junit.*\.xml.

Add a line in prow config under the viewers section of spyglass of the following form:

"myartifactregexp": ["my-view-name"]

The next time a job is viewed that contains artifacts matched by your regexp, your view should display.

See the GoDoc for more details and examples.

Config

Spyglass is currently disabled by default. To enable it, add the --spyglass arg to your deck deployment.

To provide any use, Spyglass must by configured. Its config takes the following form:

deck:
  spyglass:
    size_limit: 500e+6 # 500MB
    viewers:
      "started.json|finished.json": ["metadata"]
      "build-log.txt": ["buildlog"]
      "artifacts/junit.*\\.xml": ["junit"] # Remember to escape your '\' in yaml strings!

More formally, it is a single spyglass object under the top-level deck object that may contain fields viewers and size_limit. viewers is a map of string->[]string where the key must be a valid golang regular expression and the value is a list of viewer names that consume the artifacts matched by the corresponding regular expression. size_limit is the maximum artifact size spyglass will try to read in entirety before failing.

You can’t perform that action at this time.