hq

jq's not as sophisticated cousin but for HTML.

Installation

# with npm
npm install -g @cpclermont/hq

# with yarn
yarn global add @cpclermont/hq

Usage

HTML isn't as well structured as JSON. But we can do some fun stuff with CSS selectors and attribute selectors.

hq commands all start with a CSS selector and are then optionally passed into a transformation:

# example settings
url="https://www.webpagetest.org"
cssSelector="head > script[src]:not([defer]):not([async])"

# examples
curl -s $url | hq "$cssSelector"
curl -s $url | hq "$cssSelector | html"
curl -s $url | hq "$cssSelector | text"
curl -s $url | hq "$cssSelector | innerHTML"
curl -s $url | hq "$cssSelector | attr(src)"
curl -s $url | hq "$cssSelector | attr(src, href)"

Transformations

html (alias: outerHTML) returns the outerHTML of the node.
text returns the text in the node.
innerHTML returns the innerHTML of the node 😬.
attr(attrs) returns the values of the attributes(comma separated). The output of unmatched attributes is skipped, every attribute is output to a new line.

This exists so you can scrape URLs from different node types that might live in different attributes.

e.g.
```
curl -s https://www.webpagetest.org | hq '
  head > script[src]:not([defer]):not([async]),
  head > link[rel=stylesheet]
  | attr(src, href)'
```
Where we want to list render blocking scripts and stylesheets in the order they are found.

Examples

# Get URLs of scripts that are render blocking
curl -s https://www.webpagetest.org | hq '
  head > script[src]:not([defer]):not([async])
  | attr(src)'

# Get URLs of scripts and stylesheets that are render blocking
curl -s https://www.webpagetest.org | hq '
  head > script[src]:not([defer]):not([async]),
  head > link[rel=stylesheet]
  | attr(src, href)'

Notes

This is scrappy and nowhere near as good as jq. But it's better than nothing :)

License

MIT

charlespwd/hq

ErrorLooks like something went wrong!

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Languages

Error
Looks like something went wrong!

Packages