--selector-all to screenshot of all elements matching a selector

[simonw]

This then made me think that I actually want the same ability for just raw CSS selectors. But what should that look like?

Originally posted by @simonw in #43 (comment)

[simonw]

I found myself wanting to take a screenshot of e.g. all of the SVGs on a page - so a box that is drawn around all of those elements.

Right now the -s/--selector option takes just the first element that matches a selector - though you can use it multiple times to specify other selectors too:

shot-scraper https://simonwillison.net/ \
  -s '#bighead' -s .overband \
  -o bighead-multi-selector.png

I think a --selector-all option may be the best way to do this.

Then I could have --js-selector-all as the equivalent for this new feature:

• Selecting elements for screenshotting based on tag content #43

[simonw]

Open questions: How should that work in the multi YAML format?

[simonw]

Relevant code:

shot-scraper/shot_scraper/cli.py

Lines 655 to 665 in f662746

	def _selector_javascript(selectors, padding=0):
	selector_to_shoot = "shot-scraper-{}".format(secrets.token_hex(8))
	selector_javascript = textwrap.dedent(
	"""
	new Promise(takeShot => {
	let padding = %s;
	let minTop = 100000000;
	let minLeft = 100000000;
	let maxBottom = 0;
	let maxRight = 0;
	let els = %s.map(s => document.querySelector(s));

It uses querySelector() but for this feature it could use querySelectorAll().

[simonw]

I like that --selector-all is to querySelectorAll() as --selector is to querySelector().

[simonw]

I'm going to add selector_all: and selectors_all: as the YAML keys, similar to selector: and selectors:.

(Going with Attorneys General pluralization here.)

[simonw]

With the new feature:

shot-scraper https://simonwillison.net/ --selector-all '.day' --padding 10

[simonw]

And for shot-scraper multi:

echo '- output: selectors-all.png
  url: https://simonwillison.net/
  selectors_all:
  - .day
  - .entry:nth-of-type(1)
  padding: 20' | shot-scraper multi -

Produced this:

[simonw]

Also manually tested with this:

echo '- output: selector-all.png
  url: https://simonwillison.net/
  selector_all:
  - .day
  padding: 20' | shot-scraper multi -