selenium-get-url

This tool fetches an input URL and saves it to an output path.

This tool is a bit like the command curl or command wget, both of which can fetch a web page by using the command line.

This tool runs selenium-webdriver and loads chromedriver.

To verify you have chromedriver:

chromedriver --version

Output version should be 142 or higher:

ChromeDriver 142.0.7444.59 (4b8153ab58d3c3f4c9f7e4baad9616ecf80db5fa-refs/branch-heads/7444_52@{#4})

Troubleshooting

If you are using macOS and get an error message like this:

• Apple could not verify "chromedriver" is free of malware that may harm your Mac or compromise your privacy.

Then remove all extended attributes, including the quarantine attribute:

sudo xattr -c $(which chromedriver)

If you are get an error message like this:

• permission denied

Then change the mode of the permissions to be executable:

sudo chmod +x $(which chromedriver)

Usage

One URL:

selenium-get-url https://example.com

Multiple URLs:

selenium-get-url https://example.com https://google.com

Process URLs via stdin:

echo "https://example.com" | selenium-get-url

Process URLs via stdin file:

cat urls.txt | selenium-get-url