Skip to content

Scape firmware metadata from 18 vendors and download corresponding firmware images. Save in MySQL database for InfoSec research purposes.

License

Notifications You must be signed in to change notification settings

amosproj/amos2022ws01-firmware-scraper

Repository files navigation

Firmware Scraper (AMOS SS 2022)

Pytest Suite

Logo of the project Logo of the project

TLDR;

This Selenium-based firmware scraper gathers firmware data from 18 vendors. Our automated solution scrapes metadata and downloads corresponding firmwares. The results can be used for InfoSec research.

Installation

Option 1: On your local machine (Ubuntu only)

git clone https://github.com/amosproj/amos2022ws01-firmware-scraper
# or download and unpack .zip amos2022ws01-firmware-scraper
cd amos2022ws01-firmware-scraper
./install.sh
source .venv/bin/activate

# Make sure MySQL server is running
sudo systemctl start mysql.service

export MYSQL_USER=<your username>
export MYSQL_PASSWORD=<your password>
export LOG_LEVEL=DEBUG

# Start the scraper:
python -m src.core

Option 2: Docker

Requirements: Docker should be installed on your machine.

git clone https://github.com/amosproj/amos2022ws01-firmware-scraper
#or download and unpack .zip amos2022ws01-firmware-scraper
cd amos2022ws01-firmware-scraper
docker-compose up --build

About

Scape firmware metadata from 18 vendors and download corresponding firmware images. Save in MySQL database for InfoSec research purposes.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published