FetchData Class

The FetchData class is part of the services package (package org.example.globalgoodsindex.services) and is responsible for scraping data from external source: Numbeo - a website that provides cost-of-living information. It contains methods to scrape salary data and product price data for different countries. These methods use Jsoup for HTML parsing and data extraction.

The class provides static methods that return the scraped data in well-structured formats, ready for further processing or display in the application.

Methods

scrapeSalaries

Purpose:

Scrapes the average monthly net salary (after tax) for countries from the Numbeo website and returns the data sorted alphabetically by country.

Description:

• Connects to the Numbeo website using Jsoup and retrieves salary data from a table.
• Extracts the country name and salary, parsing the salary as a double after removing commas.
• Invalid salary entries are skipped, and an error message is logged.
• The list of Salary objects is sorted alphabetically by country name before returning.

Return Value:

• A List<Salary> of countries and their corresponding salaries, sorted alphabetically.
• Returns null if there's a connection issue or invalid data.

Example Usage:

List<Salary> salariesData = FetchData.scrapeSalaries();

Error Handling:

• Logs errors for invalid salary formats or connection issues.

---

scrapeProducts

Purpose:

Scrapes product price data for various countries from the Numbeo website and returns the data sorted alphabetically by country.

Description:

• Connects to the Numbeo website using Jsoup and constructs a URL based on the itemId values from the Goods enum, which lists the available goods.
• Extracts product names (headers) and their prices for each country from the table.
• If the price is missing, it is replaced with a placeholder ("-"). Special characters are removed from the headers for consistency (e.g., spaces are replaced with underscores).
• The data is sorted alphabetically by country name before returning it as a list of lists.

Return Value:

• A List<List<String>> containing the product column headers and their prices for each country, sorted alphabetically by country name.
• Returns null if there's a connection issue or invalid data.

Example Usage:

List<List<String>> productsData = FetchData.scrapeProducts();

Error Handling:

• Logs errors for connection issues, missing or incomplete product rows, and any parsing errors.