Skip to content

Data warehouse integration project developed in the context of 'Integration and data Interoperability' course at college

Notifications You must be signed in to change notification settings

Xiphereal/IeiWarehouse

Repository files navigation

Bibliographic references Warehouse

Project context

This application is a college project for the Interoperability and data Integration course. The porpuse was to learn how to develop a demo application integrating (pun intended) several techniques for data integration we learned about during the course: REST APIs, warehousing, data source wrappers, semantical mappings between the data sources and our data schema, etc.

Most of the relevant info about the developing process is in this repo (PRs, issues, etc.).

What did I learn

  • Perform data integration with warehousing.
  • Extract data from JSON object.
  • Design and implement the needed mappings between the data sources and our canonical schema.
  • Basic usage of Selenium 4 for data scraping at the Google Scholar website.
  • Basic usage of Gradle:
    • Building a multi-module based project.
    • Run the different task manually and understanding the Gradle environment.
    • Different notations and implications for defining dependencies.
  • Using the MySQL workbench for modeling the database schema and generate the SQL script from it. Also reverse de process, obtaning the database schema from a SQL script.
  • Implementation of basic REST API with Spring Boot.
  • Develop a basic web frontend with Vaadin.

About

Data warehouse integration project developed in the context of 'Integration and data Interoperability' course at college

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages