Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
There are two types of storage.
Raw file storage
Raw file storage stores the response received as-is, this is done automatically with FileManager. You may implement your own raw file storage using the interface:
Otherwise, we provide MysqlFileManager which utilise the local filesystem as storage backed by a MySQL record. this can be found in
Information storage stores the information extracted from a web page and is often structured and clean. For example, the information of this book can be stored in this format:
- Title = "A Light in the Attic"
- Price = 51.77
- Tax = 0.00
- Stock = 0
- UPC = "a897fe39b1053632"
For this purpose, you can use any storage engine (database). Some examples of databases include: MariaDB, MongoDB, MySQL, PostgreSQL. You may also use Object/Relational Mapping (ORM) frameworks such as Java Persistence API or Hibernate to simplify your project.
Check out the Wiki for the respective section for more details