Skip to content


Ween Jiann edited this page Mar 11, 2019 · 3 revisions

There are two types of storage.

Raw file storage

Raw file storage stores the response received as-is, this is done automatically with FileManager. You may implement your own raw file storage using the interface:

Otherwise, we provide MysqlFileManager which utilise the local filesystem as storage backed by a MySQL record. this can be found in

Information storage

Information storage stores the information extracted from a web page and is often structured and clean. For example, the information of this book can be stored in this format:

  • Title = "A Light in the Attic"
  • Price = 51.77
  • Tax = 0.00
  • Stock = 0
  • UPC = "a897fe39b1053632"

For this purpose, you can use any storage engine (database). Some examples of databases include: MariaDB, MongoDB, MySQL, PostgreSQL. You may also use Object/Relational Mapping (ORM) frameworks such as Java Persistence API or Hibernate to simplify your project.

Check out the Wiki for the respective section for more details

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.