Support importing Excel and JSON files project (GSoC 2023)

[IamEzio]

Primary Mentor: Dominykas @dmos62
Secondary Mentor: Anish @Anish9901
Contributor: Anshuman @IamEzio

Objective

Mathesar UI allows users to import data from CSV and TSV files. We aim to expand this functionality into importing JSON and Excel files too. The project will allow users to import JSON and Excel files and create a table from the imported data. The user can preview the table, remove/rename columns, change data types, and more. If the imported file is not in a suitable format for creating a table, we will develop algorithms to convert the file into a suitable format. If the algorithm doesn’t work, the system will refuse to import the file and give the user a readable error.
See detailed description

Completed Tasks

Importing JSON

The functionality for importing JSON is completed.

• Users can upload JSON files in the current import UI seamlessly.
• Invalid JSON formats are handled and appropriate errors are returned to the frontend.
• Column data type guessing functionality works on JSON files too.
• Tests have been added for all the new code.

See Documentation
Related Meta ticket: #2895

Importing Excel

The functionality for importing Excel is completed.

• Users can upload Excel files in the current import UI seamlessly.
• Invalid Excel formats are handled and appropriate errors are returned to the frontend.
• Column data type guessing functionality works on Excel files too.
• Tests have been added for all the new code.

Related Meta ticket: #3027

Additional Tasks

• Error when imported data file has null or duplicate id value #3009

Pending Tasks

• Updating the way we handle datafiles in the backend #3030
• Front end implementation for importing JSON and Excel files #3031
  • Related PR: Adhere frontend to new importable filetypes (excel and json) #3211

[seancolsen]

Closing since the GSoC project is over