Export to file library behaviour does not match 'download file' behaviour

[aaron-manning]

Describe the Bug

Scenario: Tables with many columns are used in Directus that are managed externally and haven't been fully configured in Directus (by adding them to directus_collections, directus_relations, directus_fields etc). Users need the ability to export and re-import exported data.

If you are developing an application where admin configuration is not a high priority, the above scenario is a likely one.
Point Directus at a database and it should just work for simple columns (at least those without relations) without needing to add tedious config for each table and field.

Ideally it would work with relations without any additional config, with config added simply to tweak things rather than be necessary to make Directus work.
Like how Hasura scans your database schema and doesn't need you to add config (or create the tables and fields inside Hasura itself) to make it work properly. It detects column types and relations and just works.

To Reproduce

Exporting data from non or semi-configured tables does work in Directus, but the behaviour differs in breaking ways between exporting to file or to download.

If all these fields are removed (by clicking the cross on each), the CSV will download ALL the columns (which should ideally always be the default behaviour).

This is an OK workaround for not having a complete table/field config and needing a full export.

The problem is that this behaviour is different when downloading to file system, which is required when there are more than 2500 rows.
If I remove all those fields, the export fails (but works when downloading to file and downloads all columns!)
If I set at least one field, the export succeeds, but requires the user to manually add each field.

I understand that, for partially configured tables, the UI pulls in a few fields automatically, but for a table with 100+ columns and only one configured field (to allow an admin to add some related data using the Directus UI, for example), the admin cannot simply remove that field (removing all the fields from the list) to download all columns, they would have to manually add the other 99 columns to get a full export. This is a very ugly workflow.

In my use-case, it makes no sense for admins to need to select exported columns. One can assume that whatever is exported is likely to be re-imported again at some point, which makes this a dangerous default behaviour as a user may export rows, delete the originals and then re-import the data at a later date, wrongfully assuming they have a complete CSV when they do not.

I'm aware that I can workaround this issue by manually adding config for each table, field and relationship, but that developer experience leaves a lot to be desired. It would be amazing to have Directus work like Hasura and not require so much configuration.

Thanks!

Errors Shown

No response

What version of Directus are you using?

9.14.1

What version of Node.js are you using?

16.15.1

What database are you using?

postgres 14.3.2

What browser are you using?

Chrome

How are you deploying Directus?

Docker

[rijkvanzanten]

Linear: ENG-222