Feature: customizable fields display for documents, saved views & dashboard widgets

[shamoon]

Proposed change

This is a continuation of #6424 which importantly adds toggling fields etc to /documents page and for saved views (not just on the dashboard)

This PR allows for customization of document display fields, saved views and dashboard widgets (screenshots below), including:

• Choosing display mode (table / small card / large card)
• Choose which fields to display for all display modes
• Settings per saved view (and dashboard widget) and for the documents list
• Including custom fields

Closes #4607
Closes #5568
Closes #4826
Kinda Closes #5298

Type of change

• Bug fix: non-breaking change which fixes an issue.
• New feature / Enhancement: non-breaking change which adds functionality. Please read the important note above.
• Breaking change: fix or feature that would cause existing functionality to not work as expected.
• Documentation only.
• Other. Please explain:

Checklist:

• I have read & agree with the contributing guidelines.
• If applicable, I have included testing coverage for new code in this PR, for backend and / or front-end changes.
• If applicable, I have tested my code for new features & regressions on both mobile & desktop devices, using the latest version of major browsers.
• If applicable, I have checked that all tests pass, see documentation.
• I have run all pre-commit hooks, see documentation.
• I have made corresponding changes to the documentation as needed.
• I have checked my modifications for any breaking changes.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.10%. Comparing base (7a0334f) to head (cc345c2).

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #6439      +/-   ##
==========================================
+ Coverage   97.05%   97.10%   +0.04%     
==========================================
  Files         423      429       +6     
  Lines       16964    17255     +291     
  Branches     1300     1346      +46     
==========================================
+ Hits        16465    16756     +291     
+ Misses        499      497       -2     
- Partials        0        2       +2     

Flag	Coverage Δ
backend	95.83% <100.00%> (+0.01%)	⬆️
frontend	98.59% <100.00%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[shamoon]

Went ahead and made the display_fields a simple JSON field with validation, the old way with dynamic choices was cool but it caused Django to try and create migrations for custom fields, Im good either way if you prefer the DynamicOrderedMultipleChoiceField but this seemed simpler.

[theomega]

Great job, I love this feature!
As promised here is my feedback:

• Hiding "Title" on list view make breaks the whole table layout. Suddenly the "tags" are in the "Owner" column. Maybe the easiest thing would be to not allow hiding the Title?

• The "Show" dropdown contains "Notes" even if the Notes feature is disabled.
• I found the font-size in the "Show" dropdown inconsistent with the other dropdowns right next to it. All other dropdown have normal font sizes where as the "Show" dropdown has a smaller font size.
• When in "cards" display mode in the list (the middle button), then the behaviour of the "Added" and "Created" fields in the "Show" Dropdown is not super intuitive: They are both checked as default, when I unselect "Created", then nothing gets hidden, but the dates swap. It took my some digging in the code to understand that if I unselect "Created", but leave "Added" selected, then the "Added" date is chosen instead. Why not display really both dates if both are selected in the "Show" dropdown?
• I'm unsure if the "Share" field works as intentended in the "cards" display mode: Nothing changes if I enable or disable this field. I have shared a document (using the "Share Links" feature), but independent if I enable or disable "Share" in the dropdown, nothing displays. Same for the list mode. Maybe I'm misunderstand what "Share" should display.
• The "Show" dropdown does not respect permissions, so if the user does not have the right permissions to even see the data, we should hide the field in the dropdown
• One change that was in my (closed) PR was also hiding the "Owner" field in the "Sort" dropdown if the user does not have the permissions to even see the owners. Not sure if we should do this here in this PR or if I should open a separate one to dev for this.

[theomega]

One thing I'm completely lost about: Is this feature expected to work with saved views? So are the saved views expected to store which columns they display?
I can't get this to work:

If I reduce the columns displayed to only the "title" and then save the view, however I recall the view (side menu or via the "Views" dropdown), again all columns/fields are displayed. I think this is a saving problem, if I check the Django Admin for the Saved Views, they all have null inside their "Document display fields". So somehow the fields are not stored when saving the view.

I think something is really broken here (if this is even supposed to work): When creating the view with the POST request, the UI doesn't even provide the display_fields field in the request, this is where the null in the database comes from.

[shamoon]

Addressed those, thanks for testing.

The shared button shows the little note if a document is shared:

As far as the saved views goes, yes that absolutely should work, and I cant reproduce a problem with it. See screenshots with just title:

If no fields are specified it uses a default set (same as current)

[shamoon]

When creating the view with the POST request, the UI doesn't even provide the display_fields field in the request, this is where the null in the database comes from.

I cant repro that either:

[theomega]

Thanks for the quick fixes:

1. I think the hiding of tags and title is still not working for the table: Check this out if you are disabling both title and tags, then the owner is in the "document type" column:

2. As for the not storing the enabled shown fields when saving a view, I can still 100% reproduce this in both Safari and Chrome. Something is weird here if you can't see it. I will add some debugging output to the source maybe I can find out where the issue is.

3. Also the share thing doesn't work: I have a document that was definitifely shared (also in the Django Admin I see the Shared Links entries) but still it doesn't show up on the document list. Or maybe I'm looking at the wrong feature? The api endpoint also returns is_shared_by_requester: false for all my documents, maybe I really don't get the feature :-D.

[shamoon]

Shared is for permissions not share links. Table thing is fixed.

[theomega]

OK, then all good with the shared thing.

As for saving the columns:

I think this is the code that is supposed to construct what is sent to the backend when saving a view, correct?
https://github.com/paperless-ngx/paperless-ngx/blob/feature-dynamic-fields-display/src-ui/src/app/components/document-list/document-list.component.ts#L262

This doesn't contain the display_fields field at all. Am I looking and testing the right code?

[shamoon]

Ah, we were talking about two different things, you mean saving a new view, oh yea great catch.

[shamoon]

Alright, thats sorted too thanks 👍

[theomega]

Oh, I feel stupid now, I didn't know you can actually edit the Views in the "Settings". So I have never tested that :-D

I just spotted another bug: I just tried modifying an existing view in the display mode, but it doesn't even allow me to save.

How to reproduce:

• Save a new view
• Bug (as discussed): display_fields is not stored
• Now load the view (i.e. through the sidebar)
• Modify the displayed fields using the "Show" dropdown

Observe:

• You can't save/update the view, the only option is to create another view
• The "Save [viewname]" button is grayed out:

Expected:

• I should be able to save/update the view with the new fields to show

[theomega]

Sorry, forget about that bug report, you just seem to have also fixed that....

[theomega]

Then this looks really awesome to me, nothing more from my side after testing this.

[tomihbk]

Dear @shamoon, many thanks for this amazing feature! Does this PR also include the possibility of sorting by custom fields ?

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new discussion or issue for related concerns. See our contributing guidelines for more details.