Add new metadata editing interface

[sams96]

So far all I've done is add a new empty view but I'd like some suggestions about how we design this interface. As a starting point I've attached a screenshot of the new view with digiKam's metadata editing interface pasted on top, and some other bits I added into the side.

Any suggestions for interface design will be helpful, and if anyone wants to help me to implement it let me know.

Thanks

Edit:
I just wanted to add links to the discussions on the mailing list for anyone who wants to catch up
https://www.mail-archive.com/darktable-dev@lists.darktable.org/msg04528.html
https://www.mail-archive.com/darktable-user@lists.darktable.org/msg07760.html

[phweyland]

Hi Sam,
My 2 cents here. Not sure about what you want to show here. But the digiKam metedata process is not my favourite one.
As commented in another thread, it helps to have a lean process keeping the tags visible "on line" (i.e. browsing or selecting images). So the list of metadata should be visible (and modifiable) on the right panel.
Pop-up window should be reserved for specific and rare operations like selection of the metadata the users wants to use and see on the right panel for example), but not for setting metadata themselves on images.

[sams96]

Hi, my mock up is somewhat based on @aurelienpierre's suggestion in the mailing list, having a separate view rather than trying to squeeze everything into a module in the sidebar. I just thought to use digiKam's interface as a starting point for how to lay out the different fields in the view.

[aurelienpierre]

Yeah, @hanatos agreed on IRC too about having a special view for that. The point being, a metadata editor/digital assets management is a text processor, and you need proper spacing to work on that comfortably, keeping in mind that some people will need to edit metadata in batch for large sets of pictures.

As for the UI mockup, it's ok but I would look toward photo-based social media (500px or Flickr) as a model, instead of another software. I remember Flickr web had something rather efficient to bulk edit metadata, not sure what it looks like now. I don't have more ideas to suggest for now. Maybe beginning with a clear specification book would help (what is the goal, what are the constraints, what are the info we want to edit there, how much do we want to copy/paste between pics, how much time should a picture's metadata editing take, what are the most common field to edit ?, etc.), before any mockup.

Do you have design experience @sams96 ?

[phweyland]

about having a special view for that

I agree with the view, that was not my point.
My point was that editing directly the metadata on right panel is a lot faster that to have to open a pop-up each time (that what is shown above if I'm not mistaken).

[sjjh]

One take away from "The DAM book" for me was the different situations in which meta data editing is used (e.g. bulk during import, selected set of images during triage, individual image during editing). In my understanding in the discussion on the mailing list and on here different people talk about different situations and thus come up with different requirements/ideas -- which leads to confusion & feature requests that seem to conflict. :)

I thus would suggest to start with e.g. writing user stories or a work flow description* to define when meta data editing is done and which goal the user wants to fulfil (at this exact step) (the why). That hopefully would give a clearer basis to discuss how the requirement could be fulfilled best (e.g. which data/fields, panel/pop-up/view, ...).

*knowing that there's not the one workflow

[aurelienpierre]

My point was that editing directly the metadata on right panel is a lot faster that to have to open a pop-up each time (that what is shown above if I'm not mistaken).

But, because the EXIF module has to sit in a side panel, we can't put all the EXIF fields in there. I think the popup from Digikam was just intended to figure the center panel mockup.

I thus would suggest to start with e.g. writing user stories or a work flow description* to define when meta data editing is done and which goal the user wants to fulfil (at this exact step) (the why). That hopefully would give a clearer basis to discuss how the requirement could be fulfilled best (e.g. which data/fields, panel/pop-up/view, ...).

Imagine a news photojournalist or a wedding photographer bulk-adding credits and copyright, but also fixing date and GPS tags for a large set of pictures. It can involve fixing a bad hour/date set in-camera, keywords for stock archives websites, categories and darktable/XMP tags for internal/external archieving. Possibly importing XMP tags from Lightroom or so.

The professional/time-constrained workflow would be:

1. bulk cull the pictures in lighttable
2. bulk edit in darkroom
3. bulk edit metadata in DAM view
4. bulk export in lighttable (possibly choosing wich metadata not to embed in the file, especially removing GPS tags for privacy on the web).

[sams96]

Do you have design experience @sams96 ?

Nope.

With regards to different workflows, my current plan is to go ahead with @aurelienpierre's idea because it seems like the best way to try and fit everything in so while it might not be the most efficient for everybody's workflows it should work for everyone. We could also squeeze some more of the most used into the current metadata module for people who would prefer that, but that can be done afterwards.

[aurelienpierre]

nope.

Not a problem. 3 friend's pieces of advice then:

1. the more time you spend on clearly defining the problem you are trying to solve, the easier the solution. Most newbies will jump too soon on the "easy-looking" solution with a not-so-clear idea of what they are trying to do, and discover too late they ran into a one-way.
2. pen and paper are you best friends. Do lists, draw mockups, and plan everything ahead of writing you first line of code.
3. begin by identifying SMART tasks to perform, for example: expert users should be able to edit title, copyright, description and keywords in 1 min for a set of 5 pictures. Then, you can explore for example what sort of interaction (keyboard shortcuts, tabs click, or sliding panels) fits the best the goal to switch from field to field

[sams96]

So I made a new mock up, again by taking bits from digiKam but that's just because they have most of the fields already so I thought that would be a lot easier than making all the boxes myself. The basic idea is to have 3 columns with all the different fields and then have the options for which types of metadata to apply them to (with some kind of indicator of what's available for each when ticked). There's a lot of fields from the centre and right hand columns missing because digiKam didn't have them. I'd also like to add something to shift the dates/times by a given a given amount.

There is some fields from IPTC especially but also some in XMP like the full contact details, release and expiration dates, and envelope details which could maybe go in another tab or something but I'd be interested to hear from people that actually use those.

[sjjh]

I can only second the advice from @aurelienpierre to think about the requirements before even doing mock-up's. :) If you want to provide a good usability[1], think about the different user types by e.g. applying personas and their tasks by e.g. using user stories and discuss these artefacts with some developers and users before progressing from the problem space to the solution space.

Just in case you haven't stumbled over redmine, there are some issues/ideas hidden as well, e.g.:
https://redmine.darktable.org/issues/8670
https://redmine.darktable.org/issues/8698
https://redmine.darktable.org/issues/10798
And just recently someone has apparently started working on the tagging module, as read on the developer mailing list

Thanks for your work!

[1] according to the ISO definition: The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.

[phweyland]

Working on a similar subject, tags, I have some difficulty to imagine a good way to let the user choose what he wants to export with its images or not.
From my standpoint the choice should be made at the export time (possibly with presets). But it seems that will imply a lot of code refactoring.
A simpler way is to put the choices on configuration, making them unfortunately global to all types of exportation.

Such a user choice / control will have to be made also for the metadata, object of this PR.
What is your view on this ? What is your plan ?

[Nilvus]

I think we could/should start from the actual metadata editor module, seeing it as a basic/standard metadata editor with some main fields only (5 we already have and maybe, or not, some few others). So a quick way as it already exist to edit basic fields on images and permit to let users use the module they know. On that, we could simply add a link to an advanced metadata editor with ideas explained here. And with that solving some actual Github and redmine issues by the way.

Beginning by that (solving possible existing issues and eventually add some improvements on actual basic module) could permit to take time (even if it's not for the next 3.0) to see best choice possible on how to implement this advanced metadata editor discussed here.

[phweyland]

I started to put some contribution here but as that may be an interesting subject of discussion for a wider audience I've made a post on pixls.
In short to turn more complex the metadata in dt as planned here is not the way I would go to. I've tried to explain why on the post above.

[phweyland]

Some design data. My database counts 52.000 images.
The table metadata counts 121.000 records. My name appears 99.000 times. Divided by 3 different fields, it remains 3 times 33.000 times exactly the same info.

[phweyland]

here is the list of metadata schemas, each of them contains several (if not tens of) items which often are redundant with other schemas:

Of course they are all useful for some people and can be useless for others.

So the first point is: why not let the user select the ones he would like to work with in the UI ?
We could list all the metadata of each schema (I hope exiv2 can do that for us) and let the user check mark those of his choice. That's a good way to deal with the quantity and to help people to focus on what is important for them.

The second critical point (I may be repeating myself) is to control the output on the exported file.

• the metadata which should not be exported
• those which should be calculated or derived from others (for example Xmp.dc.rights could be a combination of some text, $(CREATOR) and $(YEAR), City could be copied into Iptc.Application2.City, Xmp.photoshop.City, Xmp.iptcExt.City, ...). This should avoid to force the user to enter several times the same information. Maybe here we could reuse (/expand) the pseudo language already in place for file naming in export module (some metadata are already in the list) ?

So maybe a feasible small steps development path:

• (1) turn selectable the metadata we want to see/edit in UI (this can be applicable to image information panel too) within the metadata we have today.
• (2) create a configuration tool to export what we want (with what we have today): geotag, hierarchical tag, private, synonyms, dt history, dt development, ...
• (3) then expand (1) to other existing metadata (as above)
• (4) add to (2) the meta language to feed automatically some metadata with existing info (copyright, .., but also crop info as I've seen requested somewhere)
• (5) a side work could be to redesign a bit the metadata database in order to avoid to copy every time (for a given metadata) the same text when it already exists (as tagging module does in fact).

Any thoughts ?

[sams96]

Hi guys, sorry but I'm getting a bit busier for a while so I'm putting this project on hold for a while. I'd like to leave this PR open though so hopefully we can slowly collect some more suggestions until I get back to it.

@phweyland your suggestions look great, I will definitely talk to you more about it when I get back to working on it.

[TurboGit]

Sure, well keep this open, there is no urgency anyway. I'll mark it WIP.

[pphotography]

There are programs out there that are true experts when it comes to mass-edit data, those are spreadsheet programs.
They cover a wide-range of use cases for that:

• You can edit tons of data at the same time. Libreoffice says the maximum number of cells per sheet is 1,073,741,824 (2^30) in their calc application. Assuming you have 1 million photographs with 100 fields of metadata you can edit those 100,000,000 fields at the same time.

• You can filter the view to the data. For every row you can filter by direct names or use wildcards, you can manually select certain ranges for date/time or number values and so on.

• You can change values for different rows (aka images here) by copying one cell, select multiple other cells and just paste the data into multiple cells at the same time

• You can navigate easily with arrow keys. With addidional keystrokes you can get to the first/last row or column

• You can search & replace, in the whole worksheet or in a selection

I know it would be a big task to implement these things in a darktable view, but what about the option of building a metadata editor for single images in darktable like above and implementing a CSV export/import in the background for mass-edits?

[GLLM]

Hi !
I find this a great idea 👍

One suggestion : what I miss most in Digikam (and had when I was a windows user on LR) : the ability to copy/paste metadata from a pictures to (an)other(s).
This is especially handy when you do panorama merging, HDR stacking, focus stacking and you want the output picture to have the same metadata (EXIT, ITPC, GPS, ...) as one of the source pictures.

Thanks !
M_LL_G

[aurelienpierre]

I know it would be a big task to implement these things in a darktable view, but what about the option of building a metadata editor for single images in darktable like above and implementing a CSV export/import in the background for mass-edits?

Spreadsheets spread shit. Seriously, these softwares are jokes. When you want to do serious stats, you use R or Python (try Pandas, it will change your life). When you want to do serious databases, you use SQL. When you want to do serious accounting, you use whatever specialized soft suits you. When you want to do nothing serious and want to spare you the trouble of learning another soft, you use spreadsheets. Their UI is aweful, they are stupid slow when things get ugly, it's really not a sensible workflow. dt already uses SQLite for database, we should abuse it and wrap a nice little interface on it.

One suggestion : what I miss most in Digikam (and had when I was a windows user on LR) : the ability to copy/paste metadata from a pictures to (an)other(s).

Well, that's probably 12 lines of C and 5 lines of SQL commands. Wiring the UI to that will take more work.

[github-actions]

This pull request did not get any activity in the past 30 days and will be closed in 7 days if no update occurs. Please verify it has no conflicts with the master branch and rebase if needed.