Make it possible to upload new files to Wikimedia Commons

[wetneb]

When editing MediaInfo entities, we should also be able to create new ones. As a first prototype, we would add a field to specify a path to a local file (to be then generalized to remote URLs), which:

• if the entity is marked as new, uploads the new file alongside the structured metadata
• if the entity is existing, replace the existing file by the provided file

This interface would then be aligned to @lozanaross' design, which requires the introduction of a notion of template.
This will cover the generation of Wikitext.

[trnstlntk]

After receiving a question re: how we will actually implement this (at which stage will the structured data be added to the file, and how can this process be made as simple as possible?): this Phabricator ticket may be of interest. (Basically, UploadWizard adds SDC after a file was uploaded, if I understand correctly)

[thadguidry]

I think we would want to perform the batch process as efficiently as possible with existing API's?
So that might actually mean that in the backend we perform these steps in this order (and the user doesn't have to know we do it this way):

A. upload all files
B. then one by one apply structured data to each file that was uploaded in step A.

So then on the user facing side in OpenRefine, I can envision 2 ways to show progress to a user:

1. total batch completed (hide the truth) (and not worry about showing them that step A completed, but only showing them step B's progress)
   or
2. files uploaded progress count (22/100) as well as showing structured data applied count (1/100) and bar progress. (be truthful to the user and show them the order of what we actually do on the backend)

@wetneb Also, I'd like to see a link to @lozanaross' design in the original comment, because I didn't see any design for OpenRefine here: https://phabricator.wikimedia.org/T294938

[DominicBM]

It's not just about efficiency, but the 2-step approach probably introduces some risk and more complicated workflow to avoid problems they can cause.

I am not 100% sure of OpenRefine's plans here, but I think you have a lot of similar needs to what I expressed in my comment on Phabricator just now at https://phabricator.wikimedia.org/T224214#7846111

I suggested a way of how this could be implemented in the MediaWiki API. Would be great if the OpenRefine team could also comment on that ticket from your perspective to help WMF prioritize.

[wetneb]

On my side I was just planning to add the SDC straight after uploading the file (indeed with two API actions, which is a priori not a big deal for us, although I can totally see the point of wanting a single API action).

[thadguidry]

@DominicBM I also felt the same way and agree the 2 step approach, while aligned to file transport mechanisms with HTTP/FTP on the far backend, does run the risk of putting things "out of sync".

[wetneb]

@lozanaross could you post a link to your wireframes for this workflow?

[lozanaross]

@wetneb Yes - so to answer some of the comments above, I've now closed & updated the phabricator issue, which somehow was lingering even though the results of the research were published a long time ago. For what you're describing here - i.e. how the API calls work "in the background" - I'm not sure what wireframes are best to be linked - wireframes only show the "user-facing" side of things, not how things are done in the background. We have the "lo-fi" versions which were published back in January & linked to in the midpoint report and the phabricator ticket now. And we have the "high-fi" versions, but they are more about presenting distinct design patterns than showing a complete workflow. I'll be updating these anyway, based on some of the feedback from the recent user sessions + survey. Should have a new and more complete version next week.

[thadguidry]

@lozanaross Thanks, very helpful