Transform picture (crop/rotate/etc) within the app

[ujjwalagrawal17]

The app should have image editing features.

Desirable features:

• Crop (especially lossless resize à la jpegtran)
• Rotate (90, 180, 270 degrees)
• Modify EXIF
• Blur faces/number plates/etc for privacy reasons (see Editor to correctly blur faces and vehicle plate numbers #190)

Undesirable features:

• Color filters such as Sajuno/Manglow/Palacia/Anax/etc (original colors are better)
• Flip horizontally/vertically (misleading)
• Black and white, grayscale, sepia, etc (loss of information)
• Rescale, for instance 3000x2000 to 300x200 or vice versa (original size is the best)
• Stickers
• Pencil, text

Debated:

• White balance
• Brightness
• HDR (sounds difficult to implement though)

[maskaravivek]

I have been always thinking of having editing options in the app. I couldn't find image editing library that we can simply plug in the app. Am not sure how much effort we can put to write the whole editing feature by ourselves.

@nicolas-raoul would definitely have some insights on this.

[tanvidadu]

javacv library can be used for this purpose. Though I am not sure the effort needed to write the whole editing feature using this library.

[knight-shade]

DS photo editor SDK seems to be a good option. Or do we want to develop from scratch?

Update
@maskaravivek Link: https://www.dsphotoeditor.com/

[maskaravivek]

Can you include a link for it.

[misaochan]

An additional question for @nicolas-raoul ,@whym , or anyone else familiar with Commons policies - do we WANT to encourage people to edit their photos before they put them up on Commons?

Definitely no stickers or 'write' options though, please. ;)

[whym]

I believe the recommended way to do it is to have both - upload the unedited version first and then upload the edited (cropped, whitebalanced, etc) version, too, either as a separate file or by overwriting. This will allow someone else with a better editing skill to re-do the editing later, and until then the file remains as the version the uploader want it to be.

Not sure if there is a clearly spelled out policy about it, but the closest one I can find is https://commons.wikimedia.org/wiki/Commons:Overwriting_existing_files#Unedited_versions .

I don't think uploading the original is strictly required, and in practice, lots of people upload edited versions only, though.

[97balakrishnan]

Yeah i agree with @whym . Even if we provide editing options in-app , I don't think that features like filters,special effects and stickers will be necessary for this app. The basic enhancements/adjustments such as brighgtness,contrast,sharpness are enough and Cropping should be given the first priority.

[nicolas-raoul]

We all seem to agree that only a subset of operations are necessary, and that some operations are undesirable, so I added them to the body of the issue, anyone feel free to modify or ask to modify.

Upload the original + overwrite with the edited version sounds like a good approach, to avoid multiplication of similar files while allowing a more skilled editor to edit the picture better later (the sad reality is that many pictures with poor white balance are used on many articles, and there are not enough Wikigraphists, so uploading only the original is not very pragmatic either).

Pros of implementing our own edition features:

• We implement operations the safe way, for instance we can limit how much brightness gets added.
• We don't offer undesirable features such as sepia, so users won't be tempted.

Cons:

• Takes time to implement and maintain.
• Bigger APK.

DS photo editor SDK can not be used as it is not open source.

[97balakrishnan]

Instead of overwriting, I think that the Auto Adjust feature if available can be automatically applied if someone uploads an image.(Which automatically adjusts brightness,contrast etc).
That's also a good option to be considered

[nicolas-raoul]

@97balakrishnan Would you have a link to this "Auto Adjust" feature? Or is this something that you propose to create in Wikimedia Commons?

[97balakrishnan]

I suggest we can use SDKs to automatically adjust the pic before it gets uploaded to Wikimedia Commons without the intervention of the user. Something like this.
https://imagizer.com/docs#auto-fix

[Jatin0312]

I think for now adding just crop image options would work out well. Since cropping image is an essential edit image feature in any media related application.

[ilgazer]

@97balakrishnan imagizer API also cannot be used as it is not open source.

In my opinion, EXIF modification and anonymization(#70, #181, #1685) would be more useful as a global setting rather than an edit option. A user who would want to remove his camera serial id from a picture would probably like to remove it from every picture. The global setting behaviour is partially implemented in #1685.

Should a new issue for EXIF modification be opened and have the issues referenced above be merged into it?

[misaochan]

In my opinion, EXIF modification and anonymization(#70, #181, #1685) would be more useful as a global setting rather than an edit option. A user who would want to remove his camera serial id from a picture would probably like to remove it from every picture. The global setting behaviour is partially implemented in #1685. Should a new issue for EXIF modification be opened and have the issues referenced above be merged into it?

I agree that global settings would be useful for anonymization. It would be good if we could have an indicator in the upload screen (ShareActivity) that that setting is on, though - as we have had issues with users enabling a setting and then forgetting that they did. :)

A new issue to discuss global settings for EXIF modification would be useful indeed.

[ilgazer]

In my opinion, we should further reduce the scope of this edit interface to these 4 features:

• Crop (especially lossless resize à la jpegtran)
• Rotate (90, 180, 270 degrees)
• White balance
• Brightness

The EXIF editing interface should be completely seperate in my opinion. The proposed "Location data avaliable" indicator in #1687 and another indicator for "You have EXIF data for this picture" that can be clicked and open different interfaces would be more useful.

[nicolas-raoul]

"You have EXIF data for this picture" : almost all uploads have EXIF though.

[misaochan]

@nicolas-raoul To clarify, in my suggestion I didn't mean that the indicator would tell them whether the picture has EXIF data or not. Rather, IMO it should say something like, "Anonymization setting enabled" (if indeed the user enabled that global setting), so that people don't upload pictures forgetting that they have it on, and then complain that all their uploads have no EXIF. Alternatively, perhaps if the setting is enabled we can have an icon (mask?) that shows up on the upload screen.

I agree with the 4 features that @ilgazer proposed. For accessing the EXIF interface, perhaps we can just include that option in the expanded FAB that we already have currently (for zoom etc)?

[nicolas-raoul]

Note for the future: On Commons, a best practice is to first upload your unedited picture, and then overwrite it with the cropped/brightened/rotated/etc version. Benefit: if the picture becomes popular, other people can rework it, for instance rotate it differently. Having the source file means these derivatives can be of much higher quality than if the picture had been rotated two times for instance. So, we might want to do the same from the app: upload the original and then the edited version. The original should have the privacy edits applied, though, and that might in some cases include the cropping (maybe I cropped the picture because it was showing my nameplate). While we don't have to worry too much about this for this first implementation, let's keep it in mind and maybe implement it in the far future.

[vanshikaarora]

a best practice is to first upload your unedited picture, and then overwrite

@nicolas-raoul So should it be like we go to the image details activity from contributions and there display an item in the menu for editing the picture.

Or

While uploading the picture just before the upload activity or just after the upload activity we can ask the user whether they would like to edit the photo?

[nicolas-raoul]

@vanshikaarora The second. In a next phase, after the user has edited the image, we may want to internally upload both the original and the edited picture, but the user would see this as a single upload.

[vanshikaarora]

@vanshikaarora The second. In a next phase, after the user has edited the image, we may want to internally upload both the original and the edited picture, but the user would see this as a single upload.

@nicolas-raoul I have used the following library https://mindorks.com/android/store/Image-Croppers/arthurhub/android-image-cropper and that works pretty well for the current gradle versions.

In case of multiple image's:

One way is to create an activity that will show thumbnail of selected images then the user can select image to edit (or even more than one, the user will be redirected back to this same activity after editing image).

@nicolas-raoul @maskaravivek Can you please pitch in your ideas :)

[maskaravivek]

I was thinking if we can add an edit icon in the middle section which currently has the location icon.

Clicking on the edit icon can open a separate activity(say EditActivity) where the user can crop the image and return back to this activity(UploadActivity) once done. The EditActivity can later include other edit options.

• EditActivity can be started using startActivityForResult
• UploadActivity can implement onActivityResult to handle edited image.

[vanshikaarora]

I was thinking if we can add an edit icon in the middle section which currently has the location icon.

@maskaravivek that would definitely be better I can add the edit icon just next to location icon and that will take the user to the edit screen. 👍

Do you agree with the current cropping library implemented or will you prefer some other one?

[maskaravivek]

The library looks good to me. It looks to be quite popular and is still being maintained. :)

[maskaravivek]

Good work @vanshikaarora! Will review the changes tonight. :)

[vanshikaarora]

Thanks @maskaravivek, do share your feedback's after review :)

[neslihanturan]

The solution of this issue is almost ready at #2542 . But the PR is sleeping for a while. It can be a nice task for a beginner (not for a first time contribute but if you feel comfortable enough in repository) to fix conflicts of existing solution. So that we can include it to our codebase.

[madhurgupta10]

@neslihanturan Can I take up this issue?

[nicolas-raoul]

In order to prioritize editing features, I did some research about JPEG quality loss in popular editing apps (mostly Google Photos and Samsung Gallery), and published my findings here: https://github.com/lossless-jpg/data

In short: cropping/rotating/blurring a picture is unnecessarily lossy even with popular apps. 😱
That means not only lower quality but also (because lossy editors often save with an unnecessarily high quality setting) bigger file sizes, which costs unnecessary hosting money to Wikimedia, money that would be better spent on other things.

So, I am now even more in favor of implementing this (possibly as a GSoC?).
I am currently looking for an Android library for lossless JPEG transformation.

[Ayan-10]

@nicolas-raoul I made previous year's GSoC proposal with this issue. While researching about its implementation I came across some good image transformation libraries. One of them is uCrop. Don't know you checked this library or not. But I know some SDK's and libraries regarding this and I can add them in https://github.com/lossless-jpg/data. I read the documentation but I have some queries regarding that.

1. While comparing the images how can I determine a lossless transformation. If any red spot occurs in any transformation that means that a lossy transformation and To be a lossless one the comparison image should contain 0% red color. Is it right?
2. How can I determine EXIF is kept or lost for a transformation?

Can you please resolve these queries?

Thanks in advance.

[nicolas-raoul]

@Ayan-10

uCrop does not mention it is lossless (which always means it is lossy in my experience so far), but it is indeed worth trying by installing their sample app https://play.google.com/store/apps/details?id=com.yalantis.ucrop.sample

1. To determine whether a crop was lossless or not, please follow steps 1, 2, 4, 6 of https://github.com/lossless-jpg/data#methodology
2. To determine whether EXIF was preserved, you can use the Copy all button in https://play.google.com/store/apps/details?id=com.aminbeheshti.exifviewer for both original and transformed, then compare with a diff tool.

Thanks!

[Ayan-10]

@nicolas-raoul Okay got it. I will check the libraries that I know. The playstore link of uCrop seems not working.

1. I already read 1, 2, 4, 5, 6 steps. I was asking that after the 6th step when we get an image for each transformation from https://online-image-comparison.com which you are saving with the _comparison suffix. By that _comparison suffix image how should I judge whether the image transformation is lossless or not?

2. Got it. Thanks for the explanation.

[nicolas-raoul]

@Ayan-10 Ah sorry! If the comparison image has any red pixel, then the transformation was lossy.

[Rishavgupta12345]

@nicolas-raoul @4D17Y4 According to https://github.com/commons-app/commons-app-documentation/blob/master/android/Students.md can i consider this project to work on in GSOC 2022?

[nicolas-raoul]

@Rishavgupta12345 If for any reason you are strongly motivated by this issue, you can choose it. But otherwise I would suggest #4764. You can also submit a proposal for each.

[nicolas-raoul]

https://github.com/bkhall/AndroidMediaUtil (a port of IJG code) implements lossless rotate and crop: https://github.com/bkhall/AndroidMediaUtil/blob/master/src/android/mediautil/image/jpeg/LLJTran.java#L188

[Rishavgupta12345]

@4D17Y4 @madhurgupta10 please review my GSOC proposal on Phabricator https://phabricator.wikimedia.org/T303273

Gsoc proposal submission date is from 4th-17th April 2022

[madhurgupta10]

@Rishavgupta12345 I will not be able to mentor this year, so feedback from @nicolas-raoul and @4D17Y4 on your proposal would be the most valuable.

[nicolas-raoul]

jpegtran can be used in our Android app via JNI, here is an example doing that: https://github.com/kamemak/ajpegtran_example

And here is a jpegtran fork that performs blurring without reducing the quality of other parts of the image: https://github.com/kamemak/jpegtran_pixelization

With this, I am all for implementing crop/rotate/blur inside our app. :-)

We should first make that code into a library (the developer would probably accept such a pull request), to not force all developers to install the NDK.

[shankarpriyank]

@nicolas-raoul
Maybe jpegtran is not the best option, I have not tried it but please have a look at this thread.
Waiting time of 15 seconds is not really a good UX

[nicolas-raoul]

@shankarpriyank Do you mean that LLJTran may be faster? If you mean something else, please post the facts here and be explicit, thanks :-) Also, where does the 15 seconds figure comes from? jpegtran takes around one second to transform a large picture.

[shankarpriyank]

@nicolas-raoul In this thread if you read the top-rated answer, a developer is telling that it is taking around 15 seconds for the transformation to complete.
I did not test anything myself, just sharing what's written in that thread.

I guess I will try making a sample out and report out my findings.

[shankarpriyank]

Hey @nicolas-raoul I am making sample/poc app for lossless transformations, just wanted to confirm once if this issue can be fixed as a project under GSoC 23

[nicolas-raoul]

Hi @shankarpriyank, thanks for the question! There is an official GSoC task (making upload more reliable), but since you seem to already have experience in the non-trivial field of lossless transformations, you are welcome to create a proposal based on this GitHub issue. Thank you for your interest!

[shankarpriyank]

Hi @nicolas-raoul. I made a sample/poc app and tested it. The transformations are very fast, practically there is no delay.
Although during the process of making that app, there is one challenge that I identified. We need to discuss the flow of the transformations or the permission handling. Basically with the security updates coming in android now if you want to edit any media file which is not of the app itself the app will need to ask for extra permissions(https://developer.android.com/training/data-storage/shared/media#update-other-apps-files).
But there is no need for this permission if you do not want to modify the original file. We can just save a new transformed image. I also had a look at #2542 the pr seems really nice and I think we can follow the same approach to implement the crop feature.
Summing it up we are looking at Crop/Rotate/Blur and we have an implementation strategy ready for Crop and Rotate.

Please let me know your thoughts

[nicolas-raoul]

@shankarpriyank

I am very happy to hear that performance is good!

About permissions: We already have the picture's full data, for instance we read its binary data to find out whether the picture was downloaded from Facebook or not. So I don't think permissions will be an issue to implement this feature.

Thank you for checking Vanshika's work! As you know, two additional challenges here will be to:

• Implement the same using only lossless transforms.
• Implement blurring a part of the image. For inspiration, this app implements blurring while leaving other parts of the picture lossless: https://play.google.com/store/apps/details?id=jp.kame.jpegcropper Of course you are very welcome to come up with your own UI ideas.

[shankarpriyank]

@nicolas-raoul
Reading data is one thing and editing it is different if we want to transform the original image I think there will be permission issues coming up, please have a look at this link - https://developer.android.com/training/data-storage/shared/media#update-other-apps-files
Sorry, I maybe wrong but can we crop images without bearing any loss?

[nicolas-raoul]

@shankarpriyank

The good news is that we do not need to open files in write mode. If we have all bits from the image, then we can for instance copy it to another array or temporary file, modify that array as much as we want, then upload that array to the Wikimedia Commons server (in other words: upload the modified file) with no permission issues.

can we crop images without bearing any loss?

Loss in the blurred part is acceptable, but loss far from the blurred parts is not acceptable. Please read this section to understand what is expected. This can be implemented this way:

1. Split the JPEG image into many 8x8 JPEG images. Zero information is lost in this step.
2. Blur only the blocks that that users wants to blur.
3. Put all of the blocks back together. Zero information is lost in this step for non-blurred blocks.
4. Update (or just remove if easier) the preview often embedded in the EXIF, as it still shows the private information.

If this sounds like too much work for a GSoC, feel free to split your GSoC proposal into two parts:

1. Crop and rotate: Prototype, proper implementation, tests, documentation, merge.
2. Blur: Prototype, then the rest as much as time allows.

[shankarpriyank]

Hey @nicolas-raoul I made a proposal on phabricator, I would be really grateful I you could provide me with some feedback