Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for negative signs and optimization of exclusions #76

Closed
wants to merge 5 commits into from
Closed

Conversation

Dielee
Copy link
Contributor

@Dielee Dielee commented Nov 19, 2020

No description provided.

@@ -76,14 +76,22 @@ def rotate_image(input_file, output_file, angle=90):
:return: void
Rotates image and saves result
"""

if (angle == 0):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should move those conditions out for separation of concerns:
the function should just rotate the image unconditionally; the logic whether to rotate the image should be outside.
Furthermore I have mixed feelings about reverting the sign of the rotation. What was the use-case for that?

So basically

if angle != 0:
  rotate_image(input_file, output_file, angle)

What do you think? 😃

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @monolidth said, this should rotate the image in the correct possition. Maybe it's not trivial.

print(ORANGE + '\t~: ' + RESET + 'Rotate image' + RESET)
with WandImage(filename=input_file) as img:
with img.clone() as rotated:
rotated.rotate(angle)
rotated.save(filename=output_file)


def sharpen_image(input_file, output_file):
def sharpen_image(input_file, output_file, angle):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see that you added angle to pass it to rotate_image, but I think we should rather move rotate_image out of sharpen_image and remove the angle parameter here. It's easier to test this way and again ensures separation of concerns.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated this with last commit, thanks for your input! 👍

@@ -212,9 +225,9 @@ def main():
img = grayscale_image(img)
img = remove_noise(img)
img = deskew(img)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of img[0] and img[1] you can also destructure return parameters like so:

[output, angle] = deskew(img)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated this with last commit, thanks for your input! 👍

@mre
Copy link
Member

mre commented Nov 19, 2020

Thanks for the PR @Dielee. Added some comments. 😊

@monolidth
Copy link
Member

I take a look at the PR as well. Thanks for the work you provided but I have serious concerns.

The problem is not the pull request. There was a reason why I didn't add the auto rotation because it is non-trivial.

I came across of variety of solution, to detect the rotation of the image (and the text).

Sum brightnesses

Idea:

  1. Find contours and filter using contour area
  2. Draw filtered contours onto mask
  3. Split image horizontally or vertically based on orientation
  4. Count number of pixels in each half

Code

import cv2
import sys
import numpy as np

def detect_angle(image):
    mask = np.zeros(image.shape, dtype=np.uint8)
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    blur = cv2.GaussianBlur(gray, (3,3), 0)
    adaptive = cv2.adaptiveThreshold(blur,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV,15,4)

    cnts = cv2.findContours(adaptive, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    cnts = cnts[0] if len(cnts) == 2 else cnts[1]

    for c in cnts:
        area = cv2.contourArea(c)
        if area < 45000 and area > 20:
            cv2.drawContours(mask, [c], -1, (255,255,255), -1)

    mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
    h, w = mask.shape
    
    # Horizontal
    if w > h:
        left = mask[0:h, 0:0+w//2]
        right = mask[0:h, w//2:]
        left_pixels = cv2.countNonZero(left)
        right_pixels = cv2.countNonZero(right)
        return 0 if left_pixels >= right_pixels else 180
    # Vertical
    else:
        top = mask[0:h//2, 0:w]
        bottom = mask[h//2:, 0:w]
        top_pixels = cv2.countNonZero(top)
        bottom_pixels = cv2.countNonZero(bottom)
        return 90 if bottom_pixels >= top_pixels else 270

if __name__ == '__main__':
    image = cv2.imread(sys.argv[1])
    angle = detect_angle(image)
    print(angle)

Thus, this worked astonishing bad. The intuition is that nearly white or grey pixels get close to black because the average of the pixels in the kernel is calculated every time. Therefore, this does not work as good as on images with high contrast.
Thus, this worked astonishing bad.

image

Using min rectangle

In short, the algorithm that is now in the dev branch uses a different approach. It first calculate a convex hull over the receipt (as a shape of an rectangle).

image

After it calculates the angle but you might be noticed that not all angles are correctly identified. Since all images are taken in the landscape the current result is 3/6 (which is unacceptable.

Therefore, I suggest closing the PR. In short, because the current detection is not safe and since most of the user takes an image in landscape (for large objects).

As discusses, maybe we should add an angle tag in the config.yml.

Regards,

William

@Dielee
Copy link
Contributor Author

Dielee commented Nov 19, 2020

Sure you can close this PR. It's maybe not trivial.
No problem, but please add 1051c4d

@monolidth
Copy link
Member

Oh good work @Dielee. Can you maybe provide an new PR?

Regards,
William

@Dielee
Copy link
Contributor Author

Dielee commented Nov 19, 2020

Only with commit 1051c4d ?

@monolidth
Copy link
Member

Yes, please.

@Dielee Dielee closed this Nov 19, 2020
@Dielee
Copy link
Contributor Author

Dielee commented Nov 19, 2020

Done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants