Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Colorimetry plan #979

Closed
jywarren opened this Issue Apr 2, 2019 · 31 comments

Comments

Projects
None yet
6 participants
@jywarren
Copy link

commented Apr 2, 2019

I'm documenting all of the steps required to go through a dual color analysis sequence for one of the #707 real-world use cases: colorimetric comparison!

Working from this post, i found a sample image of a colorimetric badge that reacted to formaldehyde, where one side got darker over a few hours.

Here's the test image:

badgelittle

Note: we assume that we can guide people to photograph the badge with consistent framing. That may be its own design challenge! We can show examples, and we can also show an overlay on the initial step's preview, so people can tell if their images are properly framed.

My process was to crop the image so that we see only color from one of the two sides, so I used the crop module, but it had some bugs (that I think we can solve and then protect with better tests!).

Screen Shot 2019-04-02 at 7 21 08 PM

The issues were mostly to do with the units in the x and y values I input. The image is 244x184 pixels, but here are the values I had to enter to get the left and right sides nicely cropped:

So, 130 for the left, and 500 for the right. Which can't be quite right!

Process

So, I think we could basically approach this use case in 2 ways:

  1. We could have the starting module trigger 2 sequences, one for left and one for right. Each crops, then averages. Once both are done, we compare, for example, the average red channel of each.
  2. We run the left side (crop and average), then use the import-image module to re-import the original image, then run the right side. At the end, we look back and fetch the outputs from the two average steps (left and right), and compare the red channels of each.

In both case we display the ratio of the red channel (for example) between the two sides.

Doing this manually, i get:

Left:

image

Right:

image

Next steps

So to list out the bugs we need to address, i think:

  1. the main one is the x,y coordinate issue in the Crop module
  2. there's a bad crop bug which inverts the image, here: #213

Then, I think we can try making a new demo file in /examples/, which displays the ratio from the above process!

@jywarren

This comment has been minimized.

Copy link
Author

commented Apr 3, 2019

Here is the "all in one sequence" approach!

And here's a link to sequencer for this, although the GitHub comment encoding messes it up, but copy-paste it into a browser, and drag the sample image in to the top and the import-image step: https://sequencer.publiclab.org/examples/#steps=crop{x:260|y:110|w:50|h:50},average,import-image,crop{x:500|y:110|w:50|h:50},average

image

@jywarren jywarren referenced this issue Apr 3, 2019

Open

List of real world use cases to aim for #707

1 of 3 tasks complete
@lohitha02

This comment has been minimized.

Copy link
Member

commented Apr 4, 2019

@jywarren what do you say about automating the input values of x and y instead of typing the coordinates. Like how about having a click and drag option. So you can click and drag until where you need to crop and when you do so the values of x and y will get updated automatically.

@shapironick

This comment has been minimized.

Copy link

commented Apr 4, 2019

at https://github.com/publiclab/SmART-Form we overlaid a semi-transparent image of the optimal placement of the badge over the camera feed to guide users (worked well in beta and field testing) and I think but am not sure that we had an auto-cropping.

I think the scaled bitmap should be around lines 999 -1016 here https://github.com/publiclab/SmART-Form/blob/master/Android/app/src/main/java/edu/osu/siyang/smartform/Fragment/TestFragment.java

@jywarren

This comment has been minimized.

Copy link
Author

commented Apr 4, 2019

Hi @lohitha02 actually yes, there is supposed to be a "drag to select" interface which is currently broken -- @publiclab/is-reviewers any ideas what happened here? I opened an issue at #987

For the working demo, we should be able, as @shapironick says, to have an overlay to guide people. Thanks Nick.

This would change for different types of colorimetric tests, but this is a good initial set to work with. And different tests might compare the 2 patches in different ways - how red they are, say, or how dark they are.

And thanks Nick, indeed here it averages RGB channels and compares the brightness of the two patches, so our code should do the same: https://github.com/publiclab/SmART-Form/blob/master/Android/app/src/main/java/edu/osu/siyang/smartform/Fragment/TestFragment.java#L965-L1009

The overlay would be pretty easy to add. The other thing we could do later is to add QR codes at 3 positions around the patches, and use those to auto-crop, to smooth things out even more.

@jywarren jywarren referenced this issue Apr 5, 2019

Merged

Added doc for grid-overlay #983

0 of 4 tasks complete
@Divy123

This comment has been minimized.

Copy link

commented Apr 7, 2019

Hi @jywarren !
I would love to work upon it!!!
So basically what I have understood from the previous discussions is that we can have a sequence defined in the sequencer where we can have the badge image set to following steps:

  1. Crop the left part and average the pixles of resulting image.
  2. Crop the right part and average the pixels of resulting image.
  3. Finally we can output the ratio of the red channel of both the results and display it to the user.
    Just want to ask one thing how can we display the ratio, like if we have to build some UI for it?
    Also I think meta-modules can be uselful for this use-case.
    What do you think?
@jywarren

This comment has been minimized.

Copy link
Author

commented Apr 8, 2019

@VibhorCodecianGupta

This comment has been minimized.

Copy link

commented Apr 8, 2019

@jywarren Yes, I believe making a separate micro service for this would be ideal. Passing in images (encoded or otherwise) and using two IS instances to process and then output the comparison would make this easy to implement AND easy to use.
The first step to this would be fixing the crop module, I'll be on it as soon as I draft a proposal.
Also, a little late on my SoC proposal, apologies! Will roll one out by the end of the day, and hope that I'm not already too late for your reviews!

@Divy123 Divy123 self-assigned this Apr 8, 2019

@Divy123

This comment has been minimized.

Copy link

commented Apr 8, 2019

@jywarren I have made some changes to the proposal to include the calorimetric issue to my proposal which I really find is a strong use case for our organisation to proceed with.
Please have a look: https://publiclab.org/notes/lit2017001/04-01-2019/soc-proposal-image-sequencer-v3-boosting-the-performance

@VibhorCodecianGupta VibhorCodecianGupta self-assigned this Apr 15, 2019

@VibhorCodecianGupta

This comment has been minimized.

Copy link

commented Apr 15, 2019

@Divy123 Could we collaborate here? This task requires breaking down into multiple issues, I think it'll speed up things this way :D

@Divy123 Divy123 referenced this issue Apr 16, 2019

Merged

Fixes crop module bug #1019

4 of 4 tasks complete
@Divy123

This comment has been minimized.

Copy link

commented Apr 16, 2019

@jywarren

This comment has been minimized.

Copy link
Author

commented Apr 18, 2019

Crop fixed in #1019!!!

@Divy123

This comment has been minimized.

Copy link

commented Apr 18, 2019

@jywarren proceeding further with this one.
I think I can make a repo on github and make you a collaborator?
Or if we can make a repo in Public Lab?

@VibhorCodecianGupta

This comment has been minimized.

Copy link

commented Apr 18, 2019

Cool idea. Let's have a Public Lab repo, a separate colourimetry tool.
@jywarren on your discretion, shall we collaborate?

@Divy123

This comment has been minimized.

Copy link

commented Apr 18, 2019

Sure, we can collaborate..

@jywarren

This comment has been minimized.

Copy link
Author

commented Apr 18, 2019

@VibhorCodecianGupta

This comment has been minimized.

Copy link

commented Apr 18, 2019

@jywarren don't think we have access to creating a repo, you'll have to do that I'm afraid.

@jywarren

This comment has been minimized.

Copy link
Author

commented Apr 18, 2019

@HarshKhandeparkar

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

This is just the name of thr repo right? Doesn't have to be technical? Should it have the colorimetry in its name or can it be called something like rainbowmetry.js?

@jywarren

This comment has been minimized.

Copy link
Author

commented Apr 18, 2019

@jywarren

This comment has been minimized.

Copy link
Author

commented Apr 18, 2019

@Divy123

This comment has been minimized.

Copy link

commented Apr 19, 2019

@jywarren so to proceed forward with this, what I was thinking is we can provide a UI such that it includes:

  1. Load Image similar to IS
  2. After loadidng image, crop module is applied automatically to the image.
  3. After the image is cropped, that means as soon as the user applies the changes for the crop, average module is applied.
  4. At this step we can store the red channel avg value.
  5. Once this is over we can trigger the import-imaage module and then again simlarly steps 3 and 4.
  6. At last we can display the raio in a simple box or some suitable UI field.

What do you think upon this?

@HarshKhandeparkar

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

I can work on the UI.

@HarshKhandeparkar

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

I can also help with the algorithm.

@HarshKhandeparkar

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

I can also GPU accelerate it if needed.

@HarshKhandeparkar

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

If we have similar images, we can select settings for one of them using the UI, we can parallelize colorimetry on all the images including the forst one after that. Paralellization can be done on a gpu. Thoughts? @jywarren please transfer this issue to the colorimetry repo so that we can discuss there.

@HarshKhandeparkar HarshKhandeparkar changed the title Bugs in crop module re: colorimetry, Colorimetry plan Colorimetry plan Apr 19, 2019

@Divy123

This comment has been minimized.

Copy link

commented Apr 19, 2019

Ya sure @HarshKhandeparkar !!
@jywarren what are your views upon this?
#979 (comment)

@Divy123

This comment has been minimized.

Copy link

commented Apr 19, 2019

@jywarren please share your opinion !!

@jywarren

This comment has been minimized.

Copy link
Author

commented Apr 19, 2019

I left some notes in publiclab/colorimetry#1 that may be relevant! Let's try to stay in sync by opening a few issues over there? I think it's possible to do a bare-bones UI with an image select and a display of the 2 numbers (and the ratio maybe?), with an example image of how you should photo the test badge.

Let's build out some issues over there! Thanks!

@HarshKhandeparkar

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

@jywarren GitHub has a transfer issue feature. Transfer this issue there.

@VibhorCodecianGupta

This comment has been minimized.

Copy link

commented Apr 19, 2019

Discussion shifted to publiclab/colorimetry#2

@HarshKhandeparkar

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

Closing...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.