Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grainy and false color image with ASI120MC #180

Closed
RobertG73 opened this issue May 13, 2020 · 30 comments
Closed

Grainy and false color image with ASI120MC #180

RobertG73 opened this issue May 13, 2020 · 30 comments

Comments

@RobertG73
Copy link

Hi, i have a strange issue with images taken by my ASI 120 MC. There are two types of bad images which appears totally random and also last for hours when appears. in some nights doesn't appears at all... I attach a photo with both issues. Anyone knows what could be the problem and how to fix it?
Both cases disappear only if i reboot the PI and if i only restart the allsky service this problem doesn't disappear.
allsky issue

@Weather42
Copy link

Weather42 commented May 21, 2020

I've just started using Allsky, and I have almost the same camera (mine is an ASI120MC-S). I had the same issue last night...all of a sudden about mid way through the night (based on the video), it suddenly changed to a blueish tint. And towards morning it got grainy, almost looks like it is looking through a screendoor.

@RobertG73
Copy link
Author

RobertG73 commented May 21, 2020

Today this problem started earlier and i am sure now it is because of camera. I attach two images, image(taken by camera) and image-processed(by allsky software. as it can be seen the problem appears from the image delivered by camera. I am thinking to one solution to detect this issues in openCV and when is detected to reset the ASI camera. But as my programming skills is almost equal with zero, i must learn how to do this. :)
image
image-processed

@phil-zenger
Copy link

phil-zenger commented May 21, 2020

Try using a Dark Subtraction file that is the same resolution and your pictures. So, for the ASI120MC camera, your highest resolution is 1280 X 960. I have attached a Dark file with that exact resolution. Then copy that file to your /home/pi/allsky directory. You then have two options. 1. rename the file to Dark.png or edit your config.sh setting to DARK_FRAME = dark_1280_960.png.
Dark_1280_960

--

@Dentonknifeworks
Copy link

Is this not the same as doing a dark frame within the GUI?
I get the same type of images on mine and I have done the dark frame capture within the GUI.
Brad

@RobertG73
Copy link
Author

It is not because of dark file. I am using a good "dark.png" image without any problems. What you see in pictures is not related to dark images. Appears and disappears randomly even on the "image" taken from the camera before it was processed with the dark. It is some kind of blue "grid" with totally different hot pixels from the dark taken.

@RobertG73
Copy link
Author

I forgot to mention, i am using three darks, taken at 10, 20 and 30 degrees darks changed when chip sensor exceed +/- 5 degrees around this temperature values. But it has nothing to do with dark changing. This problem appears totally random and is a camera issue(in my opinion).

@Dentonknifeworks
Copy link

Well last night I started getting the same issue. It started the run with the blue/red/green dots and went all night. Powering it done and playing with the camera settings did nothing to fix it. During the day when the gain and exposure is at 0 no problems. Tonight I am going to limit the exposure and see if that helps.

@RobertG73
Copy link
Author

In this evening i have seen the camera with that blue image full of hot pixels. Quick, in this state i make a dark frame. I used this dark to substract the hot pixels in the "startrails" software with very good results. So, if i will catch a fireball on this bad state of the camera i have the posibility now to clean the image and see the stars, even if the image remain bluish.

@Dentonknifeworks
Copy link

I am still doing testing on mine I finding that the Dome is degrading my photos a lot.
Need a better dome then the one I got off of Amazon.

Some of my first runs without the dome looked very good. I have the dome off as of 6-2-20 it ran last night and has a lot less hot pixels. Tonight going to run at gain 100 and auto expos.\ no dome.

My photos are at www.dentonknifeworks.com

@RobertG73
Copy link
Author

I have (fortunatelly) a glass dome. Acrilic ones, in time, become less transparent and will make the quality of images lower, but i think it is noting to do with hot pixels. The images will become a bit blurry, i think. The real problem might be the temperature of the sensor... i think. I am thinking to place a small fan to cool down the ASI camera. Now, because ambiental temperature raised, i have more frequently this problem with bluish image and hot pixels. I will check this...

@Dentonknifeworks
Copy link

Dentonknifeworks commented Jun 4, 2020

After more testing I agree the dome is not producing the Hot pixels.
What I did lastnight was redid the dark photo. I found that just turning it on and waiting a min is not the way to do it. I set the camera settings no auto settings, cover the camera turn on the dark frame and watch the live view. I let it run for at least 5min I found as I watch it and the camera ramped up I could see the hot pixels in the live view. I then turned off dark frame and looked at the dark.png in the allsky folder and found the hot pixels. The one I had before did not show the hot pixels. The end video looks better now all be it still some hot pixels are still showing.
I have not figured out if the dark frame is only used during the stacking and nothing else.
I am going to pull the images and stack them with the dark frame and see what I get with DSS.
Them Galss domes are kinda $$ ............. as to Temp mine has been running around 30 to 40C and here in Texas its getting Hot now....

@RobertG73
Copy link
Author

I have seen the same. When the blue image appears, also new hot pixels(a lot of them) are shown, This ones, does not fit the usual dark and you need a new dark for processing this. I think this is a hardware problem related with debayering process inside the camera. I am thinking to detect when this appears and reset the USB, because the problem dissapear only if power off/on the camera. Also, in the next days, i will relocate the fan which cool down the PI to cool simultaneously both, pi and the camera.

@robertpascale
Copy link

The problem I see with the dark frames is that they vary with the exposure length.
Some nights, my exposures are very short - could be only a few seconds (lots of clouds, moon is out and light pollution), and other nights which are much darker, (up to 30 seconds). The dark frames for 30 seconds and the dark frames for a few seconds are vastly different.
On top of that, the ASI120MC-S is not a cooled camera. The hot pixels at sensor temperatures of 30C vs 10C are very different. We can't really control the temperature variable, but unless the entire dark capture routine is re-written to create some kind of library (based on temperature and exposure length), and then some kind of 'smart' dark subtraction which uses the meta-data to pick the best dark frame, you basically have to take fresh darks whenever there is a big variation in either temp or exposure length.

@Dentonknifeworks
Copy link

Well at this point it looks to me that the ASI120MC-S in this texas heat is not good for Allsky.
My camera temps average 40c to 60c during the night and Hot pixels are real bad. Anything above 4 secs are at gain of 50 the image is full of hot pixels. Now what dont make sense to me is I can do a 30 Sec image with different software with the same camera on a telescope and dont have the hot pixels.

@Dentonknifeworks
Copy link

image-20200612225004
This is what I am getting every night now.
dark
My current dark file

@robertpascale
Copy link

I've found the entire dark subtraction process with this software to be a bit of a hit & miss exercise.
I had worse looking subs than you.
I deleted my dark, and just went without, and it was vastly improved.
When I get some time, I want to try and write my own dark library modification routine (I have created an issue in this regard). I would just try and make your darks as close to the same duration and gain/temperature as your typical night (though with the camera being uncooled this is going to be very diffcult).

@Dentonknifeworks
Copy link

Ok so I renamed the dark.png and let it run last night no difference at all it like the dark file is not doing anything at all.

@robertpascale
Copy link

That sounds like a useful result. I don't think you have to restart the service, but did you try that?
If you did, then I would try and make a fresh dark, and see how that goes.

The code does the following when it makes a dark frame:
convert "$FULL_FILENAME" "$DARK_FRAME" -compose minus_src -composite -type TrueColor "$FILENAME-processed.$EXTENSION"
so you could try it at the command line with your current dark and a sample image and see what you get:
convert whatever.png dark.png -compose minus_src -composite -type TrueColor output.png

Make sure your config file specifies the dark name (it should be dark.png)

@robertpascale
Copy link

Looking at your dark, and the image, they looks like they are vastly different (ie maybe you did the dark when it was cooler?). Given the temperatures you're talking about (40+ degree sensor is pretty hot) - I can guarantee you're going to be way cooler hanging off the back of a telescope vs being in a box.
I suspect I'm going to have the same issues as you when I get to summer. I think the only options are to get a cooled camera, or to code up a temperature sensitive dark library.

I looked at this a little - there are two main issues. The code library being used does not store meta data in the image files (unlike a FITS file for example). So neither the exposure length, nor the sensor temperature are stored in the image files. This makes coming up with a routine a bit messy. I'm thinking that either there needs to be a routine to encode the information in to the filename (which causes other issues), or adding a library to embed metadata in to the images (like EXIF data), which could then be used to be able to dynamically apply an appropriately scaled dark.

@RobertG73
Copy link
Author

RobertG73 commented Jun 15, 2020

I am using three darks. From the original "capture.cpp" i export in a text file the sensor temperature. Then, PHP read the value from the file and is changing the dark.png with one of three darks taken at 10, 20, 30 degree celsius +/- 5 degrees hysteresis around these values. But as i said, that bluish and pixeled image does not match any of that darks which are all good and is working very well in allsky dark substraction when the camera is working normally. Didn't had time to make a code to detect that blue image and switch to that specific dark which is good in that condition.

@Dentonknifeworks
Copy link

I am thinking your on the right track on the Temp. I added a fan to the box today I will see if it runs cooler tonight. At looking at the frames taken looks like 40c to 50c on temps. If the fan dont work I will put the cam on the scope and see what temps it gets too and recheck the bad pixels on it.

@Dentonknifeworks
Copy link

Well the fan helped on the temp now runing at 30 - 33c.
New photos look a little better but the old tear to the right side is still a random issue.
I have yet to play with the darks yet thanks for the code I will play with them.
Here is the new hot pixels photo at lower temps.
image-20200617215236
image-20200617215311

@michaeloconnell78
Copy link

michaeloconnell78 commented Jul 18, 2020

@RobertG73 I have the exact same issue as you.
The colour of the image suddenly changes toward the blue during the middle of the night for no reason at all.
It's like flicking a switch. No transition. Just sudden change from one frame to the next.
Hot pixels that were green become blue or red .
Hot pixels that were red (very few) become green.
A host of new red hot pixels are suddenly created.
image-20200718005759
image-20200718005807

@michaeloconnell78
Copy link

Got worse again last night. I'm now getting banding appear on the RHS of the image when it goes all blue, pixely and ignores the dark. Of no use like this.
image-20200807235006
image-20200808034143

@michaeloconnell78
Copy link

And this morning's image:
image

@maserowik
Copy link

For my past experience with the asi120. It not a good fit for a raspberry pi. Had nothing but issues with it. Worked with vendor and now I upgraded it to a asi224. Much better quality and works better.

@michaeloconnell78
Copy link

Thanks.
However, I'm slow to invest in another camera for this project.
I have a spare DMK camera.
Anyone got that to work on the Pi for this project?

@dmnkhhn
Copy link

dmnkhhn commented Aug 12, 2020

Unfortunately not, I’m having the same issues with my ASI120MC-S on a Pi 4 4 GB. The cam works fine on a second Pi 4 as a guiding camera with the INDI drivers.

@Dentonknifeworks
Copy link

Dentonknifeworks commented Aug 12, 2020

I put my ASI120MC-S back on the telescope and works fine. So I went with a HQ pi Cam and used the converted code.
No more problems hot pixels or torn video. The HQ pi Cam is not as sensitive as the ASI but its not bad. I have seen some located in darker Skys then I have and there are picking up the milky way.
This is a frame from last night the green tent needs to be adjusted. There is one problem with the HQ pi cam and that the auto exposure don't work in the modified software. So each night I adjust it for the conditions at the time. This shot was 20 sec at max gain of the camera I was hoping to catch some meteors last-night.
image-20200802214857
Here is last nights video.
http://www.dentonknifeworks.com/allsky/videos/allsky-20200811.mp4

@EricClaeys
Copy link
Collaborator

This issue is from an older release of the AllSky software; the current release uses a dark library with dark files based on sensor temperature which improves dark subtraction. Other issues above might be hardware related.
If you think this problem might still be relevant, please test it with the newest version and submit a new issue if needed. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants