Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

961 protect from empty camera image #967

Merged
merged 11 commits into from
Dec 23, 2021

Conversation

Ezward
Copy link
Contributor

@Ezward Ezward commented Dec 17, 2021

This PR addresses Issue 961 and a little more. Basically our cameras could return None images if they did not start correctly. The warmup was just a sleep(2) and no check was actually made to know that the camera started. This could result in hard to understand downstream errors.

  • This PR fixes the 'PICAM', 'CSIC' and 'WEBCAM' camera drivers to check the the camera starts and is returning non-None images at initialization time so any camera initialization issue is caught early and with a clear error message.
  • There are other camera configurations that have NOT been addressed by this PR, but should be looked at in a future PR; 'V4L' and 'CVCAM'.
  • this PR also addresses the originally reported bug directly by protecting cv_control.py template from a None image.
  • In addition it makes the debug window optional; it can be turned on with a command line argument. This was done because the cv_control.py template could not be used in a headless OS because it tried to open a window.
  • In addition it adds a CAMERA_INDEX configuration value that can be used to choose the camera index in the 'WEBCAM' and 'CVCAM' configurations when there is more than one camera connected.

- prior code just waited a couple of seconds.
- this new code now starts asking for frames and will wait
  until a frame is return or a timeout happens
- Generally this is a more reliable and faster way to warm
  the camera; if it is going to work it will return a frame
  quickly.  So start up is faster.
- The warmup code also declares a better  error if the camera can't
  start; it raises the error in the camera part rather than
  downstream where the image is used or later in the pipeline.
  That makes it easier to diagnose what is going on.
- if the camera returns a None image the code would
  crash.  That could happen with the prior CISC camera
  if it did not start properly.  However, that error
  was confusing.
- Now this code will simply ignore an None image.
- added --debug flag. if this is passed to 'drive' command
  then debug window is shown.  Otherwise it does not.
- this fixes an issue when using a headless OS on the
  donkeycar; open cv cannot create a debug window and so crashes.
- this guarantees we can produce frames before the camera
  is used
- also detects if pygame is not installed and prints instructions
- if both a CSIC camera and a USB camera are connected then
  there will be more than one camera available
- The current code always just choose index 0
- This change allows the index to be chose with
  cfg.CAMERA_INDEX, which defaults to zero
Copy link
Contributor

@DocGarbanzo DocGarbanzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also squash your commits into one?

print('PiCamera loaded.. .warming camera')
time.sleep(2)
# get the first frame or timeout
print('PiCamera loaded...')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you convert your print into logger.info?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did do.

print('PiCamera opened.. .warming camera')
warming_time = time.time() + 5 # quick after 5 seconds
while self.frame is None and time.time() < warming_time:
print(".", end="")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how to do this w/ logging though...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... actually I figured out a way to do it like this:

import logging
import time

# handler = logging.StreamHandler()
dfmt = "%d %b %Y %H:%M:%S"
format = '%(asctime)s %(message)s'
logging.basicConfig(format=format, datefmt=dfmt, level='INFO')
logger = logging.getLogger(__name__)
handler = logging.StreamHandler()


if __name__ == "__main__":
    logger.info('Starting')
    logger.propagate = False
    logger.addHandler(handler)
    handler.terminator = ""
    for i in range(20):
        logger.info(f'-' if i % 2 == 0 else '>')
        time.sleep(0.3)
    logger.info('\n')
    logger.removeHandler(handler)
    logger.propagate = True
    logger.info('bye')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't see this. I've made a change that is satisfactory I think.

@Ezward
Copy link
Contributor Author

Ezward commented Dec 18, 2021

Also added support for CAMERA_INDEX config when using CVCAM

import os
import time
import numpy as np
from PIL import Image
import glob
from donkeycar.utils import rgb2gray

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are already setting the logging level in the car app (in complete.py) to INFO already. Because this goes to the root logger, you would optimally set this only once where you start the application. There is no harm with this line, but if we decide to dynamically change the logger level per argument or config then we have to find and remove all these local settings as they might interfere/overwrite the application settings.

Copy link
Contributor

@DocGarbanzo DocGarbanzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks for the comprehensive documentation of the change.

Copy link
Contributor

@DocGarbanzo DocGarbanzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually - could you still squash all your commits into one, so we only see a single commit in the repo after merging?

Copy link
Contributor

@DocGarbanzo DocGarbanzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with @Ezward - squash merging should be fine here.

@DocGarbanzo DocGarbanzo merged commit 3c0e5b0 into dev Dec 23, 2021
@Ezward Ezward deleted the 961-protect-from-empty-camera-image branch January 5, 2022 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants