-
Notifications
You must be signed in to change notification settings - Fork 8.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How the visiond works? #31
Comments
hi,me too tks |
They apparently use opencv. Q: there a better forum for questions? Does anyone know where to discuss? -Sorry to barge in. |
|
@geohot Is it possible for Comma to release a visiond skeleton that does the following:
i.e. visiond with no neural networks, the community can write other agents just using opencl. |
You can already do this with the released visiond. See how UI reads the image from visiond, it's open source. As for linking OpenCV, that should just work the same as on a desktop on NEOS. |
The problem is that if I run visiond just to get the camera feed from it I am not sure I would have enough processing power to try to reproduce it. As I understand things it would continue to run the network, save the videos, etc. |
If all you plan to use is OpenCV none of that will have any effect, since OpenCV is CPU only. Network and videos are done off CPU. |
I was thinking of using opencl on the adreno GPU for now ... ... But after your comment I realized that I will have a full Nvidia GTX 1070 on the back of the car, so I can develop for tensorflow during the next few months and in the near future hopefully we all switch to the oneplus [1] that comes with a Snapdragon 835 for tensorflow support on CPU, GPU and DSP [2] [1] https://oneplus4.net/oneplus-5-release-date/ |
To answer the original poster and maybe close the issue.
|
|
Issue #4 mention the API. |
From objdump on visiond (running from an nvidia TX2):
|
Adds in the 0x214 message removal
visiond is now open source. |
Cool. I found it in https://github.com/commaai/openpilot/tree/devel/selfdrive/visiond and will definitely give it a go now. :) |
20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 20c7fd6
20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 20c7fd6
20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 20c7fd6
20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 20c7fd6
20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 20c7fd6
6511990 log focus state 8efe413 qlog radarState eeb3d56 Add liveLocationKalman to qlogs 128b3f9 add networkStrength to thermal (#36) 2e5cbfc Create and init message in one line with `new_message` in messaging (#35) 4589107 not everyone likes gpstime 17363e9 support for end of log sentinel (#34) f6e9345 val valid is confusing bb2cc75 fix duplicate ordinals 0c38fc9 Add blindspot cereal values (#26) bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 6511990
6511990 log focus state 8efe413 qlog radarState eeb3d56 Add liveLocationKalman to qlogs 128b3f9 add networkStrength to thermal (#36) 2e5cbfc Create and init message in one line with `new_message` in messaging (#35) 4589107 not everyone likes gpstime 17363e9 support for end of log sentinel (#34) f6e9345 val valid is confusing bb2cc75 fix duplicate ordinals 0c38fc9 Add blindspot cereal values (#26) bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 6511990
6511990 log focus state 8efe413 qlog radarState eeb3d56 Add liveLocationKalman to qlogs 128b3f9 add networkStrength to thermal (#36) 2e5cbfc Create and init message in one line with `new_message` in messaging (#35) 4589107 not everyone likes gpstime 17363e9 support for end of log sentinel (#34) f6e9345 val valid is confusing bb2cc75 fix duplicate ordinals 0c38fc9 Add blindspot cereal values (#26) bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 6511990
6511990 log focus state 8efe413 qlog radarState eeb3d56 Add liveLocationKalman to qlogs 128b3f9 add networkStrength to thermal (#36) 2e5cbfc Create and init message in one line with `new_message` in messaging (#35) 4589107 not everyone likes gpstime 17363e9 support for end of log sentinel (#34) f6e9345 val valid is confusing bb2cc75 fix duplicate ordinals 0c38fc9 Add blindspot cereal values (#26) bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 6511990
6511990 log focus state 8efe413 qlog radarState eeb3d56 Add liveLocationKalman to qlogs 128b3f9 add networkStrength to thermal (#36) 2e5cbfc Create and init message in one line with `new_message` in messaging (#35) 4589107 not everyone likes gpstime 17363e9 support for end of log sentinel (#34) f6e9345 val valid is confusing bb2cc75 fix duplicate ordinals 0c38fc9 Add blindspot cereal values (#26) bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 6511990
6511990 log focus state 8efe413 qlog radarState eeb3d56 Add liveLocationKalman to qlogs 128b3f9 add networkStrength to thermal (#36) 2e5cbfc Create and init message in one line with `new_message` in messaging (#35) 4589107 not everyone likes gpstime 17363e9 support for end of log sentinel (#34) f6e9345 val valid is confusing bb2cc75 fix duplicate ordinals 0c38fc9 Add blindspot cereal values (#26) bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request #31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) git-subtree-dir: cereal git-subtree-split: 6511990
bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request commaai#31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c0 improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) eba4349 put liveLocation in qlogs 6c4735f add fixed fingerprintSource 4c50d26 GitHub actions (#25) 988783f Merge pull request #28 from commaai/power_monitoring b43fd06 Added offroad power usage ab32956 Merge pull request #27 from commaai/ublox-hw-status 2e7f4b9 Added ublox hw status 23abef4 add networkType to thermal 3f01dcf fix bad naming 4999e62 unconfusing and accessible to all 8f9aa8f add fingerprintSource to carParams 5794030 better put 06f8d15 standalone monitorstate (#23) 6eb1e36 qlog at model rate c38fb57 LateralParams 0c45f20 add uncertainty event name 01942b8 add TODO b74a456 don't hardcode the lists ed5a4bf add face stds 396a2bb add can error counter to controlsState c6b5c73 Switch default to msgq (#21) a457ffa Fix indentation in readme.md a1fc8c7 explicitly mention Python for syntax colouring (#20) 19e2393 Fix expected for cameraOdometry and liveCalibration e7d2f97 Add radar comm issue error db64cd4 Reserve safety #21 for VAG PQ35/PQ46/NMS (#19) 79d638d separate honda safety models between Bosch Giraffe and Bosch Nidec 2614a65 better name b6b84cd add longitudinal 78f5934 Add canRxErrs to health 6758899 qlog liveCalibration df80b87 add more stuff to fw log in CarParams a87805a fix doxs 4746b20 got doxed 21cf3f5 build on mac 31ac47c Add carUnrecognized event b8382bb steerLimitTimer should be car dependent 9a22968 add pa0 temp to ThermalData f6f0f60 Add stock Fcw to carState b608683 no l/r distinction for LDW 555f48d Add ldw alert 8e8b4a4 Remove plusFrame socket in favor of UiLayoutState 3410325 log stock AEB events 2219f2b Add warning about not using cython version of sec_since_boot 8f1a512 for legacy-testing reasons, better to define the used percent instead of avail e86d954 adding low memory event ad23834 remove TODO d0962b3 log mem available and cpu perc in thermald 3b753be Implement error handling and exceptions (#18) a7d5bb7 add explicit dependencies on services.h 1ba6467 fix linter c7d215b Added communityFeatureDisallowed event 492140a Added communityFeature bit detection to CarParams 266a5fe log Panda fault types 347a866 Switch from polling on FIFOs to signal (#12) e25bba7 no need to double build the objects fe43a99 20Hz for radar time step is very standard 2aabf1e Added radar time step to car params e8ae908 Generate capnp for java 57126a2 cereal_shared da655cd Add uptime to health f6a8e39 add test with multiple subscribers 84b3af5 comment out the debugging 4b9c942 added power save state to health packet 66be370 run python unittest in ci 52c6db8 Run scons in CI (#14) 9414615 do need it, but only for arm 2856c37 remove gnustl_shared 7f05ee6 fix apks e3a6bde Revert "no more makefiles" 487fbd0 don't rely on BASEDIR, and add zmq library 223e37a no more makefiles da2ed11 don't link the wrong one fe9fe2a scons builds the python lib now 2f81135 err, it can't build services.h 57b03f8 now we shouldn't need that yaml crap everywhere f8e5327 bridge builds with services.h 2b0cb60 noOutput safety mode is now called silent 83880d5 add msgq tests bcad184 msgq: dont block when fifo does not exists b4b26782 Default to zmq 473e291 fix compilation in docker 30aaadd msgq: try again when no timeout on poll but also no message c4f2ad5 msgq: make sure read_fifos is initalized so we dont close random fds 4e513a85 msgq: dont clean up uninitialized sockerts c008b63 also remove the fifo from disk ef64eb2 MSGQ stability improvements when opening and closing lots of queues e147abc Revert "Revert "deprecate irpwr"" 932dc32 Revert "deprecate irpwr" a684415 disengage ec27e18 capnpc also generated the header files ee52ab9 deprecate irpwr 301c74c8 Merge branch 'master' of github.com:commaai/cereal 6da7d55 add front frame a5944eb add conflate parameter for SubSocket::create ca8df17 Add fault status to health ef4ded0 add conflate support in SubSocket constructor 7fd314a update scons build file 93d814e add saturated flags to indi and lqr logs 50302fe add steeringRateLimited to car.capnp 05e3513 add msgq readme a6759a9 faster make 94b7377 Add struct to log FW version 64ce0b5 add scons build dc9ad18 add debug print statement on SIGINT 4a61269 Merge pull request #10 from commaai/msgq 4873449 use recv one or none after poll a054864 default to msgq fbc4a4c oops bad number 5067cf4 add meta cbd0286 fix export prefix and make shared library world readable c273054 add c exports for jni usage e77f41e zmq already sets the errno correctly 3196cf6 Fix service list path in bridge d35515a add all msgq files, but dont use as default a68a38f Don't delete context from python side only bd46c22 Revert "zmq_ctx_term is blocking" a1fc26b zmq_ctx_term is blocking 0902182 remote address support 21a3536 only delete subsocket when created by same object 34df735 remove extra underscore from __dealloc__ c8748f8 fix internal refs 79b2fbf fixups 23ad256 import messaging and services git-subtree-dir: cereal git-subtree-split: bd9a877
bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request commaai#31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (#30) bb13121 add honda ecus (#29) eba4349 put liveLocation in qlogs 6c4735f add fixed fingerprintSource 4c50d26 GitHub actions (#25) 988783f Merge pull request #28 from commaai/power_monitoring b43fd06 Added offroad power usage ab32956 Merge pull request #27 from commaai/ublox-hw-status 2e7f4b9 Added ublox hw status 23abef4 add networkType to thermal 3f01dcf fix bad naming 4999e62 unconfusing and accessible to all 8f9aa8f add fingerprintSource to carParams 5794030 better put 06f8d15 standalone monitorstate (#23) 6eb1e36 qlog at model rate c38fb57 LateralParams 0c45f20 add uncertainty event name 01942b8 add TODO b74a456 don't hardcode the lists ed5a4bf add face stds 396a2bb add can error counter to controlsState c6b5c73 Switch default to msgq (#21) a457ffa Fix indentation in readme.md a1fc8c7 explicitly mention Python for syntax colouring (#20) 19e2393 Fix expected for cameraOdometry and liveCalibration e7d2f97 Add radar comm issue error db64cd4 Reserve safety #21 for VAG PQ35/PQ46/NMS (#19) 79d638d separate honda safety models between Bosch Giraffe and Bosch Nidec 2614a65 better name b6b84cd add longitudinal 78f5934 Add canRxErrs to health 6758899 qlog liveCalibration df80b87 add more stuff to fw log in CarParams a87805a fix doxs 4746b20 got doxed 21cf3f5 build on mac 31ac47c Add carUnrecognized event b8382bb steerLimitTimer should be car dependent 9a22968 add pa0 temp to ThermalData f6f0f60 Add stock Fcw to carState b608683 no l/r distinction for LDW 555f48d Add ldw alert 8e8b4a4 Remove plusFrame socket in favor of UiLayoutState 3410325 log stock AEB events 2219f2b Add warning about not using cython version of sec_since_boot 8f1a512 for legacy-testing reasons, better to define the used percent instead of avail e86d954 adding low memory event ad23834 remove TODO d0962b3 log mem available and cpu perc in thermald 3b753be Implement error handling and exceptions (#18) a7d5bb7 add explicit dependencies on services.h 1ba6467 fix linter c7d215b Added communityFeatureDisallowed event 492140a Added communityFeature bit detection to CarParams 266a5fe log Panda fault types 347a866 Switch from polling on FIFOs to signal (#12) e25bba7 no need to double build the objects fe43a99 20Hz for radar time step is very standard 2aabf1e Added radar time step to car params e8ae908 Generate capnp for java 57126a2 cereal_shared da655cd Add uptime to health f6a8e39 add test with multiple subscribers 84b3af5 comment out the debugging 4b9c942 added power save state to health packet 66be370 run python unittest in ci 52c6db8 Run scons in CI (#14) 9414615 do need it, but only for arm 2856c37 remove gnustl_shared 7f05ee6 fix apks e3a6bde Revert "no more makefiles" 487fbd0 don't rely on BASEDIR, and add zmq library 223e37a no more makefiles da2ed11 don't link the wrong one fe9fe2a scons builds the python lib now 2f81135 err, it can't build services.h 57b03f8 now we shouldn't need that yaml crap everywhere f8e5327 bridge builds with services.h 2b0cb60 noOutput safety mode is now called silent 83880d5 add msgq tests bcad184 msgq: dont block when fifo does not exists b4b2678 Default to zmq 473e291 fix compilation in docker 30aaadd msgq: try again when no timeout on poll but also no message c4f2ad5 msgq: make sure read_fifos is initalized so we dont close random fds 4e513a8 msgq: dont clean up uninitialized sockerts c008b63 also remove the fifo from disk ef64eb2 MSGQ stability improvements when opening and closing lots of queues e147abc Revert "Revert "deprecate irpwr"" 932dc32 Revert "deprecate irpwr" a684415 disengage ec27e18 capnpc also generated the header files ee52ab9 deprecate irpwr 301c74c Merge branch 'master' of github.com:commaai/cereal 6da7d55 add front frame a5944eb add conflate parameter for SubSocket::create ca8df17 Add fault status to health ef4ded0 add conflate support in SubSocket constructor 7fd314a update scons build file 93d814e add saturated flags to indi and lqr logs 50302fe add steeringRateLimited to car.capnp 05e3513 add msgq readme a6759a9 faster make 94b7377 Add struct to log FW version 64ce0b5 add scons build dc9ad18 add debug print statement on SIGINT 4a61269 Merge pull request #10 from commaai/msgq 4873449 use recv one or none after poll a054864 default to msgq fbc4a4c oops bad number 5067cf4 add meta cbd0286 fix export prefix and make shared library world readable c273054 add c exports for jni usage e77f41e zmq already sets the errno correctly 3196cf6 Fix service list path in bridge d35515a add all msgq files, but dont use as default a68a38f Don't delete context from python side only bd46c22 Revert "zmq_ctx_term is blocking" a1fc26b zmq_ctx_term is blocking 0902182 remote address support 21a3536 only delete subsocket when created by same object 34df735 remove extra underscore from __dealloc__ c8748f8 fix internal refs 79b2fbf fixups 23ad256 import messaging and services git-subtree-dir: cereal git-subtree-split: bd9a877
openpilot v0.8.2 release
3bf5936a Added bmw safety enum 6511990 log focus state 8efe413 qlog radarState eeb3d56 Add liveLocationKalman to qlogs 128b3f9 add networkStrength to thermal (commaai#36) 2e5cbfc Create and init message in one line with `new_message` in messaging (commaai#35) 4589107 not everyone likes gpstime 17363e9 support for end of log sentinel (commaai#34) f6e9345 val valid is confusing bb2cc75 fix duplicate ordinals 0c38fc9 Add blindspot cereal values (commaai#26) bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request commaai#31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c0 improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (commaai#30) bb13121 add honda ecus (commaai#29) git-subtree-dir: cereal git-subtree-split: 3bf5936a70b5a10350b581d600600dc19375a233
3bf5936a Added bmw safety enum 6511990 log focus state 8efe413 qlog radarState eeb3d56 Add liveLocationKalman to qlogs 128b3f9 add networkStrength to thermal (commaai#36) 2e5cbfc Create and init message in one line with `new_message` in messaging (commaai#35) 4589107 not everyone likes gpstime 17363e9 support for end of log sentinel (commaai#34) f6e9345 val valid is confusing bb2cc75 fix duplicate ordinals 0c38fc9 Add blindspot cereal values (commaai#26) bd9a877 pulse desire and e2e 20c7fd6 Add subaru pre-Global safety mode 522ff85 Merge pull request commaai#31 from commaai/good_location_packet ab07f22 deprecate b03c2c5 already exists 166418c0 improvements 97373f9 or rigor in american 365abba rigour 25eaf9d add espDisabled to carState (commaai#30) bb13121 add honda ecus (commaai#29) git-subtree-dir: cereal git-subtree-split: 3bf5936a70b5a10350b581d600600dc19375a233
…maai#31717)" (commaai#31719) * Revert "Revert "Hyundai Azera: allow fingerprinting without comma power" (commaai#31…" This reverts commit 1ffc58a. * we should get EPS * add EPS to non-logging can 0 query
Hi,
I am curious the internal of visiond, I want to do similar work but not in selfdrive domain, can anyone answer this questions:
thanks,
The text was updated successfully, but these errors were encountered: