Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Udp (Battery) Camera Support #199

Merged

Conversation

QuantumEntangledAndy
Copy link
Collaborator

@QuantumEntangledAndy QuantumEntangledAndy commented Aug 25, 2021

Depends on #185


This PR adds support for UDP based cameras such as battery camera.

  • It uses nom/cookie factory to de/ser BcUdp packets
  • It generalises the BcConnection to allow either udp or tcp though a BcSource enum
  • It performs udp discovey based on UID
    • TODO: Add support for starting tcp based on UID discovery (next PR)
  • Discovery of ip from UID can happen in own of two ways
    • Local: Using a UDP broadcast to ports 2015 and 2018
    • Remote: Connecting to the the reolink p2p servers and requesting the ip for a UID

To use UDP the only change in the config is to set the address field to the uid

[[cameras]]
address = "MYUID"

This PR also updates the dissector

  • To Support UDP
  • Perform Heurestic detection of Bc packets so we don't need to set the port in the dissector
  • Adds luacheck to the github workflow to ensure that future dissector PRs are valid lua

This is another big PR but it is ultimately one feature, if you want me to break it down into smaller PRs please let me know

@QuantumEntangledAndy QuantumEntangledAndy marked this pull request as ready for review August 31, 2021 14:04
@QuantumEntangledAndy
Copy link
Collaborator Author

I separated this PR from the mqtt PR so that it can be merged independantly

@QuantumEntangledAndy
Copy link
Collaborator Author

QuantumEntangledAndy commented Aug 31, 2021

Also thought I'd try and describe UDP

UDP has two stages, Discovery and Transmit

Discovery

The discovery stage negociates the connection with the camera. 4 pieces of information must be shared for a connection to start.

  1. Client ID (a random number we decided)
  2. Camera ID (their random number)
  3. SID (fixed for a camera)
  4. MTU. Seems to be constant, but maybe should be found via mtu path discovery instead

The discovery stage is contained in crates/core/src/bc_protocol/connection/udpconn/discover.rs

Transmit stage

The transmit stage represenets and encasulated stream of Bc packets

The Udp packets can come in any order and are not guaranteed to be received. As a result crates/core/src/bc_protocol/connection/udpconn/transmit.rs is much more complex than its TCP equivalent.

There are two loops in the UDP transmit which represent the receive and send part of the connection.

Recieved packets are put into a buffer. When sufficent contigious packets are received then these packets are pushed into a crossbeam channel hooked up to the Read trait. Packet acknowledgements that are received from the camera are used to drop messages from the resend buffer (see below)

The send loop handles sending packets and acknowledgments. The Write trait pushes packets into a crossbeam channel that is hooked up to this loop. The loop pulls these packets of the channel and sends them for transmision. The camera is not guaranteed to received them so they are also added into a resend buffer until droped by a Packet acknowledgment from the camera (see above). This loop will resend any un-acknowledged packets that are still in the resend buffer.

Copy link
Owner

@thirtythreeforty thirtythreeforty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't finished reading through the UDP implementation itself, so more comments to come - quite a lot of SLoC - but this is really cool. Honestly I think this will make Neolink a much bigger deal than it is now, because a lot more people have battery-powered cams than do the PoE models, since it's a lower barrier to entry.

README.md Show resolved Hide resolved
crates/core/src/bc_protocol/ping.rs Outdated Show resolved Hide resolved
crates/core/src/bcudp/crc.rs Outdated Show resolved Hide resolved
crates/core/src/bcudp/model.rs Outdated Show resolved Hide resolved
crates/core/src/bcudp/model.rs Show resolved Hide resolved
crates/core/src/bcudp/xml_crypto.rs Show resolved Hide resolved
dissector/udp.md Outdated Show resolved Hide resolved
dissector/udp.md Show resolved Hide resolved
sample_config.toml Outdated Show resolved Hide resolved
sample_config.toml Outdated Show resolved Hide resolved
src/utils.rs Outdated Show resolved Hide resolved
crates/core/src/bc_protocol.rs Outdated Show resolved Hide resolved
crates/core/src/bc_protocol/connection/udpconn/transmit.rs Outdated Show resolved Hide resolved
crates/core/src/bcudp/de.rs Outdated Show resolved Hide resolved
dissector/udp.md Outdated Show resolved Hide resolved
dissector/udp.md Show resolved Hide resolved
dissector/udp.md Outdated Show resolved Hide resolved
dissector/udp.md Outdated Show resolved Hide resolved
@Subtixx
Copy link

Subtixx commented Sep 11, 2021

Using an Reolink Argus 3 results in a parsing error (yes im sure my credentials are correct.):

[2021-09-11T14:33:57Z INFO  neolink::rtsp] storage shed: Connecting to camera at UID: *UID*
[2021-09-11T14:33:57Z INFO  neolink::rtsp] storage shed: Connecting to camera at UID: *UID*
[2021-09-11T14:33:57Z INFO  neolink::rtsp] storage shed: Logging in
[2021-09-11T14:33:57Z INFO  neolink::rtsp] storage shed: Logging in
[2021-09-11T14:33:57Z ERROR neolink_core::bc_protocol::connection::bcconn] Deserialization error: Deserialization error
[2021-09-11T14:33:57Z ERROR neolink_core::bc_protocol::connection::bcconn] caused by: Parsing error

@QuantumEntangledAndy
Copy link
Collaborator Author

@Subtixx this is a known error that is dealt with in #198 once that gets merged it should be fine.

Argus 3 uses a newer encryption algorithm that we need to account for.

@Subtixx
Copy link

Subtixx commented Sep 11, 2021

Thanks and sorry I didnt see that one. So is there any neolink binary that I can use for the Argus 3 right now?

@QuantumEntangledAndy
Copy link
Collaborator Author

Yeah my udp_docker branch should work.

@Subtixx
Copy link

Subtixx commented Sep 11, 2021

It doesn't but oh well (says auth failed even though the credentials are correct, even verified by logging out of the app & back in). Can't report it anywhere, and this is the wrong place to do this.

@QuantumEntangledAndy
Copy link
Collaborator Author

Maybe open an issue on my fork and we can discuss it there. It's getting late in my timezone now so won't be able to help much until tomorrow though.

Copy link
Owner

@thirtythreeforty thirtythreeforty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I have some remaining concerns with code duplication in the various subcommands, but I think overall the approach is fine and can go ahead and merge to get the other PRs moving. Thanks for the work!

@thirtythreeforty
Copy link
Owner

And just to make sure I don't understate this: this is super super cool to have working.

@QuantumEntangledAndy QuantumEntangledAndy merged commit f10f9f9 into thirtythreeforty:master Oct 29, 2021
@QuantumEntangledAndy
Copy link
Collaborator Author

QuantumEntangledAndy commented Oct 29, 2021

I'm so glad we can merge this.

Was a lot of work to pull of and must say thank you to those that also supported the dev. By providing me with remote access to their cameras and setting up vpns and generally sticking through my insistent tests about UDP broadcasts. Also thanks to the testers that tried their best to stress test this :)

@thirtythreeforty
Copy link
Owner

Yes! Seriously cool, and I think it'll expand the scope of who's interested in the software by an order of magnitude - everyone seems to have cheap battery cams, PoE hardwired setups are a bit rarer.

@MasterPlexus
Copy link

Hi. Is this also part of the current downloadable binary for armhf? As far as I understand this should also work with Argus 2, but at the moment I'm not able to use it for that Cams..

@QuantumEntangledAndy
Copy link
Collaborator Author

@MasterPlexus Can you open a new issue and give up some more details such as your log files when in debug mode.

export RUST_LOG=debug

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants