Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protocol Updates #83

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
5baacbe
Implement bc make ext xml
QuantumEntangledAndy Oct 1, 2020
3acd0b4
Updates to the readme
QuantumEntangledAndy Oct 5, 2020
4ff53b7
Refractored bc connection to treat binary as a generic payload
QuantumEntangledAndy Oct 8, 2020
31aefdb
Rename bin_offset to payload_offset
QuantumEntangledAndy Oct 8, 2020
d0f760e
Refractor to apply ext xml before payload_offset
QuantumEntangledAndy Oct 8, 2020
edf2f25
Fixes from rebase
QuantumEntangledAndy Oct 15, 2020
d92cb38
Rust fmted
QuantumEntangledAndy Oct 15, 2020
71de7da
Base encryption solely off legacy login reply
QuantumEntangledAndy Oct 29, 2020
6e1cf2f
Work on switching encoffset with channelid and msg_mum
QuantumEntangledAndy Nov 16, 2020
25b561c
More chances to confrom to new protocol
QuantumEntangledAndy Nov 26, 2020
dda603a
Add message handle and status codes and handel encryption correctly
QuantumEntangledAndy Nov 26, 2020
2103007
Merge branch 'master'
QuantumEntangledAndy Nov 26, 2020
9604ecb
Merge branch 'error_updates'
QuantumEntangledAndy Nov 26, 2020
a961829
Bug fixes from merges
QuantumEntangledAndy Nov 26, 2020
882d34d
Merge branch 'error_updates' into protocol_updates
QuantumEntangledAndy Nov 29, 2020
2e04405
Format error cause nicely
QuantumEntangledAndy Dec 9, 2020
772ba86
Bug fix and preperation for new encryption protocol
QuantumEntangledAndy Apr 14, 2021
a2ca630
Merge branch 'master' of github.com:thirtythreeforty/neolink into pro…
QuantumEntangledAndy Apr 15, 2021
f479c82
Added firmware update to messages.md
QuantumEntangledAndy Apr 15, 2021
02f4cab
Added AES encryption
QuantumEntangledAndy Apr 17, 2021
1227ff1
Refactor AesKey to only contain key bytes
thirtythreeforty Apr 17, 2021
30fad08
Bug fixes and making serde respect the status code to send
QuantumEntangledAndy Apr 17, 2021
f4630f4
Treat AES encrypt and decrypt differently
QuantumEntangledAndy Apr 17, 2021
eda598c
Restore timeout duration
QuantumEntangledAndy Apr 17, 2021
ce9f3e7
Removed extra init method for context
QuantumEntangledAndy Apr 17, 2021
0dc2569
Removed docs for prototype mqtt
QuantumEntangledAndy Apr 17, 2021
4721d36
Use context to keep track of binary
QuantumEntangledAndy Apr 17, 2021
949192b
Use pure Rust AES implementation
thirtythreeforty Apr 17, 2021
6ac4b70
cargo fmt
thirtythreeforty Apr 17, 2021
05cb032
Bump nom version
QuantumEntangledAndy Apr 18, 2021
9692aa8
Update tests to the new protocol model
QuantumEntangledAndy Apr 18, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
363 changes: 220 additions & 143 deletions Cargo.lock

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
aes = "0.6"
cfb-mode = "0.6"
cookie-factory = "0.3"
crossbeam = "0.7"
env_logger = "*"
Expand All @@ -23,7 +25,7 @@ log = { version = "0.4" }
indoc = "0.3"
itertools = "0.9"
md5 = "0.7"
nom = "5.1"
nom = "6.1.2"
regex = "1"
serde = { version = "1.0", features = ["derive"] }
socket2 = "0.3"
Expand Down
181 changes: 126 additions & 55 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,19 @@

![CI](https://github.com/thirtythreeforty/neolink/workflows/CI/badge.svg)

Neolink is a small program that acts as a proxy between Reolink IP cameras and normal RTSP clients.
Certain cameras, such as the Reolink B800, do not implement ONVIF or RTSP, but instead use a proprietary "Baichuan" protocol only compatible with their apps and NVRs (any camera that uses "port 9000" will likely be using this protocol).

Neolink allows you to use NVR software such as Shinobi or Blue Iris to receive video from these cameras instead.
Neolink is a small program that acts as a proxy between Reolink IP cameras and
normal RTSP clients.
Certain cameras, such as the Reolink B800, do not implement ONVIF or RTSP, but
instead use a proprietary "Baichuan" protocol only compatible with their apps
and NVRs (any camera that uses "port 9000" will likely be using this protocol).
Neolink allows you to use NVR software such as Shinobi or Blue Iris to receive
video from these cameras instead.
The Reolink NVR is not required, and the cameras are unmodified.
Your NVR software connects to Neolink, which forwards the video stream from the camera.
Your NVR software connects to Neolink, which forwards the video stream from the
camera.

The Neolink project is not affiliated with Reolink in any way; everything it does has been reverse engineered.
The Neolink project is not affiliated with Reolink in any way; everything it
does has been reverse engineered.

## Supported cameras

Expand All @@ -21,7 +26,8 @@ Currently it has been tested on the following cameras:
- E1
- Lumus

Neolink does not support other cameras such as the RLC-420, since they already [provide native RTSP](https://support.reolink.com/hc/en-us/articles/360007010473-How-to-Live-View-Reolink-Cameras-via-VLC-Media-Player).
Neolink does not support other cameras such as the RLC-420, since they already
[provide native RTSP](https://support.reolink.com/hc/en-us/articles/360007010473-How-to-Live-View-Reolink-Cameras-via-VLC-Media-Player).

## Installation

Expand All @@ -39,27 +45,36 @@ Builds are provided for the following platforms:

### Windows/Linux

1. [Install Gstreamer][gstreamer] from the most recent MSI installer on Windows, or your package manager on Linux.
1. [Install Gstreamer][gstreamer] from the most recent MSI installer on Windows,
or your package manager on Linux.

2. If you are using Windows, add the following to your `PATH` environment variable:

```
%GSTREAMER_1_0_ROOT_X86_64%\bin
```
```
%GSTREAMER_1_0_ROOT_X86_64%\bin
```

Note that if you use Chocolatey to install Gstreamer, it does this automatically.
**Note:** If you use Chocolatey to install Gstreamer, it does this
automatically.

3. Download and extract a [prebuilt binary from continuous integration][ci-download]
(click on the topmost commit for the most recent build).

3. Download and unpack Neolink from the links above.
1. Note: you can also click on [this link][ci-download] to see all historical builds.
You will need to be logged in to GitHub to download directly from the builds page.
4. Write a configuration file for your cameras. See the section below.

5. Launch Neolink from a shell, passing your configuration file:

```
neolink --config my_config.toml
```
```bash
neolink --config my_config.toml
```

6. Connect your NVR software to Neolink's RTSP server.
The default URL is `rtsp://127.0.0.1:8554/your_camera_name` if you're running it on the same computer.

The default URL is `rtsp://127.0.0.1:8554/your_camera_name` if you're running
it on the same computer.
If you run it on a different server, you may need to open firewall ports.
See the "Viewing" section below for more troubleshooting.

Expand All @@ -78,122 +93,178 @@ neolink --config my_config.toml
A Docker image is also available containing Neolink and all its dependencies.
The image is `thirtythreeforty/neolink`.
Port 8554 is exposed, which is the default listen port.
You must mount a configuration file (see below) into the container at `/etc/neolink.toml`.
You must mount a configuration file (see below) into the container at
`/etc/neolink.toml`.

Here is a sample launch commmand:

```
```bash
docker run \
-p 8554:8554 \
--restart=on-failure \
--volume=$PWD/config.toml:/etc/neolink.toml \
thirtythreeforty/neolink
```

The Docker image is "best effort" and intended for advanced users; questions about running Docker are outside the scope of Neolink.
The Docker image is "best effort" and intended for advanced users; questions
about running Docker are outside the scope of Neolink.

## Configuration

**Note**: for a more comprehensive setup tutorial, refer to the [Blue Iris setup walkthrough in `docs/`][blue-iris-setup] (which is probably also helpful even with other NVR software).
**Note**: for a more comprehensive setup tutorial, refer to the
[Blue Iris setup walkthrough in `docs/`][blue-iris-setup] (which is probably
also helpful even with other NVR software).

[blue-iris-setup]: docs/Setting%20Up%20Neolink%20For%20Use%20With%20Blue%20Iris.md

Copy and modify the `sample_config.toml` to specify the address, username, and password for each camera (if there is no password, you can omit that line).
Each `[[cameras]]` block creates a new camera; the `name` determines the RTSP path you should connect your client to.
Currently Neolink cannot auto-detect cameras like the official clients do; you must specify their IP addresses directly.
Copy and modify the `sample_config.toml` to specify the address, username, and
password for each camera (if there is no password, you can omit that line).

Each `[[cameras]]` block creates a new camera; the `name` determines the RTSP
path you should connect your client to.
Currently Neolink cannot auto-detect cameras like the official clients do; you
must specify their IP addresses directly.

By default the H265 video format is used. Some cameras, for example E1, provide H264 streams. To use these you must specify `format = "h264"` in the `[[cameras]]` config.
Soon this will be auto-detected, and you will not have to know or care about the format.
By default the H265 video format is used. Some cameras, for example E1, provide
H264 streams. To use these you must specify `format = "h264"` in the
`[[cameras]]` config. Soon this will be auto-detected, and you will not have to know or care about
the format.

By default, the HD stream is available at the RTSP path `/name` or `/name/mainStream`, and the SD stream is available at `/name/subStream`.
You can use only the HD stream by adding `stream = "mainStream"` to the `[[cameras]]` config, or only the SD stream with `stream = "subStream"`.
By default, the HD stream is available at the RTSP path `/name` or
`/name/mainStream`, and the SD stream is available at `/name/subStream`.
You can use only the HD stream by adding `stream = "mainStream"` to the
`[[cameras]]` config, or only the SD stream with `stream = "subStream"`.

**Note**: The B400/D400 models only support a single stream at a time, so you must add this line to sections for those cameras.
**Note**: The B400/D400 models only support a single stream at a time, so you
must add this line to sections for those cameras.

By default Neolink serves on all IP addresses on port 8554.
You can modify this by changing the `bind` and the `bind_port` parameter.
You only need one `bind`/`bind_port` setting at the top of the config file.

You can enable `rtsps` (TLS) by adding a `certificate = "/path/to/pem"` to the top section of the config file. This PEM should contain by the certificate and the key used for the server. If TLS is enabled all connections must use `rtsps`. You can also control client side TLS with the config option `tls_client_auth = "none|request|require"`; in this case the client should present a certificate signed by the server's CA.
You can enable `rtsps` (TLS) by adding a `certificate = "/path/to/pem"` to the
top section of the config file. This PEM should contain by the certificate
and the key used for the server. If TLS is enabled all connections must use
`rtsps`. You can also control client side TLS with the config option
`tls_client_auth = "none|request|require"`; in this case the client should
present a certificate signed by the server's CA.

TLS is disabled by default.

You can password-protect the Neolink server by adding `[[users]]`` sections to the configuration file:
You can password-protect the Neolink server by adding `[[users]]` sections to
the configuration file:

```
[[users]]
name: someone
pass: somepass
```
you also need to add the allowed users into each camera by adding the following to `[[cameras]]`.

you also need to add the allowed users into each camera by adding the following
to `[[cameras]]`.

```
permitted_users = ["someone", "someoneelse"]
```

Anywhere a username is accepted it can take any username or one of the following special values.
Anywhere a username is accepted it can take any username or one of the
following special values.

- `anyone` means any user with a valid user/pass
- `anonymous` means no user/pass required

The default `permitted_users` list is:
- `[ "anonymous"]` if no `[[users]]` were given in the config meaning no authentication required to connect.
- `[ "anyone" ]` if `[[users]]` were provided meaning any authourised users can connect.

You can change the Neolink log level by setting the `RUST_LOG` environment variable (not in the configuration file) to one of `error`, `warn`, `info`, `debug`, or `trace`:
- `[ "anonymous"]` if no `[[users]]` were given in the config meaning no
authentication required to connect.

```
- `[ "anyone" ]` if `[[users]]` were provided meaning any authourised users can
connect.

You can change the Neolink log level by setting the `RUST_LOG` environment
variable (not in the configuration file) to one of `error`, `warn`, `info`,
`debug`, or `trace`:

```sh
set RUST_LOG=debug
```

On Linux:

```
```bash
export RUST_LOG=debug
```

## Viewing

Connect your RTSP client to the stream with the name you provided in the configuration file.
Again, the default URL is `rtsp://127.0.0.1:8554/your_camera_name` if you're running it on the same computer as the client.
Connect your RTSP client to the stream with the name you provided in the
configuration file.

Again, the default URL is `rtsp://127.0.0.1:8554/your_camera_name` if you're
running it on the same computer as the client.
The smaller SD video is `rtsp://127.0.0.1:8554/your_camera_name/subStream`.

4K cameras send large video "key frames" once every few seconds and the client must have a receive buffer large enough to store the entire frame.
If your client's buffer size is configurable (like Blue Iris), ensure it's set to 20MB, which should ensure plenty of headroom.
4K cameras send large video "key frames" once every few seconds and the client
must have a receive buffer large enough to store the entire frame.
If your client's buffer size is configurable (like Blue Iris), ensure it's set
to 20MB, which should ensure plenty of headroom.

## Stability

Neolink has had minimal testing, but it seems to be very reliable in multiple users' testing.
Neolink has had minimal testing, but it seems to be very reliable in multiple
users' testing.

The formats of all configuration files and APIs is subject to change as required while it is pre-1.0.
The formats of all configuration files and APIs is subject to change as required
while it is pre-1.0.

## Development

Neolink is written in Rust, and binds to Gstreamer to provide RTSP server functionality.
To compile, ensure you have the Rust compiler, Gstreamer, and gst-rtsp-server installed.
Neolink is written in Rust, and binds to Gstreamer to provide RTSP server
functionality.

To compile, ensure you have the Rust compiler, Gstreamer, and gst-rtsp-server
installed.

Then simply run:

```
```bash
cargo build
```

from this top directory.

### Baichuan Protocol

The "port 9000" protocol used by Reolink and some Swann cameras is internally referred to as the Baichuan protocol; this is the company based in China that is known internationally as Reolink.
The "port 9000" protocol used by Reolink and some Swann cameras is internally
referred to as the Baichuan protocol; this is the company based in China that
is known internationally as Reolink.

This protocol is a slightly convoluted header-data format, and appears to have
been upgraded several times.

The modern variant uses obfuscated XML commands and sends ordinary H.265 or
H.264 video streams encapsulated in a custom header.

This protocol is a slightly convoluted header-data format, and appears to have been upgraded several times.
The modern variant uses obfuscated XML commands and sends ordinary H.265 or H.264 video streams encapsulated in a custom header.
More details about the on-the-wire protocol will come later.

### Baichuan dissector

A Wireshark dissector is available for the BC wire protocol in the `dissector` directory.
It dissects the BC header and also allows viewing the deobfuscated XML in command messages.
To use it, copy or symlink it into your Wireshark plugin directory; typically this is `~/.local/lib/wireshark/plugins/` under Linux.
A Wireshark dissector is available for the BC wire protocol in the `dissector`
directory.

It dissects the BC header and also allows viewing the deobfuscated XML in
command messages.
To use it, copy or symlink it into your Wireshark plugin directory; typically
this is `~/.local/lib/wireshark/plugins/` under Linux.

Currently the dissector does not attempt to decode the Baichuan "extension" messages except `binaryData`.
Currently the dissector does not attempt to decode the Baichuan "extension"
messages except `binaryData`.
This will change in the future as reverse engineering needs require.

## License

Neolink is free software, released under the GNU Affero General Public License v3.
This means that if you incorporate it into a piece of software available over the network, you must offer that software's source code to your users.
Neolink is free software, released under the GNU Affero General Public License
v3.

This means that if you incorporate it into a piece of software available over
the network, you must offer that software's source code to your users.
1 change: 0 additions & 1 deletion clippy.toml

This file was deleted.

12 changes: 6 additions & 6 deletions dissector/baichuan.lua
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ unknown = ProtoField.int8("baichuan.unknown", "unknown", base.DEC)
msg_handle = ProtoField.int8("baichuan.message_handle", "messageHandle", base.DEC)
status_code = ProtoField.int16("baichuan.status_code", "status_code", base.DEC)
message_class = ProtoField.int32("baichuan.msg_class", "messageClass", base.DEC)
f_bin_offset = ProtoField.int32("baichuan.bin_offset", "binOffset", base.DEC)
f_payload_offset = ProtoField.int32("baichuan.payload_offset", "binOffset", base.DEC)
username = ProtoField.string("baichuan.username", "username", base.ASCII)
password = ProtoField.string("baichuan.password", "password", base.ASCII)

Expand All @@ -31,7 +31,7 @@ bc_protocol.fields = {
encrypt_xml,
status_code,
message_class,
f_bin_offset,
f_payload_offset,
username,
password,
}
Expand Down Expand Up @@ -187,7 +187,7 @@ function get_header(buffer)
local msg_type = buffer(4, 4):le_uint()
if header_len == 24 then
bin_offset = buffer(20, 4):le_uint() -- if NHD-805/806 legacy protocol 30 30 30 30 aka "0000"
status_code = buffer(16, 2):le_uint()
status_code = buffer(16, 2):le_uint()
else
encrypt_xml = buffer(16, 1):le_uint()
end
Expand Down Expand Up @@ -234,13 +234,13 @@ function process_header(buffer, headers_tree)
header:add_le(channel_id, buffer(12, 1))
header:add_le(stream_id, buffer(13, 1))
:append_text(stream_text)
header:add_le(unknown, buffer(14, 1))
header:add_le(unknown, buffer(14, 1))
header:add_le(msg_handle, buffer(15, 1))

header:add_le(message_class, buffer(18, 2)):append_text(" (" .. header_data.class .. ")")

if header_data.header_len == 24 then
header:add_le(status_code, buffer(16, 2))
header:add_le(status_code, buffer(16, 2))
header:add_le(f_bin_offset, buffer(20, 4))
else
header:add_le(encrypt_xml, buffer(16, 1))
Expand Down
Loading