v0.7.1 -> v0.7.4 upgrade problems

[jlpoolen]

Performing an upgrade and ran into this:

jlpoole@pi:/usr/local/src/moonfire-nvr/server $ cargo build --release
    Updating git repository `https://github.com/scottlamb/mylog`
    Updating crates.io index
    Updating git repository `https://github.com/scottlamb/mp4-rust`
  Downloaded inout v0.1.2
error: failed to parse manifest at `/home/jlpoole/.cargo/registry/src/github.com-1285ae84e5963aae/inout-0.1.2/Cargo.toml`

Caused by:
  feature `edition2021` is required

  this Cargo does not support nightly features, but if you
  switch to nightly channel you can add
  `cargo-features = ["edition2021"]` to enable this feature
jlpoole@pi:/usr/local/src/moonfire-nvr/server $

[jlpoolen]

log of my termination of moonfire, then update attempt:

http://salemdata.us/moonfire-nvr/upgrade.log.html

[scottlamb]

Check your Rust version vs the current build instructions. I bumped the minimum to 1.56 in the Moonfire 0.7.2 release.

Good news is that after you upgrade to 1.56, cargo will pay attention to a rust-version field in Cargo.toml, so you'll get a clearer error message next time. E.g. if I add this field requiring the 1.70 release (which won't exist until about 60 weeks from now):

[slamb@nuc ~/git/moonfire-nvr/server]$ cargo build
error: package `moonfire-nvr v0.7.4 (/home/slamb/git/moonfire-nvr/server)` cannot be built because it requires rustc 1.70 or newer, while the currently active rustc version is 1.60.0

[jlpoolen]

I upgraded rust (took some doing as I had to remove previous before installer would upgrade and no hints on the Rust site about "uninstalling"):

 jlpoole@pi:~ $ rustc -V
 rustc 1.60.0 (7737e0b5c 2022-04-04)
 jlpoole@pi:~ $

ran the install instructions:

After building "server" and under directory "server": sudo install -m 755 target/release/moonfire-nvr /usr/local/bin
After building "ui" and under main directory: sudo rsync --recursive --delete --chmod=D755,F644 ui/build/ /usr/local/lib/moonfire-nvr/ui

Then proceeded with a normal startup and got this:

jlpoole@pi:~ $ nvr run
E20220413 19:49:58.331 main moonfire_nvr] Exiting due to error: unable to read /etc/moonfire-nvr.toml
caused by: No such file or directory (os error 2)

Backtrace:
   0: backtrace::backtrace::libunwind::trace
             at .cargo/registry/src/github.com-1285ae84e5963aae/backtrace-0.3.64/src/backtrace/libunwind.rs:93:5
      backtrace::backtrace::trace_unsynchronized
             at .cargo/registry/src/github.com-1285ae84e5963aae/backtrace-0.3.64/src/backtrace/mod.rs:66:5
      backtrace::backtrace::trace
             at .cargo/registry/src/github.com-1285ae84e5963aae/backtrace-0.3.64/src/backtrace/mod.rs:53:14
   1: backtrace::capture::Backtrace::create
             at .cargo/registry/src/github.com-1285ae84e5963aae/backtrace-0.3.64/src/capture.rs:176:9
      backtrace::capture::Backtrace::new_unresolved
             at .cargo/registry/src/github.com-1285ae84e5963aae/backtrace-0.3.64/src/capture.rs:170:9
   2: failure::backtrace::internal::InternalBacktrace::new
             at .cargo/registry/src/github.com-1285ae84e5963aae/failure-0.1.8/src/backtrace/internal.rs:46:44
   3: failure::backtrace::Backtrace::new
             at .cargo/registry/src/github.com-1285ae84e5963aae/failure-0.1.8/src/backtrace/mod.rs:121:35
      <failure::backtrace::Backtrace as core::default::Default>::default
             at .cargo/registry/src/github.com-1285ae84e5963aae/failure-0.1.8/src/backtrace/mod.rs:140:13
   4: <failure::error::error_impl::ErrorImpl as core::convert::From<F>>::from
             at .cargo/registry/src/github.com-1285ae84e5963aae/failure-0.1.8/src/error/error_impl.rs:19:17
      <failure::error::Error as core::convert::From<F>>::from
             at .cargo/registry/src/github.com-1285ae84e5963aae/failure-0.1.8/src/error/mod.rs:36:18
   5: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/result.rs:2064:27
      moonfire_nvr::cmds::run::read_config
             at /usr/local/src/moonfire-nvr/server/src/cmds/run/mod.rs:132:18
      moonfire_nvr::cmds::run::run
             at /usr/local/src/moonfire-nvr/server/src/cmds/run/mod.rs:138:18
   6: moonfire_nvr::Args::run
             at /usr/local/src/moonfire-nvr/server/src/main.rs:68:29
      moonfire_nvr::main
             at /usr/local/src/moonfire-nvr/server/src/main.rs:143:9
   7: core::ops::function::FnOnce::call_once
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:227:5
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:122:18
   8: std::rt::lang_start::{{closure}}
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/rt.rs:145:18
   9: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:259:13
      std::panicking::try::do_call
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/rt.rs:128:48
      std::panicking::try::do_call
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/rt.rs:128:20
  10: main
  11: __libc_start_main
             at /home/pi/dev/glibc-buster/csu/libc-start.c:308:16

jlpoole@pi:~ $

[scottlamb]

That's new with 0.7.3 0.7.2 also. See the changelog. You probably want your /etc/moonfire-nvr.toml to look like this:

[[binds]]
ipv4 = "0.0.0.0:8080"

[[binds]]
unix = "/var/lib/moonfire-nvr/sock"
own_uid_is_privileged = true

[jlpoolen]

Thank you. The additional /etc/moonfire-nvr.toml allowed me to commence the program.

However, when I visit the URL, I encounterd this error message:

 --ui-dir not configured; no static files available.

Since I have not upgraded since fall of 2021, I searched using "ui-dir" at https://github.com/scottlamb/moonfire-nvr/blob/master/guide/build.md and at https://github.com/scottlamb/moonfire-nvr/blob/master/CHANGELOG.md and the term "ui-dir" does not appear in either. Yet, the steps I took to upgrade which had before worked and now include the above, suggest I've missed something.

[jlpoolen]

I considered your "own_uid_is_privileged" and supposed that I need to run as sudo, though it was in my account as jlpoole. I had created an account of moonfire some time ago, but ran into problems with permissions and have since been running nvr run as jlpoole. Still, running with sudo did not change the outcome of the ui error above.

[scottlamb]

It probably logged something like this first?

                    warn!(
                        "Unable to load --ui-dir={}; will serve no static files: {}",
                        d.display(),
                        e
                    );

(I need to update those messages, which refer to the commandline arg form that no longer exists.)

It defaults to /usr/local/lib/moonfire-nvr/ui as --ui-dir used to. If you were overriding that, the new version is to add something like this to the beginning of /etc/moonfire-nvr.toml (before any [[bind]] lines):

ui_dir = "/override/path"

The own_uid_is_privileged is something else. It means that when running as the same effective uid as the moonfire-nvr server itself (whether that's moonfire-nvr, jlpoole, or whatever), you can send a request over the Unix domain socket (e.g. via curl --unix-socket /var/lib/moonfire-nvr/sock http://nvr/api/...) and have it be always considered authenticated without specifying a session cookie. I'm planning to have all the configuration happen with the server running (rather than shutting it down and going into moonfire-nvr config), and that will be a way of bootstrapping authentication.

[jlpoolen]

I checked my Linux console to see if the warn would appear there nothing, so you must be referring to the error messages in Firefox?

Any rate, here's what's on my system:

jlpoole@pi:~ $ ls -la /usr/local/lib/moonfire-nvr
total 44
drwxr-xr-x 4 root root 4096 Apr 13 19:44 .
drwxr-xr-x 7 root root 4096 Aug 19  2021 ..
-rw-r--r-- 1 root root 1353 Apr 13 19:44 asset-manifest.json
-rw-r--r-- 1 root root  380 Apr 13 19:44 asset-manifest.json.gz
drwxr-xr-x 2 root root 4096 Apr 13 19:44 favicons
-rw-r--r-- 1 root root 1022 Apr 13 19:44 index.html
-rw-r--r-- 1 root root  547 Apr 13 19:44 index.html.gz
-rw-r--r-- 1 root root   69 Apr 13 19:44 robots.txt
-rw-r--r-- 1 root root  513 Apr 13 19:44 site.webmanifest
-rw-r--r-- 1 root root  289 Apr 13 19:44 site.webmanifest.gz
drwxr-xr-x 5 root root 4096 Apr 13 19:44 static
jlpoole@pi:~ $ ls -la /usr/local/lib/moonfire-nvr/static
total 20
drwxr-xr-x 5 root root 4096 Apr 13 19:44 .
drwxr-xr-x 4 root root 4096 Apr 13 19:44 ..
drwxr-xr-x 2 root root 4096 Apr 13 19:44 css
drwxr-xr-x 2 root root 4096 Apr 13 19:44 js
drwxr-xr-x 2 root root 4096 Apr 13 19:44 media
jlpoole@pi:~ $ ls -la /usr/local/lib/moonfire-nvr/static/ui
ls: cannot access '/usr/local/lib/moonfire-nvr/static/ui': No such file or directory
jlpoole@pi:~ $ ls /usr/local/src/moonfire-nvr/ui
build  node_modules  package.json  package-lock.json  public  README.md  src  tsconfig.json
jlpoole@pi:~ $ ls /usr/local/src/moonfire-nvr/ui/build
asset-manifest.json  asset-manifest.json.gz  favicons  index.html  index.html.gz  robots.txt  site.webmanifest  site.webmanifest.gz  static
jlpoole@pi:~ $
[moonfire]0:bash*  

And here is a screenshot of Firefox with the developer Tools:

[scottlamb]

I checked my Linux console to see if the warn would appear there nothing, so you must be referring to the error messages in Firefox?

No, I mean a server-side log message. Huh. Not sure yet what's going on then.

[scottlamb]

Could you paste output of ls -laFR /usr/local/lib/moonfire-nvr/ui?

[jlpoolen]

 jlpoole@pi:~ $ ls -laFR /usr/local/lib/moonfire-nvr/ui
 ls: cannot access '/usr/local/lib/moonfire-nvr/ui': No such file or directory
 jlpoole@pi:~ $

But, this fixed it:

jlpoole@pi:~ $ ls -la /usr/local/lib/moonfire-nvr
total 44
drwxr-xr-x 4 root root 4096 Apr 13 19:44 .
drwxr-xr-x 7 root root 4096 Aug 19  2021 ..
-rw-r--r-- 1 root root 1353 Apr 13 19:44 asset-manifest.json
-rw-r--r-- 1 root root  380 Apr 13 19:44 asset-manifest.json.gz
drwxr-xr-x 2 root root 4096 Apr 13 19:44 favicons
-rw-r--r-- 1 root root 1022 Apr 13 19:44 index.html
-rw-r--r-- 1 root root  547 Apr 13 19:44 index.html.gz
-rw-r--r-- 1 root root   69 Apr 13 19:44 robots.txt
-rw-r--r-- 1 root root  513 Apr 13 19:44 site.webmanifest
-rw-r--r-- 1 root root  289 Apr 13 19:44 site.webmanifest.gz
drwxr-xr-x 5 root root 4096 Apr 13 19:44 static
jlpoole@pi:~ $ ln -s /usr/local/lib/moonfire-nvr /usr/local/lib/moonfire-nvr/ui
ln: failed to create symbolic link '/usr/local/lib/moonfire-nvr/ui': Permission denied
jlpoole@pi:~ $ sudo ln -s /usr/local/lib/moonfire-nvr /usr/local/lib/moonfire-nvr/ui
jlpoole@pi:~ $

conclusion: I was one directory off at some point and the error did not manifest itself until an upgrade.

Sorry for the confusion.

[jlpoolen]

I'm nervous about my "fix"... it might come back to bite me later on, so I'll move everything into a directory named "ui".

[scottlamb]

btw, in general: upgrades from version 0.x.y to version 0.x.z don't change the database schema, so you can safely revert if the new version gives you trouble.

[jlpoolen]

Thanks. The UI retrieval is working, nothing in the linux console seems out of the ordinary, in fact, it seems a bit cleaner given the errors my cameras and/or network introduce.

I have encountered the Live view is not working: ws close: 1006. Happens in both Chrome and Firefox on Windows 7. Those error message look like the ones when I have returned to a Live View only to find things have timed-out.

So I'm retracing my steps of the upgrade and confirming the version of required software. If my backtracking doesn't reveal something I've mangled, I'll create anew ticket re: Live View. (I've shared this Live View with about 10 people now -- not a great time to go offline.) If there is any debug mode or log that the server has, that could be helpful. The

[scottlamb]

I have encountered the Live view is not working: ws close: 1006.

Argh, that's my fault. It's a problem with my 0.7.3 security fix (changelog entry, commit). If your clients use URLs with a non-default port number, the live stream will never authenticate properly. Rather than the server accepting the websocket upgrade, it will send back a HTTP 403 with a body like this:

Permission denied: cross-origin request forbidden (request host "slamb-workstation:8080", origin host "slamb-workstation")

...which the browser UI won't show. It'll just give the classic "ws close: 1006" error.

Fix coming shortly.

[jlpoolen]

I don't know if this factors in, but I am having my pfSense reroute, "port forward", port 8080 packets to the Raspberry Pi unit for my domain salemdata.us (and now salemdata.net which the squatters abandoned after I previously let it go -- yippeee).

[scottlamb]

Fixed in fd7438d.

[jlpoolen]

same problem. here's a log while I tried the UI and still encountered the ws close: 1006

http://salemdata.us/moonfire-nvr/moonfire-nvr_20220413_Wed_2154_script.log.html

[jlpoolen]

I've restarted the server, with script logging, so you can use your account to look: http;//salemdata.us:8080

[scottlamb]

You're not running the fixed version. From Wireshark, I see it's returning this error:

Permission denied: cross-origin request forbidden (request host "salemdata.us:8080", origin host "salemdata.us")

which doesn't match the fd74e8d format (note the origin host vs origin):

moonfire-nvr/server/src/web/live.rs

Lines 56 to 61 in fd7438d

	bail_t!(
	PermissionDenied,
	"cross-origin request forbidden (request host {:?}, origin {:?})",
	host_hdr,
	origin_hdr
	);

[jlpoolen]

You're right. So here a log of what I just did now. The Live View has no errors and not images. I'll leave the server running for your benefit and I have the log being captured by script to preserve colors.

jlpoole@pi:~ $ cd /usr/local/src/moonfire-nvr/
jlpoole@pi:/usr/local/src/moonfire-nvr $ git pull
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 17 (delta 9), reused 17 (delta 9), pack-reused 0
Unpacking objects: 100% (17/17), done.
From https://github.com/scottlamb/moonfire-nvr
   13bfb76..fd7438d  master     -> origin/master
Updating 13bfb76..fd7438d
Fast-forward
 CHANGELOG.md                  |   7 +++++++
 server/src/web/live.rs        | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------
 server/src/web/mod.rs         |   2 +-
 server/src/web/static_file.rs |   7 +++----
 4 files changed, 87 insertions(+), 35 deletions(-)
jlpoole@pi:/usr/local/src/moonfire-nvr $ cd ui
jlpoole@pi:/usr/local/src/moonfire-nvr/ui $ npm install

up to date, audited 1808 packages in 13s

209 packages are looking for funding
  run `npm fund` for details

17 vulnerabilities (5 moderate, 11 high, 1 critical)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
jlpoole@pi:/usr/local/src/moonfire-nvr/ui $ npm run build

> ui@0.1.0 build
> react-scripts build && gzipper compress --exclude=png,woff2 --remove-larger ./build

Creating an optimized production build...
Compiled successfully.

File sizes after gzip:

  208.2 kB  build/static/js/main.64532c01.js
  621 B     build/static/css/main.159fcde9.css

The project was built assuming it is hosted at /.
You can control this with the homepage field in your package.json.

The build folder is ready to be deployed.
You may serve it with a static server:

  serve -s build

Find out more about deployment here:

  https://cra.link/deployment

gzipper: 11 files have been compressed. (1s 143.613467ms)
jlpoole@pi:/usr/local/src/moonfire-nvr/ui $ cd ..
jlpoole@pi:/usr/local/src/moonfire-nvr $  sudo rsync --recursive --delete --chmod=D755,F644 ui/build/ /usr/local/lib/moonfire-nvr/ui
jlpoole@pi:/usr/local/src/moonfire-nvr $ ls -la /usr/local/lib/moonfire-nvr/ui
total 44
drwxr-xr-x 4 root root 4096 Apr 13 21:53 .
drwxr-xr-x 3 root root 4096 Apr 13 20:53 ..
-rw-r--r-- 1 root root 1353 Apr 13 21:53 asset-manifest.json
-rw-r--r-- 1 root root  380 Apr 13 21:53 asset-manifest.json.gz
drwxr-xr-x 2 root root 4096 Apr 13 21:53 favicons
-rw-r--r-- 1 root root 1022 Apr 13 21:53 index.html
-rw-r--r-- 1 root root  547 Apr 13 21:53 index.html.gz
-rw-r--r-- 1 root root   69 Apr 13 21:53 robots.txt
-rw-r--r-- 1 root root  513 Apr 13 21:53 site.webmanifest
-rw-r--r-- 1 root root  289 Apr 13 21:53 site.webmanifest.gz
drwxr-xr-x 5 root root 4096 Apr 13 20:54 static
jlpoole@pi:/usr/local/src/moonfire-nvr $ cd ~
jlpoole@pi:~ $ tmux new -s moonfire

    mention #206 in v0.7.4 changelog
jlpoole@pi:/usr/local/src/moonfire-nvr $ cd ui
jlpoole@pi:/usr/local/src/moonfire-nvr/ui $ npm install

up to date, audited 1808 packages in 13s

209 packages are looking for funding
  run `npm fund` for details

17 vulnerabilities (5 moderate, 11 high, 1 critical)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
jlpoole@pi:/usr/local/src/moonfire-nvr/ui $ npm run build

> ui@0.1.0 build
> react-scripts build && gzipper compress --exclude=png,woff2 --remove-larger ./build

Creating an optimized production build...
Compiled successfully.

File sizes after gzip:

  208.2 kB  build/static/js/main.64532c01.js
  621 B     build/static/css/main.159fcde9.css

The project was built assuming it is hosted at /.
You can control this with the homepage field in your package.json.

The build folder is ready to be deployed.
You may serve it with a static server:

  serve -s build

Find out more about deployment here:

  https://cra.link/deployment

gzipper: 11 files have been compressed. (1s 136.072596ms)
jlpoole@pi:/usr/local/src/moonfire-nvr/ui $ cd ..
jlpoole@pi:/usr/local/src/moonfire-nvr $ sudo rsync --recursive --delete --chmod=D755,F644 ui/build/ /usr/local/lib/moonfire-nvr/ui
jlpoole@pi:/usr/local/src/moonfire-nvr $ ls -la /usr/local/lib/moonfire-nvr/ui
total 44
drwxr-xr-x 4 root root 4096 Apr 13 22:12 .
drwxr-xr-x 3 root root 4096 Apr 13 20:53 ..
-rw-r--r-- 1 root root 1353 Apr 13 22:12 asset-manifest.json
-rw-r--r-- 1 root root  380 Apr 13 22:12 asset-manifest.json.gz
drwxr-xr-x 2 root root 4096 Apr 13 22:12 favicons
-rw-r--r-- 1 root root 1022 Apr 13 22:12 index.html
-rw-r--r-- 1 root root  547 Apr 13 22:12 index.html.gz
-rw-r--r-- 1 root root   69 Apr 13 22:12 robots.txt
-rw-r--r-- 1 root root  513 Apr 13 22:12 site.webmanifest
-rw-r--r-- 1 root root  289 Apr 13 22:12 site.webmanifest.gz
drwxr-xr-x 5 root root 4096 Apr 13 20:54 static
jlpoole@pi:/usr/local/src/moonfire-nvr $ date
Wed 13 Apr 2022 10:12:57 PM PDT
jlpoole@pi:/usr/local/src/moonfire-nvr $
[moonfire]0:bash*                                                     

[jlpoolen]

It's getting late. I'm okay just letting this run... the capturing and preservation are important, the Live View is a nice-to-have... but, very enjoyable. This can be looked at later by you at your convenience. I'm probably going to retire in the next half hour.

[scottlamb]

The fix is in the Rust server code, not the UI. I don't see a rebuild there, and I just checked again; you're still on the old version.

[jlpoolen]

Yes, I had to rebuild the server and did, and I ran through the build steps for the ui, though I'm not sure if it resulted in any change.

Live View is now working.

Thank you.

Log of rebuild: http://salemdata.us/moonfire-nvr/rebuild_20220413_Wed_2223_script.log.html

I'll remove the logs exposed in this ticket in 36 hours.