Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Input process (PID 3879352) shut down before stream coming online, aborting #57

Open
nerijus opened this issue Jul 23, 2022 · 4 comments
Open

Input process (PID 3879352) shut down before stream coming online, aborting #57

nerijus opened this issue Jul 23, 2022 · 4 comments

Comments

@nerijus
Copy link

nerijus commented Jul 23, 2022

Mistserver 3.1 on CentOS Stream 9. We restarted the service:

Jul 23 06:07:12 server systemd[1]: Stopping MistServer...
Jul 23 06:07:12 server MistController[3804293]: [2022-07-23 06:07:12] MistController (3804293) CONF: Controller shutting down because of user request (received shutdown signal)
Jul 23 06:07:13 server MistController[3804293]: Killed all processes, wrote config to disk. Exiting.
Jul 23 06:07:22 server systemd[1]: mistserver.service: State 'final-sigterm' timed out. Killing.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804411 (MistInTS) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804412 (MistInTS) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804414 (MistInBuffer) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804415 (MistInBuffer) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804419 (MistInTS) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804420 (MistInTS) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804422 (MistInBuffer) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804423 (MistInBuffer) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804431 (MistInTS) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804432 (MistInTS) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804434 (MistInBuffer) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3804435 (MistInBuffer) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3850985 (MistInTS) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3850986 (MistInTS) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3850988 (MistInBuffer) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Killing process 3850989 (MistInBuffer) with signal SIGKILL.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Failed with result 'timeout'.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804412 (MistInTS) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804414 (MistInBuffer) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804415 (MistInBuffer) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804420 (MistInTS) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804422 (MistInBuffer) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804423 (MistInBuffer) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804431 (MistInTS) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804432 (MistInTS) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804434 (MistInBuffer) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804435 (MistInBuffer) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3850985 (MistInTS) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3850986 (MistInTS) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3850988 (MistInBuffer) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3850989 (MistInBuffer) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: Stopped MistServer.

After restart a lof of such messages started to appear:

Jul 23 06:07:56 server MistController[3879038]: [2022-07-23 06:07:56] MistInTS:bf5002 (3879375) WARN: A pull process for this stream is already running
Jul 23 06:07:56 server MistController[3879038]: [2022-07-23 06:07:56] MistInTS:bf5002 (3879377) WARN: A pull process for this stream is already running
Jul 23 06:07:56 server MistController[3879038]: [2022-07-23 06:07:56] MistOutHTTPTS:bf5012 (3879251) FAIL: Input process (PID 3879346) shut down before stream coming online, aborting.
Jul 23 06:07:56 server MistController[3879038]: [2022-07-23 06:07:56] MistOutHTTPTS:bf5012 (3879251) FAIL: onFail 'bf5012': Stream open failed
Jul 23 06:07:56 server MistController[3879038]: [2022-07-23 06:07:56] MistOutHTTPTS:bf5012 (3879251) FAIL: onFail 'bf5012': Could not connect to stream
Jul 23 06:07:56 server MistController[3879038]: [2022-07-23 06:07:56] MistInTS:bf5012 (3879378) WARN: A pull process for this stream is already running
Jul 23 06:07:56 server MistController[3879038]: [2022-07-23 06:07:56] MistInTS:bf5002 (3879380) WARN: A pull process for this stream is already running
Jul 23 06:07:57 server MistController[3879038]: [2022-07-23 06:07:57] MistOutHTTPTS:bf5012 (3879306) FAIL: Input process (PID 3879352) shut down before stream coming online, aborting.
Jul 23 06:07:57 server MistController[3879038]: [2022-07-23 06:07:57] MistOutHTTPTS:bf5012 (3879306) FAIL: onFail 'bf5012': Stream open failed
Jul 23 06:07:57 server MistController[3879038]: [2022-07-23 06:07:57] MistOutHTTPTS:bf5012 (3879306) FAIL: onFail 'bf5012': Could not connect to stream
Jul 23 06:07:57 server MistController[3879038]: [2022-07-23 06:07:57] MistInTS:bf5012 (3879381) WARN: A pull process for this stream is already running

Only after 2nd restart they disappeared.
@nerijus
Copy link
Author

nerijus commented Jul 23, 2022

Streams are like this:

    "bf5012": {
      "always_on": false,
      "name": "bf5012",
      "processes": [],
      "realtime": false,
      "source": "tsudp://224.2.2.2:5012",
      "stop_sessions": false
    },

@nerijus
Copy link
Author

nerijus commented Jul 24, 2022 •
edited
Loading

It happened again. Do I understand correctly that some old processes are still running and somehow interfere?

@Rokamun
Copy link
Contributor

Rokamun commented Jul 27, 2022

Hey Nerijus,

Looks like the processes kept active as the time-out on trying to kill the processes was met:
Jul 23 06:07:22 server systemd[1]: mistserver.service: Failed with result 'timeout'.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804412 (MistInTS) remains running after unit stopped.
Jul 23 06:07:22 server systemd[1]: mistserver.service: Unit process 3804414 (MistInBuffer) remains running after unit stopped.

The 2nd boot of MistServer would indeed see processes active here, since a buffer and input process are active. It would try to reconnect at that point. That will definitely interfere at that point as I'd have to assume both processes wouldn't be working properly at that point.

A method to absolutely make sure it's dead would be to use the MistServer 3.x command nuke.
' "nuke_stream": "stream name here" '
(MistServer manual 7.1.11 )
This should completely kill off the process and clean up any shared memory it might've left behind.

This is happening on a clean CentOS Stream 9? I'll verify if we're seeing the same behaviour as this does look like somehting we'll want to fix or double check.

@nerijus
Copy link
Author

nerijus commented Jul 27, 2022

Yes, it is a clean CentOS Stream 9 install. I did not report the whole story yet, will do now. 3 nights in a row at about 02:20 Mistserver started to consume RAM (unfortunately I cannot tell the exact process name), Grafana graph:
image
We are getting several channels via multicast ("source": "tsudp://224.2.2.2:5012") and I suspect one of the channels started to transmit something which triggered a bug in Mistserver. Log:

Jul 24 02:08:55 server MistController[3969490]: [2022-07-24 02:08:55] MistInTS (3969600) WARN: Packet loss detected (354 != 384), throwing away data to compensate
Jul 24 02:18:23 server MistController[3969490]: [2022-07-24 02:18:23] MistOutHTTPTS:bf5010 (4003391) WARN: Dropping HEVC track 1@k51171 (nextP=51149, lastP=51149): disappeared from metadata
Jul 24 02:18:23 server MistController[3969490]: [2022-07-24 02:18:23] MistInTS (4003341) FAIL: No PES packets to parse (2)
Jul 24 02:18:48 server MistController[3969490]: [2022-07-24 02:18:48] MistController (3969490) ACCS: Session <5e37e85c9308fb7e436a1123257b0a83> bf5010 (HTTPTS) from 66.206.x.x ended after 24564s, avg 251KB/s up 0KB/s down. Tags: [UA:Streamer 21.09.1]
Jul 24 02:20:40 server MistController[3969490]: [2022-07-24 02:20:40] MistOutHTTPTS:bf5018 (4008027) WARN: Dropping HEVC track 1@k116424 (nextP=116404, lastP=116404): disappeared from metadata
Jul 24 02:20:42 server MistController[3969490]: [2022-07-24 02:20:42] MistInTS (3969607) FAIL: No PES packets to parse (2)
Jul 24 02:20:44 server MistController[3969490]: [2022-07-24 02:20:44] MistOutHTTPTS:bf5002 (3969628) WARN: Dropping HEVC track 1@k116437 (nextP=4294967295, lastP=116410): disappeared from metadata
Jul 24 02:20:44 server MistController[3969490]: [2022-07-24 02:20:44] MistOutHTTPTS:bf5002 (4004948) WARN: Dropping HEVC track 1@k116437 (nextP=4294967295, lastP=116410): disappeared from metadata
Jul 24 02:20:44 server MistController[3969490]: [2022-07-24 02:20:44] MistOutHTTPTS:bf5002 (3969582) WARN: Dropping HEVC track 1@k116437 (nextP=4294967295, lastP=116410): disappeared from metadata
Jul 24 02:20:47 server MistController[3969490]: [2022-07-24 02:20:47] MistInTS (3969600) FAIL: No PES packets to parse (2)
Jul 24 02:21:01 server MistController[3969490]: [2022-07-24 02:21:01] MistOutHTTPTS:bf5004 (3969570) WARN: Dropping HEVC track 1@k116472 (nextP=4294967295, lastP=116406): disappeared from metadata
Jul 24 02:21:01 server MistController[3969490]: [2022-07-24 02:21:01] MistOutHTTPTS:bf5004 (4008036) WARN: Dropping HEVC track 1@k116472 (nextP=4294967295, lastP=116406): disappeared from metadata
Jul 24 02:21:01 server MistController[3969490]: [2022-07-24 02:21:01] MistOutHTTPTS:bf5004 (3969625) WARN: Dropping HEVC track 1@k116472 (nextP=4294967295, lastP=116406): disappeared from metadata
Jul 24 02:21:01 server MistController[3969490]: [2022-07-24 02:21:01] MistOutHTTPTS:bf5004 (4008572) WARN: Dropping HEVC track 1@k116472 (nextP=4294967295, lastP=116406): disappeared from metadata
Jul 24 02:21:01 server MistController[3969490]: [2022-07-24 02:21:01] MistController (3969490) ACCS: Session <3bdd9300d04a618d3fdf46e2ca20d8c2> bf5002 (HTTPTS) from 195.201.x.x ended after 22842s, avg 251KB/s up 0KB/s down.
Jul 24 02:21:03 server MistController[3969490]: [2022-07-24 02:21:03] MistInTS (3969589) FAIL: No PES packets to parse (2)
Jul 24 02:21:12 server MistController[3969490]: [2022-07-24 02:21:12] MistController (3969490) ACCS: Session <f9963dbf79038dd7729b4ef59ed60f52> bf5018 (HTTPTS) from 195.201.x.x ended after 19171s, avg 251KB/s up 0KB/s down. Tags: [UA:Streamer 21.09.1]
Jul 24 02:21:37 server MistController[3969490]: [2022-07-24 02:21:37] MistController (3969490) ACCS: Session <cb62f4df429532a930bf98d0ffeed47d> bf5004 (HTTPTS) from 66.206.x.x ended after 18561s, avg 251KB/s up 0KB/s down.
Jul 24 02:22:37 server MistController[3969490]: [2022-07-24 02:22:37] MistOutHTTPTS:bf5012 (3971014) WARN: Dropping HEVC track 2@k116671 (nextP=116654, lastP=116654): disappeared from metadata
Jul 24 02:22:37 server MistController[3969490]: [2022-07-24 02:22:37] MistOutHTTPTS:bf5012 (3971002) WARN: Dropping HEVC track 2@k116671 (nextP=116654, lastP=116654): disappeared from metadata
Jul 24 02:22:40 server MistController[3969490]: [2022-07-24 02:22:40] MistInTS (3969577) FAIL: No PES packets to parse (2)
Jul 24 02:23:01 server MistController[3969490]: [2022-07-24 02:23:01] MistController (3969490) ACCS: Session <70a29160c0f51e1957d7017087c2ce0d> bf5012 (HTTPTS) from 160.119.x.x ended after 56022s, avg 251KB/s up 0KB/s down.
Jul 24 02:49:45 server MistController[3969490]: [2022-07-24 02:49:45] MistController (3969490) UPDR: Self-compiled build. Updater disabled.
Jul 24 02:49:49 server MistController[3969490]: [2022-07-24 02:49:49] MistInTS (3969577) WARN: Packet loss detected (354 != 384), throwing away data to compensate
Jul 24 02:49:49 server MistController[3969490]: [2022-07-24 02:49:49] MistInTS (3969577) WARN: Packet loss detected (354 != 384), throwing away data to compensate
Jul 24 02:49:49 server MistController[3969490]: [2022-07-24 02:49:49] MistInTS (3969600) WARN: Packet loss detected (354 != 384), throwing away data to compensate
Jul 24 02:49:49 server MistController[3969490]: [2022-07-24 02:49:49] MistInTS (3969589) WARN: Packet loss detected (200 != 384), throwing away data to compensate
Jul 24 02:49:49 server MistController[3969490]: [2022-07-24 02:49:49] MistInTS (3969607) WARN: Packet loss detected (354 != 384), throwing away data to compensate
Jul 24 02:49:49 server MistController[3969490]: [2022-07-24 02:49:49] MistInTS (3969577) WARN: Packet loss detected (354 != 384), throwing away data to compensate
Jul 24 02:49:49 server MistController[3969490]: [2022-07-24 02:49:49] MistInTS (3969600) WARN: Packet loss detected (200 != 384), throwing away data to compensate
Jul 24 02:49:49 server MistController[3969490]: [2022-07-24 02:49:49] MistInTS (3969589) WARN: Packet loss detected (170 != 384), throwing away data to compensate
Jul 24 02:49:50 server MistController[3969490]: [2022-07-24 02:49:50] MistInTS (3969577) WARN: Packet loss detected (200 != 384), throwing away data to compensate
      lots of repeating messages
Jul 24 03:09:03 server MistController[3969490]: [2022-07-24 03:09:03] MistInTS (4003341) WARN: Packet loss detected (200 != 384), throwing away data to compensate
Jul 24 03:09:03 server MistController[3969490]: [2022-07-24 03:09:03] MistInTS (3969600) WARN: Packet loss detected (200 != 384), throwing away data to compensate
Jul 24 03:09:03 server MistController[3969490]: [2022-07-24 03:09:03] MistInTS (3969589) WARN: Packet loss detected (200 != 384), throwing away data to compensate
Jul 24 03:09:05 server systemd[1]: Stopping MistServer...
Jul 24 03:09:05 server MistController[3969490]: [2022-07-24 03:09:05] MistController (3969490) CONF: Controller shutting down because of user request (received shutdown signal)
Jul 24 03:09:06 server MistController[3969490]: Killed all processes, wrote config to disk. Exiting.
Jul 24 03:09:15 server systemd[1]: mistserver.service: State 'final-sigterm' timed out. Killing.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969576 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969577 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969579 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969580 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969588 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969589 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969591 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969592 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969599 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969600 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969602 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969603 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969606 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969607 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969609 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969610 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4003340 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4003341 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4003343 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4003344 (MistInBuffer) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969601 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969608 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969617 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 3969620 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4003342 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4003345 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4024166 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4024277 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4024281 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4024309 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Killing process 4024397 (MistInTS) with signal SIGKILL.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Failed with result 'timeout'.
Jul 24 03:09:15 server systemd[1]: Stopped MistServer.
Jul 24 03:09:15 server systemd[1]: mistserver.service: Consumed 2h 45min 2.369s CPU time.
Jul 24 03:09:15 server systemd[1]: Started MistServer.
Jul 24 03:09:18 server MistController[4026809]: [2022-07-24 03:09:18] MistController (4026809) CONF: Controller started
Jul 24 03:09:18 server MistController[4026809]: [2022-07-24 03:09:18] MistController (4026809) CONF: Starting connector: {"acceptable":"2","connector":"TSSRT"}
Jul 24 03:09:18 server MistController[4026809]: [2022-07-24 03:09:18] MistController (4026809) CONF: Starting connector: {"cert":"/etc/mistserver/stream.pem","connector":"HTTPS","key":"/etc/mistserver/stream.key","port":443,"pubaddr":[]}
Jul 24 03:09:18 server MistController[4026809]: [2022-07-24 03:09:18] MistController (4026809) CONF: Starting connector: {"connector":"DTSC"}
Jul 24 03:09:18 server MistController[4026809]: [2022-07-24 03:09:18] MistController (4026809) CONF: Starting connector: {"connector":"HTTP"}
Jul 24 03:09:18 server MistController[4026809]: [2022-07-24 03:09:18] MistController (4026809) CONF: Starting connector: {"connector":"RTMP","port":1936}
Jul 24 03:09:18 server MistController[4026809]: [2022-07-24 03:09:18] MistController (4026809) CONF: Starting connector: {"connector":"RTSP"}
Jul 24 03:09:19 server MistController[4026809]: [2022-07-24 03:09:19] MistInTS:bf5002 (4026907) WARN: A pull process for this stream is already running
Jul 24 03:09:20 server MistController[4026809]: [2022-07-24 03:09:20] MistController (4026809) UPDR: Self-compiled build. Updater disabled.
Jul 24 03:09:37 server MistController[4026809]: [2022-07-24 03:09:37] MistOutHTTPS (4026876) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:37 server MistController[4026809]: [2022-07-24 03:09:37] MistOutHTTPS (4026877) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:37 server MistController[4026809]: [2022-07-24 03:09:37] MistOutHTTPS (4026880) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:37 server MistController[4026809]: [2022-07-24 03:09:37] MistOutHTTPS (4026878) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:37 server MistController[4026809]: [2022-07-24 03:09:37] MistOutHTTPS (4026879) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:37 server MistController[4026809]: [2022-07-24 03:09:37] MistOutHTTPS (4026883) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:37 server MistController[4026809]: [2022-07-24 03:09:37] MistOutHTTPS (4026875) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:38 server MistController[4026809]: [2022-07-24 03:09:38] MistOutHTTPS (4026874) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:38 server MistController[4026809]: [2022-07-24 03:09:38] MistOutHTTPS (4026912) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:40 server MistController[4026809]: [2022-07-24 03:09:40] MistOutHTTPS (4026933) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:41 server MistController[4026809]: [2022-07-24 03:09:41] MistOutHTTPS (4026947) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:41 server MistController[4026809]: [2022-07-24 03:09:41] MistOutHTTPS (4026948) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:41 server MistController[4026809]: [2022-07-24 03:09:41] MistOutHTTPS (4026949) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:41 server MistController[4026809]: [2022-07-24 03:09:41] MistOutHTTPS (4026951) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:41 server MistController[4026809]: [2022-07-24 03:09:41] MistOutHTTPS (4026946) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:41 server MistController[4026809]: [2022-07-24 03:09:41] MistOutHTTPS (4026950) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:41 server MistController[4026809]: [2022-07-24 03:09:41] MistOutHTTPS (4026954) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:42 server MistController[4026809]: [2022-07-24 03:09:42] MistOutHTTPS (4026955) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:44 server MistController[4026809]: [2022-07-24 03:09:44] MistOutHTTPS (4026972) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:45 server MistController[4026809]: [2022-07-24 03:09:45] MistOutHTTPS (4026975) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:45 server MistController[4026809]: [2022-07-24 03:09:45] MistOutHTTPS (4026980) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:45 server MistController[4026809]: [2022-07-24 03:09:45] MistOutHTTPS (4026981) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:45 server MistController[4026809]: [2022-07-24 03:09:45] MistOutHTTPS (4026983) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:45 server MistController[4026809]: [2022-07-24 03:09:45] MistOutHTTPS (4026982) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:45 server MistController[4026809]: [2022-07-24 03:09:45] MistOutHTTPS (4026979) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:45 server MistController[4026809]: [2022-07-24 03:09:45] MistOutHTTPS (4026986) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:46 server MistController[4026809]: [2022-07-24 03:09:46] MistOutHTTPS (4027000) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:46 server MistController[4026809]: [2022-07-24 03:09:46] MistOutHTTPS (4026978) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:48 server MistController[4026809]: [2022-07-24 03:09:48] MistOutHTTPS (4027011) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5012 (4027159) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5012 (4027161) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5012 (4027164) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5012 (4027166) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5012 (4027168) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5012 (4027170) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5012 (4027172) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027174) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027176) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5012 (4027178) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027180) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027182) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027184) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027186) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027188) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027190) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027192) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027194) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027196) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027198) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027200) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027202) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027204) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027206) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027209) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027211) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027213) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027215) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027217) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027219) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027221) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5002 (4027223) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027225) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027227) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027231) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5004 (4027233) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistOutHTTPTS:bf5012 (4027114) FAIL: Input process (PID 4027172) shut down before stream coming online, aborting.
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistOutHTTPTS:bf5012 (4027114) FAIL: onFail 'bf5012': Stream open failed
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistOutHTTPTS:bf5012 (4027114) FAIL: onFail 'bf5012': Could not connect to stream
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistInTS:bf5012 (4027238) WARN: A pull process for this stream is already running
Jul 24 03:09:49 server MistController[4026809]: [2022-07-24 03:09:49] MistOutHTTPS (4027015) ERROR: Sending SIGKILL to remaining 1 children
Jul 24 03:09:50 server MistController[4026809]: [2022-07-24 03:09:50] MistOutHTTPTS:bf5012 (4027031) FAIL: Input process (PID 4027178) shut down before stream coming online, aborting.

After 2nd restart it started to work. I have a full log if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nerijus @Rokamun