FURY Streaming System Proposal #437

devmessias · 2021-05-27T19:48:33Z

related with #435

related with fury-gl#435

pep8speaks · 2021-05-27T19:48:35Z

Hello @devmessias! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2022-04-13 14:53:03 UTC

Co-authored-by: devmessias <messias.physics@gmail.com> Co-authored-by: filipinascimento <filipinascimento@gmail.com>

Doing that way because the FuryStreamClient will have a lot of more behaviors, like user interaction.

This can be used for example to store/read user interactions

@filipinascimento

This uses a circular queue as proposed by @filipinascimento

It seems that vtk uses left-bottom coordinates

macOS dosen't use fork to achieve multiprocessing. Therefore, this create some issues for specific python versions. The solution it's to pass just the buffers to the childreen process

For example, using ngrok ./ngrok http --subdomain=something 8000

Fix zoom factor for throttling

iframe mode, background color and interaction can be defined through url parameters Example: http://localhost:8000/?interaction=1&iframe=0&background=black

…uffers This allows you to start a webrtc server in the main process

This will avoid the first rendering to be just a blank screen inside the browser

devmessias · 2021-08-17T14:56:23Z

Widget demo do not work on windows. May also need fixes for the timers.

Fixed by the following commits: 36ed049, 183badd and 91f3256

docs/tutorials/04_stream/viz_interaction.py

Garyfallidis

This is the first round of my reviews. More asap.

Garyfallidis · 2021-09-28T13:18:28Z

docs/tutorials/04_stream/viz_interaction.py

+Linux
+
+
+`apt install libavdevice-dev libavfilter-dev libopus-dev libvpx-dev pkg-config`


I do not see ffmpeg here. Is that okay?

Ok, I forgot. I'll address that

Did you address it?

it seems that for mac, ffmpeg, opus or libvpx are not needed. I've just pipinstall aiortc

Garyfallidis · 2021-09-28T13:24:27Z

docs/tutorials/04_stream/viz_interaction.py

+                None,
+                stream.img_manager.info_buffer,
+                None,
+                stream_interaction.circular_queue.head_tail_buffer,


The args can be simplified very much by just providing as input stream and stream_interaction. Then you can grab the other properties from the inside your function.

This will not work on MacOs. Unfortunately, MacOs seems to have some problems sending a pickle to a child process

Otherwise, create two classes one for shared memory and one for raw array.

docs/tutorials/04_stream/viz_widget.py

fury/stream/widget.py

Garyfallidis · 2021-09-28T13:47:38Z

fury/stream/widget.py

+        self.ms_jpeg = ms_jpeg
+        self._host = host
+        if port is None:
+            port = np.random.randint(7000, 8888)


Is this safe to do?

Yes, I think so. @filipinascimento do you have any considerations?

This do not check for existing open ports, it is safe, but can fail if the selected port is already open.

We can check this. But this operation takes some time.

skoudoro

Hi @devmessias,

See below for some comments, I still need to spend time on fury.stream.tools

fury/stream/client.py

to improve readability

To improve the code readability and design

filipinascimento · 2022-04-10T09:48:22Z

fury/stream/tools.py

+    def fix_register(name, rtype):
+        if rtype == "shared_memory":
+            return
+        return resource_tracker._resource_tracker.register(


self is undefined here. Results in a silent error when the server is running in another process.

This is how we can do monkey patching on python. This is an open bug in CPython implementation regarding shared memory

Alto this is a silent ERROR or silent Warning? What is the msg?

Indeed this results in a warning caused by an unraisable exception, which I'm not sure if it is unraisable because it is running during garbage collector, destroy or because it runs in another process.

What you can do is call ls -ls /dev/shm and see if some resource was not properly released.

Ok, I've tested this, and the patch works on mac. The problem is that, for some reason, pytest checks this self and gives an NameError. On macs, /dev/shm does not exist, but I've checked for memory usage and everything seems to work nicely on Python 3.8.

Adding a NameError try catch, as you suggested, gets rid of the warning for when running pytest. Memory is reclaimed. so everything should be fine. The problem does not happen on Windows.

Done at 25b3979
Maybe is good to describe this issue on cypthon repo. python/cpython#82300

skoudoro · 2022-04-12T15:11:42Z

Also, do not forget to address all the pep8 listed above

filipinascimento

I think the PR is ready (may need a pass from Serge and Elef to make sure everything is ok with coding style). We can improve the documentation incrementally later on.

fury/stream/client.py

Garyfallidis

Hi @devmessias Please see comments and also address pep 8 issues. Keep pushing this is getting super close to be merged! FURY streaming is on the way! Go go team!

docs/tutorials/04_stream/viz_interaction.py

fury/stream/client.py

fury/stream/widget.py

requirements/optional.txt

It dosen't provides real improvements

python bug related to python/cpython#82300

Garyfallidis · 2022-04-22T16:54:54Z

Congratulations @devmessias This important PR is finally merged. Great job!

devmessias · 2022-04-22T23:29:34Z

Great news! Thanks @Garyfallidis , @skoudoro and @filipinascimento

Garyfallidis · 2022-04-23T00:42:08Z

Next step @devmessias is the creation of furyspeed package that will accelerate streaming speed @skoudoro

adds a simple FuryStreamClient

634a328

related with fury-gl#435

devmessias changed the title ~~adds a simple FuryStreamClient~~ FuryStreamClient proposal May 27, 2021

devmessias marked this pull request as draft May 27, 2021 19:48

devmessias mentioned this pull request May 27, 2021

Fury/VTK Streaming: webrtc/rtmp #435

Closed

devmessias and others added 25 commits May 27, 2021 17:02

allows to stop the stream if requested

fce8e07

fix identation

a14228c

a simple example using shared mem. and streaming client

8626370

webrtc streaming server

5c3fcf2

Co-authored-by: devmessias <messias.physics@gmail.com> Co-authored-by: filipinascimento <filipinascimento@gmail.com>

reorganize stream files and folders

e8044bc

Doing that way because the FuryStreamClient will have a lot of more behaviors, like user interaction.

this separates the js script of webrtcstream into a module

0c919a7

implements a circular queue using shared buffers

af66c6d

This can be used for example to store/read user interactions

a porposal for user interaction for furystream

72436bc

This uses a circular queue as proposed by @filipinascimento

removes old stuff

bfecc9b

simplifies the use of the streaming system

8b12ff2

implements mouse interaction

03a4827

fixes mouse movement in Y-axis

7eab5a7

It seems that vtk uses left-bottom coordinates

this avoid pickle issues in OSX

be0b65f

macOS dosen't use fork to achieve multiprocessing. Therefore, this create some issues for specific python versions. The solution it's to pass just the buffers to the childreen process

fix typo

50fec39

simplifies the use of the StreamInteraction Class

c56d3f0

allows to run the stream inside of a online server

38f1f9b

For example, using ngrok ./ngrok http --subdomain=something 8000

Added support for n-buffering

16d26ca

Removed bounce effect on safari

9d858b6

Added mouse wheel throtling

c743f63

Fix zoom factor for throttling

improves the organization, add iframe mode

504e17f

iframe mode, background color and interaction can be defined through url parameters Example: http://localhost:8000/?interaction=1&iframe=0&background=black

allows to create a FuryStreamClient and Interaction with predefined b…

4548565

…uffers This allows you to start a webrtc server in the main process

Added support for broadcast using MediaRelay

54bcf33

websocket mouse interaction

c873c3a

allows to use the webrtc and interaction using different webservers

fa8779a

this will call render after starting the streaming

4bafc90

This will avoid the first rendering to be just a blank screen inside the browser

devmessias force-pushed the feature_fury_stream_client branch from cff02af to eb85624 Compare August 15, 2021 01:18

Garyfallidis reviewed Sep 28, 2021

View reviewed changes

docs/tutorials/04_stream/viz_interaction.py Outdated Show resolved Hide resolved

Garyfallidis reviewed Sep 28, 2021

View reviewed changes

skoudoro reviewed Sep 28, 2021

View reviewed changes

fury/stream/client.py Outdated Show resolved Hide resolved

fury/stream/client.py Outdated Show resolved Hide resolved

fury/stream/client.py Show resolved Hide resolved

fury/stream/client.py Outdated Show resolved Hide resolved

devmessias added 2 commits October 5, 2021 10:26

declare PY_VERSION_8 in constants.py

71915c9

to improve readability

refactor: break web_server between SharedMem and RawArrays

051d42f

To improve the code readability and design

devmessias force-pushed the feature_fury_stream_client branch from e1390da to 051d42f Compare April 5, 2022 18:11

devmessias added 5 commits April 5, 2022 15:41

refactor: web server location

902b52c

fix: open just the MJPEG if webrtc is not available

995c322

fix: tests after refactoring

135461d

refactor: simplify the callback_vtk

bc64731

refactor: isAlive is depreacted

9e74cb4

filipinascimento reviewed Apr 10, 2022

View reviewed changes

devmessias mentioned this pull request Apr 12, 2022

This PR creates an actor to draw labels using texture maps #489

Open

filipinascimento approved these changes Apr 12, 2022

View reviewed changes

fury/stream/client.py Outdated Show resolved Hide resolved

Garyfallidis requested changes Apr 12, 2022

View reviewed changes

devmessias added 5 commits April 13, 2022 09:25

docs: fix typos

c36f863

docstrings: fix pep8

fb45d77

perf: remove opencv for img decoding

609a439

It dosen't provides real improvements

docs: minor improvements

8473418

fix: pytest silent errors in MacOs regarding shm

25b3979

python bug related to python/cpython#82300

devmessias force-pushed the feature_fury_stream_client branch from 4a8f49f to 25b3979 Compare April 13, 2022 13:44

fix: pep8

3fbaf24

devmessias force-pushed the feature_fury_stream_client branch from b58807f to 3fbaf24 Compare April 13, 2022 14:52

Garyfallidis merged commit 063aea9 into fury-gl:master Apr 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FURY Streaming System Proposal #437

FURY Streaming System Proposal #437

devmessias commented May 27, 2021

pep8speaks commented May 27, 2021 •

edited

Loading

devmessias commented Aug 17, 2021

Garyfallidis left a comment

Garyfallidis Sep 28, 2021

devmessias Sep 28, 2021

skoudoro Apr 12, 2022

filipinascimento Apr 12, 2022

filipinascimento Apr 12, 2022

Garyfallidis Sep 28, 2021

devmessias Sep 28, 2021

Garyfallidis Oct 5, 2021

Garyfallidis Sep 28, 2021

devmessias Sep 30, 2021

filipinascimento Apr 5, 2022

devmessias Apr 5, 2022

skoudoro left a comment

filipinascimento Apr 10, 2022

devmessias Apr 10, 2022

devmessias Apr 10, 2022 •

edited

Loading

filipinascimento Apr 10, 2022

devmessias Apr 10, 2022 •

edited

Loading

filipinascimento Apr 11, 2022

filipinascimento Apr 11, 2022

devmessias Apr 13, 2022

skoudoro commented Apr 12, 2022

filipinascimento left a comment

Garyfallidis left a comment

Garyfallidis commented Apr 22, 2022

devmessias commented Apr 22, 2022

Garyfallidis commented Apr 23, 2022

		Linux


		`apt install libavdevice-dev libavfilter-dev libopus-dev libvpx-dev pkg-config`

FURY Streaming System Proposal #437

FURY Streaming System Proposal #437

Conversation

devmessias commented May 27, 2021

pep8speaks commented May 27, 2021 • edited Loading

Comment last updated at 2022-04-13 14:53:03 UTC

devmessias commented Aug 17, 2021

Garyfallidis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skoudoro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devmessias Apr 10, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devmessias Apr 10, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skoudoro commented Apr 12, 2022

filipinascimento left a comment

Choose a reason for hiding this comment

Garyfallidis left a comment

Choose a reason for hiding this comment

Garyfallidis commented Apr 22, 2022

devmessias commented Apr 22, 2022

Garyfallidis commented Apr 23, 2022

pep8speaks commented May 27, 2021 •

edited

Loading

devmessias Apr 10, 2022 •

edited

Loading

devmessias Apr 10, 2022 •

edited

Loading