Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added method addMoreFrames #39

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

kimhongyeon
Copy link

Issue: #38

Fast Image Sequence Renderer, as it stands, is difficult to add frames to once initialized. It requires reinitialization to add more frames, which is inefficient and complicates the code since new images keep coming in over a short period, characteristic of streaming.

So, I have added theaddMoreFrames method.

When addMoreFrames is called while the image sequence is running after the instance has been created, the frames and sources are restructured, allowing images to be added seamlessly.

To implement this, I moved some of the initialization logic from the constructor into a struct method. By calling struct in the constructor, it retains the same logic as before.

In addMoreFrames, after modifying this.options.frames, the struct method is called, followed by the destruct method to temporarily release resources like the canvas. Then, the original logic is executed with a few branching points. For example, when calling drawingLoop, it resumes from the previous playback point.

As a result, when the image sequence needs to grow continuously like streaming, calling the addMoreFrames method allows it to function like video streaming.

reindernijhoff

This comment was marked as duplicate.

Copy link
Member

@reindernijhoff reindernijhoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need an addMoreFrames function? Is this because you don't know exactly how many frames the sequence will have in advance? Or is this because you want to preload additional frames later? Or is there any other reason?
In your current pull request, you recreate all frames and input sources without destructing the old ones. This could cause memory leaks but is also, as far as I can estimate, no faster than an entire destruct/re-init of the sequence renderer.

@kimhongyeon
Copy link
Author

kimhongyeon commented Jul 8, 2024

Why do you need an addMoreFrames function? Is this because you don't know exactly how many frames the sequence will have in advance? Or is this because you want to preload additional frames later? Or is there any other reason?
In your current pull request, you recreate all frames and input sources without destructing the old ones. This could cause memory leaks but is also, as far as I can estimate, no faster than an entire destruct/re-init of the sequence renderer.

Here are the reasons for needing the addMoreFrames method:

  • It is designed to handle situations where additional frames are being received in real-time. For example, implementing a video live streaming feature with an image sequence.
  • For instance, images are continuously being captured in real-time from somewhere, and someone needs to view these captured images continuously in real-time.
  • Therefore, I want to initialize the FastImageSequence with the existing image sequence and play it. Then, naturally append new image sequences received from the server in real-time.

Implementation method:

  • Initially, I tried to resolve this by adding new image sequences to the frames and sources member variables, but since the original source code was not implemented with such considerations, I couldn’t solve it that way.
  • If you follow the internal flow of the addMoreFrames method in detail, you will see that it calls the destruct method. You mentioned that the frames and input sources were being recreated without destructing them, but internally, they are indeed being destructed. If this is not the case, please let me know.

As you suggested, destructing/re-initializing the FastImageSequence from the usage part is also a method, but the usage code would become ugly and hard to understand. More importantly, the video played with the image sequence may not be smooth whenever new frames are added.

As you can see below, in situations where frames are added, it is handled to play continuously and smoothly. Additionally, it branches to moreFrame to help with performance as much as possible.

https://github.com/kimhongyeon/fast-image-sequence/blob/5eb35966d080c18bcb962c04aca7b3a97060cb35/src/lib/FastImageSequence.ts#L373-L377
스크린샷 2024-07-09 오전 7 31 50

On the other hand, if the new image sequences being received from the server in real-time keep coming in at intervals smaller than a second, the addMoreFrames method will be called repeatedly. Considering this, it might be a good idea to handle such situations using throttling or debouncing.

@kimhongyeon kimhongyeon force-pushed the add-method-addMoreFrames branch from 5eb3596 to dc5283a Compare August 7, 2024 23:25
@kimhongyeon
Copy link
Author

When will this PR be merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants