Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fffmpeg does not scale. #6

Closed
tomp736 opened this issue Feb 24, 2023 · 1 comment · Fixed by #7
Closed

Fffmpeg does not scale. #6

tomp736 opened this issue Feb 24, 2023 · 1 comment · Fixed by #7
Assignees

Comments

@tomp736
Copy link
Owner

tomp736 commented Feb 24, 2023

When processing media to generate thumbnails FfmpegService retrieves all available media. If multiple instances of pymp_server exist that are designated the role of processing media, they are not aware of who is processing what, and will all potentially (usually) process the same media.

The expected behavior is that when scaling this service, it will allow for media to be processed faster by means of splitting the work across the instances with the FfmpegService.

@tomp736 tomp736 self-assigned this Feb 24, 2023
@tomp736
Copy link
Owner Author

tomp736 commented Feb 24, 2023

Implement Queue using redis lists with lpop and rpush.

https://redis.io/docs/data-types/lists/

  • add a new redis data access class (RedisMediaProcessQueue) that uses a redis list along with rpop and lpush methods implemented. The list elements will be media_info objects.
  • during the existing scan that is done by the media_registry service, it will push media_info items onto the queue if they need to be processed (meaning, the media has no thumbnail or metadata present).
  • during the existing scan that is done by the ffmpeg service, it will pop media_info from the new redis list and process the media.

@tomp736 tomp736 linked a pull request Feb 27, 2023 that will close this issue
@tomp736 tomp736 closed this as completed Feb 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant