Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to synchronize videos on import #379

Merged
merged 40 commits into from Jul 14, 2023
Merged

Conversation

philipqueen
Copy link
Collaborator

@philipqueen philipqueen commented May 1, 2023

This PR is to add an option to the Import Videos" wizard that will run the imported videos through the SkellySynchronize` code that will synchronize them by cross corrolating their audio streams and clipping them so they are synchronized and have the exact same number of frames.

We'll know it works when a user can load freemocap select 'import videos' and load in a folder full of (say) GoPro videos that get synchronized and saved to a sycnhronzied_videos folder to be processed just like any other freemocap recording.

Tasks to be done for this to work :

freemocap side

  • add a synchronize videos by their audio track checkbox (or parameter tree?) or a paramter tree pops up after they click "import" if that button is checked?) to the Import Videos wizard.

SkellySynchronize side

  • save out some kind of diagnostic information to a TOML
  • OPTIONAL - Make some kind of visual debug output so that we can check quality
  • OPTIONAL - - Run via parameter tree model (for now just run with default values)
  • OPTIONAL - fix the phone video resoultion bug (but focus on GoPro) - mostly fixed, will sometimes flip a vertical video upside down

skelly_synchronize repo: https://github.com/freemocap/skelly_synchronize
this would close #385

@philipqueen
Copy link
Collaborator Author

I've added a checkbox to synchronize videos, and added a handling function that just prints whether the button is checked or not to verify if it works.

I tried adding another page, but what we're calling a "wizard" is actually just a QDialog, and I haven't found anything on adding another page to a QDialog. I tried making it a real wizard, but ran into the trouble of dealing with the click connect on line 448 in freemocap_main_window.py when the function its connecting to is inside a WizardPage instead of a Wizard.

The takeaway is we either need to put some work into making a real wizard, or I can add all of the synchronize options onto the main window. Currently there aren't really options, so I can just have it run skelly synchronize on the folder when you click continue

@philipqueen
Copy link
Collaborator Author

I added some additional widgets that open when you click on the synchronize box, which I think is the right option for dealing with the synchronization parameters.
Screen Shot 2023-05-04 at 11 52 32 AM

@philipqueen
Copy link
Collaborator Author

Also, I thought my last post mentioned this but if the synchronization button is clicked, it will run skelly synchronize on the videos before importing them, so this PR is minimally viable

@philipqueen
Copy link
Collaborator Author

I've changed the layout by putting all of the synchronization information in an hbox. Let me know @jonmatthis if there are any other aesthetic changes to make. (Ignore the coloring, I'm using changes from the recoloring PR because the dialog currently doesn't show up on my computer without them)

I've also added proper error handling to skelly_synchronize if videos don't have an audio file.

Screen Shot 2023-05-09 at 4 10 41 PM

@philipqueen
Copy link
Collaborator Author

Skelly_synchronize now outputs a debug plot.

debug_plot

Because opencv's videowriter does not support audio, the synchronized videos do not have audio files. So what is being displayed in the "After Cross Correlation" section is the audio files from the raw videos trimmed according to the same process as the videos. This isn't ideal, but it's better than no debug, and there are checks to make sure the videos are trimmed properly.

@philipqueen
Copy link
Collaborator Author

As of today this is non-blocking (runs the synchronization in a thread) and it prints the logging messages from skelly_synchronize in the console.
Screen Shot 2023-06-20 at 2 52 55 PM

Copy link
Collaborator

@aaroncherian aaroncherian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got this to work successfully on two long GoPro videos, as seen below:

debug_plot

pyproject.toml Show resolved Hide resolved
@philipqueen philipqueen added the ready for review Pull request is ready for final review label Jul 14, 2023
Copy link
Member

@jonmatthis jonmatthis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woo!

@philipqueen philipqueen merged commit 2e97908 into main Jul 14, 2023
2 checks passed
@philipqueen philipqueen deleted the philip/synchronize_videos branch July 14, 2023 19:48
jeff-cn pushed a commit to jeff-cn/freemocap that referenced this pull request Nov 23, 2023
* Add space to readme

* remove dummy commit on readme

* rename import videos wizard

* synchronize videos by audio checkbox

* synchronizing, but to wrong place

* working messy synchronization

* working dummy synchronize parameters

* better naming+desrciptions in parameter dialog

* disambiguate synchronize video function

* change message, remove dummy variable addition

* remove debugging print statements

* add skelly_synchronize to pyproject dependencies

* Change synchronize layout in import videos dialog

* Remove unnecessary QWidget import

* synchronize videos directly into session folder

* Separate button message and label with parentheses

* make synchronization run in a thread

* pin aniposelib and toml versions

* Make sure active recording is set after synching

* DRYing _handle_import_videos

* move debug plotting out of thread

* remove mediapip pin to conform to main

* Add word wrap to synchronization message

* remove unnecessary import

---------

Co-authored-by: jonmatthis <jonmatthis@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review Pull request is ready for final review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Import GoPro with sync
3 participants