Skip to content

Commit

Permalink
Refactor of global timeline effects, to address a regression with glo…
Browse files Browse the repository at this point in the history 
…bal/timeline Mask/Transitions no longer working correctly. This was caused by an optimization that broke the general behavior of the global transitions.
  • Loading branch information
@jonoomph
jonoomph committed May 18, 2021
1 parent 6a004ed commit 441cb18
Show file tree
Hide file tree
Showing 5 changed files with 86 additions and 24 deletions.
36 changes: 33 additions & 3 deletions src/Clip.cpp
View file
Expand Up @@ -335,15 +335,36 @@ std::shared_ptr<Frame> Clip::GetFrame(int64_t frame_number)

// Get the original frame and pass it to GetFrame overload
std::shared_ptr<Frame> original_frame = GetOrCreateFrame(frame_number);
return GetFrame(original_frame, frame_number);
return GetFrame(original_frame, frame_number, NULL);
}
else
// Throw error if reader not initialized
throw ReaderClosed("No Reader has been initialized for this Clip. Call Reader(*reader) before calling this method.");
}

// Use an existing openshot::Frame object and draw this Clip's frame onto it
// Create an openshot::Frame object for a specific frame number of this reader.
std::shared_ptr<Frame> Clip::GetFrame(std::shared_ptr<openshot::Frame> background_frame, int64_t frame_number)
{
// Check for open reader (or throw exception)
if (!is_open)
throw ReaderClosed("The Clip is closed. Call Open() before calling this method.");

if (reader)
{
// Adjust out of bounds frame number
frame_number = adjust_frame_number_minimum(frame_number);

// Get the original frame and pass it to GetFrame overload
std::shared_ptr<Frame> original_frame = GetOrCreateFrame(frame_number);
return GetFrame(original_frame, frame_number, NULL);
}
else
// Throw error if reader not initialized
throw ReaderClosed("No Reader has been initialized for this Clip. Call Reader(*reader) before calling this method.");
}

// Use an existing openshot::Frame object and draw this Clip's frame onto it
std::shared_ptr<Frame> Clip::GetFrame(std::shared_ptr<openshot::Frame> background_frame, int64_t frame_number, openshot::TimelineInfoStruct* options)
{
// Check for open reader (or throw exception)
if (!is_open)
Expand All @@ -367,9 +388,18 @@ std::shared_ptr<Frame> Clip::GetFrame(std::shared_ptr<openshot::Frame> backgroun
// TODO: Handle variable # of samples, since this resamples audio for different speeds (only when time curve is set)
get_time_mapped_frame(original_frame, new_frame_number);

// Apply effects to the frame (if any)
// Apply local effects to the frame (if any)
apply_effects(original_frame);

// Apply global timeline effects (i.e. transitions & masks... if any)
if (timeline != NULL && options != NULL) {
if (options->is_top_clip) {
// Apply global timeline effects (only to top clip... if overlapping)
Timeline* timeline_instance = (Timeline*) timeline;
original_frame = timeline_instance->apply_effects(original_frame, options->timeline_frame_number, Layer());
}
}

// Apply keyframe / transforms
apply_keyframes(original_frame, background_frame->GetImage());

Expand Down
29 changes: 22 additions & 7 deletions src/Clip.h
View file
Expand Up @@ -214,25 +214,40 @@ namespace openshot {
/// Look up an effect by ID
openshot::EffectBase* GetEffect(const std::string& id);

/// @brief Get an openshot::Frame object for a specific frame number of this timeline. The image size and number
/// @brief Get an openshot::Frame object for a specific frame number of this clip. The image size and number
/// of samples match the source reader.
///
/// @returns A new openshot::Frame object
/// @param frame_number The frame number (starting at 1) of the clip or effect on the timeline.
/// @param frame_number The frame number (starting at 1) of the clip
std::shared_ptr<openshot::Frame> GetFrame(int64_t frame_number) override;

/// @brief Get an openshot::Frame object for a specific frame number of this timeline. The image size and number
/// of samples can be customized to match the Timeline, or any custom output. Extra samples will be moved to the
/// next Frame. Missing samples will be moved from the next Frame.
/// @brief Get an openshot::Frame object for a specific frame number of this clip. The image size and number
/// of samples match the background_frame passed in and the timeline (if available).
///
/// A new openshot::Frame objects is returned, based on a copy from the source image, with all keyframes and clip effects
/// rendered.
/// rendered/rasterized.
///
/// @returns The modified openshot::Frame object
/// @param background_frame The frame object to use as a background canvas (i.e. an existing Timeline openshot::Frame instance)
/// @param frame_number The frame number (starting at 1) of the clip or effect on the timeline.
/// @param frame_number The frame number (starting at 1) of the clip. The image size and number
/// of samples match the background_frame passed in and the timeline (if available)
std::shared_ptr<openshot::Frame> GetFrame(std::shared_ptr<openshot::Frame> background_frame, int64_t frame_number);

/// @brief Get an openshot::Frame object for a specific frame number of this clip. The image size and number
/// of samples match the background_frame passed in and the timeline (if available).
///
/// A new openshot::Frame objects is returned, based on a copy from the source image, with all keyframes and clip effects
/// rendered/rasterized.
///
/// @returns The modified openshot::Frame object
/// @param background_frame The frame object to use as a background canvas (i.e. an existing Timeline openshot::Frame instance)
/// @param frame_number The frame number (starting at 1) of the clip on the timeline. The image size and number
/// of samples match the timeline.
/// @param options The openshot::TimelineInfoStruct pointer, with more details about this specific timeline clip,
/// such as, if it's a top clip, and what the absolute timeline frame number is. This info is used to apply global
/// transitions and masks, if needed.
std::shared_ptr<openshot::Frame> GetFrame(std::shared_ptr<openshot::Frame> background_frame, int64_t frame_number, openshot::TimelineInfoStruct* options);

/// Open the internal reader
void Open() override;

Expand Down
20 changes: 10 additions & 10 deletions src/Timeline.cpp
View file
Expand Up @@ -444,7 +444,7 @@ std::shared_ptr<Frame> Timeline::apply_effects(std::shared_ptr<Frame> frame, int
}

// Get or generate a blank frame
std::shared_ptr<Frame> Timeline::GetOrCreateFrame(std::shared_ptr<Frame> background_frame, Clip* clip, int64_t number)
std::shared_ptr<Frame> Timeline::GetOrCreateFrame(std::shared_ptr<Frame> background_frame, Clip* clip, int64_t number, openshot::TimelineInfoStruct* options)
{
std::shared_ptr<Frame> new_frame;

Expand All @@ -456,7 +456,7 @@ std::shared_ptr<Frame> Timeline::GetOrCreateFrame(std::shared_ptr<Frame> backgro
ZmqLogger::Instance()->AppendDebugMethod("Timeline::GetOrCreateFrame (from reader)", "number", number, "samples_in_frame", samples_in_frame);

// Attempt to get a frame (but this could fail if a reader has just been closed)
new_frame = std::shared_ptr<Frame>(clip->GetFrame(background_frame, number));
new_frame = std::shared_ptr<Frame>(clip->GetFrame(background_frame, number, options));

// Return real frame
return new_frame;
Expand All @@ -477,9 +477,15 @@ std::shared_ptr<Frame> Timeline::GetOrCreateFrame(std::shared_ptr<Frame> backgro
// Process a new layer of video or audio
void Timeline::add_layer(std::shared_ptr<Frame> new_frame, Clip* source_clip, int64_t clip_frame_number, int64_t timeline_frame_number, bool is_top_clip, float max_volume)
{
// Get the clip's frame, composited on top of the current timeline frame
// Create timeline options (with details about this current frame request)
TimelineInfoStruct* options = new TimelineInfoStruct();
options->timeline_frame_number = timeline_frame_number;
options->is_top_clip = is_top_clip;

// Get the clip's frame, composited on top of the current timeline frame
std::shared_ptr<Frame> source_frame;
source_frame = GetOrCreateFrame(new_frame, source_clip, clip_frame_number);
source_frame = GetOrCreateFrame(new_frame, source_clip, clip_frame_number, options);
delete options;

// No frame found... so bail
if (!source_frame)
Expand All @@ -488,12 +494,6 @@ void Timeline::add_layer(std::shared_ptr<Frame> new_frame, Clip* source_clip, in
// Debug output
ZmqLogger::Instance()->AppendDebugMethod("Timeline::add_layer", "new_frame->number", new_frame->number, "clip_frame_number", clip_frame_number, "timeline_frame_number", timeline_frame_number);

/* Apply effects to the source frame (if any). If multiple clips are overlapping, only process the
* effects on the top clip. */
if (is_top_clip) {
source_frame = apply_effects(source_frame, timeline_frame_number, source_clip->Layer());
}

/* COPY AUDIO - with correct volume */
if (source_clip->Reader()->info.has_audio) {
// Debug output
Expand Down
8 changes: 4 additions & 4 deletions src/Timeline.h
View file
Expand Up @@ -198,10 +198,7 @@ namespace openshot {
std::vector<openshot::Clip*> find_intersecting_clips(int64_t requested_frame, int number_of_frames, bool include);

/// Get a clip's frame or generate a blank frame
std::shared_ptr<openshot::Frame> GetOrCreateFrame(std::shared_ptr<Frame> background_frame, openshot::Clip* clip, int64_t number);

/// Apply effects to the source frame (if any)
std::shared_ptr<openshot::Frame> apply_effects(std::shared_ptr<openshot::Frame> frame, int64_t timeline_frame_number, int layer);
std::shared_ptr<openshot::Frame> GetOrCreateFrame(std::shared_ptr<Frame> background_frame, openshot::Clip* clip, int64_t number, openshot::TimelineInfoStruct* options);

/// Compare 2 floating point numbers for equality
bool isEqual(double a, double b);
Expand Down Expand Up @@ -249,6 +246,9 @@ namespace openshot {
/// @param effect Add an effect to the timeline. An effect can modify the audio or video of an openshot::Frame.
void AddEffect(openshot::EffectBase* effect);

/// Apply global/timeline effects to the source frame (if any)
std::shared_ptr<openshot::Frame> apply_effects(std::shared_ptr<openshot::Frame> frame, int64_t timeline_frame_number, int layer);

/// Apply the timeline's framerate and samplerate to all clips
void ApplyMapperToClips();

Expand Down
17 changes: 17 additions & 0 deletions src/TimelineBase.h
View file
Expand Up @@ -31,8 +31,25 @@
#ifndef OPENSHOT_TIMELINE_BASE_H
#define OPENSHOT_TIMELINE_BASE_H

#include <cstdint>


namespace openshot {
/**
* @brief This struct contains info about the current Timeline clip instance
*
* When the Timeline requests an openshot::Frame instance from a Clip, it passes
* this struct along, with some additional details from the Timeline, such as the absolute
* timeline frame number requested, and if this clip is above or below overlapping clips, etc...
* This info can help determine if a Clip should apply global effects from the Timeline, such
* as a global Transition/Mask effect.
*/
struct TimelineInfoStruct
{
int64_t timeline_frame_number; ///< The absolute frame number from the timeline we are requesting
bool is_top_clip; ///< Is clip on top (if overlapping another clip)
};

/**
* @brief This class represents a timeline (used for building generic timeline implementations)
*/
Expand Down

0 comments on commit 441cb18

Please sign in to comment.