Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timestamps class v2 #69

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
01e202e
Add timestamps class
moi15moi Jan 8, 2023
f9ea3c5
Remove format in the method frames_to_ms
moi15moi Jan 14, 2023
933e7fa
[Timestamps] Correct set numerator for from_fps
moi15moi Jan 14, 2023
7a9a010
Update __init__.py
moi15moi Jan 14, 2023
05cae97
[timestamps] Format with Black
moi15moi Jan 14, 2023
e20a23c
Add test for timestamps
moi15moi Jan 14, 2023
d848757
Add data timestamps file
moi15moi Jan 14, 2023
aab9981
[Timestamps] Clean import
moi15moi Jan 14, 2023
7b90aa4
Update shift and make_time with new class Timestamps
moi15moi Jan 15, 2023
0bc224c
Add video test for timestamps
tkarabela Jan 29, 2023
e93c453
Timestamps fixes
tkarabela Jan 29, 2023
f637db8
Timestamps - fix mypy and Python 3.7 support
tkarabela Jan 29, 2023
84a5b55
[Timestamps\ms_to_frames] Remove default value for time_type
moi15moi Mar 5, 2023
a3375a0
[Timestamps\frames_to_ms] Remove default value for time_type
moi15moi Mar 5, 2023
3a17394
[microdvd] Remove unnecessary comment
moi15moi Mar 5, 2023
0b9d3f3
[tests\test_timestamps] Add time_type to test
moi15moi Mar 5, 2023
1ded5b0
[tests\test_time] Add time_type to test
moi15moi Mar 5, 2023
c8e2faf
[ssa_event] Correct test for shifting and raise correctly an error wh…
moi15moi Mar 5, 2023
13ff899
[test_ssafile\test_shift] Add an event to be able to test properly
moi15moi Mar 5, 2023
95385dc
[timestamps\TimeType] Add documentation
moi15moi Mar 5, 2023
b99d2cd
[.gitignore] Add vscode folder
moi15moi Mar 25, 2023
e32c7e5
[timestamps] Use simpler algorithm for ms_to_frames and frames_to_ms.…
moi15moi Mar 25, 2023
b87a05a
[tests\data\timestamps\xxx.ass] Remove negative timecode tests
moi15moi Mar 25, 2023
28e681a
[tests\test_timestamps] Add some tests
moi15moi Mar 25, 2023
fed590e
[timestamps] Replace round by math.floor + 0.5
moi15moi Mar 25, 2023
7d6e1e2
[timestamps] ms_to_frames use an proved algorithm
moi15moi Mar 26, 2023
4f40876
[test_time] Handle negative ms/frame
moi15moi Mar 26, 2023
f265d00
[ssaevent] If the shift create negative ms, make it 0
moi15moi Mar 26, 2023
3490eaf
[test_cli] Correct test value
moi15moi Mar 26, 2023
ecb1e16
[microdvd] Moved from_fps to be able to raise correctly error
moi15moi Mar 26, 2023
c97aa49
[test_microdvd] Correct wrong test end value
moi15moi Mar 26, 2023
dfe9594
Update Proof algorithm - ms_to_frames.md
moi15moi Mar 27, 2023
3580e8f
[timestamps] Round the pts_time
moi15moi May 3, 2023
ab3295a
Merge remote-tracking branch 'upstream/master' into Add-timestamps-cl…
moi15moi May 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
sudo apt update && sudo apt install ffmpeg
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Typecheck with mypy
Expand Down
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -66,4 +66,7 @@ target/
temp/

# mypy
.mypy_cache/
.mypy_cache/

# vscode
.vscode/
64 changes: 64 additions & 0 deletions docs/Proof algorithm - ms_to_frames.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# ms_to_frames - Explanation

To convert an frame to an ms, here is the formula: $$ms = round(frame * {denominator \over numerator})$$

Important to note, here the rounding method round up, so, if it encounter $round(x.5)$, it will become $x + 1$

From the previous equation, we can deduce this:
$$ms - 0.5 \le frame * {denominator \over numerator} < ms + 0.5$$

And from the previous inequation, we can isolate $frame$ like this:
$$(ms - 0.5) * {numerator \over denominator} \le frame < (ms + 0.5) * {numerator \over denominator}$$

But, don't forget, $frame \in \mathbb{N}$. This means we need to take the **integer** between the 2 bounds.

If there is no integer, this means that the 2 bounds are the same frame, so we can take one of the 2 bounds and floor it.

## Example of frame corresponding ms
```math
\begin{gather}
fps = 24000/1001 \\
Frame_0 : [0, 42[ ms \\
Frame_1 : [42, 83[ ms \\
Frame_2 : [83, 125[ ms \\
Frame_3 : [125, 167[ ms
\end{gather}
```

## Example where the 2 bounds are the same frame
```math
\begin{gather}
ms = 82 \\
numerator = 24000/1001 \\
denominator = 1000 \\
1.95404 \le frame < 1.97802
\end{gather}
```
So, for $ms = 82$, the $frame = 1$

## Example where the 2 bounds aren't the same frame
```math
\begin{gather}
ms = 83 \\
numerator = 24000/1001 \\
denominator = 1000 \\
1.97802 \le frame < 2.002
\end{gather}
```
So, for $ms = 83$, the $frame = 2$

We need an algorithm to do that.
There is probably many ways how to do that, but here is what i think is the easiest:
```py
# We use the upper bound
upper_bound = (ms + 0.5) * numerator / denominator
# Then, we trunc the result
trunc_frame = int(upper_bound)

# If the upper_bound equals to the trunc_frame, this means that we don't respect the inequation because it is "greater than", not "greater than or equals".
# So if it happens, this means we need to return the previous frame
if upper_bound == trunc_frame:
return trunc_frame - 1
else:
return trunc_frame
```
1 change: 1 addition & 0 deletions pysubs2/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
from . import time, formats, cli, whisper
from .exceptions import *
from .common import Color, Alignment, VERSION
from .timestamps import Timestamps, TimeType

#: Alias for :meth:`SSAFile.load()`.
load = SSAFile.load
Expand Down
30 changes: 21 additions & 9 deletions pysubs2/microdvd.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
from functools import partial
from numbers import Real
from typing import Union
import re
from .exceptions import UnknownFPSError
from .ssaevent import SSAEvent
from .ssastyle import SSAStyle
from .formatbase import FormatBase
from .substation import parse_tags
from .time import ms_to_frames, frames_to_ms
from .timestamps import Timestamps, TimeType

#: Matches a MicroDVD line.
MICRODVD_LINE = re.compile(r" *\{ *(\d+) *\} *\{ *(\d+) *\}(.+)")
Expand All @@ -20,8 +21,13 @@ def guess_format(cls, text):
return "microdvd"

@classmethod
def from_file(cls, subs, fp, format_, fps=None, **kwargs):
def from_file(cls, subs, fp, format_, fps:Union[None,Real,Timestamps] = None, **kwargs):
"""See :meth:`pysubs2.formats.FormatBase.from_file()`"""
if isinstance(fps, Real):
timestamps = Timestamps.from_fps(fps)
elif isinstance(fps, Timestamps):
timestamps = fps

for line in fp:
match = MICRODVD_LINE.match(line)
if not match:
Expand All @@ -35,15 +41,17 @@ def from_file(cls, subs, fp, format_, fps=None, **kwargs):
# it as text of the first subtitle. In that case, we skip
# this auxiliary subtitle and proceed with reading.
try:
fps = float(text)
fps = float(text) # type: ignore[assignment]
subs.fps = fps
continue
except ValueError:
raise UnknownFPSError("Framerate was not specified and "
"cannot be read from "
"the MicroDVD file.")
timestamps = Timestamps.from_fps(fps) # type: ignore[arg-type]
continue

start, end = map(partial(frames_to_ms, fps=fps), (fstart, fend))
start = timestamps.frames_to_ms(fstart, TimeType.START)
end = timestamps.frames_to_ms(fend, TimeType.END)

def prepare_text(text):
text = text.replace("|", r"\N")
Expand All @@ -63,7 +71,7 @@ def style_replacer(match: re.Match) -> str:
subs.append(ev)

@classmethod
def to_file(cls, subs, fp, format_, fps=None, write_fps_declaration=True, apply_styles=True, **kwargs):
def to_file(cls, subs, fp, format_, fps:Union[None,Real,Timestamps] = None, write_fps_declaration=True, apply_styles=True, **kwargs):
"""
See :meth:`pysubs2.formats.FormatBase.to_file()`

Expand All @@ -80,7 +88,10 @@ def to_file(cls, subs, fp, format_, fps=None, write_fps_declaration=True, apply_

if fps is None:
raise UnknownFPSError("Framerate must be specified when writing MicroDVD.")
to_frames = partial(ms_to_frames, fps=fps)
elif isinstance(fps, Real):
timestamps = Timestamps.from_fps(fps)
elif isinstance(fps, Timestamps):
timestamps = fps

def is_entirely_italic(line: SSAEvent) -> bool:
style = subs.styles.get(line.style, SSAStyle.DEFAULT_STYLE)
Expand All @@ -104,7 +115,8 @@ def is_entirely_italic(line: SSAEvent) -> bool:
if apply_styles and is_entirely_italic(line):
text = "{Y:i}" + text

start, end = map(to_frames, (line.start, line.end))
start = timestamps.ms_to_frames(line.start, TimeType.START)
end = timestamps.ms_to_frames(line.end, TimeType.END)

# XXX warn on underflow?
if start < 0: start = 0
Expand Down
42 changes: 36 additions & 6 deletions pysubs2/ssaevent.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
import dataclasses
import re
import warnings
from typing import Optional, Dict, Any, ClassVar
import dataclasses
from numbers import Real
from typing import Optional, Dict, Any, ClassVar, Union

from .common import IntOrFloat
from .time import ms_to_str, make_time
from .timestamps import Timestamps, TimeType


@dataclasses.dataclass(repr=False, eq=False, order=False)
Expand Down Expand Up @@ -106,16 +108,44 @@ def plaintext(self, text: str):
self.text = text.replace("\n", r"\N")

def shift(self, h: IntOrFloat=0, m: IntOrFloat=0, s: IntOrFloat=0, ms: IntOrFloat=0,
frames: Optional[int]=None, fps: Optional[float]=None):
frames: Optional[int]=None, fps: Optional[Union[Real,Timestamps]]=None):
"""
Shift start and end times.

See :meth:`SSAFile.shift()` for full description.

"""
delta = make_time(h=h, m=m, s=s, ms=ms, frames=frames, fps=fps)
self.start += delta
self.end += delta
if frames is None and fps is None:
delta = make_time(h=h, m=m, s=s, ms=ms)
self.start += delta
self.start = max(self.start, 0)
self.end += delta
self.end = max(self.end, 0)
elif frames is None or fps is None:
raise ValueError("Both fps and frames must be specified")
else:
if isinstance(fps, Real):
timestamps = Timestamps.from_fps(fps)
elif isinstance(fps, Timestamps):
timestamps = fps
else:
raise TypeError("Unexpected type for fps")

start_frame = timestamps.ms_to_frames(self.start, TimeType.START)
end_frame = timestamps.ms_to_frames(self.end, TimeType.END)

start_frame += frames
end_frame += frames

try:
self.start = timestamps.frames_to_ms(start_frame, TimeType.START)
except ValueError:
self.start = 0

try:
self.end = timestamps.frames_to_ms(end_frame, TimeType.END)
except ValueError:
self.end = 0

def copy(self) -> "SSAEvent":
"""Return a copy of the SSAEvent."""
Expand Down
11 changes: 6 additions & 5 deletions pysubs2/ssafile.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,19 @@
import io
from io import open
from itertools import chain
from numbers import Real
import os.path
import logging
from typing import Optional, List, Dict, Iterable, Any, overload, Iterator
from typing import Optional, List, Dict, Iterable, Any, Union, overload, Iterator


from .common import IntOrFloat
from .formats import autodetect_format, get_format_class, get_format_identifier
from .substation import is_valid_field_content
from .ssaevent import SSAEvent
from .ssastyle import SSAStyle
from .time import make_time, ms_to_str
from .timestamps import Timestamps


class SSAFile(MutableSequence):
Expand Down Expand Up @@ -245,7 +248,7 @@ def to_file(self, fp: io.TextIOBase, format_: str, fps: Optional[float]=None, **
# ------------------------------------------------------------------------

def shift(self, h: IntOrFloat=0, m: IntOrFloat=0, s: IntOrFloat=0, ms: IntOrFloat=0,
frames: Optional[int]=None, fps: Optional[float]=None):
frames: Optional[int]=None, fps: Optional[Union[Real,Timestamps]]=None):
"""
Shift all subtitles by constant time amount.

Expand All @@ -262,10 +265,8 @@ def shift(self, h: IntOrFloat=0, m: IntOrFloat=0, s: IntOrFloat=0, ms: IntOrFloa
ValueError: Invalid fps or missing number of frames.

"""
delta = make_time(h=h, m=m, s=s, ms=ms, frames=frames, fps=fps)
for line in self:
line.start += delta
line.end += delta
line.shift(h=h, m=m, s=s, ms=ms, frames=frames, fps=fps)

def transform_framerate(self, in_fps: float, out_fps: float):
"""
Expand Down
47 changes: 24 additions & 23 deletions pysubs2/time.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
from collections import namedtuple
from numbers import Real
import re
from typing import Optional, List, Tuple, Sequence
from typing import Optional, Tuple, Sequence, Union, cast
from pysubs2.common import IntOrFloat
from .timestamps import Timestamps, TimeType

#: Pattern that matches both SubStation and SubRip timestamps.
TIMESTAMP = re.compile(r"(\d{1,2}):(\d{1,2}):(\d{1,2})[.,](\d{1,3})")
Expand All @@ -13,7 +15,7 @@


def make_time(h: IntOrFloat=0, m: IntOrFloat=0, s: IntOrFloat=0, ms: IntOrFloat=0,
frames: Optional[int]=None, fps: Optional[float]=None):
frames: Optional[int]=None, fps: Optional[Union[Real,Timestamps]]=None, time_type: Optional[TimeType] = None):
"""
Convert time to milliseconds.

Expand All @@ -30,10 +32,15 @@ def make_time(h: IntOrFloat=0, m: IntOrFloat=0, s: IntOrFloat=0, ms: IntOrFloat=
2000

"""
if frames is None and fps is None:
if frames is None and fps is None and time_type is None:
return times_to_ms(h, m, s, ms)
elif frames is not None and fps is not None:
return frames_to_ms(frames, fps)
elif frames is not None and fps is not None and time_type is not None:
if isinstance(fps, Real):
timestamps = Timestamps.from_fps(fps)
elif isinstance(fps, Timestamps):
timestamps = fps

return timestamps.frames_to_ms(frames, time_type)
else:
raise ValueError("Both fps and frames must be specified")

Expand Down Expand Up @@ -82,46 +89,40 @@ def times_to_ms(h: IntOrFloat=0, m: IntOrFloat=0, s: IntOrFloat=0, ms: IntOrFloa
return int(round(ms))


def frames_to_ms(frames: int, fps: float) -> int:
def frames_to_ms(frames: int, fps: float, time_type: TimeType) -> int:
"""
Convert frame-based duration to milliseconds.

Arguments:
frames: Number of frames (should be int).
fps: Framerate (must be a positive number, eg. 23.976).

Returns:
Number of milliseconds (rounded to int).

Raises:
ValueError: fps was negative or zero.

"""
if fps <= 0:
raise ValueError(f"Framerate must be a positive number ({fps}).")

return int(round(frames * (1000 / fps)))
"""
return Timestamps.from_fps(cast(Real, fps)).frames_to_ms(frames, time_type)


def ms_to_frames(ms: IntOrFloat, fps: float) -> int:
def ms_to_frames(ms: IntOrFloat, fps: float, time_type: TimeType) -> int:
"""
Convert milliseconds to number of frames.

Arguments:
ms: Number of milliseconds (may be int, float or other numeric class).
fps: Framerate (must be a positive number, eg. 23.976).

Returns:
Number of frames (int).

Raises:
ValueError: fps was negative or zero.

"""
if fps <= 0:
raise ValueError(f"Framerate must be a positive number ({fps}).")

return int(round((ms / 1000) * fps))
"""
return Timestamps.from_fps(cast(Real, fps)).ms_to_frames(int(ms), time_type)


def ms_to_times(ms: IntOrFloat) -> Tuple[int, int, int, int]:
Expand Down
Loading