Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add SRFolderMultipleGTDataset #355

Merged
merged 5 commits into from
Jun 15, 2021

Conversation

ckkelvinchan
Copy link
Member

@ckkelvinchan ckkelvinchan commented Jun 4, 2021

Motivation

Currently, MMEditing does not support training video SR models with custom datasets. Users can only trained on pre-determined datasets such as REDS and Vimeo-90K.

Modification

This PR adds support to training recurrent networks (e.g. BasicVSR, IconVSR) on custom datasets. Changes are as follows:

  1. Add SRFolderMultipleGTDataset, which allows users to train and test on their own datasets.
  2. SRTestMultipleGTDataset is deprecated. All existing uses of SRTestMultipleGTDataset are replaced with SRFolderMultipleGTDataset, but we still keep SRTestMultipleGTDataset to avoid any unexpected compatibility issue. It can be removed in the future.

@codecov
Copy link

codecov bot commented Jun 4, 2021

Codecov Report

Merging #355 (6dc27f6) into master (7622cf8) will increase coverage by 0.32%.
The diff coverage is 92.25%.

❗ Current head 6dc27f6 differs from pull request most recent head b39420b. Consider uploading reports for the commit b39420b to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master     #355      +/-   ##
==========================================
+ Coverage   80.11%   80.44%   +0.32%     
==========================================
  Files         181      183       +2     
  Lines        9496     9760     +264     
  Branches     1385     1417      +32     
==========================================
+ Hits         7608     7851     +243     
- Misses       1698     1717      +19     
- Partials      190      192       +2     
Flag Coverage Δ
unittests 80.42% <92.25%> (+0.32%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmedit/apis/train.py 18.00% <5.26%> (-2.99%) ⬇️
mmedit/models/backbones/sr_backbones/dic_net.py 98.20% <98.20%> (ø)
mmedit/datasets/__init__.py 100.00% <100.00%> (ø)
mmedit/datasets/sr_folder_multiple_gt_dataset.py 100.00% <100.00%> (ø)
mmedit/datasets/sr_test_multiple_gt_dataset.py 100.00% <100.00%> (ø)
mmedit/models/__init__.py 100.00% <100.00%> (ø)
mmedit/models/backbones/__init__.py 100.00% <100.00%> (ø)
mmedit/models/backbones/sr_backbones/__init__.py 100.00% <100.00%> (ø)
mmedit/models/extractors/__init__.py 100.00% <100.00%> (ø)
mmedit/models/extractors/feedback_hour_glass.py 100.00% <100.00%> (ø)
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7622cf8...b39420b. Read the comment docs.

@innerlee
Copy link
Contributor

innerlee commented Jun 4, 2021

Can the custom be avoided in the class name?

@ckkelvinchan
Copy link
Member Author

Not sure if SRMultipleGTDataset would be ambiguous, as there are different multiple-gt datasets in MMEditing.

@innerlee
Copy link
Contributor

innerlee commented Jun 4, 2021

Custom does not add any meanful info.

@ckkelvinchan
Copy link
Member Author

Custom does not add any meanful info.

How about SRMultipleGTFolderDataset? It corresponds to SRFolderDataset for image.

After this PR, there will be another named SRVideoFolderDataset, which is for sliding-window framework.


@DATASETS.register_module()
class SRCustomMultipleGTDataset(BaseSRDataset):
"""Custom dataset for video super resolution for recurrent networks.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docstring failed to explain Multiple.
Besides, This sentence is a bit difficult to understand (... for video super resolution for recurrent networks.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is explained in line 15: The dataset loads several LQ (Low-Quality) frames and GT (Ground-Truth) frames.. There are multiple GT frames.

pipeline (list[dict | callable]): A sequence of data transformations.
scale (int): Upsampling scale ratio.
num_input_frames (None | int): The number of frames per iteration.
If None, the whole clip are extracted. If it is an integer
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If None, the whole clip will be extracted.

pipeline (list[dict | callable]): A sequence of data transformations.
scale (int): Upsampling scale ratio.
num_input_frames (None | int): The number of frames per iteration.
If None, the whole clip are extracted. If it is an integer
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please delete an integer.

gt_path=self.gt_folder,
key=sequence.replace(f'{self.lq_folder}/', ''),
num_input_frames=num_input_frames,
sequence_length=int(sequence_length)))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sequence_length = len(glob.glob(osp.join(sequence, '*.png')))
len(__obj: Sized) -> int
No need to use int(sequence_length)

@ckkelvinchan ckkelvinchan changed the title [Feature] Add SRCutomMultipleGTDataset [Feature] Add SRFolderMultipleGTDataset Jun 14, 2021
@@ -27,6 +28,10 @@ class SRTestMultipleGTDataset(BaseSRDataset):
def __init__(self, lq_folder, gt_folder, pipeline, scale, test_mode=True):
super().__init__(pipeline, scale, test_mode)

warnings.warn('"SRTestMultipleGTDataset" have been deprecated and '
'will be removed in future release. Please use '
'"SRFolderMultipleGTDataset" instead.')
Copy link
Contributor

@innerlee innerlee Jun 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Details see https://github.com/open-mmlab/mmediting/pull/355

@innerlee innerlee merged commit 089460c into open-mmlab:master Jun 15, 2021
@ckkelvinchan ckkelvinchan deleted the custom_video branch July 8, 2021 03:12
Yshuo-Li pushed a commit to Yshuo-Li/mmediting that referenced this pull request Jul 15, 2022
* Add SRCutomMultipleGTDataset

* Rename and minor updates

* Fix

* Add dataset

* Modify warnings for SRTestMultipleGTDataset
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants