Moco v2 #2757

lezhang0912 · 2023-04-14T14:07:24Z

添加moco_v2, moco_clas模型

…it_net

paddle-bot · 2023-04-14T14:07:35Z

Thanks for your contribution!

lezhang0912 · 2023-04-25T13:58:10Z

moco-v2-clas复现精度67.6，和原paper精度一致

lezhang0912 · 2023-05-17T11:34:32Z

@TingquanGao 请review代码

ppcls/arch/backbone/legendary_models/resnet.py

ppcls/configs/ImageNet/MoCo/MoCoV2_r50.yaml

ppcls/configs/ImageNet/MoCo/MoCo_clas.yaml

ppcls/arch/init_weight.py

ppcls/data/dataloader/imagenet_dataset.py

ppcls/engine/engine.py

ppcls/engine/train/train_iter_two_samples.py

ppcls/arch/backbone/legendary_models/resnet.py

TingquanGao · 2023-05-20T06:02:59Z

代码提交前用pre-commit了么？主要是编码风格。

lezhang0912 · 2023-05-20T07:33:03Z

代码提交前pre-commit全部pass

…ImageNetDataset

lezhang0912 · 2023-05-20T09:22:50Z

@TingquanGao 已经按照上述建议完成项目修改，请再次review

ppcls/configs/ImageNet/MoCo/MoCoV2_r50.yaml

ppcls/configs/ImageNet/MoCo/MoCo_clas.yaml

ppcls/data/dataloader/moco_imagenet_dataset.py

TingquanGao · 2023-05-29T06:30:00Z

验收通过：

pretrain：
训练命令：python -m paddle.distributed.launch --gpus="0,1,2,3" tools/train.py -c ./ppcls/configs/ImageNet/MoCo/MoCoV2_r50.yaml
结果：
[2023/05/24 19:32:07] ppcls INFO: [Train][Epoch 200/200][Avg]top1: 0.87063, top5: 0.94182, CELoss: 6.55722, loss: 6.55722

fine-tune：
训练命令：python -m paddle.distributed.launch --gpus="0,1,2,3" tools/train.py -c ./ppcls/configs/ImageNet/MoCo/MoCo_clas.yaml
结果：
[2023/05/25 10:37:34] ppcls INFO: [Eval][Epoch 100][best metric: 0.6761819124221802]

TingquanGao · 2023-05-29T06:33:15Z

ppcls/data/preprocess/ops/randaugment.py

这个文件好像不需要修改？

好的，已恢复原来更改 8cf3bc2

TingquanGao · 2023-06-01T10:51:21Z

ppcls/data/preprocess/__init__.py

@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

+from paddle.vision.transforms import ToTensor, Normalize, RandomHorizontalFlip, RandomResizedCrop, Transpose


Transpose 这个好像并没有用到？

已改，ea714d2

ppcls/data/preprocess/ops/operators.py

TingquanGao · 2023-06-01T10:53:11Z

ppcls/configs/ImageNet/MoCo/MoCoV2_r50.yaml

+
+# model architecture
+Arch:
+  name: moco_v2


这样吧：
moco_v2 -> MoCoV2

ppcls/configs/ImageNet/MoCo/MoCo_clas.yaml

TingquanGao · 2023-06-01T10:54:12Z

ppcls/arch/backbone/model_zoo/moco.py

+    return model
+
+
+def moco_v2(backbone, neck, head, pretrained=False, use_ssld=False):


moco_v2 -> MoCoV2

TingquanGao · 2023-06-01T10:54:19Z

ppcls/arch/backbone/model_zoo/moco.py

+            layer._use_global_stats = True
+
+
+def moco_v1(backbone, neck, head, pretrained=False, use_ssld=False):


moco_v1 -> MoCoV1

TingquanGao · 2023-06-01T10:54:42Z

ppcls/arch/backbone/model_zoo/moco_clas.py

+            param.trainable = False
+
+
+def moco_clas(backbone, head, pretrained=False, use_ssld=False):


moco_clas -> MoCo_finetune

已改，b2fe78a

TingquanGao · 2023-06-01T11:07:51Z

ppcls/data/preprocess/ops/operators.py

+class GaussianBlur(object):
+    """Gaussian blur augmentation in SimCLR https://arxiv.org/abs/2002.05709"""
+
+    def __init__(self, sigma=[.1, 2.], _PIL=False):


这里的 _PIL=False 参数，改成 backend="cv2" 吧，和其他预处理算子用法一致。

已改，ea714d2

TingquanGao · 2023-06-01T11:08:15Z

ppcls/data/preprocess/ops/operators.py

+            x = x.filter(ImageFilter.GaussianBlur(radius=sigma))
+            return x
+        else:
+            import cv2


cv2不需要再import了，上面应该import过

已改，ea714d2

ppcls/data/preprocess/ops/operators.py

ppcls/arch/backbone/__init__.py

ppcls/arch/backbone/model_zoo/moco_clas.py

TingquanGao

一点小问题，麻烦再改一下吧。

…ult backbend in GaussianBlur func to cv2

TingquanGao

LGTM

lezhang0912 · 2023-06-02T13:06:40Z

thank you

lezhang0912 added 20 commits March 18, 2023 07:20

add moco-v2 rfc

ec41276

momo-v2 model

9acf327

momo-v2 model modify. merge nech and head to moco.py

4a451eb

moco-v2前向传播完成

abc2a13

moco-v2代码改造

6231407

解决super().init_res is depreacted error, Super().init_res==>super().in…

0009cb6

…it_net

增加了kaiming初始化方式

3b21aaf

修改设计文档，重构ImageNetDataset类别，增加了同时返回两个样本的选项

aa8dd4a

修改设计文档，重构ImageNetDataset类别，增加了同时返回两个样本的选项

b1a888a

增加RandomGraysacle 同时支持输入nd.array和PIL.Image.Image

65eeaca

增加GaussianBlur

1855672

重构ImageDataset类，使之兼容同时返回两个sample(MoCo训练需要)

ea749a6

增加train_iter_two_samples.py文件(MoCo训练需要的train函数)

08818d3

去除moco模型head网络自带的loss函数和acc函数

0b5a960

修改MoCo-V2配置文件

a182dc7

增加moco_clas模型，支持paddleclas项目原生的骨干网络

9a93a23

添加moco_v2 moco_clas yaml配置文件

ea36552

修改moco_clas配置文件

980cd1d

修改moco_clas.py中logger.info bug

e28388e

添加配置 DistributedBatchSampler

986ebaa

paddle-bot bot added contributor status: proposed labels Apr 14, 2023

修改moco_clas.yaml文件

9407722

lezhang0912 mentioned this pull request May 17, 2023

【PaddlePaddle Hackathon 第四期】任务总览 PaddlePaddle/Paddle#51281

Closed

TingquanGao reviewed May 20, 2023

View reviewed changes

lezhang0912 added 8 commits May 20, 2023 15:56

解决super().init_res bug问题，必须将其改为super().init_net， resnet相关model才能正常调用

70f1550

fix remove dali

fee6963

fix unnecessay comments

4185491

新建MoCoImageNetDataset类，还原ImageNetDataset类更改

b6ecdfe

fix MoCoImageNetDataset class super() bug

d29eb81

去除自定义init_weight.py，使用项目tool.initializer.py中对应的初始化方法代替

df2482e

修改moco_clas函数，在arch配置pretrained_model配置项

07a8f01

在MoCo_clas.yaml文件中 arch下增加 pretrained_model配置项，并统一修改dataset name为MoCo…

00e5527

…ImageNetDataset

fix method _load_pretrained() bug

00a83a1

lezhang0912 requested a review from TingquanGao May 20, 2023 09:36

TingquanGao reviewed May 24, 2023

View reviewed changes

ppcls/configs/ImageNet/MoCo/MoCoV2_r50.yaml Outdated Show resolved Hide resolved

ppcls/configs/ImageNet/MoCo/MoCo_clas.yaml Outdated Show resolved Hide resolved

ppcls/data/dataloader/moco_imagenet_dataset.py Outdated Show resolved Hide resolved

lezhang0912 added 2 commits May 27, 2023 14:45

fix ImageNetDataset -> MoCoImageNetDataset

a0a2132

remove seed and distributed in config files

c57e88a

lezhang0912 requested a review from TingquanGao May 27, 2023 06:50

TingquanGao reviewed Jun 1, 2023

View reviewed changes

ppcls/arch/backbone/__init__.py Outdated Show resolved Hide resolved

ppcls/arch/backbone/model_zoo/moco_clas.py Outdated Show resolved Hide resolved

TingquanGao reviewed Jun 1, 2023

View reviewed changes

lezhang0912 added 3 commits June 1, 2023 19:52

fix randaugment.py change, rollback origin state

8cf3bc2

fix -> MoCo_V1, MoCo_V2; -> MoCo_finetune; --moco_finetune.py

b2fe78a

fix removing Tranpose, import cv2 and PIL, besides, changing the defa…

ea714d2

…ult backbend in GaussianBlur func to cv2

lezhang0912 requested a review from TingquanGao June 1, 2023 12:40

TingquanGao approved these changes Jun 2, 2023

View reviewed changes

TingquanGao merged commit 122f7f9 into PaddlePaddle:dev_hackathon4 Jun 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Moco v2 #2757

Moco v2 #2757

lezhang0912 commented Apr 14, 2023

paddle-bot bot commented Apr 14, 2023

lezhang0912 commented Apr 25, 2023

lezhang0912 commented May 17, 2023

TingquanGao commented May 20, 2023

lezhang0912 commented May 20, 2023

lezhang0912 commented May 20, 2023

TingquanGao commented May 29, 2023

TingquanGao May 29, 2023

lezhang0912 Jun 1, 2023 •

edited

TingquanGao Jun 1, 2023

lezhang0912 Jun 1, 2023 •

edited

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023 •

edited

lezhang0912 Jun 1, 2023

TingquanGao Jun 1, 2023

lezhang0912 Jun 1, 2023

TingquanGao Jun 1, 2023

lezhang0912 Jun 1, 2023

TingquanGao left a comment •

edited

TingquanGao left a comment

lezhang0912 commented Jun 2, 2023

		return model


		def moco_v2(backbone, neck, head, pretrained=False, use_ssld=False):

		layer._use_global_stats = True


		def moco_v1(backbone, neck, head, pretrained=False, use_ssld=False):

		param.trainable = False


		def moco_clas(backbone, head, pretrained=False, use_ssld=False):

Moco v2 #2757

Moco v2 #2757

Conversation

lezhang0912 commented Apr 14, 2023

paddle-bot bot commented Apr 14, 2023

lezhang0912 commented Apr 25, 2023

lezhang0912 commented May 17, 2023

TingquanGao commented May 20, 2023

lezhang0912 commented May 20, 2023

lezhang0912 commented May 20, 2023

TingquanGao commented May 29, 2023

Choose a reason for hiding this comment

lezhang0912 Jun 1, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lezhang0912 Jun 1, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TingquanGao Jun 1, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TingquanGao left a comment • edited

Choose a reason for hiding this comment

TingquanGao left a comment

Choose a reason for hiding this comment

lezhang0912 commented Jun 2, 2023

lezhang0912 Jun 1, 2023 •

edited

lezhang0912 Jun 1, 2023 •

edited

TingquanGao Jun 1, 2023 •

edited

TingquanGao left a comment •

edited