Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Byol(190) #2

Open
wants to merge 23 commits into
base: byol
Choose a base branch
from
Open

Byol(190) #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
55538bf
More linear evaluation benchmarks (#376)
QuentinDuval Jul 26, 2021
5f86ae1
Add support for Kinetics 700 automatic download script (#174)
QuentinDuval Jul 28, 2021
8800989
Fix warm up all reduce call (#382)
mannatsingh Jul 30, 2021
6b3add3
Update july news (#383)
prigoyal Aug 2, 2021
474a1d7
correctly rely on config to decide whether or not to flatten the feat…
QuentinDuval Aug 3, 2021
3660267
RegNet256Gf linear eval configs
QuentinDuval Aug 3, 2021
5a622f4
Conversion to sliced checkpoints now supports PathManager (#387)
QuentinDuval Aug 4, 2021
554aa15
FSDP Activation Checkpointing improvements (#176)
QuentinDuval Aug 6, 2021
43f230c
LARS exclude BN: use a flag instead of a function (#388)
jingli9111 Aug 6, 2021
79161e9
Fixed typo on website/pages/en/index.js (#389)
xcastilla Aug 6, 2021
d77d6c8
Instance Retrieval Improvements (#380)
iseessel Aug 9, 2021
b9bb004
Fix gem logic, reverse cropping/transformation order. (#379)
iseessel Aug 9, 2021
b4e6aa4
Optional crop bbx (#378)
iseessel Aug 9, 2021
db63a8f
Refactor instance retrieval optional normalization (#381)
iseessel Aug 9, 2021
61123d4
Added BYOL implementation
pranavsinghps1 Aug 10, 2021
8ac03f2
initialised BYOL hook
pranavsinghps1 Aug 10, 2021
425a106
Added comments and type hints in byol_hooks
pranavsinghps1 Aug 11, 2021
8764ae1
Added comments and type hints in byol_hooks
pranavsinghps1 Aug 11, 2021
5b4b3f6
Added comments and type hints in byol_loss
pranavsinghps1 Aug 11, 2021
2451ccf
Added references from BYOL paper in byol_8node_resnet.yaml
pranavsinghps1 Aug 11, 2021
3a458d2
Minor Changes
pranavsinghps1 Aug 11, 2021
9ff5847
Linter Fixes
pranavsinghps1 Aug 11, 2021
b511be6
Included feedback on comments
pranavsinghps1 Aug 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

## What's New
Below we share, in reverse chronological order, the updates and new releases in VISSL. All VISSL releases are available [here](https://github.com/facebookresearch/vissl/releases).

- **[July 2021]**: [Fully Sharded Data Parallel](https://github.com/facebookresearch/fairscale/pull/413) integrated in VISSL and announced in [blog](https://engineering.fb.com/2021/07/15/open-source/fsdp/).
- **[May 2021]**: [DINO: Emerging Properties in Self-Supervised Vision Transformers](https://arxiv.org/pdf/2104.14294.pdf) code released.
- **[May 2021]**: VISSL relicensed under [MIT License](LICENSE).
- **[May 2021]**: [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://arxiv.org/abs/2103.03230) code released.
Expand Down
26 changes: 26 additions & 0 deletions configs/config/benchmark/linear_image_classification/clevr_count/models/regnet128Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 4
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[3, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[2, 1]]]],
["avgpool", ["Identity", []]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
name: regnet_y_128gf
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 2904, "dims": [8712, 8]}],
["eval_mlp", {"in_channels": 7392, "dims": [14784, 8]}],
["eval_mlp", {"in_channels": 7392, "dims": [7392, 8]}],
["mlp", {"dims": [7392, 8]}],
]
28 changes: 28 additions & 0 deletions configs/config/benchmark/linear_image_classification/clevr_count/models/regnet256Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 3
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[2, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[1, 2]]]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
depth: 27
w_0: 640
w_a: 230.83
w_m: 2.53
group_width: 373
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 4103, "dims": [8206, 8]}],
["eval_mlp", {"in_channels": 10444, "dims": [20888, 8]}],
["eval_mlp", {"in_channels": 10444, "dims": [10444, 8]}],
]
26 changes: 26 additions & 0 deletions configs/config/benchmark/linear_image_classification/clevr_dist/models/regnet128Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 4
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[3, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[2, 1]]]],
["avgpool", ["Identity", []]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
name: regnet_y_128gf
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 2904, "dims": [8712, 6]}],
["eval_mlp", {"in_channels": 7392, "dims": [14784, 6]}],
["eval_mlp", {"in_channels": 7392, "dims": [7392, 6]}],
["mlp", {"dims": [7392, 6]}],
]
28 changes: 28 additions & 0 deletions configs/config/benchmark/linear_image_classification/clevr_dist/models/regnet256Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 3
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[2, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[1, 2]]]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
depth: 27
w_0: 640
w_a: 230.83
w_m: 2.53
group_width: 373
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 4103, "dims": [8206, 6]}],
["eval_mlp", {"in_channels": 10444, "dims": [20888, 6]}],
["eval_mlp", {"in_channels": 10444, "dims": [10444, 6]}],
]
26 changes: 26 additions & 0 deletions configs/config/benchmark/linear_image_classification/dsprites/models/regnet128Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 4
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[3, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[2, 1]]]],
["avgpool", ["Identity", []]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
name: regnet_y_128gf
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 2904, "dims": [8712, 16]}],
["eval_mlp", {"in_channels": 7392, "dims": [14784, 16]}],
["eval_mlp", {"in_channels": 7392, "dims": [7392, 16]}],
["mlp", {"dims": [7392, 16]}],
]
28 changes: 28 additions & 0 deletions configs/config/benchmark/linear_image_classification/dsprites/models/regnet256Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 3
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[2, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[1, 2]]]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
depth: 27
w_0: 640
w_a: 230.83
w_m: 2.53
group_width: 373
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 4103, "dims": [8206, 16]}],
["eval_mlp", {"in_channels": 10444, "dims": [20888, 16]}],
["eval_mlp", {"in_channels": 10444, "dims": [10444, 16]}],
]
26 changes: 26 additions & 0 deletions configs/config/benchmark/linear_image_classification/euro_sat/models/regnet128Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 4
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[3, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[2, 1]]]],
["avgpool", ["Identity", []]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
name: regnet_y_128gf
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 2904, "dims": [8712, 10]}],
["eval_mlp", {"in_channels": 7392, "dims": [14784, 10]}],
["eval_mlp", {"in_channels": 7392, "dims": [7392, 10]}],
["mlp", {"dims": [7392, 10]}],
]
28 changes: 28 additions & 0 deletions configs/config/benchmark/linear_image_classification/euro_sat/models/regnet256Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 3
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[2, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[1, 2]]]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
depth: 27
w_0: 640
w_a: 230.83
w_m: 2.53
group_width: 373
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 4103, "dims": [8206, 10]}],
["eval_mlp", {"in_channels": 10444, "dims": [20888, 10]}],
["eval_mlp", {"in_channels": 10444, "dims": [10444, 10]}],
]
26 changes: 26 additions & 0 deletions configs/config/benchmark/linear_image_classification/gtsrb/models/regnet128Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 4
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[3, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[2, 1]]]],
["avgpool", ["Identity", []]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
name: regnet_y_128gf
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 2904, "dims": [8712, 43]}],
["eval_mlp", {"in_channels": 7392, "dims": [14784, 43]}],
["eval_mlp", {"in_channels": 7392, "dims": [7392, 43]}],
["mlp", {"dims": [7392, 43]}],
]
28 changes: 28 additions & 0 deletions configs/config/benchmark/linear_image_classification/gtsrb/models/regnet256Gf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# @package _global_
config:
METERS:
name: accuracy_list_meter
accuracy_list_meter:
num_meters: 3
topk_values: [1]
MODEL:
FEATURE_EVAL_SETTINGS:
LINEAR_EVAL_FEAT_POOL_OPS_MAP: [
["res4", ["AdaptiveAvgPool2d", [[2, 1]]]],
["res5", ["AdaptiveAvgPool2d", [[1, 2]]]],
["avgpool", ["Identity", []]],
]
TRUNK:
NAME: regnet
REGNET:
depth: 27
w_0: 640
w_a: 230.83
w_m: 2.53
group_width: 373
HEAD:
PARAMS: [
["eval_mlp", {"in_channels": 4103, "dims": [8206, 43]}],
["eval_mlp", {"in_channels": 10444, "dims": [20888, 43]}],
["eval_mlp", {"in_channels": 10444, "dims": [10444, 43]}],
]
Loading