Pretrained models #33

prabhatnagarajan · 2020-07-31T06:42:24Z

Resolves #2

We can see that test_pretrained_models is being run in both the CPU and GPU tests:

2020-07-20 15:59:15.347601 STDOUT 2105] | [SUCCESS] /pfrl/tests/utils_tests/test_pretrained_models.py (17 passed, 15 deselected, 1 warnings in 20.78 seconds)

and

[SUCCESS] /pfrl/tests/utils_tests/test_pretrained_models.py (15 passed, 17 deselected, 1 warnings in 19.12 seconds)

(https://ci.preferred.jp/pfrl.cpu/59865/) and https://ci.preferred.jp/pfrl.gpu/59868/

…o pretrained_models

prabhatnagarajan · 2020-07-31T06:44:12Z

DQN Performance comparison:

Network: Best

Game	PFRL Score	Pretrained Demo Score
AirRaid	6020.3	5755.2
Alien	1976.3	2416.7
Amidar	976.6	176.9
Assault	3542.6	4428.6
Asterix	5715.3	6061.7
Asteroids	1596.0	1407.4
Atlantis	97512.8	3781000.0
BankHeist	663.2	664.4
BattleZone	5144.4	11000.0
BeamRider	7146.6	8066.0
Berzerk	658.8	498.9
Bowling	55.5	55.5
Boxing	89.3	88.9
Breakout	352.5	347.6
Carnival	5249.2	5614.8
Centipede	5058.7	5648.1
ChopperCommand	4737.2	3500.0
CrazyClimber	103234.4	99348.4
DemonAttack	9208.2	9227.7
DoubleDunk	-10.9	-10.0
Enduro	307.4	788.7
FishingDerby	14.3	10.0
Freeway	20.6	30.6
Frostbite	1388.4	831.8
Gopher	7947.2	8907.7
Gravitar	471.9	492.7
Hero	19588.2	19945.5
IceHockey	-2.7	-3.7
Jamesbond	765.8	648.3
JourneyEscape	-1713.3	-1774.7
Kangaroo	8345.6	7705.9
Krull	5679.2	7097.7
KungFuMaster	27362.2	28609.5
MontezumaRevenge	0.6	0.0
MsPacman	2776.4	2727.0
NameThisGame	7279.9	8659.1
Phoenix	9406.4	9938.3
Pitfall	-4.7	-5.8
Pong	20.0	19.5
Pooyan	3446.6	3482.9
PrivateEye	2196.5	334.8
Qbert	10675.1	10785.6
Riverraid	7554.0	7854.7
RoadRunner	36572.8	37285.6
Robotank	47.5	61.0
Seaquest	6252.0	6232.2
Skiing	-12426.3	-12934.0
Solaris	1396.3	5180.0
SpaceInvaders	1609.8	1516.9
StarGunner	57293.9	56108.6
Tennis	-1.8	0.0
TimePilot	5802.8	4909.6
Tutankham	148.2	96.7
UpNDown	11110.7	10255.9
Venture	517.8	382.6
VideoPinball	14376.7	10813.3
WizardOfWor	2202.2	1173.3
YarsRevenge	6602.9	10898.7
Zaxxon	6191.1	6573.1

DQN performance comparison

Network: final

Game	PFRL Score	Pretrained Demo Score
AirRaid	2033.3	5647.0
Alien	1641.9	1768.8
Amidar	780.7	176.5
Assault	1748.8	1481.3
Asterix	856.5	106.7
Asteroids	710.1	1017.3
Atlantis	124120.1	46063.4
BankHeist	495.1	559.8
BattleZone	5693.8	3764.7
BeamRider	8563.3	7585.0
Berzerk	525.9	798.2
Bowling	31.8	30.9
Boxing	76.6	80.1
Breakout	296.7	297.8
Carnival	4825.1	5012.4
Centipede	906.2	518.2
ChopperCommand	3214.2	1933.3
CrazyClimber	93779.0	102579.2
DemonAttack	10979.3	10041.8
DoubleDunk	-14.4	-14.8
Enduro	486.0	518.4
FishingDerby	-17.4	-16.2
Freeway	19.9	30.0
Frostbite	793.5	78.2
Gopher	4309.8	3202.6
Gravitar	382.7	179.5
Hero	12359.2	13817.7
IceHockey	-9.9	-7.9
Jamesbond	612.6	860.2
JourneyEscape	-4079.3	-6418.2
Kangaroo	3487.5	4448.8
Krull	5238.3	5122.8
KungFuMaster	13000.3	9811.9
MontezumaRevenge	0.0	0.0
MsPacman	2315.6	1822.8
NameThisGame	6933.5	7225.2
Phoenix	5500.8	6946.4
Pitfall	-377.2	-1139.0
Pong	19.4	17.7
Pooyan	2382.4	2856.5
PrivateEye	-25.8	-32.2
Qbert	8511.3	3450.4
Riverraid	3739.3	6981.2
RoadRunner	25939.4	37971.2
Robotank	59.0	54.9
Seaquest	4528.7	5918.7
Skiing	-20050.9	-13228.7
Solaris	1486.1	623.8
SpaceInvaders	1226.7	841.1
StarGunner	47830.9	19652.1
Tennis	-7.5	-1.0
TimePilot	2676.6	1180.4
Tutankham	70.1	85.0
UpNDown	4126.0	3019.9
Venture	293.3	308.0
VideoPinball	10737.1	12731.6
WizardOfWor	530.0	506.7
YarsRevenge	2488.0	4254.9
Zaxxon	2309.6	139.0

prabhatnagarajan · 2020-12-10T15:40:06Z

/test

pfn-ci-bot · 2020-12-10T15:40:13Z

Successfully created a job for commit f7ecb95:

Dashboard for commit f7ecb95

prabhatnagarajan · 2020-12-10T16:24:02Z

Algorithm: DDPG
Final network

Environment	PFRL Score	Pretrained Score
HalfCheetah-v2	9750.57	8625.39
Hopper-v2	1577.20	3459.48
Walker2d-v2	2098.46	1909.11
Ant-v2	753.00	847.16
Reacher-v2	-5.64	-5.05
InvertedPendulum-v2	844.17	1000.0
InvertedDoublePendulum-v2	6464.58	9201.52

Best network

Environment	PFRL Score	Pretrained Score
HalfCheetah-v2	10262.97	8719.39
Hopper-v2	3521.07	3509.89
Walker2d-v2	3932.74	2657.8
Ant-v2	1532.35	1546.07
Reacher-v2	-2.97	-3.93
InvertedPendulum-v2	1000.00	1000.0
InvertedDoublePendulum-v2	6558.32	9321.6

prabhatnagarajan · 2020-12-10T16:46:05Z

/test

pfn-ci-bot · 2020-12-10T16:46:11Z

Successfully created a job for commit 09c43e3:

Dashboard for commit 09c43e3

prabhatnagarajan · 2020-12-11T05:31:11Z

/test

pfn-ci-bot · 2020-12-11T05:31:16Z

Successfully created a job for commit 570e467:

Dashboard for commit 570e467

prabhatnagarajan · 2020-12-11T07:28:14Z

/test

pfn-ci-bot · 2020-12-11T07:28:20Z

Successfully created a job for commit 6555e3d:

Dashboard for commit 6555e3d

muupan · 2020-12-14T09:51:32Z

Why is TD3's Walker2d-v2 result blank?

prabhatnagarajan · 2020-12-14T09:56:48Z

Not sure, I ran these results individually so I must have overlooked it. I just ran those pretrained models and updated the results.

pfrl/utils/pretrained_models.py

Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>

prabhatnagarajan · 2020-12-16T04:56:34Z

/test

pfn-ci-bot · 2020-12-16T04:56:40Z

Successfully created a job for commit 719dfce:

Dashboard for commit 719dfce

prabhatnagarajan added 29 commits July 2, 2020 02:01

Updates pretrained_models file

4070fe7

Adds env variable

24510e1

Merge branch 'master' into pretrained_models

fb55db2

Removes the 'Currently unsupported' caveat

61b279e

Updates readmes with --load-pretrained example command

0d347e3

Adds pretrained-type option for a3c

cc2ee3a

Removes exception throw

3fa1013

modifies path

00650c1

Adds Gcloud for model downloads

c496c07

Removes marker

27c8840

Adds A3C tests

bbab734

Applies black

504e194

Removes some of the zips from the tests

7b4ef54

Adds obs and action size

1eeb028

Applies black

ae7fe6b

Adds DQN and IQN tests

5441f98

Adds numpy import

7588799

Adds Rainbow test

28e5def

Adds DDPG tests and applies black

24aa568

Adds other models to download script, adds TRPO test

10192cd

Adds PPO pretrained test

b188403

Adds TD3 tests

3d815c9

Adds SAC tests

b9b2fe3

Uses posixpath

3ae2f88

Applies black

c3e6aa7

Addresses flakes

2e6427a

Applies black

d396f65

Merge branch 'pretrained_models' of https://github.com/pfnet/pfrl int…

9caf054

…o pretrained_models

Merge branch 'master' into pretrained_models

8d7f754

prabhatnagarajan changed the base branch from pretrained_models to master December 10, 2020 15:33

Applies black

09c43e3

Applies isort

570e467

Amends command

6555e3d

prabhatnagarajan marked this pull request as ready for review December 11, 2020 07:28

muupan self-requested a review December 14, 2020 10:09

muupan requested changes Dec 16, 2020

View reviewed changes

pfrl/utils/pretrained_models.py Outdated Show resolved Hide resolved

pfrl/utils/pretrained_models.py Outdated Show resolved Hide resolved

pfrl/utils/pretrained_models.py Outdated Show resolved Hide resolved

prabhatnagarajan and others added 3 commits December 16, 2020 12:14

Update pfrl/utils/pretrained_models.py

be0cf29

Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>

Update pfrl/utils/pretrained_models.py

d273d1e

Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>

Moves fork reference to the docstrings

719dfce

prabhatnagarajan requested a review from muupan December 16, 2020 04:56

muupan approved these changes Dec 16, 2020

View reviewed changes

muupan merged commit 4e45b8d into pfnet:master Dec 16, 2020

prabhatnagarajan deleted the pretrained_models branch December 16, 2020 11:09

muupan added enhancement New feature or request example Modification or addition of examples labels Dec 16, 2020

muupan added this to the v0.2.0 milestone Dec 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pretrained models #33

Pretrained models #33

prabhatnagarajan commented Jul 31, 2020 •

edited

Loading

prabhatnagarajan commented Jul 31, 2020

prabhatnagarajan commented Dec 10, 2020

pfn-ci-bot commented Dec 10, 2020

prabhatnagarajan commented Dec 10, 2020 •

edited

Loading

prabhatnagarajan commented Dec 10, 2020

pfn-ci-bot commented Dec 10, 2020

prabhatnagarajan commented Dec 11, 2020

pfn-ci-bot commented Dec 11, 2020

prabhatnagarajan commented Dec 11, 2020

pfn-ci-bot commented Dec 11, 2020

muupan commented Dec 14, 2020

prabhatnagarajan commented Dec 14, 2020

prabhatnagarajan commented Dec 16, 2020

pfn-ci-bot commented Dec 16, 2020

Pretrained models #33

Pretrained models #33

Conversation

prabhatnagarajan commented Jul 31, 2020 • edited Loading

prabhatnagarajan commented Jul 31, 2020

prabhatnagarajan commented Dec 10, 2020

pfn-ci-bot commented Dec 10, 2020

prabhatnagarajan commented Dec 10, 2020 • edited Loading

prabhatnagarajan commented Dec 10, 2020

pfn-ci-bot commented Dec 10, 2020

prabhatnagarajan commented Dec 11, 2020

pfn-ci-bot commented Dec 11, 2020

prabhatnagarajan commented Dec 11, 2020

pfn-ci-bot commented Dec 11, 2020

muupan commented Dec 14, 2020

prabhatnagarajan commented Dec 14, 2020

prabhatnagarajan commented Dec 16, 2020

pfn-ci-bot commented Dec 16, 2020

prabhatnagarajan commented Jul 31, 2020 •

edited

Loading

prabhatnagarajan commented Dec 10, 2020 •

edited

Loading