Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem with loading trained model #17

Closed
hn2 opened this issue Nov 29, 2020 · 21 comments
Closed

Problem with loading trained model #17

hn2 opened this issue Nov 29, 2020 · 21 comments
Labels
bug Something isn't working

Comments

@hn2
Copy link

hn2 commented Nov 29, 2020 •
edited

I am trying to load a trained model with CQL.load_model(..full model [path).
I first got fname is missing I tried fname=..full_model_path
I then got self is missing
I added self It still doesn't load the model.
no attribute 'impl' ...
@hn2 hn2 added the bug Something isn't working label Nov 29, 2020
@hn2
Copy link
Author

hn2 commented Nov 29, 2020 •
edited

from d3rlpy.algos import DiscreteCQL
from d3rlpy.datasets import get_cartpole
from d3rlpy.metrics.scorer import evaluate_on_environment

obtain dataset

dataset, env = get_cartpole()

setup algorithm

DiscreteCQL = DiscreteCQL(n_epochs=1)

'''

train

DiscreteCQL.fit(dataset.episodes, n_epochs=1)

DiscreteCQL.save_model('DiscreteCQLModel')
'''
DiscreteCQL.load_model(fname='DiscreteCQLModel')

evaluate trained algorithm

evaluate_on_environment(env, render=True)(DiscreteCQL)

When trying to load this trained model ('DiscreteCQLModel') I get:

d3rlpy\base.py", line 244, in load_model
self.impl.load_model(fname)

AttributeError: 'NoneType' object has no attribute 'load_model'

Problem is here:

def load_model(self, fname):
""" Load neural network parameters.

    .. code-block:: python

        algo.load_model('model.pt')

    Args:
        fname (str): source file path.

    """
    self.impl.load_model(fname)

@takuseno
Copy link
Owner

@hn2 Hello, thank you for reporting the issue and sorry for the inconvenience. I confirmed that there is a bug when loading the model in your way. Please do work around as follows for a moment.

cql = DiscreteCQL(<give parameters>)

cql.create_impl(dataset.get_observation_shape(), dataset.get_action_size())

cql.load_model('<path-to-your-model>')

I'll prepare more sophisticated ways to do this in the next release. Thank you.

@takuseno
Copy link
Owner

Or please do this.

cql = DiscreteCQL.from_json('<path-to-params.json>')
cql.load('<path-to-model>')

Please replace <path-to-params.json> with the JSON file named 'params.json' that d3rlpy saves in d3rlpy_logs/<experiment directory>. This is the cleanest way so far.

@hn2
Copy link
Author

hn2 commented Nov 29, 2020 via email

@takuseno
Copy link
Owner

The json file is automatically saved in the log directory. The purpose of this file is to save the configuration of the algorithm (not the model parameters such as weights and bias). You can use the json file to initialize the algorithm object as it was.

@hn2
Copy link
Author

hn2 commented Nov 29, 2020 via email

@takuseno
Copy link
Owner

takuseno commented Nov 29, 2020 •
edited

You can change that via logdir and experiment_name. Please see details at here.

@hn2
Copy link
Author

hn2 commented Nov 29, 2020 via email

@takuseno
Copy link
Owner

@hn2 I'm afraid to say this, but I'd like you to learn Python a bit more because I cannot support how to code Python. Please write as below.

cql = CQL.from_json('E:\QuantConnect\my-algos\drl-forex-offline\d3rlpy_logs\CQL_20201129130908\params.json')
cql.load(v_offline_model_file_name)

@takuseno
Copy link
Owner

takuseno commented Nov 29, 2020 •
edited

And this will be useful to figure out how to save and load models.

@hn2
Copy link
Author

hn2 commented Nov 29, 2020 via email

@takuseno
Copy link
Owner

There is a bug, but you can load your model with work arounds.

option 1

Initialize PyTorch models manually.

cql = CQL()
cql.create_impl(dataset.get_observation_shape(), dataset.get_action_size())
cql.load_model(<path-to-model>)

option 2

initialize PyTorch's models via JSON.

cql = CQL.from_json(<path-to-params.json>)
cql.load_model(<path-to-model>)

@hn2
Copy link
Author

hn2 commented Nov 29, 2020 via email

@takuseno
Copy link
Owner

I'm not working on this project as a part of my full-time project. But, I'll release the next version in 2 weeks.

@hn2
Copy link
Author

hn2 commented Nov 29, 2020 via email

@takuseno
Copy link
Owner

takuseno commented Dec 2, 2020

The next release includes new methods build_with_dataset and build_with_env to solve your problem, which is already available in master branch. In your case, you'll be able to do as follows:

# create algorithm object
cql = CQL()

# build neural networks with shapes of your dataset
cql.build_with_dataset(dataset)

# or build with your environment that must have observation_space and action_space as OpenAI does
# cql.build_with_env(env)

# load the model
cql.load_model(<path-to-your-model>)

I'll let you know when it's released.

@hn2
Copy link
Author

hn2 commented Dec 2, 2020 via email

@takuseno
Copy link
Owner

takuseno commented Dec 3, 2020

It looks like the observation shape of your environment is not valid.

@hn2
Copy link
Author

hn2 commented Dec 4, 2020 via email

@hn2
Copy link
Author

hn2 commented Dec 17, 2020 via email

@takuseno
Copy link
Owner

Closes this since the v0.41 supports build_with_dataset and build_with_env.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@takuseno @hn2