Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add layers for save/load op #8711

Merged
merged 3 commits into from
Mar 5, 2018
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 69 additions & 1 deletion python/paddle/fluid/layers/nn.py
Original file line number Diff line number Diff line change
Expand Up @@ -3206,7 +3206,7 @@ def one_hot(input, depth):
operator.

Args:
input(Tensor/LodTensor): A Tensor/LodTensor of indices, last dimension must be 1.
input(variable): A Tensor/LodTensor of indices, last dimension must be 1.
depth(scalar): an interger defining the depth of the one hot dimension.

Returns:
Expand Down Expand Up @@ -3265,3 +3265,71 @@ def autoincreased_step_counter(counter_name=None, begin=1, step=1):
counter.stop_gradient = True

return counter


def save(x, file_path, overwrite=True):
"""
Saves a variable as a file.

Args:
x(variable): The Tensor/LoDTensor to be saved.
file_path(str): The file path where the variable will be saved.
overwrite(bool): Whether or not cover the given file when it has already
existed. If it's set 'False' and the file is existed, a runtime
error will be thrown.
"""
helper = LayerHelper("save", **locals())
helper.append_op(
type="save",
inputs={"input": x},
outputs={},
args={"file_path": file_path,
"overwrite": overwrite})


def save_combine(x, file_path, overwrite=True):
"""
Saves a variable as a file.

Args:
x(list): A list of Tensor/LoDTensor to be saved together in a single file.
file_path(str): The file path where variables will be saved.
overwrite(bool): Whether or not cover the given file when it has already
existed. If it's set 'False' and the file is existed, a runtime
error will be thrown.
"""
helper = LayerHelper("save_combine", **locals())
helper.append_op(
type="save_combine",
inputs={"input": x},
outputs={},
args={"file_path": file_path,
"overwrite": overwrite})


def load(out, file_path):
"""
Args:
out(variable): The variable to be read from the disk file.
file_path(str): The path of the disk file.
"""
helper = LayerHelper("load", **locals())
helper.append_op(
type="load",
inputs={},
output={"Out": out},
args={"file_path": file_path})


def load_combine(out, file_path):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need better doc about the difference between save/load and save/load_combine

Copy link
Collaborator Author

@JiayiFeng JiayiFeng Mar 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main difference between save/load and combine save/load is that the combine save can take a list of variables and dump all of them into a single file, while normal save can only save one variable to one file.

You are right, I can make the difference clearer.

"""
Args:
out(list): The list of variables to be read from the disk file.
file_path(str): The path of the disk file.
"""
helper = LayerHelper("load_combine", **locals())
helper.append_op(
type="load_combine",
inputs={},
output={"Out": out},
args={"file_path": file_path})