Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove python function wrapper. #6012

Merged
merged 15 commits into from
Aug 25, 2021
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
45 changes: 45 additions & 0 deletions oneflow/api/python/framework/doc.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
/*
Copyright 2020 The OneFlow Authors. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
#include <pybind11/pybind11.h>

#include "oneflow/api/python/of_api_registry.h"
#include "oneflow/api/python/framework/throw.h"

namespace py = pybind11;

namespace oneflow {

py::object AddFunctionDoc(py::object f, const std::string& doc_string) {
static std::vector<std::string> all_doc_strings;
all_doc_strings.emplace_back(doc_string);
const char* doc_str = all_doc_strings.back().c_str();
PyObject* obj = f.ptr();
if (PyCFunction_Check(obj)) {
auto* f = (PyCFunctionObject*)obj;
if (f->m_ml->ml_doc) {
THROW(RuntimeError) << "function " << f->m_ml->ml_name << " already has a docstring.";
}
f->m_ml->ml_doc = doc_str;
} else {
THROW(RuntimeError) << "function is " << Py_TYPE(obj)->tp_name << ", not a valid PyCFunction.";
}
f.inc_ref();
return f;
}

} // namespace oneflow

ONEFLOW_API_PYBIND11_MODULE("", m) { m.def("add_doc", &oneflow::AddFunctionDoc); }
16 changes: 16 additions & 0 deletions python/oneflow/F/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"""
Copyright 2020 The OneFlow Authors. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from oneflow._oneflow_internal.F import *
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个 F 名字空间,其实作用是对应了 torch 中的 torch._C 吧?
我们是未来会改成 _C 吗?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

目前感觉没太大必要,和torch的_C也不是完全对应。torch的接口导出比较分散,有些直接导出到torch,有些是torch._C,我们都统一到F

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的。

6 changes: 2 additions & 4 deletions python/oneflow/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,12 +107,10 @@ def _SyncOnMasterFn():
atexit.register(_SyncOnMasterFn)
del atexit
del oneflow

import oneflow.F
import oneflow.framework.docstr as docstr
from oneflow.framework.docstr.utils import register_docstr

register_docstr()
del register_docstr
del docstr
from oneflow.autograd import grad_enable, no_grad, inference_mode, is_grad_enabled
import oneflow.nn.image
import oneflow.nn.modules.acosh
Expand Down
10 changes: 3 additions & 7 deletions python/oneflow/framework/docstr/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,9 @@
See the License for the specific language governing permissions and
limitations under the License.
"""
_function_docstr = {}


def add_docstr(fun, docstr: str):
_function_docstr[fun] = docstr
import oneflow._oneflow_internal


def register_docstr():
for (fun, docstr) in _function_docstr.items():
setattr(fun, "__doc__", docstr)
def add_docstr(fun, docstr: str):
oneflow._oneflow_internal.add_doc(fun, docstr)
hjchen2 marked this conversation as resolved.
Show resolved Hide resolved
45 changes: 0 additions & 45 deletions python/oneflow/framework/functional.py

This file was deleted.

2 changes: 0 additions & 2 deletions python/oneflow/framework/register_class_method_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
limitations under the License.
"""
import oneflow._oneflow_internal
import oneflow.framework.functional as functional
import oneflow.framework.generator as generator
import oneflow.framework.op_expr_util as op_expr_util
import oneflow.framework.tensor as tensor_util
Expand All @@ -23,4 +22,3 @@
def RegisterMethod4Class():
tensor_util.RegisterMethods()
op_expr_util.RegisterMethod4UserOpExpr()
functional.RegisterFunctionalApis()
4 changes: 4 additions & 0 deletions tools/generate_functional_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,11 @@
namespace functional = one::functional;

ONEFLOW_API_PYBIND11_MODULE("F", m) {{
py::options options;
options.disable_function_signatures();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

禁止pybind自动生成doc


{1}
options.enable_function_signatures();
}}

}} // namespace oneflow
Expand Down