Skip to content

[SPARK-14605][ML][PYTHON] Changed Python to use unicode UIDs for spark.ml Identifiable #12368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jkbradley wants to merge 3 commits into from
Closed

[SPARK-14605][ML][PYTHON] Changed Python to use unicode UIDs for spark.ml Identifiable #12368

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4a4d577
Changed Python to use unicode UIDs for spark.ml Identifiable classes
jkbradley Apr 13, 2016
34acb6f
added unit test which failed before fix
jkbradley Apr 13, 2016
5b5860a
cleaned up issues left over from refactoring before PR
jkbradley Apr 14, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion python/pyspark/ml/param/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -469,10 +469,11 @@ def _resetUid(self, newUid):
Changes the uid of this instance. This updates both
the stored uid and the parent uid of params and param maps.
This is used by persistence (loading).
:param newUid: new uid to use
:param newUid: new uid to use, which is converted to unicode
:return: same instance, but with the uid and Param.parent values
updated, including within param maps
"""
newUid = unicode(newUid)
self.uid = newUid
newDefaultParamMap = dict()
newParamMap = dict()
Expand Down
2 changes: 2 additions & 0 deletions python/pyspark/ml/tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -618,6 +618,8 @@ def test_linear_regression(self):
lr_path = path + "/lr"
lr.save(lr_path)
lr2 = LinearRegression.load(lr_path)
self.assertEqual(lr.uid, lr2.uid)
self.assertEqual(type(lr.uid), type(lr2.uid))
self.assertEqual(lr2.uid, lr2.maxIter.parent,
"Loaded LinearRegression instance uid (%s) did not match Param's uid (%s)"
% (lr2.uid, lr2.maxIter.parent))
Expand Down
5 changes: 3 additions & 2 deletions python/pyspark/ml/util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@

if sys.version > '3':
basestring = str
unicode = str

from pyspark import SparkContext, since
from pyspark.mllib.common import inherit_doc
Expand Down Expand Up @@ -67,10 +68,10 @@ def __repr__(self):
@classmethod
def _randomUID(cls):
"""
Generate a unique id for the object. The default implementation
Generate a unique unicode id for the object. The default implementation
concatenates the class name, "_", and 12 random hex chars.
"""
return cls.__name__ + "_" + uuid.uuid4().hex[12:]
return unicode(cls.__name__ + "_" + uuid.uuid4().hex[12:])


@inherit_doc
Expand Down
2 changes: 1 addition & 1 deletion python/pyspark/ml/wrapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ def __init__(self, java_model=None):
"""
super(JavaModel, self).__init__(java_model)
if java_model is not None:
self.uid = java_model.uid()
self._resetUid(java_model.uid())

def copy(self, extra=None):
"""
Expand Down