Skip to content

Commit

Permalink
[SPARK-25072][PYSPARK] Forbid extra value for custom Row
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?

Add value length check in `_create_row`, forbid extra value for custom Row in PySpark.

## How was this patch tested?

New UT in pyspark-sql

Closes #22140 from xuanyuanking/SPARK-25072.

Lead-authored-by: liyuanjian <liyuanjian@baidu.com>
Co-authored-by: Yuanjian Li <xyliyuanjian@gmail.com>
Signed-off-by: Bryan Cutler <cutlerb@gmail.com>
  • Loading branch information
2 people authored and BryanCutler committed Sep 6, 2018
1 parent 3b6591b commit c84bc40
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 0 deletions.
4 changes: 4 additions & 0 deletions python/pyspark/sql/tests.py
Expand Up @@ -277,6 +277,10 @@ def test_struct_field_type_name(self):
struct_field = StructField("a", IntegerType())
self.assertRaises(TypeError, struct_field.typeName)

def test_invalid_create_row(self):
row_class = Row("c1", "c2")
self.assertRaises(ValueError, lambda: row_class(1, 2, 3))


class SQLTests(ReusedSQLTestCase):

Expand Down
3 changes: 3 additions & 0 deletions python/pyspark/sql/types.py
Expand Up @@ -1500,6 +1500,9 @@ def __contains__(self, item):
# let object acts like class
def __call__(self, *args):
"""create new Row object"""
if len(args) > len(self):
raise ValueError("Can not create Row with fields %s, expected %d values "
"but got %s" % (self, len(self), args))
return _create_row(self, args)

def __getitem__(self, item):
Expand Down

0 comments on commit c84bc40

Please sign in to comment.