[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Python - part 1 #41561

panbingkun · 2023-06-13T01:38:59Z

What changes were proposed in this pull request?

Add following functions:

char
btrim
char_length
character_length
chr
contains
elt
find_in_set
like
ilike
lcase
ucase
~~len: Because it conflicts with Python keywords, and we already have length~~
left
right

to:

Scala API
Python API
Spark Connect Scala Client
Spark Connect Python Client

Why are the changes needed?

for parity

Does this PR introduce any user-facing change?

Yes, new functions.

How was this patch tested?

Add New UT.

…on - part 1

zhengruifeng · 2023-06-15T01:46:17Z

sql/core/src/main/scala/org/apache/spark/sql/functions.scala

+   * @since 3.5.0
+   */
+  def contains(left: Column, right: Column): Column = withExpr {
+    ContainsExpressionBuilder.build("contains", Seq(left.expr, right.expr))


why not Contains(left, right)?

Because Contains(left, right) only supports StringType, not include BinaryType.

spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala

Lines 574 to 578 in f44d923

object ContainsExpressionBuilder extends StringBinaryPredicateExpressionBuilderBase {

override protected def createStringPredicate(left: Expression, right: Expression): Expression = {

Contains(left, right)

}

}

spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala

Lines 503 to 518 in f44d923

trait StringBinaryPredicateExpressionBuilderBase extends ExpressionBuilder {

override def build(funcName: String, expressions: Seq[Expression]): Expression = {

val numArgs = expressions.length

if (numArgs == 2) {

if (expressions(0).dataType == BinaryType && expressions(1).dataType == BinaryType) {

BinaryPredicate(funcName, expressions(0), expressions(1))

} else {

createStringPredicate(expressions(0), expressions(1))

}

} else {

throw QueryCompilationErrors.wrongNumArgsError(funcName, Seq(2), numArgs)

}

}

protected def createStringPredicate(left: Expression, right: Expression): Expression

}

I see, then I have another question:
does it works with df.select(contains(col("a"), col("b")))?
in this case the underlying expressions are not resolved, and I guess it can not get the datatype

Yes, that's right, so do we only support stringType here?

I think we should only support StringType here, and document this limitation (contains in sql support both StringType and BinaryType).

also cc @HyukjinKwon @beliefer @cloud-fan

Yeah. I see the limit. Because scala API wrapped column with unresolved expression, it seems we can't reuse the *ExpressionBuilder here.

zhengruifeng · 2023-06-15T01:49:47Z

python/pyspark/sql/functions.py

@@ -8159,6 +8159,382 @@ def to_number(col: "ColumnOrName", format: "ColumnOrName") -> Column:
    return _invoke_function_over_columns("to_number", col, format)


+@try_remote_functions
+def char(n: "ColumnOrName") -> Column:


i suggest we use col: "ColumnOrName" for similar cases in python

e.g.

spark/python/pyspark/sql/functions.py

Lines 7268 to 7339 in 79100ce

@try_remote_functions

def upper(col: "ColumnOrName") -> Column:

"""

Converts a string expression to upper case.

.. versionadded:: 1.5.0

.. versionchanged:: 3.4.0

Supports Spark Connect.

Parameters

----------

col : :class:`~pyspark.sql.Column` or str

target column to work on.

Returns

-------

:class:`~pyspark.sql.Column`

upper case values.

Examples

--------

>>> df = spark.createDataFrame(["Spark", "PySpark", "Pandas API"], "STRING")

>>> df.select(upper("value")).show()

+------------+

|upper(value)|

+------------+

| SPARK|

| PYSPARK|

| PANDAS API|

+------------+

"""

return _invoke_function_over_columns("upper", col)

@try_remote_functions

def lower(col: "ColumnOrName") -> Column:

"""

Converts a string expression to lower case.

.. versionadded:: 1.5.0

.. versionchanged:: 3.4.0

Supports Spark Connect.

Parameters

----------

col : :class:`~pyspark.sql.Column` or str

target column to work on.

Returns

-------

:class:`~pyspark.sql.Column`

lower case values.

Examples

--------

>>> df = spark.createDataFrame(["Spark", "PySpark", "Pandas API"], "STRING")

>>> df.select(lower("value")).show()

+------------+

|lower(value)|

+------------+

| spark|

| pyspark|

| pandas api|

+------------+

"""

return _invoke_function_over_columns("lower", col)

@try_remote_functions

def ascii(col: "ColumnOrName") -> Column:

zhengruifeng · 2023-06-15T01:52:33Z

also cc @beliefer

beliefer

Please update python/docs/source/reference/pyspark.sql/functions.rst too.

beliefer · 2023-06-15T04:38:49Z

connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/functions.scala

+   * @group string_funcs
+   * @since 3.5.0
+   */
+  def like(str: Column, pattern: Column): Column = Column.fn("like", str, pattern)


Please add def like(str: Column, pattern: Column, escapeChar: Column): Column.

@beliefer @zhengruifeng
I don't think it's necessary to add this function for the following reasons:

https://spark.apache.org/docs/latest/sql-ref-functions-builtin.html#string-functions

It only has two parameter usage forms. If a third parameter needs to be used, its usage form is:

rather than like(str, pattern, escapeChar)

In my first local version, I actually implemented this method and reported an error when executing 'SPARK_GENERATE_GOLDEN_FILES=1 build/sbt "connect/testOnly org.apache.spark.sql.connect.ProtoToParsedPlanTestSuite"'

It triggers the following detection logic:

spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala

Lines 139 to 145 in f44d923

val params = Seq.fill(expressions.size)(classOf[Expression])

val f = constructors.find(_.getParameterTypes.toSeq == params).getOrElse {

val validParametersCount = constructors

.filter(_.getParameterTypes.forall(_ == classOf[Expression]))

.map(_.getParameterCount).distinct.sorted

throw QueryCompilationErrors.wrongNumArgsError(

name, validParametersCount, params.length)

Yeah. We still could support it. Please refer the nth_value. You must treat it on special way at the server side.

if escapeChar was not supported in SQL syntax, I think we can also hold on supporting it in functions.

If we want to support it now, for the Connect side, we need to add it in SparkConnectPlanner#transformUnregisteredFunction

Yes, I have already added it.

@zhengruifeng SQL syntax has been supported escapeChar.

oh, I see, so let's support it in functions too

beliefer · 2023-06-15T04:39:26Z

connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/functions.scala

+   * @group string_funcs
+   * @since 3.5.0
+   */
+  def ilike(str: Column, pattern: Column): Column = Column.fn("ilike", str, pattern)


beliefer · 2023-06-15T04:44:03Z

connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/PlanGenerationTestSuite.scala

+    fn.btrim(fn.col("g"))
+  }
+
+  functionTest("btrim with trim") {


btrim with specified trim string

beliefer · 2023-06-15T04:44:55Z

connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/PlanGenerationTestSuite.scala

+    fn.find_in_set(fn.col("g"), fn.col("g"))
+  }
+
+  functionTest("like") {


Please add the test cases with escape character.

beliefer · 2023-06-15T04:45:03Z

connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/PlanGenerationTestSuite.scala

+    fn.like(fn.col("g"), fn.col("g"))
+  }
+
+  functionTest("ilike") {


beliefer · 2023-06-15T04:46:51Z

python/pyspark/sql/functions.py

@@ -8159,6 +8159,382 @@ def to_number(col: "ColumnOrName", format: "ColumnOrName") -> Column:
    return _invoke_function_over_columns("to_number", col, format)


+@try_remote_functions
+def char(n: "ColumnOrName") -> Column:


beliefer · 2023-06-15T04:48:53Z

python/pyspark/sql/functions.py

+    Parameters
+    ----------
+    left : :class:`~pyspark.sql.Column` or str
+        Input column or strings.


Shall we describe it more detail?

beliefer · 2023-06-15T04:49:03Z

python/pyspark/sql/functions.py

+    left : :class:`~pyspark.sql.Column` or str
+        Input column or strings.
+    right : :class:`~pyspark.sql.Column` or str
+        Input column or strings.


beliefer · 2023-06-15T04:50:51Z

python/pyspark/sql/functions.py

+    Parameters
+    ----------
+    inputs : :class:`~pyspark.sql.Column` or str
+        Input column or strings.


Input columns or strings.

…on - part 1

panbingkun · 2023-06-16T01:45:29Z

python/pyspark/sql/connect/functions.py

+ucase.__doc__ = pysparkfuncs.ucase.__doc__
+
+
+def py_len(str: "ColumnOrName") -> Column:


Conflicting with Python keywords len

ok, let us also skip this one, since we have length

panbingkun · 2023-06-16T01:49:46Z

sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala

+    // TODO
+    val df = Seq(("Spark SQL", "Spark")).toDF("a", "b")
+    checkAnswer(df.selectExpr("contains(a, b)"), Seq(Row(true)))
+    checkAnswer(df.select(contains(lit("Spark SQL"), lit("Spark"))), Seq(Row(true)))


Why not df.select(contains(col("a"), col("b"))), butcontains(lit("Spark SQL"), lit("Spark"))), because here it can't get the datatype.

…on - part 1

beliefer

LGTM except three comment.

beliefer · 2023-06-17T03:08:38Z

python/pyspark/sql/functions.py

+    """
+    Returns a boolean. The value is True if right is found inside left.
+    Returns NULL if either input expression is NULL. Otherwise, returns False.
+    Both left or right must be of STRING or BINARY type.


We should update the comments too.

+1, let's also add Notes here, you can refer to

spark/python/pyspark/sql/functions.py

Lines 9318 to 9321 in 1b12094

Notes

-----

The position is not zero based, but 1 based index. Returns 0 if the given

value could not be found in the array.

beliefer · 2023-06-17T03:20:28Z

python/pyspark/sql/functions.py

+
+@try_remote_functions
+def like(
+    str: "ColumnOrName", pattern: "ColumnOrName", escapeChar: Optional["str"] = None


escapeChar: Optional["ColumnOrName"] ?

in this case, since escapeChar is a Char in the expression constructors, I feel it is a bit confusing if we treat str input as a column name.

shall we follow the Char types in expression? or directly use Column for both scala and python

also cc @HyukjinKwon

At two weeks ago, I thought escapeChar: Optional["str"] = None is OK. But I'm not sure now.

beliefer · 2023-06-17T03:20:40Z

python/pyspark/sql/functions.py

+
+@try_remote_functions
+def ilike(
+    str: "ColumnOrName", pattern: "ColumnOrName", escapeChar: Optional["str"] = None


zhengruifeng · 2023-06-17T03:22:47Z

python/pyspark/sql/tests/test_functions.py

@@ -67,7 +67,7 @@ def test_function_parity(self):

        # For functions that are named differently in pyspark this is the mapping of their
        # python name to the JVM equivalent
-        py_equiv_jvm = {"create_map": "map"}
+        py_equiv_jvm = {"create_map": "map", "py_len": "len"}


I think we don't need this change any more

beliefer · 2023-06-17T03:23:03Z

sql/core/src/main/scala/org/apache/spark/sql/functions.scala

+   * @group string_funcs
+   * @since 3.5.0
+   */
+  def ilike(str: Column, pattern: Column, escapeChar: Char): Column = withExpr {


escapeChar: Column

beliefer · 2023-06-17T03:23:18Z

sql/core/src/main/scala/org/apache/spark/sql/functions.scala

+   * @group string_funcs
+   * @since 3.5.0
+   */
+  def like(str: Column, pattern: Column, escapeChar: Char): Column = withExpr {


escapeChar: Column

spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala

Line 125 in b97ce8b

case class Like(left: Expression, right: Expression, escapeChar: Char)

Do we need to implement escapeChar as a Column here? @zhengruifeng

zhengruifeng · 2023-06-17T03:24:38Z

sql/core/src/main/scala/org/apache/spark/sql/functions.scala

+   * @note
+   *   This is different from the `contains` method in SQL that supports both STRING and BINARY
+   *   type.


Suggested change

* @note

* This is different from the `contains` method in SQL that supports both STRING and BINARY

* type.

* @note Only STRING type is supported in this function, while `contains` in SQL supports both STRING and BINARY.

zhengruifeng · 2023-06-17T03:29:11Z

python/pyspark/sql/functions.py

+    """
+    Returns a boolean. The value is True if right is found inside left.
+    Returns NULL if either input expression is NULL. Otherwise, returns False.
+    Both left or right must be of STRING or BINARY type.


+1, let's also add Notes here, you can refer to

spark/python/pyspark/sql/functions.py

Lines 9318 to 9321 in 1b12094

Notes

-----

The position is not zero based, but 1 based index. Returns 0 if the given

value could not be found in the array.

…on - part 1

panbingkun · 2023-06-17T08:18:30Z

Update all, exclude:
1.In Scala side: escapeChar: Char -> escapeChar: Column
2.In Python side: escapeChar: Optional["str"] -> Optional["ColumnOrName"]

beliefer · 2023-06-17T10:55:00Z

Update all, exclude:
1.In Scala side: escapeChar: Char -> escapeChar: Column
2.In Python side: escapeChar: Optional["str"] -> Optional["ColumnOrName"]

OK. Please wait for @HyukjinKwon 's opinion.

panbingkun · 2023-06-17T11:34:19Z

@beliefer There is a small bug in PR SPARK-43938

Maybe need to exclude 'getbit' in DataFrameFunctionsSuite

beliefer · 2023-06-17T11:58:35Z

@panbingkun Please wait #41640

zhengruifeng · 2023-06-17T12:53:25Z

Update all, exclude: 1.In Scala side: escapeChar: Char -> escapeChar: Column 2.In Python side: escapeChar: Optional["str"] -> Optional["ColumnOrName"]

thanks for the update, my last suggestion would be

1.In Scala side: escapeChar: Char -> escapeChar: Column
2.In Python side: escapeChar: Optional["str"] -> Optional["Column"].

I still feel it would be ambiguous if treat str as the column name here; so I suggest to use Optional["Column"] and then support other types later (if we support a type now, it is nearly impossible to remove it)

…on - part 1

panbingkun · 2023-06-18T04:36:49Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala

@@ -125,6 +125,13 @@ abstract class StringRegexExpression extends BinaryExpression
 case class Like(left: Expression, right: Expression, escapeChar: Char)
  extends StringRegexExpression {



For support Column, this construction method has to be added.

I think we can use escapeChar.eval() in the functions side.

Because escapeChar is a feature provided myself. I know some background.
The escape char must be a constant. So we didn't support it as expression.
I agree the comment wrote by @zhengruifeng . Please check escapeChar in the functions API.

beliefer · 2023-06-18T09:30:45Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala

@@ -125,6 +125,13 @@ abstract class StringRegexExpression extends BinaryExpression
 case class Like(left: Expression, right: Expression, escapeChar: Char)
  extends StringRegexExpression {



Because escapeChar is a feature provided myself. I know some background.
The escape char must be a constant. So we didn't support it as expression.
I agree the comment wrote by @zhengruifeng . Please check escapeChar in the functions API.

beliefer · 2023-06-18T09:31:45Z

sql/core/src/main/scala/org/apache/spark/sql/functions.scala

+   * @since 3.5.0
+   */
+  def like(str: Column, pattern: Column, escapeChar: Column): Column = withExpr {
+    new Like(str.expr, pattern.expr, escapeChar.expr)


We can check escapeChar here.

…on - part 1

panbingkun · 2023-06-18T11:15:17Z

Update all done.

beliefer · 2023-06-18T12:20:04Z

core/src/main/resources/error/error-classes.json

@@ -395,6 +395,11 @@
          "Input to <functionName> should all be the same type, but it's <dataType>."
        ]
      },
+      "ESCAPE_CHAR_WRONG_TYPE" : {
+        "message" : [
+          "EscapeChar should be a string literal."


'EscapeChar' should be a string literal of length one.

beliefer · 2023-06-18T12:23:06Z

sql/core/src/main/scala/org/apache/spark/sql/functions.scala

+      case escape @ Literal(_, StringType) =>
+        Like(str.expr, pattern.expr, escape.eval().toString.charAt(0))


Suggested change

case escape @ Literal(_, StringType) =>

Like(str.expr, pattern.expr, escape.eval().toString.charAt(0))

case StringLiteral(v) if v.size == 1 =>

Like(str.expr, pattern.expr, v.charAt(0))

…on - part 1

beliefer

LGTM expect one comment.

beliefer · 2023-06-19T02:21:11Z

core/src/main/resources/error/error-classes.json

@@ -395,6 +395,11 @@
          "Input to <functionName> should all be the same type, but it's <dataType>."
        ]
      },
+      "ESCAPE_CHAR_WRONG_TYPE" : {


After a new thought, we can simplify to INVALID_ESCAPE_CHAR.

…on - part 1

zhengruifeng · 2023-06-19T09:26:11Z

I am going to merge this PR after all tests pass, and I just notice that we can support BINARY via call_udf (see #41659), please help check it and send a follow-up PR if it works.

panbingkun · 2023-06-19T09:36:49Z

I am going to merge this PR after all tests pass, and I just notice that we can support BINARY via call_udf (see #41659), please help check it and send a follow-up PR if it works.

👌🏻, let me to check it

zhengruifeng · 2023-06-19T12:37:32Z

@panbingkun @beliefer thank you guys!

merged to master

panbingkun added 2 commits June 12, 2023 20:05

test

8b4d0b5

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

996b566

…on - part 1

github-actions bot added CONNECT CORE PYTHON SQL labels Jun 13, 2023

panbingkun added 3 commits June 13, 2023 09:49

Merge branch 'master' into SPARK-43942

1e01472

Merge branch 'master' into SPARK-43942

1167546

Merge branch 'master' into SPARK-43942

3036603

zhengruifeng reviewed Jun 15, 2023

View reviewed changes

beliefer reviewed Jun 15, 2023

View reviewed changes

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

fdb73b1

…on - part 1

panbingkun commented Jun 16, 2023

View reviewed changes

panbingkun added 3 commits June 16, 2023 11:00

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

27460c7

…on - part 1

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

0ae5779

…on - part 1

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

d183c2a

…on - part 1

panbingkun requested a review from zhengruifeng June 17, 2023 02:11

beliefer approved these changes Jun 17, 2023

View reviewed changes

beliefer reviewed Jun 17, 2023

View reviewed changes

zhengruifeng reviewed Jun 17, 2023

View reviewed changes

beliefer reviewed Jun 17, 2023

View reviewed changes

zhengruifeng approved these changes Jun 17, 2023

View reviewed changes

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

11f4ff6

…on - part 1

Merge branch 'master' into SPARK-43942

e9c4de8

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

c54194d

…on - part 1

panbingkun commented Jun 18, 2023

View reviewed changes

panbingkun requested a review from zhengruifeng June 18, 2023 04:37

beliefer reviewed Jun 18, 2023

View reviewed changes

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

912bf90

…on - part 1

beliefer reviewed Jun 18, 2023

View reviewed changes

panbingkun added 2 commits June 18, 2023 21:50

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

0cddc2d

…on - part 1

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

de518d0

…on - part 1

beliefer approved these changes Jun 19, 2023

View reviewed changes

panbingkun added 3 commits June 19, 2023 11:59

Merge branch 'master' into SPARK-43942

1b7d037

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

dfffe8e

…on - part 1

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Pyth…

9bee3e4

…on - part 1

zhengruifeng approved these changes Jun 19, 2023

View reviewed changes

zhengruifeng closed this in 476c58e Jun 19, 2023

	object ContainsExpressionBuilder extends StringBinaryPredicateExpressionBuilderBase {
	override protected def createStringPredicate(left: Expression, right: Expression): Expression = {
	Contains(left, right)
	}
	}

	trait StringBinaryPredicateExpressionBuilderBase extends ExpressionBuilder {
	override def build(funcName: String, expressions: Seq[Expression]): Expression = {
	val numArgs = expressions.length
	if (numArgs == 2) {
	if (expressions(0).dataType == BinaryType && expressions(1).dataType == BinaryType) {
	BinaryPredicate(funcName, expressions(0), expressions(1))
	} else {
	createStringPredicate(expressions(0), expressions(1))
	}
	} else {
	throw QueryCompilationErrors.wrongNumArgsError(funcName, Seq(2), numArgs)
	}
	}

	protected def createStringPredicate(left: Expression, right: Expression): Expression
	}

	@try_remote_functions
	def upper(col: "ColumnOrName") -> Column:
	"""
	Converts a string expression to upper case.

	.. versionadded:: 1.5.0

	.. versionchanged:: 3.4.0
	Supports Spark Connect.

	Parameters
	----------
	col : :class:`~pyspark.sql.Column` or str
	target column to work on.

	Returns
	-------
	:class:`~pyspark.sql.Column`
	upper case values.

	Examples
	--------
	>>> df = spark.createDataFrame(["Spark", "PySpark", "Pandas API"], "STRING")
	>>> df.select(upper("value")).show()
	+------------+
	\|upper(value)\|
	+------------+
	\| SPARK\|
	\| PYSPARK\|
	\| PANDAS API\|
	+------------+
	"""
	return _invoke_function_over_columns("upper", col)


	@try_remote_functions
	def lower(col: "ColumnOrName") -> Column:
	"""
	Converts a string expression to lower case.

	.. versionadded:: 1.5.0

	.. versionchanged:: 3.4.0
	Supports Spark Connect.

	Parameters
	----------
	col : :class:`~pyspark.sql.Column` or str
	target column to work on.

	Returns
	-------
	:class:`~pyspark.sql.Column`
	lower case values.

	Examples
	--------
	>>> df = spark.createDataFrame(["Spark", "PySpark", "Pandas API"], "STRING")
	>>> df.select(lower("value")).show()
	+------------+
	\|lower(value)\|
	+------------+
	\| spark\|
	\| pyspark\|
	\| pandas api\|
	+------------+
	"""
	return _invoke_function_over_columns("lower", col)


	@try_remote_functions
	def ascii(col: "ColumnOrName") -> Column:

	val params = Seq.fill(expressions.size)(classOf[Expression])
	val f = constructors.find(_.getParameterTypes.toSeq == params).getOrElse {
	val validParametersCount = constructors
	.filter(_.getParameterTypes.forall(_ == classOf[Expression]))
	.map(_.getParameterCount).distinct.sorted
	throw QueryCompilationErrors.wrongNumArgsError(
	name, validParametersCount, params.length)

		ucase.__doc__ = pysparkfuncs.ucase.__doc__


		def py_len(str: "ColumnOrName") -> Column:

	Notes
	-----
	The position is not zero based, but 1 based index. Returns 0 if the given
	value could not be found in the array.

		@@ -125,6 +125,13 @@ abstract class StringRegexExpression extends BinaryExpression
		case class Like(left: Expression, right: Expression, escapeChar: Char)
		extends StringRegexExpression {

		case escape @ Literal(_, StringType) =>
		Like(str.expr, pattern.expr, escape.eval().toString.charAt(0))

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Python - part 1 #41561

[SPARK-43942][CONNECT][PYTHON] Add string functions to Scala and Python - part 1 #41561

Conversation

panbingkun commented Jun 13, 2023 • edited Loading

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Choose a reason for hiding this comment

panbingkun Jun 15, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhengruifeng commented Jun 15, 2023

beliefer left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

panbingkun Jun 15, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

beliefer left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

panbingkun commented Jun 17, 2023

beliefer commented Jun 17, 2023

panbingkun commented Jun 17, 2023 • edited Loading

beliefer commented Jun 17, 2023

zhengruifeng commented Jun 17, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

beliefer Jun 18, 2023 • edited Loading

Choose a reason for hiding this comment

beliefer Jun 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

panbingkun commented Jun 18, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

beliefer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhengruifeng commented Jun 19, 2023

panbingkun commented Jun 19, 2023

zhengruifeng commented Jun 19, 2023

panbingkun commented Jun 13, 2023 •

edited

Loading

panbingkun Jun 15, 2023 •

edited

Loading

beliefer left a comment •

edited

Loading

panbingkun Jun 15, 2023 •

edited

Loading

beliefer left a comment •

edited

Loading

panbingkun commented Jun 17, 2023 •

edited

Loading

beliefer Jun 18, 2023 •

edited

Loading

beliefer Jun 18, 2023 •

edited

Loading