Skip to content

Commit

Permalink
[SPARK-40796][BUILD][FOLLOW-UP] Fix Mypy check on unused "type: ignore"
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

The mypy check in master seems to be broken:

<img width="676" alt="Screen Shot 2022-10-17 at 1 09 13 PM" src="https://user-images.githubusercontent.com/1938382/196273759-64372862-6caa-4a9e-b700-b665c7ff7e6c.png">

(see it on apache#38279 and apache#38275, also reproducible locally).

This PR propose to remove the relevant `type: ignore` comments.

### Why are the changes needed?

Fix python lint check.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

UT

Closes apache#38287 from amaliujia/test_python_lint.

Authored-by: Rui Wang <rui.wang@databricks.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
amaliujia authored and SandishKumarHN committed Dec 12, 2022
1 parent 0ab837c commit d934018
Showing 1 changed file with 5 additions and 11 deletions.
16 changes: 5 additions & 11 deletions python/pyspark/sql/connect/plan.py
Expand Up @@ -322,15 +322,11 @@ def _convert_measure(
) -> proto.Aggregate.AggregateFunction:
exp, fun = m
measure = proto.Aggregate.AggregateFunction()
measure.function.name = fun # type: ignore[attr-defined]
measure.name = fun
if type(exp) is str:
measure.function.arguments.append( # type: ignore[attr-defined]
self.unresolved_attr(exp)
)
measure.arguments.append(self.unresolved_attr(exp))
else:
measure.function.arguments.append( # type: ignore[attr-defined]
cast(Expression, exp).to_plan(session)
)
measure.arguments.append(cast(Expression, exp).to_plan(session))
return measure

def plan(self, session: Optional["RemoteSparkSession"]) -> proto.Relation:
Expand All @@ -339,13 +335,11 @@ def plan(self, session: Optional["RemoteSparkSession"]) -> proto.Relation:

agg = proto.Relation()
agg.aggregate.input.CopyFrom(self._child.plan(session))
agg.aggregate.measures.extend( # type: ignore[attr-defined]
agg.aggregate.result_expressions.extend(
list(map(lambda x: self._convert_measure(x, session), self.measures))
)

gs = proto.Aggregate.GroupingSet() # type: ignore[attr-defined]
gs.aggregate_expressions.extend(groupings)
agg.aggregate.grouping_sets.append(gs) # type: ignore[attr-defined]
agg.aggregate.grouping_expressions.extend(groupings)
return agg

def print(self, indent: int = 0) -> str:
Expand Down

0 comments on commit d934018

Please sign in to comment.