Skip to content

issues Search Results · repo:zero323/pyspark-stubs language:Python

Filter by

157 results
 (83 ms)

157 results

inzero323/pyspark-stubs (press backspace or delete to remove)

zero323/pyspark-stubs
Allow latest version of pyspark

Currently, pyspark-stubs is limiting pyspark to have version pyspark =3.0.0.dev0, 3.1.0 However, we would like to install the latest version of pyspark (3.2.1) which has addressed the Log4J vulnerability. ...
  • jruales
  • 2
  • Opened 
    on Mar 25, 2022
  • #557

zero323/pyspark-stubs
Wrong type in Dataframe.write.parquet

MyPy complains when the partitionBy argument in DataFrame.parquet.write is of type str, although the pyspark docs says that str should be ok. error: Argument partitionBy to parquet of DataFrameWriter ...
  • ghost
  • Opened 
    on Dec 16, 2021
  • #555

zero323/pyspark-stubs
DataFrameLike does not have to_sql method

The DataFrameLike class needs to have to_sql method. Otherwise an unnecessary type conversion will be required after toPandas to use to_sql.
wontfix
  • consciousgaze
  • 2
  • Opened 
    on Sep 12, 2021
  • #553

zero323/pyspark-stubs
How to handle java backend stubs

Do you have a recommendation for how I can add stub support for java private methods? As an example, the following is a valid line of code but mypy is unable to resolve the ._gateway java backend private ...
question
wontfix
  • JeroenSchmidt
  • 2
  • Opened 
    on Jul 7, 2021
  • #550

zero323/pyspark-stubs
`pyspark.rdd.RDD.histogram`'s `buckets` argument is incomplete

The type hint for pyspark.rdd .RDD.histogram s buckets argument should be Union[int, List[T], Tuple[T]] From pyspark source: if isinstance(buckets, int): ... elif isinstance(buckets, (list, tuple)): ...
  • tpvasconcelos
  • Opened 
    on Jun 29, 2021
  • #548

zero323/pyspark-stubs
pyspark-stubs installed pyspark-2.4.4 and corrupt pre-installed pyspark-3.0.0

Describe the bug I just installed conda install -c conda-forge pyspark-stubs, then pyspark failed to use and shows: $ python etl.py Traceback (most recent call last): File etl.py , line 4, in module ...
  • fxrcode
  • 3
  • Opened 
    on Sep 23, 2020
  • #538

zero323/pyspark-stubs
[SPARK-32835] Add withField method to the pyspark Column class

External URLs - JIRA URL: https://issues.apache.org/jira/browse/SPARK-32835 - Commit URL: https://github.com/apache/spark/pull/29699
3.1
  • zero323
  • Opened 
    on Sep 16, 2020
  • #534

zero323/pyspark-stubs
Drop hasSummary from LinearRegressionTrainingSummary, GeneralizedLinearRegressionTrainingSummary and LogisticRegressionSummary

Method is inherited from HasTrainingSummary so shouldn t be annotated here.
3.0
3.1
  • zero323
  • Opened 
    on Sep 15, 2020
  • #531

zero323/pyspark-stubs
RandomForestRegressor.{__init__, setParams} are missing leafCol

3.1
  • zero323
  • Opened 
    on Sep 15, 2020
  • #528

zero323/pyspark-stubs
[SPARK-32798] Make unionByName optionally fill missing columns with nulls in PySpark

External URLs - JIRA URL: https://issues.apache.org/jira/browse/SPARK-32798 - Commit URL: https://github.com/apache/spark/pull/29657
3.1
  • zero323
  • Opened 
    on Sep 14, 2020
  • #526
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Press the
/
key to activate the search input again and adjust your query.
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Restrict your search to the title by using the in:title qualifier.
Issue search results · GitHub