-
Notifications
You must be signed in to change notification settings - Fork 695
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
can i use RDD s in PySpark? #394
Comments
Currently Python API supports GeoSparkSQL. GeoSpark core Python API is in the development phase, I should finish it within month. |
Hi there,
My name is Roy, from Toronto. I had sent you an email at
pawel93kocinski@gmail.com just to get to know you.
I am working on geospatial big data and came across your work which looks
really great. Wondering if I could talk to to you for a minute about this.
My watsapp is +16478581118. peaese feel free to shoot me a message.
Thanks,
Roy
…On Fri, Nov 1, 2019 at 6:05 AM Paweł Kociński ***@***.***> wrote:
Currently Python API supports GeoSparkSQL. GeoSpark core Python API is in
the development phase, I should finish it within month.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#394>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFCDQCIGCMOKFIOC45HUW3TQRP5M7ANCNFSM4JG644XQ>
.
|
jiayuasu
pushed a commit
that referenced
this issue
Feb 3, 2020
* Fix Issue, unread block data (#408) * Add GeoSpark core Python API, version beta. * Fix issue with additional else statement. * Add WkbReader to direct imports, Fix issue with version tests. * Add geo_pyspark version 0.3.0. * Add geo_pyspark version 0.3.0. * Update wheel file for geo_pyspark version 0.3.0. * Improve serialization process for GeoSpark Python. * Fix Issue with Adapter import. * Create example notebook for GeoPysparkSQL and GeoPysparkCore. * Delete ShowCase Notebook.ipynb * Update GeoSparkCore example notebook. * Update code for DataBricks platform support. * Add support for collect SpatialPartitionedRDD. * Add persist possibility to indexedRDD. * Add support for serializing rawSpatialRDD. * Update wheel file for geo_pyspark version 0.3.0.
jiayuasu
pushed a commit
that referenced
this issue
Feb 3, 2020
* Fix Issue, unread block data (#408) * Add GeoSpark core Python API, version beta. * Fix issue with additional else statement. * Add WkbReader to direct imports, Fix issue with version tests. * Add geo_pyspark version 0.3.0. * Add geo_pyspark version 0.3.0. * Update wheel file for geo_pyspark version 0.3.0. * Improve serialization process for GeoSpark Python. * Fix Issue with Adapter import. * Create example notebook for GeoPysparkSQL and GeoPysparkCore. * Delete ShowCase Notebook.ipynb * Update GeoSparkCore example notebook. * Update code for DataBricks platform support. * Add support for collect SpatialPartitionedRDD. * Add persist possibility to indexedRDD. * Add support for serializing rawSpatialRDD. * Update wheel file for geo_pyspark version 0.3.0. # Conflicts: # docs/tutorial/geospark-python.md # mkdocs.yml
jiayuasu
pushed a commit
that referenced
this issue
Feb 3, 2020
* Fix Issue, unread block data (#408) * Add GeoSpark core Python API, version beta. * Fix issue with additional else statement. * Add WkbReader to direct imports, Fix issue with version tests. * Add geo_pyspark version 0.3.0. * Add geo_pyspark version 0.3.0. * Update wheel file for geo_pyspark version 0.3.0. * Improve serialization process for GeoSpark Python. * Fix Issue with Adapter import. * Create example notebook for GeoPysparkSQL and GeoPysparkCore. * Delete ShowCase Notebook.ipynb * Update GeoSparkCore example notebook. * Update code for DataBricks platform support. * Add support for collect SpatialPartitionedRDD. * Add persist possibility to indexedRDD. * Add support for serializing rawSpatialRDD. * Update wheel file for geo_pyspark version 0.3.0. # Conflicts: # docs/tutorial/geospark-python.md # mkdocs.yml
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Expected behavior
Actual behavior
Steps to reproduce the problem
Settings
GeoSpark version = 1.2
Apache Spark version = 2.4
JRE version = 1.8
API type = Python
How can i import and use the following in pyspark?
Since the sql is almost impractical in terms of performance, i am trying to do it in rdds, but the imports or anything is not working.
import org.datasyslab.geospark.enums.{FileDataSplitter, GridType, IndexType}
import org.datasyslab.geospark.formatMapper.GeoJsonReader
import org.datasyslab.geospark.formatMapper.shapefileParser.ShapefileReader
import org.datasyslab.geospark.spatialOperator.JoinQuery
import org.datasyslab.geospark.spatialRDD.{CircleRDD, PolygonRDD, SpatialRDD}
import org.datasyslab.geosparksql.utils.Adapter
The text was updated successfully, but these errors were encountered: