Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat] Implement GraphAr Spark Writer for writing Spark DataFrame into GAR format files #28

Closed
acezen opened this issue Dec 12, 2022 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@acezen
Copy link
Contributor

acezen commented Dec 12, 2022

Is your feature request related to a problem? Please describe.
Implement writer of Spark tool to provide functions to generate GraphAr format files from hive table.

Describe the solution you'd like
It's better to read hive table as a spark DataFrame and use operators of DataFrame to generate the files.
The writer should include VertexWriter and EdgeWriter.

  • VertexWriter provide functions to generate chunk files of property group base on the vertex info user defined
  • EdgeWriter provide function to generate chunk files of adj list/offset/property group base on the edge info that user defined

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

@acezen acezen added the enhancement New feature or request label Dec 12, 2022
@acezen acezen added this to the Release 0.1.0 milestone Dec 13, 2022
@acezen acezen self-assigned this Dec 16, 2022
@acezen acezen pinned this issue Dec 29, 2022
@acezen acezen unpinned this issue Dec 29, 2022
@acezen
Copy link
Contributor Author

acezen commented Dec 29, 2022

The issue has been solved by pr #51

@acezen acezen closed this as completed Dec 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant