New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Creates a new function that unifies a list of GraphFrames into a single GraphFrame #10
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ilumsden
added
area-graphframe
Issues and PRs involving Hatchet's core GraphFrame datastructure and associated classes
area-utils
Issues and PRs related to Hatchets high-level API and other utility libraries
priority-normal
Normal priority issues and PRs
status-ready-for-review
This PR is ready to be reviewed by assigned reviewers
type-feature
Requests for new features or PRs which implement new features
labels
Feb 3, 2022
…phFrame.dataset after from_X functions are called if possible
ilumsden
force-pushed
the
unify_ensemble
branch
from
February 17, 2022 13:53
85fbdc6
to
5d64016
Compare
ilumsden
added
status-work-in-progress
PR is currently being worked on
and removed
status-ready-for-review
This PR is ready to be reviewed by assigned reviewers
labels
Feb 17, 2022
…e.from_caliper function
ilumsden
added
status-ready-for-review
This PR is ready to be reviewed by assigned reviewers
and removed
status-work-in-progress
PR is currently being worked on
labels
Feb 21, 2022
slabasan
reviewed
Feb 23, 2022
slabasan
force-pushed
the
develop
branch
16 times, most recently
from
August 9, 2022 05:03
b461833
to
48d44ce
Compare
Closing because this functionality is going elsewhere |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area-graphframe
Issues and PRs involving Hatchet's core GraphFrame datastructure and associated classes
area-utils
Issues and PRs related to Hatchets high-level API and other utility libraries
priority-normal
Normal priority issues and PRs
status-ready-for-review
This PR is ready to be reviewed by assigned reviewers
type-feature
Requests for new features or PRs which implement new features
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements a new function called
unify_ensemble
that takes a list ofGraphFrame
objects with equal graphs and returns a newGraphFrame
containing the data of all the inputs. In the output data, a newDataFrame
column, calleddataset
, is added that informs the user whichGraphFrame
that row came from. If thedataset
attribute of theGraphFrame
(explained below) is set, that value will be used for the corresponding rows in the output. Otherwise, the string "gframe_#" is used, with "#" being replaced by the index of theGraphFrame
in the input list.To help link output data to input data, this PR also adds a new
dataset
attribute to theGraphFrame
class and agraphframe_reader
decorator to help set this attribute. Thedataset
attribute is meant to be a string that labels theGraphFrame
. For most readers, this attribute will be set automatically by thegraphframe_reader
decorator. This decorator is meant to be applied tofrom_X
static methods in theGraphFrame
class. This decorator does 3 things:from_X
function it decoratesfrom_X
function did not set thedataset
attribute and the first argument tofrom_X
is a string, this first argument will be considered a path to the read data, and it will be used to setdataset
GraphFrame
produced byfrom_X