added flag for clean columns for joining table dataset with source dataset #302
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ywkim312
requested review from
jonglee1,
longshuicy,
navarroc,
Rashmil-1999 and
Vismayak
|
@ywkim312 Good to know that we already have a method for that. Im most curious about how fast it is? Could you have some benchmark for the SLC data I shared? |
Actually, it is very hard to do make a fair comparison because this one happens in the local machine but the service's join happens in the server side. However, using this code in pyincore is very meaningful because it will reduce the workload from the server significantly and we can avoid some errors like time out. |
longshuicy
requested changes
May 6, 2024
longshuicy
reviewed
May 6, 2024
longshuicy
approved these changes
May 6, 2024
ylyangtw
approved these changes
May 14, 2024
ywkim312
restored the
299-investigate-joincreating-shapefiles-using-pandas
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@longshuicy @jonglee1 @navarroc There was already an method for joining the table dataset with source dataset. I just added another variable to clean up the columns. If the new variable is true, it will remove all the columns from source dataset except 'geometry' and 'guid'. So eventually. the output geodataframe will only have the columns from table dataset with geometry. The user can easily convert this GeoDataFrame to shapefile or geopakage using geopands.
I am not very sure how we will handle this join process at this moment. We probably need to discuss more how to do it. However, having this PR will be helpful in some cases