You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We should get all the files from a repository, get the blocks information, and aggregate repositories by datanodes with more block of each repository.
With this information we need to create a new class called RepositoryPartition that extends the trait org.apache.spark.Partition, that will include a list of repository folders.
This partitions will be sent to each relation to create RDD partitions correctly, depending of the locality.
The text was updated successfully, but these errors were encountered:
Repositories will be in a specific folder. Example:
We should get all the files from a repository, get the blocks information, and aggregate repositories by datanodes with more block of each repository.
With this information we need to create a new class called RepositoryPartition that extends the trait org.apache.spark.Partition, that will include a list of repository folders.
This partitions will be sent to each relation to create RDD partitions correctly, depending of the locality.
The text was updated successfully, but these errors were encountered: