Repository navigation
Connected components correctness fixes
- Python
connectedComponents(algorithm="graphx")no longer stops after 31 iterations: themax_iterdefault was written2 ^ 31 - 2(XOR, not power), so graphs needing more than 31 supersteps silently returned split components randomized_contractionnow maps its output back to non-integral (e.g. string) vertex IDs whenuseLabelsAsComponents = false, instead of failing withCAST_INVALID_INPUTunder ANSI or returning wrong resultsrandomized_contractioncleans up its checkpoint directory when it fails, not only on successKCoreandrandomized_contractionregister their temp functions with qualified identifiers, fixingFunction identifier must be fully qualifiedon Spark 4.2
What's Changed
- fix: point the CC benchmark row at randomized_contraction by @james-willis in #891
- fix: map randomized_contraction output back to non-integral vertex IDs by @james-willis in #893
- fix: clean randomized contraction checkpoints on failure by @SauronShepherd in #895
- ci: run python lint on tests/ and docs/, as its own job by @james-willis in #896
- doc: Pregel Tutorial by @rjurney in #809
- fix: use qualified function identifiers for temp functions by @ledanieln in #900
- fix: connectedComponents max_iter default evaluated to 31 due to ^ precedence by @simplegaurav in #902
- docs: sync connected components docs with the code by @simplegaurav in #903
- docs: randomized_contraction always needs a checkpoint directory by @yuchaoran2011 in #909
Full Changelog: v0.12.2...v0.12.3