-
-
Notifications
You must be signed in to change notification settings - Fork 26.1k
[MRG+2] OPTICS: add extract_xi method #12077
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
107 commits
Select commit
Hold shift + click to select a range
3ab2e63
add extract method parameter
adrinjalali 7190098
change fancy to sqlnk
adrinjalali eb12095
test for invalid extract_method value
adrinjalali 58913d8
test easy dbscan
adrinjalali 4b06ba8
pep8
adrinjalali c9819d2
add extract_sqlnk
adrinjalali 61b440f
mention which parameter is used for which extact method
adrinjalali b8fcc9c
extractXi in progress
adrinjalali 8af9364
remove R inspired implementation, starting over
adrinjalali 6e4a467
first draft ready
adrinjalali b791606
improved, but not detecting the last cluster
adrinjalali 2695f2b
some edge cases solved
adrinjalali 2b4cd24
regions are almost correct, global mib is not
adrinjalali b5830e1
edges between clusters fixed
adrinjalali 08a0f9f
edges between clusters fixed
adrinjalali 8b182b8
issues of the first test case fixed
adrinjalali 0f0f157
add extract_xi to OPTICS
adrinjalali 41244a0
fix label order issue
adrinjalali 2bb2c77
add docstrings
adrinjalali 8ec6df2
missing - sign
adrinjalali 5ce8952
merge optics/choose_extractor
adrinjalali 2655edb
continue merge
adrinjalali f2fe4e7
add xi parameter to OPTICS
adrinjalali 9a65082
add tests for extract methods with no params given
adrinjalali bfbe759
add xi to docstrings
adrinjalali d599a03
fix a few more issues with extract_xi integration
adrinjalali baa94f9
add a xi test
adrinjalali bff04ee
merge master
adrinjalali f1c6cf1
merge master
adrinjalali c16abdc
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali df261c0
merge master and choose_extractor
adrinjalali 02058e9
trying to base on the latest
adrinjalali 309f4a2
trying to base on the latest
adrinjalali a9b3c7e
fix docstring
adrinjalali 76f7101
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali f2b50bd
pep8
adrinjalali a72c67e
pep8
adrinjalali 93640e0
docstring fixes
adrinjalali e80d17f
better same `xi` docstring everywhere
adrinjalali d36076f
address some of Joel's comments
adrinjalali 17de182
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali 9bd42ab
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali a7ba939
fixes and tests for smaller functions
adrinjalali 7c091c4
fixing some minor issues and adding more low level tests
adrinjalali 52b409f
fix test
adrinjalali 613bd5c
test unorered labeling
adrinjalali 15e5dd7
add and pass more tests
adrinjalali 00a2c6f
add comment
adrinjalali 29c63d4
pep8
adrinjalali edd94ff
randomize input in test
adrinjalali 86217ce
add predecessor correction
adrinjalali a077a67
pep8
adrinjalali 392d6cc
docstring cleanup and removal of extra flawed 3.b condition
adrinjalali fd39b77
merge min_cluster_size and min_samples
adrinjalali 6092a45
add edge case tests and fix an mib issue
adrinjalali 29b61b8
set the min_samples for OPTICS in common tests
adrinjalali c4629b5
pep8
adrinjalali 4c930c2
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali b9d8582
add back optics to docs
adrinjalali 08d4b58
add optics example
adrinjalali c41da4c
fine tune examples, remove prints
adrinjalali c876e1f
put back min_cluster_size
adrinjalali 175c074
fix min_samples to NN
adrinjalali 992c81e
fix examples
adrinjalali 13bc11b
expose clusters_ as array
adrinjalali 14021ad
use utils.shuffle
adrinjalali 4bd96ab
add clusters_ test
adrinjalali da1cbec
restructure and use numpy vector operations
adrinjalali ec2e379
iterate only through steep points
adrinjalali b480842
cleanup the prints
adrinjalali 10f6ec2
k, c -> klass, color
adrinjalali 9fc9935
address optics_.py comments
adrinjalali 1d924c9
fix documentation and remove *5 factor from eps
adrinjalali 5bfc27b
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali a4533fa
pep8
adrinjalali 5705daa
address some of Joel's comments
adrinjalali 09e125b
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali c79820b
more fixes and tests pass
adrinjalali 1df4a75
address more comments
adrinjalali 3c03e99
finalize fixes and address remaining comments
adrinjalali 4d84f94
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali c031196
address comments
adrinjalali 17e7784
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali a974376
more addressing comments
adrinjalali 0b8e161
fix min_cluster_size as 1
adrinjalali 293ace7
apply Hanmin's comments
adrinjalali 68ade1b
merge upstream/master
adrinjalali f61516f
fix predecessor issue and apply other comments
adrinjalali 1ffa5d4
pep8
adrinjalali ebe0da8
one of the common tests is already fixed
adrinjalali 8470539
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali f6fcfc8
update OPTICS's min_samples docstring
adrinjalali 4e2e2e7
fix docstring
adrinjalali 34f954a
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali d778dbe
apply Hanmin's review
adrinjalali 9f20ebc
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali d76b2a5
apply more comments
adrinjalali e1c6086
Merge remote-tracking branch 'upstream/master' into optics/extractXi
adrinjalali 1792605
apply hanmin's fix for predecessor correction
adrinjalali 44240d5
apply some of hanmin's style
adrinjalali 310c2a0
Revert "apply hanmin's fix for predecessor correction"
adrinjalali d83c85d
fix predecessor correction (again!)
adrinjalali c76dee7
add comment to test
adrinjalali c96a32e
rename predecessor to predecessor_plot where needed
adrinjalali ab34b53
apply more of Hanmin's style
adrinjalali be9812b
credit in whats_new
adrinjalali c1e5cde
fix assia's name
adrinjalali File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,98 @@ | ||
""" | ||
=================================== | ||
Demo of OPTICS clustering algorithm | ||
=================================== | ||
Finds core samples of high density and expands clusters from them. | ||
This example uses data that is generated so that the clusters have | ||
different densities. | ||
The :class:`sklearn.cluster.OPTICS` is first used with its Xi cluster detection | ||
method, and then setting specific thresholds on the reachability, which | ||
corresponds to :class:`sklearn.cluster.DBSCAN`. We can see that the different | ||
clusters of OPTICS's Xi method can be recovered with different choices of | ||
thresholds in DBSCAN. | ||
""" | ||
|
||
# Authors: Shane Grigsby <refuge@rocktalus.com> | ||
# Adrin Jalali <adrin.jalali@gmail.com> | ||
# License: BSD 3 clause | ||
|
||
|
||
from sklearn.cluster import OPTICS, cluster_optics_dbscan | ||
import matplotlib.gridspec as gridspec | ||
import matplotlib.pyplot as plt | ||
import numpy as np | ||
|
||
# Generate sample data | ||
|
||
np.random.seed(0) | ||
n_points_per_cluster = 250 | ||
|
||
C1 = [-5, -2] + .8 * np.random.randn(n_points_per_cluster, 2) | ||
C2 = [4, -1] + .1 * np.random.randn(n_points_per_cluster, 2) | ||
C3 = [1, -2] + .2 * np.random.randn(n_points_per_cluster, 2) | ||
C4 = [-2, 3] + .3 * np.random.randn(n_points_per_cluster, 2) | ||
C5 = [3, -2] + 1.6 * np.random.randn(n_points_per_cluster, 2) | ||
C6 = [5, 6] + 2 * np.random.randn(n_points_per_cluster, 2) | ||
X = np.vstack((C1, C2, C3, C4, C5, C6)) | ||
|
||
clust = OPTICS(min_samples=50, xi=.05, min_cluster_size=.05) | ||
|
||
# Run the fit | ||
clust.fit(X) | ||
|
||
labels_050 = cluster_optics_dbscan(reachability=clust.reachability_, | ||
core_distances=clust.core_distances_, | ||
ordering=clust.ordering_, eps=0.5) | ||
labels_200 = cluster_optics_dbscan(reachability=clust.reachability_, | ||
core_distances=clust.core_distances_, | ||
ordering=clust.ordering_, eps=2) | ||
|
||
space = np.arange(len(X)) | ||
reachability = clust.reachability_[clust.ordering_] | ||
labels = clust.labels_[clust.ordering_] | ||
|
||
plt.figure(figsize=(10, 7)) | ||
G = gridspec.GridSpec(2, 3) | ||
ax1 = plt.subplot(G[0, :]) | ||
ax2 = plt.subplot(G[1, 0]) | ||
ax3 = plt.subplot(G[1, 1]) | ||
ax4 = plt.subplot(G[1, 2]) | ||
|
||
# Reachability plot | ||
colors = ['g.', 'r.', 'b.', 'y.', 'c.'] | ||
for klass, color in zip(range(0, 5), colors): | ||
Xk = space[labels == klass] | ||
Rk = reachability[labels == klass] | ||
ax1.plot(Xk, Rk, color, alpha=0.3) | ||
ax1.plot(space[labels == -1], reachability[labels == -1], 'k.', alpha=0.3) | ||
ax1.plot(space, np.full_like(space, 2., dtype=float), 'k-', alpha=0.5) | ||
ax1.plot(space, np.full_like(space, 0.5, dtype=float), 'k-.', alpha=0.5) | ||
ax1.set_ylabel('Reachability (epsilon distance)') | ||
ax1.set_title('Reachability Plot') | ||
|
||
# OPTICS | ||
colors = ['g.', 'r.', 'b.', 'y.', 'c.'] | ||
for klass, color in zip(range(0, 5), colors): | ||
Xk = X[clust.labels_ == klass] | ||
ax2.plot(Xk[:, 0], Xk[:, 1], color, alpha=0.3) | ||
ax2.plot(X[clust.labels_ == -1, 0], X[clust.labels_ == -1, 1], 'k+', alpha=0.1) | ||
ax2.set_title('Automatic Clustering\nOPTICS') | ||
|
||
# DBSCAN at 0.5 | ||
colors = ['g', 'greenyellow', 'olive', 'r', 'b', 'c'] | ||
for klass, color in zip(range(0, 6), colors): | ||
Xk = X[labels_050 == klass] | ||
ax3.plot(Xk[:, 0], Xk[:, 1], color, alpha=0.3, marker='.') | ||
ax3.plot(X[labels_050 == -1, 0], X[labels_050 == -1, 1], 'k+', alpha=0.1) | ||
ax3.set_title('Clustering at 0.5 epsilon cut\nDBSCAN') | ||
|
||
# DBSCAN at 2. | ||
colors = ['g.', 'm.', 'y.', 'c.'] | ||
for klass, color in zip(range(0, 4), colors): | ||
Xk = X[labels_200 == klass] | ||
ax4.plot(Xk[:, 0], Xk[:, 1], color, alpha=0.3) | ||
ax4.plot(X[labels_200 == -1, 0], X[labels_200 == -1, 1], 'k+', alpha=0.1) | ||
ax4.set_title('Clustering at 2.0 epsilon cut\nDBSCAN') | ||
|
||
plt.tight_layout() | ||
plt.show() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.