Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement/select device #488

Merged
merged 59 commits into from
Jun 16, 2020
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
042bea1
rewrite selection
mtar Feb 14, 2020
590e765
fix wrong skippping in test_device.py
mtar Feb 14, 2020
1ae04f4
use variable
mtar Feb 14, 2020
65012ea
add missing references
mtar Feb 14, 2020
32a9ab5
sort functions in distance.py following stylistic guideline
mtar Feb 14, 2020
e3c3ed7
revert
mtar Feb 14, 2020
5832958
add missing device in distance.py
mtar Feb 14, 2020
4d0c63a
add second missing device
mtar Feb 14, 2020
b6b9d5e
update header
mtar Feb 14, 2020
f75b80c
add missing devices
mtar Feb 14, 2020
8eea734
add header test_devices
mtar Feb 14, 2020
8010878
remove old code
mtar Feb 14, 2020
f1ae63f
fix assign new device
mtar Feb 14, 2020
fc3b38e
fix missing heat device change on DNDarray.cpu
mtar Feb 14, 2020
1227183
Merge branch 'master' into enhancement/select-device
mtar Feb 17, 2020
bd45727
changelog
mtar Feb 17, 2020
680ad9c
Merge branch 'master' into enhancement/select-device
Markus-Goetz Feb 18, 2020
7ac3282
Merge branch 'master' into enhancement/select-device
Markus-Goetz Feb 18, 2020
df11692
refactor the 'device header' into a seperate module
mtar Mar 6, 2020
fc02990
formatting
mtar Mar 6, 2020
a44f03b
Merge branch 'master' into enhancement/select-device
mtar Mar 16, 2020
58ec0b1
change envar name & changelog
mtar Mar 23, 2020
335d955
revert heat_device back to ht_device
mtar Mar 23, 2020
768246f
move code to device.py
mtar Mar 24, 2020
c453aab
Merge branch 'master' into enhancement/select-device
mtar Mar 24, 2020
816ff85
add change requests
mtar Mar 31, 2020
613e597
Merge branch 'enhancement/select-device' of github.com:helmholtz-anal…
mtar Mar 31, 2020
9535102
Merge branch 'master' into enhancement/select-device
coquelin77 Apr 16, 2020
54f3a85
removed lasso tests added by github merge
coquelin77 Apr 16, 2020
9e8ed07
super setupclass
mtar Apr 28, 2020
0af54b1
change naming
mtar Apr 28, 2020
cfa24c7
Merge branch 'master' into enhancement/select-device
mtar Apr 28, 2020
04836d7
adjusting names
mtar Apr 28, 2020
cbdaa6b
black formatting
mtar Apr 28, 2020
17e7293
naming conventions
mtar Apr 28, 2020
2374ae8
Merge branch 'master' into enhancement/select-device
Markus-Goetz Apr 29, 2020
bac8ec6
remove unneccessary functions
mtar Apr 29, 2020
ffd6e7b
rename BasicTest completely
mtar Apr 30, 2020
b355451
Merge branch 'master' into enhancement/select-device
Markus-Goetz Apr 30, 2020
36efc80
rename BasicTest to TestCase & remove local cases
mtar Apr 30, 2020
c7c596e
Merge branch 'enhancement/select-device' of github.com:helmholtz-anal…
mtar Apr 30, 2020
a4ca8db
rename class members & update tests
mtar May 4, 2020
79925b5
remove unsused variable
mtar May 4, 2020
093ab69
update test_lasso
mtar May 4, 2020
499cc1e
Merge branch 'master' into enhancement/select-device
mtar Jun 3, 2020
1b178cd
Merge branch 'master' into enhancement/select-device
mtar Jun 4, 2020
1ffe420
Update CHANGELOG.md
mtar Jun 4, 2020
d9eac6c
remove device parameters in new tests
mtar Jun 4, 2020
f269458
Merge branch 'master' into enhancement/select-device
mtar Jun 5, 2020
6eb50f1
Merge branch 'master' into enhancement/select-device
mtar Jun 9, 2020
4e98d25
Merge branch 'master' into enhancement/select-device
mtar Jun 10, 2020
ff195ee
Merge branch 'master' into enhancement/select-device
mtar Jun 16, 2020
2d94115
add raises paragraph
mtar Jun 16, 2020
920b973
black
mtar Jun 16, 2020
6fb10f8
Merge branch 'master' into enhancement/select-device
Markus-Goetz Jun 16, 2020
9799411
update torch_device
mtar Jun 16, 2020
31f2438
Merge branch 'master' into enhancement/select-device
mtar Jun 16, 2020
fbda362
black formatting
mtar Jun 16, 2020
01eccf1
doc
mtar Jun 16, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
- [#496](https://github.com/helmholtz-analytics/heat/pull/496) New feature: flipud()
- [#498](https://github.com/helmholtz-analytics/heat/pull/498) Feature: flip()
- [#499](https://github.com/helmholtz-analytics/heat/pull/499) Bugfix: MPI datatype mapping: `torch.int16` now maps to `MPI.SHORT` instead of `MPI.SHORT_INT`
- [#488](https://github.com/helmholtz-analytics/heat/pull/488) Enhancement: Rework of the test device selection. Environment variable is now called HEAT_USE_DEVICE and has more options.

- [#501](https://github.com/helmholtz-analytics/heat/pull/501) New Feature: flatten
- [#506](https://github.com/helmholtz-analytics/heat/pull/506) Bugfix: setup.py has correct version parsing
- [#507](https://github.com/helmholtz-analytics/heat/pull/507) Bugfix: sanitize_axis changes axis of 0-dim scalars to None
Expand Down
24 changes: 10 additions & 14 deletions heat/cluster/tests/test_kmeans.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,14 @@

import heat as ht

if os.environ.get("DEVICE") == "gpu" and ht.torch.cuda.is_available():
ht.use_device("gpu")
ht.torch.cuda.set_device(ht.torch.device(ht.get_device().torch_device))
else:
ht.use_device("cpu")
device = ht.get_device().torch_device
ht_device = None
if os.environ.get("DEVICE") == "lgpu" and ht.torch.cuda.is_available():
device = ht.gpu.torch_device
ht_device = ht.gpu
ht.torch.cuda.set_device(device)
from heat.core.tests.test_suites.basic_test import BasicTest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably be called TestCase as well

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be imported as TestCase or should the name be changed in general?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name should be changed in general in my opinion



class TestKMeans(unittest.TestCase):
class TestKMeans(BasicTest):
@classmethod
def setUpClass(cls):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be explicitly called here? Should be automatically done due to inheritance

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ones without already existing setup functions can be removed. Good.

super(TestKMeans, cls).setUpClass()

def test_clusterer(self):
kmeans = ht.cluster.KMeans()
self.assertTrue(ht.is_estimator(kmeans))
Expand All @@ -38,7 +32,9 @@ def test_get_and_set_params(self):
def test_fit_iris_unsplit(self):
for split in [None, 0]:
# get some test data
iris = ht.load("heat/datasets/data/iris.csv", sep=";", split=split)
iris = ht.load(
"heat/datasets/data/iris.csv", sep=";", split=split, device=self.ht_device
)

# fit the clusters
k = 3
Expand All @@ -58,7 +54,7 @@ def test_fit_iris_unsplit(self):

def test_exceptions(self):
# get some test data
iris_split = ht.load("heat/datasets/data/iris.csv", sep=";", split=1)
iris_split = ht.load("heat/datasets/data/iris.csv", sep=";", split=1, device=self.ht_device)

# build a clusterer
k = 3
Expand Down
22 changes: 8 additions & 14 deletions heat/cluster/tests/test_spectral.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,14 @@

import heat as ht

if os.environ.get("DEVICE") == "gpu" and ht.torch.cuda.is_available():
ht.use_device("gpu")
ht.torch.cuda.set_device(ht.torch.device(ht.get_device().torch_device))
else:
ht.use_device("cpu")
device = ht.get_device().torch_device
ht_device = None
if os.environ.get("DEVICE") == "lgpu" and ht.torch.cuda.is_available():
device = ht.gpu.torch_device
ht_device = ht.gpu
ht.torch.cuda.set_device(device)
from heat.core.tests.test_suites.basic_test import BasicTest


class TestSpectral(unittest.TestCase):
class TestSpectral(BasicTest):
@classmethod
def setUpClass(cls):
super(TestSpectral, cls).setUpClass()

def test_clusterer(self):
spectral = ht.cluster.Spectral()
self.assertTrue(ht.is_estimator(spectral))
Expand Down Expand Up @@ -46,7 +40,7 @@ def test_get_and_set_params(self):

def test_fit_iris(self):
# get some test data
iris = ht.load("heat/datasets/data/iris.csv", sep=";", split=0)
iris = ht.load("heat/datasets/data/iris.csv", sep=";", split=0, device=self.ht_device)
m = 10

# fit the clusters
Expand Down Expand Up @@ -84,7 +78,7 @@ def test_fit_iris(self):
with self.assertRaises(NotImplementedError):
spectral = ht.cluster.Spectral(metric="ahalanobis", n_lanczos=m)

iris_split = ht.load("heat/datasets/data/iris.csv", sep=";", split=1)
iris_split = ht.load("heat/datasets/data/iris.csv", sep=";", split=1, device=self.ht_device)
spectral = ht.cluster.Spectral(n_lanczos=20)
with self.assertRaises(NotImplementedError):
spectral.fit(iris_split)
Loading