Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update testing infrastructure for pytest #5

Merged
merged 11 commits into from
May 4, 2016
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,13 @@ config.log
*_wrap.cc
*Lib.py
doc/html
doc/xml
doc/*.tag
doc/*.inc
doc/doxygen.conf
tests/.tests
tests/.cache
version.py
tests/ieee
examples/demangle
examples/demangle
_build.*
6 changes: 3 additions & 3 deletions doc/main.dox
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ namespace lsst { namespace utils {
This is a collection of miscellaneous utility functions, classes and macros, including:
- lsst/p_lsstSwig.i useful macros for SWIG interface files
- \ref lsst.utils.tests utilties to aid in unit testing
- \ref lsst::utils::eups::productDir provides C++ code access to the eups productDir function
(Python code simply import eups for to access this function)
- \ref lsst::utils::getPackageDir and lsst.utils.getPackageDir provide an equivalent
to the eups productDir function
*/

}} // namespace lsst::skymap
}} // namespace lsst::utils
15 changes: 8 additions & 7 deletions python/lsst/__init__.py
Original file line number Diff line number Diff line change
@@ -1,24 +1,25 @@
#
#
# LSST Data Management System
# Copyright 2008, 2009, 2010 LSST Corporation.
#
#
# This product includes software developed by the
# LSST Project (http://www.lsst.org/).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
# see <http://www.lsstcorp.org/LegalNotices/>.
#

import lsstimport, pkgutil
import lsstimport
import pkgutil
__path__ = pkgutil.extend_path(__path__, __name__)
12 changes: 6 additions & 6 deletions python/lsst/utils/__init__.py
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
#
#
# LSST Data Management System
# Copyright 2008, 2009, 2010 LSST Corporation.
#
#
# This product includes software developed by the
# LSST Project (http://www.lsst.org/).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
# see <http://www.lsstcorp.org/LegalNotices/>.
#

Expand Down
61 changes: 31 additions & 30 deletions python/lsst/utils/multithreading/SharedData.py
Original file line number Diff line number Diff line change
@@ -1,29 +1,30 @@
#
#
# LSST Data Management System
# Copyright 2008, 2009, 2010 LSST Corporation.
#
#
# This product includes software developed by the
# LSST Project (http://www.lsst.org/).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
# see <http://www.lsstcorp.org/LegalNotices/>.
#

#
from __future__ import with_statement
import threading


class SharedData(object):
"""
a lock-protected container for data that can be shared amongst threads.
Expand All @@ -32,9 +33,9 @@ class SharedData(object):
threads. In order to update or (optionally) examine the data, one must
first aquire the lock associated with the container.

This container also behaves like a threading.Container, via its
wait(), notify(), and notifyAll() functions. Also like Condition,
acquire() is reentrant.
This container also behaves like a threading.Container, via its
wait(), notify(), and notifyAll() functions. Also like Condition,
acquire() is reentrant.

SharedData instances may be used with the with statement:

Expand All @@ -43,17 +44,17 @@ class SharedData(object):
sd.blah = 1

The with statement will acquire the lock and ensure that it is released
when its block is exited.
when its block is exited.
"""

def __init__(self, needLockOnRead=True, data=None, cond=None):
"""
create and initialize the shared data
@param needLockOnRead if true (default), acquiring the lock will
@param needLockOnRead if true (default), acquiring the lock will
be needed when reading the data. This is recommended
if the data items are anything but primitive types;
otherwise, a compound data item (e.g. of type dict)
could be updated without acquiring a lock.
could be updated without acquiring a lock.
@param data a dictionary of data to initialize the container with.
This is done by calling initData(). Set this value to
False when calling from a subclass constructor; this
Expand All @@ -70,11 +71,11 @@ def __init__(self, needLockOnRead=True, data=None, cond=None):
self._cond = cond

# behave like a Condition
self.acquire = cond.acquire
self.release = cond.release
self.notify = cond.notify
self.acquire = cond.acquire
self.release = cond.release
self.notify = cond.notify
self.notifyAll = cond.notifyAll
self.wait = cond.wait
self.wait = cond.wait
self._is_owned = cond._is_owned

self._lockOnRead = needLockOnRead
Expand All @@ -85,40 +86,41 @@ def __init__(self, needLockOnRead=True, data=None, cond=None):
self._d["__"] = True

# behave like a Condition
def __enter__(self): return self._cond.__enter__();
def __exit__(self, *exc_info): return self._cond.__exit__(*exc_info);

def __enter__(self):
return self._cond.__enter__()

def __exit__(self, *exc_info):
return self._cond.__exit__(*exc_info)

def __getattribute__(self, name):
if name == "_d" or len(self._d) == 0 or not self._d.has_key(name):
if name == "_d" or len(self._d) == 0 or name not in self._d:
return object.__getattribute__(self, name)

if self._lockOnRead and not self._is_owned():
raise AttributeError("%s: lock required for read access" % name)
return self._d[name]


def __setattr__(self, name, value):
if name == "_d" or len(self._d) == 0 or name in self.__dict__.keys():
object.__setattr__(self, name, value)
return
return

if not self._is_owned():
raise AttributeError("%s: lock required for write access" % name)

self._d[name] = value


def initData(self, data):
"""
initialize the container with the data from a dictionary.
initialize the container with the data from a dictionary.
@param data a dictionary of data to initialize the container with.
Attributes will be added to this container with names
matching the given the dictionary's key names and
initialized to their corresponding values. The keys
cannot match an existing function (or internal attribute)
name.
@throws ValueError if the dictionary has a key that conflicts with
an existing function or internal attribute name.
an existing function or internal attribute name.
"""
with self._cond:
bad = []
Expand All @@ -134,8 +136,7 @@ def initData(self, data):
self._d[key] = data[key]

if len(self._d) == 0:
self._d["__"] = True
self._d["__"] = True

def dir(self):
return filter(lambda k: k != "__", self._d.keys())

17 changes: 9 additions & 8 deletions python/lsst/utils/multithreading/__init__.py
Original file line number Diff line number Diff line change
@@ -1,24 +1,25 @@
#
from __future__ import absolute_import
#
# LSST Data Management System
# Copyright 2008, 2009, 2010 LSST Corporation.
#
#
# This product includes software developed by the
# LSST Project (http://www.lsst.org/).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
# see <http://www.lsstcorp.org/LegalNotices/>.
#

from SharedData import SharedData
from lockProtection import *
from .SharedData import SharedData
from .lockProtection import *
24 changes: 13 additions & 11 deletions python/lsst/utils/multithreading/lockProtection.py
Original file line number Diff line number Diff line change
@@ -1,39 +1,40 @@
#
#
# LSST Data Management System
# Copyright 2008, 2009, 2010 LSST Corporation.
#
#
# This product includes software developed by the
# LSST Project (http://www.lsst.org/).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
# see <http://www.lsstcorp.org/LegalNotices/>.
#

"""
classes used to protect method from simultaneous access by different
threads. The primary class is LockProtected.
threads. The primary class is LockProtected.
"""
import threading

SharedLock = threading.RLock


class LockProtected(object):
"""
a class that assists in keeping methods thread-safe.

This class is intended to be enlisted as a base class to the class being
protected. A method that is to be protected from simultaneous access
protected. A method that is to be protected from simultaneous access
would include (at its beginning) a call to _checkLocked():

class MyClass(BaseClass, LockProtected):
Expand All @@ -45,7 +46,7 @@ def dangerous(self):
self._checkLocked()
...

Doing so will require that the protected class be "locked" before a
Doing so will require that the protected class be "locked" before a
protected method can be called or else an UnsafeAccessError will be
raised. Locking is done via the with statement:

Expand All @@ -66,7 +67,7 @@ def dangerous(self):
access to any protected method across multiple class instances. To
accomplish this, each class to be protected should accept a lock as
a constructor argument. Then the same lock is passed into all of the
constructors that need protection together.
constructors that need protection together.
"""

def __init__(self, lock=None):
Expand All @@ -91,13 +92,14 @@ def _checkLocked(self):
if self._lp_lock and not self._lp_lock._is_owned():
raise UnsafeAccessError()


class UnsafeAccessError(Exception):
"""
an exception that is raised when one attempts to access a protected
method without first acquiring the lock on its class.
"""

def __init__(self, msg=None):
if not msg:
msg = "Programmer Error: failed to obtain lock via with statement"
Exception.__init__(self, msg)