Refactor pyccl under an abstract CCLObject. (#934)

* added copy method

* first commit

* first commit

* moved caching functionality from global dict to inside of cached function

* improved docs and a few bug fixes

* improved docs

* improved docs

* moved in separate module

* changed default hashing behavior

* kept only caching-related stuff

* wrong Pk2D type should return TypeError

* cosmetics

* x_out in GNFW profile should trigger interp recomputation

* enable flake for constants.py

* expose CCLParameters

* return pknl if already computed

* comprehensive imports & specified __all__ for star-imports

* E402 is all over CCL and we silence it every time; get rid of it

* fixed typos

* kept only caching-related stuff

* first commit

* first commit

* stop supporting py27

* improve docs & expose stuff

* all tests import directly from tests/__init__.py

* all benchmarks import directly from tests/__init__.py

* more generous cache maxsize

* comprehensive docs for CCLObject

* document x_out behavior

* improved docs

* removed spurious import

* added convenience unwrapper

* UnlockInstance can now also unlock from kwargs

* separate CCL-default caching maxsize, policy from user-defined maxsize, policy & less code for cache decorator with or without parens

* more generous maxsize

* fixed typo in comment

* fixed docs for None maxsize

* first commit

* 'other' contains 'self'

* 'other' contains 'self'

* added bool dunder method

* correct implementation of non-commutative __dunders__ & simple psp checks

* tidy-up code for optional cosmo in Pk2D.eval

* querying C-level stuff via property

* removed requirement for gsl_params and spline_params to be const at the C-level; makes the entire PR shorter and simpler

* fix relative import

* use stored accuracy params in Cosmology.repr

* docstrings, frozen keys, reload option, tests

* flaked

* has_tsp property

* don't pickle swig spline types

* singleton implementation moved under base class

* tests for all new features

* a few more tests

* one more test

* update_parameters in Pk2D

* proper handling of updated accuracy params in Cosmology

* no need to unlock MG tracer instance

* remove huge mamba banner from installation output

* temporary bugfix for rho0 in HMCalculator

* enable func_from_name --> Func.from_name

* implemented MassDef.from_name

* HMCalculator from strings

* removed  anti-patterns

* clear all temporary vars in Cosmology

* combined type checking for

* tests for all new features & few fixes

* don't even store fastpt; reload time is negligible

* implement get global na/nk sampling without cosmo; init Pk2D from pkfunc without cosmo

* removed unnecessary test

* new docstrings

* option to call new funcs with or without cosmo

* straightforward memory optimization

* changed maxsize for calls to nonlin and linear power spectrum

* removed leftover attr

* Pk2D.from_model

* removed spurious cosmo from testing

* *sigh* py37-compatibility loses 1-line elegance

* functools.wraps to preserve docs

* more useful error when cosmicemu fails

* fix docs

* no need to create new cosmology type for a parameter-specific implementation

* optimization: reduced hash time by 30%

* simplified key-making; other minor improvements

* swapped hits/misses

* type-checking for __eq__

* fix typo

* fast-hash numbers & strings

* simplified MassDef implementations

* no need for errors to inherit from CCLObject

* compatibility for <py39

* 'fof' massdef also allowed

* import __repr__ directly into the class

* refactor repr using functools.cached_property

* fix typo

* removed Caching.toggle tests

* test against the defaults instead of manual inputs

* implemented thread lock

* updated docs

* updated warning in tracers.py

* some minor improvements

* updated readthedocs with new usage

* fixed F401

* a few simplifications

* thread safe mutation

* test seg fault

* maybe problem with ghw?

* removed cache from halo model (simple lookup is faster)

* removed option for consistent hashing; changed thread lock implementation

* tests to improve code coverage

* brought forward changes from caching

* fixed merge from default branch

* flaked

* improved coverage

* removed name property from MassDef

* immutable Pk2D

* removed i_operations from Pk2D (they're in another PR)

* immutable Cosmology

* minor improvements

* tests for CCLObject

* removed name from MassDef

* replace all swig lookups with python-level

* clear up namespace & allow T_CMB changing

* removed every trace of direct parameter assignment from code base

* Refactor: direct interface with SWIG; deprecate direct assignment; no python-level copy; no singleton.

* removed unecessary import

* save spline and gsl params in cosmo

* small improvement

* tests to improve coverage

* E303

* remove _type attribute

* simpler key creation

* temporarily fix __eq__ in MassDef

* tweak MassDef __eq__

* removed Numbers package requirement

* backwards changes

* simplify: make __eq__ not rely on __hash__; use __repr__ directly

* simplified particular Tracer implementations by removing subclasses

* introduced ABC functionality for all bases

* ABC for HaloProfile with linked abstract methods

* removed direct parameter assignment from benchmarks

* reload frozen parameters (physical constants)

* removed useless code

* correct repr

* verbose warning & added a test

* py38 compatibility

* tests to increase coverage

* fixed typo

* immutable Pk2D

* more abstractmethods

* improvements in Tracer

* sophisticated repr for Tracer

* restart coveralls

* test Tracer repr

* something wrong with github again

* added bool for Tracer

* manually rebase pk2d_extensions commit 1

* moved Tracer.__bool__ to pk2d_extensions

* Tracer chi_min and chi_max properties

* simplified implementation: distributred CCLObject instructions to CCLHalosObject

* no need to pass if docstring is provided

* introduce CCLObjectMeta to simplifiy link_abstractmethods

* implemented ObjectLock for CCLObjects

* removed spurious star import

* simplified auto_assign & usage of linked abstract methods

* test for ObjectLock repr

* improved docstrings

* fixes to homogenize merge from default branch

* removed relative pyccl imports

* minimize differences from base

* fix Cosmology docstring

* option to enable caching in testing

* fix test_profiles

* revert NFW analytic Fourier

* flake conftest

* minor changes

* removed unecessary check in tracers

* deleted benchmark init

* cleared some diffs to make it easier to review + addressed comments

* git push test

* git push test 2

* remove auto_assign; control repr via __repr_attrs__ hook

* relax __repr_attrs__ requirement in abstract classes

* implementation improvement of __repr_attrs__; fallback mechanism

* removed __linked_abstractmethods__; reverted some previous changes

* add check on instantiation

* fix tests

* covtest for HaloProfile

* fix typo

* enable central control of fancy reprs

* tests for fancy-repr

* FancyRepr --> framework pattern (disable instantiation)

* addressed comments

* added extra test

* add test for subclass-level fancy-repr disable/enable failure

benchmarks/conftest.py

@@ -0,0 +1,13 @@
+"""Config file for pytest. Allow caching for faster test completion.
+`$ pytest tests/ --use-cache`
+"""
+
+
+def pytest_addoption(parser):
+    parser.addoption("--use-cache", action="store_true", help="Enable cache.")
+
+
+def pytest_generate_tests(metafunc):
+    if metafunc.config.getoption("use_cache"):
+        import pyccl
+        pyccl.Caching.enable()

benchmarks/data/codes/cl_corr_bm/run_all.py

@@ -9,7 +9,7 @@ def run_task(window_type,bins_type,b1,b2) :
     elif bins_type=="log" :
         do_logbin=1
     else :
-        print "Wrong binning scheme "+bins_type
+        print("Wrong binning scheme "+bins_type)
 
     strout="#Cosmological parameters\n"
     strout+="omega_m= 0.3\n"
@@ -66,4 +66,3 @@ def run_task(window_type,bins_type,b1,b2) :
 run_task("histo","log",1,1)
 run_task("histo","log",1,2)
 run_task("histo","log",2,2)
-

benchmarks/data/codes/param_space.py

@@ -207,8 +207,8 @@ def generate_ccl_pspec(sample_points, root, class_data_root, zvals,
             except KeyboardInterrupt:
                 raise
             except:
-                print "--- Error running CLASS"
-                print "--- Parameters:", params
+                print("--- Error running CLASS")
+                print("--- Parameters:", params)
                 errored.append(i)
                 continue
 
@@ -221,9 +221,9 @@ def generate_ccl_pspec(sample_points, root, class_data_root, zvals,
                 np.savetxt(fname_lin, np.column_stack((k_class, pk_lin)))
 
     # Print runs that errored
-    if len(errored) > 0: print "ERRORED:"
+    if len(errored) > 0: print("ERRORED:")
     for err in errored:
-        print err
+        print(err)
 
 
 def generate_class_ini(sample_points, root, nonlinear=False, mnu=False,
@@ -448,7 +448,7 @@ def load_summary_stats(sample_points, ccl_data_root, class_data_root,
 
     # Loop over sample points in parameter space and calculate summary stats
     for i in range(N_samp):
-        print "  Loading power spectra for parameter set %05d" % i
+        print("  Loading power spectra for parameter set %05d" % i)
 
         # Get Hubble parameter, h, for rescaling CLASS P(k) to Mpc units
         h = sample_points['h'][i]
@@ -476,7 +476,7 @@ def load_summary_stats(sample_points, ccl_data_root, class_data_root,
             class_pk = pk_class_dat[:,1] / h**3.
 
             # Sanity checks
-            print ccl_pk.size, class_pk.size
+            print(ccl_pk.size, class_pk.size)
             assert ccl_pk.size == class_pk.size
 
             # Calculate fractional deviation
@@ -533,7 +533,7 @@ def ccl_summary_stats(params,
     if cache_name is not None:
         try:
             stats = np.load("%s.npy" % cache_name)
-            print "  Loaded '%s' from cache." % cache_name
+            print("  Loaded '%s' from cache." % cache_name)
             assert stats.shape == (N_samp, N_thres, N_z, N_kbins)
             return stats, params
         except:
@@ -546,7 +546,7 @@ def ccl_summary_stats(params,
     # Loop over sample points in parameter space and calculate summary stats
     for i in range(N_samp):
         trial = params['id'][i]
-        print "  Loading CCL power spectra for parameter set %05d" % i
+        print("  Loading CCL power spectra for parameter set %05d" % i)
 
         # Loop over redshift values
         for j in range(N_z):

pyccl/__init__.py

@@ -23,11 +23,15 @@
 # SWIG-generated
 from . import ccllib as lib
 
-# CCL base
+# Hashing, Caching, CCL base, Mutation locks
 from .base import (
+    CCLObject,
+    CCLHalosObject,
     Caching,
     cache,
     hash_,
+    UnlockInstance,
+    unlock_instance,
 )
 
 # Errors
@@ -167,9 +171,9 @@
 
 
 __all__ = (
-    'lib',
-    'Caching', 'cache', 'hash_',
-    'CCLParameters', 'spline_params', 'gsl_params', 'physical_constants',
+    'lib', 'Caching', 'cache', 'hash_', 'CCLObject', 'CCLHalosObject',
+    'UnlockInstance', 'unlock_instance',
+    'CCLParameters', 'physical_constants', 'gsl_params', 'spline_params',
     'CCLError', 'CCLWarning', 'CCLDeprecationWarning',
     'Cosmology', 'CosmologyVanillaLCDM', 'CosmologyCalculator',
     'growth_factor', 'growth_factor_unnorm', 'growth_rate',