Merge pull request #247 from sciris/rc1.3.0

Rc1.3.0

.github/workflows/tests.yaml

@@ -9,7 +9,7 @@ jobs:
       fail-fast: false
       max-parallel: 8
       matrix:
-        python-version: [ '3.8' ]
+        python-version: [ '3.9' ]
     name: Install and test
     steps:
       - name: Checkout sources
@@ -19,7 +19,7 @@ jobs:
           python-version: ${{ matrix.python-version }}
           architecture: x64
       - name: Install Sciris
-        run: python setup.py develop
+        run: pip install -e .
       - name: Install tests
         run: pip install pytest
       - name: Run API tests

CHANGELOG.rst

@@ -5,6 +5,54 @@ All notable changes to this project will be documented in this file.
 
 By import convention, components of the Sciris library are listed beginning with ``sc.``, e.g. ``sc.odict()``.
 
+Version 1.3.0 (2021-12-30)
+--------------------------
+
+This version contains a number of major improvements, including:
+
+#. **Better date plotting**: ``sc.dateformatter()`` has been revamped to provide compact and intuitive date plotting.
+#. **Better smoothing**: The new functions ``sc.convolve()``/``sc.gauss1d()``/``sc.gauss2d()``, and the updated ``sc.smooth()``, provide new options for smoothing data.
+#. **Simpler fonts**: ``sc.fonts()`` can both list fonts and add new ones.
+#. **Simpler options**: Need a bigger font? Just do ``sc.options(fontsize=18)``.
+
+New functions and methods
+~~~~~~~~~~~~~~~~~~~~~~~~~
+#. Added a settings module to quickly set both Sciris and Matplotlib options; e.g. ``sc.options(dpi=150)`` is a shortcut for ``pl.rc('figure', dpi=150)``, while e.g. ``sc.options(aspath=True)`` will globally set Sciris functions to return ``Path`` objects instead of strings.
+#. Added ``sc.timer()`` as a simpler and more flexible way of accessing ``sc.tic()``/``sc.toc()`` and ``sc.Timer()``.
+#. Added ``sc.convolve()``, a simple fix to ``np.convolve()`` that avoids edge effects (see update to ``sc.smooth()`` below).
+#. Added ``sc.gauss1d()`` and ``sc.gauss2d()`` as additional (high-performance) smoothing functions.
+#. Added ``sc.fonts()``, to easily list or add fonts for use in plotting.
+#. Added ``sc.dictobj()``, the inverse of ``sc.objdict()`` -- an object that acts like a dictionary (instead of a dictionary that acts like an object). Compared to ``sc.objdict()``, ``sc.dictobj()`` is lighter-weight and slightly faster but less powerful.
+#. Added ``sc.swapdict()``, a shortcut for swapping the keys and values of a dictionary.
+#. Added ``sc.loadobj2or3()``, for legacy support for loading Python 2 pickles. (Support had been removed in version 1.1.1.)
+#. Added ``sc.help()``, to quickly allow searching of Sciris' docstrings.
+
+Bugfixes
+~~~~~~~~
+#. Fixed edge effects when using ``sc.smooth()`` by using ``sc.convolve()`` instead of ``np.convolve()``.
+#. Fixed a bug with checking types when saving files via ``sc.save()``. (Thanks to Rowan Martin-Hughes.)
+#. Fixed a bug with ``output=True`` not being passed correctly for ``sc.heading()``.
+
+Improvements
+~~~~~~~~~~~~
+#. ``sc.dateformatter()`` is now an interface to a new formatter for plotting dates (``ScirisDateFormatter``). This formatter is optimized for aesthetics, combining the best aspects of Matplotlib's and Plotly's date formatters. (Thanks to Daniel Klein.)
+#. ``sc.daterange()`` now accepts an ``interval`` argument.
+#. ``sc.datedelta()`` can now return the actual delta rather than just the date.
+#. ``sc.toc()`` has more flexible printing options.
+#. ``sc.Spreadsheet()`` now keeps a copy of the opened workbook, so there is no need to reopen it for every operation.
+#. ``sc.commaticks()`` can now use non-comma separators. 
+#. Many other functions had small usability improvements, e.g. input arguments are more consistent and more flexible.
+
+Housekeeping
+~~~~~~~~~~~~
+#. ``xlrd`` has been removed as a dependency; ``openpyexcel`` is used instead, with simple spreadsheet loading now done by ``pandas``.
+#. Source files were refactored and split into smaller pieces (e.g. ``sc_utils.py`` was split into ``sc_utils.py``, ``sc_printing.py``, ``sc_datetime.py``, ``sc_nested.py``).
+
+Regression information
+~~~~~~~~~~~~~~~~~~~~~~
+#. To restore previous spreadsheet loading behavior, use ``sc.loadspreadsheet(..., method='xlrd')``.
+#. To use previous smoothing (with edge effects), use ``sc.smooth(..., legacy=True)``
+
 
 Version 1.2.3 (2021-08-27)
 --------------------------

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2020 by the Sciris Development Team
+Copyright (c) 2014-2021 by the Sciris Development Team
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

docs/_static/theme_overrides.css

@@ -6,6 +6,11 @@
     font-size: 100%;
 }
 
+/* CK: slightly wider than default (800px) */
+.wy-nav-content {
+    max-width: 920px;
+}
+
 footer p {
     font-size: 14px;
     margin-top: 0px;
@@ -17,6 +22,13 @@ h1, h2, .rst-content .toctree-wrapper p.caption, h3, h4, h5, h6, legend {
     margin-bottom: 0.5em;
 }
 
+/* CK: added toc-backref since otherwise overrides this */
+.toc-backref, .rst-content {
+    font-size: inherit !important;
+    color: inherit !important;
+    margin-bottom: inherit !important;
+}
+
 h1 {
     margin-bottom: 1.0em;
 }
@@ -27,6 +39,7 @@ h2 {
     margin-bottom: 1.0em;
 }
 
+
 h3 {
     font-size: 115%;
     color: #38761d;
@@ -45,9 +58,19 @@ div.document span.search-highlight {
     margin-bottom: 10px;
 }
 
+/* CK: alternating table row colors */
 .rst-content table.docutils:not(.field-list) tr:nth-child(2n-1) td, .wy-table-backed, .wy-table-odd td, .wy-table-striped tr:nth-child(2n-1) td {
     background-color: #acf;
 }
 tr.row-even {
     background-color: #def;
-}
+}
+
+/* CK: Change the color of code blocks */
+.highlight  { background: #D9F0FF; }
+
+/* CK: Change the color of inline code */
+code.literal {
+    color: #e01e5a !important;
+    background-color: #f6f6f6 !important;
+}

docs/_templates/footer.html

@@ -0,0 +1,4 @@
+{% extends '!footer.html' %}
+{% block extrafooter %}
+<p>Questions? Feedback? Please write to <a href="mailto:info@sciris.org" target="_top">info@sciris.org</a>.
+{% endblock %}

docs/_templates/page.html

@@ -0,0 +1,18 @@
+{% extends "!page.html" %}
+
+{% set css_files = css_files + ["_static/theme_overrides.css"] %}
+
+{% block htmltitle %}
+{{ super() }}
+
+{% if not READTHEDOCS %}
+<meta name="robots" content="none">
+{% endif %}
+
+{% endblock %}
+
+{% block extrahead %}
+
+<link href="https://fonts.googleapis.com/css?family=Lato|Montserrat|Open+Sans" rel="stylesheet">
+
+{% endblock %}

docs/conf.py

@@ -8,11 +8,6 @@
 
 import os
 import sys
-import matplotlib
-
-matplotlib.use("agg") # To ensure figures don't render
-sys.path.insert(0, os.path.abspath("../"))  # Source code dir relative to this file
-
 import sciris as sc
 
 # -- Path setup --------------------------------------------------------------
@@ -24,8 +19,8 @@
 # -- Project information -----------------------------------------------------
 
 project = 'Sciris'
-copyright = f'2020 by the Sciris team (version {sc.__version__}).'
-author = 'Sciris.org'
+copyright = f'2021 by the Sciris Development Team (version {sc.__version__})'
+author = 'Sciris Development Team'
 
 # The short X.Y version
 version = sc.__version__
@@ -43,14 +38,20 @@
 # ones.
 extensions = [
     "sphinx.ext.autodoc",  # Core Sphinx library for auto html doc generation from docstrings
-    "sphinx.ext.autosummary",  # Create neat summary tables for modules/classes/methods etc
+    "sphinx.ext.autosummary",  # Create neat summary tables for modules/classes/methods etc -- causes warnings with Napoleon however
     "sphinx.ext.viewcode",  # Add a link to the Python source code for classes, functions etc.
     "sphinx.ext.napoleon",
     "sphinx.ext.autosectionlabel",
     "sphinx_autodoc_typehints",  # Automatically document param types (less noise in class signature)
     "recommonmark",
 ]
 
+autodoc_default_options = {
+    'member-order': 'bysource',
+    'members': None,
+}
+
+autodoc_mock_imports = []
 napoleon_google_docstring = True
 
 # Configure autosummary
@@ -80,11 +81,11 @@
 html_favicon = "favicon.ico"
 html_static_path = ['_static']
 html_context = {
-    'css_files': [
-        '_static/theme_overrides.css'
-    ]
+    'rtd_url': 'https://sciris.readthedocs.io/en/latest/',
+    'theme_vcs_pageview_mode': 'edit'
 }
 
+html_last_updated_fmt = '%Y-%b-%d'
 html_show_sourcelink = True
 html_show_sphinx = False
-htmlhelp_basename = 'Sciris'
+htmlhelp_basename = 'Sciris'

sciris/__init__.py

@@ -8,11 +8,16 @@
 
 # Import everything
 from .sc_version   import *
+from .sc_settings  import *
 from .sc_utils     import *
+from .sc_printing  import *
+from .sc_datetime  import *
+from .sc_nested    import *
 from .sc_math      import *
 from .sc_odict     import *
 from .sc_dataframe import *
 from .sc_parallel  import *
 from .sc_fileio    import *
 from .sc_asd       import *
 from .sc_plotting  import *
+from .sc_colors    import *

sciris/sc_asd.py

@@ -8,8 +8,9 @@
 import numpy as np
 import numpy.random as nr
 from time import time
-from . import sc_utils as ut
-from .sc_odict import objdict
+from . import sc_utils as scu
+from . import sc_printing as scp
+from . import sc_odict as sco
 
 __all__ = ['asd']
 
@@ -158,11 +159,11 @@ def consistentshape(userinput, origshape=False):
     if args is None: # Reset if no function arguments supplied
         args = []
     elif isinstance(args, dict): # It's actually kwargs supplied
-        kwargs = ut.mergedicts(args, kwargs)
+        kwargs = scu.mergedicts(args, kwargs)
         args = []
     fval = function(x, *args, **kwargs) # Calculate initial value of the objective function
     fvalorig = fval # Store the original value of the objective function, since fval is overwritten on each step
-    xorig = ut.dcp(x) # Keep the original x, just in case
+    xorig = scu.dcp(x) # Keep the original x, just in case
 
     # Initialize history
     abserrorhistory = np.zeros(stalliters) # Store previous error changes
@@ -201,7 +202,7 @@ def consistentshape(userinput, origshape=False):
             stepsizes[choice] = stepsizes[choice] / sdec
 
         # Calculate the new value
-        xnew = ut.dcp(x) # Initialize the new parameter set
+        xnew = scu.dcp(x) # Initialize the new parameter set
         xnew[par] = newval # Update the new parameter set
         fvalnew = function(xnew, *args, **kwargs) # Calculate the objective function for the new parameter set
         eps = 1e-12 # Small value to avoid divide-by-zero errors
@@ -227,7 +228,7 @@ def consistentshape(userinput, origshape=False):
             if np.isnan(fvalnew):
                 if verbose >= 1: print('ASD: Warning, objective function returned NaN')
         if verbose > 0 and not (count % max(1, int(1.0/verbose))): # Print out every 1/verbose steps
-            orig, best, new, diff = ut.sigfig([fvalorig, fvalold, fvalnew, fvalnew-fvalold])
+            orig, best, new, diff = scp.sigfig([fvalorig, fvalold, fvalnew, fvalnew-fvalold])
             print(offset + label + f' step {count} ({time()-start:0.1f} s) {flag} (orig:{orig} | best:{best} | new:{new} | diff:{diff})')
 
         # Store output information
@@ -239,15 +240,15 @@ def consistentshape(userinput, origshape=False):
             exitreason = 'Maximum iterations reached'
             break
         if (time() - start) > maxtime:
-            strtime, strmax = ut.sigfig([(time()-start), maxtime])
+            strtime, strmax = scp.sigfig([(time()-start), maxtime])
             exitreason = f'Time limit reached ({strtime} > {strmax})'
             break
         if (count > stalliters) and (abs(np.mean(abserrorhistory)) < abstol): # Stop if improvement is too small
-            strabs, strtol = ut.sigfig([np.mean(abserrorhistory), abstol])
+            strabs, strtol = scp.sigfig([np.mean(abserrorhistory), abstol])
             exitreason = f'Absolute improvement too small ({strabs} < {strtol})'
             break
         if (count > stalliters) and (sum(relerrorhistory) < reltol): # Stop if improvement is too small
-            strrel, strtol = ut.sigfig([np.mean(relerrorhistory), reltol])
+            strrel, strtol = scp.sigfig([np.mean(relerrorhistory), reltol])
             exitreason = f'Relative improvement too small ({strrel} < {strtol})'
             break
         if stoppingfunc and stoppingfunc():
@@ -256,13 +257,13 @@ def consistentshape(userinput, origshape=False):
 
     # Return
     if verbose > 0:
-        orig, best, ratio = ut.sigfig([fvals[0], fvals[-1], fvals[-1]/fvals[0]])
+        orig, best, ratio = scp.sigfig([fvals[0], fvals[-1], fvals[-1]/fvals[0]])
         print(f'=== {label} {exitreason} ({count} steps, orig: {orig} | best: {best} | ratio: {ratio}) ===')
-    output = objdict()
+    output = sco.objdict()
     output['x'] = np.reshape(x, origshape) # Parameters
     output['fval'] = fvals[count]
     output['exitreason'] = exitreason
-    output['details'] = objdict()
+    output['details'] = sco.objdict()
     output['details']['fvals'] = fvals[:count+1] # Function evaluations
     output['details']['xvals'] = allsteps[:count+1, :]
     output['details']['probabilities'] = probabilities