docs/adding-features.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adding features\n",
    "\n",
    "## General principles\n",
    "\n",
    "niimpy is an open source project and general open source contribution guidelines apply - there is no need for us to repeat them right now.  Please use Github for communication.\n",
    "\n",
    "Contributions are welcome and encouraged.\n",
    "* You don't need to be perfect.  Suggest what you can and we will help it improve."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding an analysis\n",
    "\n",
    "* Please add documentatation to a sensor page when you add a new analysis.  This should include enough description so that someone else can understand and reproduce all relevant features - enough to describe the method for a scientific article.\n",
    "* Please add unit tests which test each relevant feature (and each claimed method feature) with a minimal example.  Each function can have multiple tests.  For examples of unit tests, see below or ``niimpy/test_screen.py``.  You can create some sample data within each test module which can be used both during development and for tests."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Common things to note\n",
    "\n",
    "* You should always use the DataFrame index to retrieve data/time values, not the `datetime` column (which is a convenience thing but not guaranteed to be there).\n",
    "  * Don't require `datetime` in your input\n",
    "  * Have any times returned in the index (unless each row needs multiple times, then do what you need)\n",
    "* Don't fail if there are extra columns passed (or missing some non-essential columns).  Look at what columns/data is passed and and use that, but don't do anything unexpected if someone makes a mistake with input data\n",
    "* Group by 'user' and 'device' columns if they are present in the input\n",
    "* Use `niimpy.util._read_sqlite_auto` function for getting data from input\n",
    "* Use `niimpy.filter.filter_dataframe` to do basic initial filterings based on standard arguments.\n",
    "* [The Zen of Python](https://www.python.org/dev/peps/pep-0020/) is always good advice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improving old functions\n",
    "- Add tests for existing functionality\n",
    "  - For every functionality it claims, there should be a minimal test for it.\n",
    "- Use `read._get_dataframe` and `filter.filter_dataframe` to handle standard arguments\n",
    "- Don't fail if unnecessary columns are not there (don't drop unneeded columns, select only the needed ones).\n",
    "- Make sure it uses the index, not the `datetime` column.  Some older functions mays still expect it so we have a difficult challenge.\n",
    "- Improve the docstring of the function: we use the [numpydoc format](https://numpydoc.readthedocs.io/en/latest/format.html)\n",
    "- Add a documentation page for these sensors, document each function and include an example.\n",
    "- Document what parameters it groups by when analyzing\n",
    "  - For example an ideal case is that any 'user' and 'device' columns are grouped by in the final output.\n",
    "- When there are things that don't work yet, you can put a TODO in the docstring to indicate that someone should come back to it later."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Example unit test\n",
    "\n",
    "You can read about testing in general in the [CodeRefinery testing lesson](https://coderefinery.github.io/testing/).\n",
    "\n",
    "First you would define some sample data.  You could reuse existing data (or data from `niimpy.sampledata`), but if data is reused too much then it becomes hard to improve test B because it will affect the data of test A.  (do share data when possible but split it when it's relevant).\n",
    "\n",
    "```python\n",
    "@pytest.fixture\n",
    "def screen1():\n",
    "    return niimpy.read_csv(io.StringIO(\"\"\"\\\n",
    "time,screen_status\n",
    "0,1\n",
    "60,0\n",
    "\"\"\"))\n",
    "```\n",
    "\n",
    "Then you can make a test function:\n",
    "\n",
    "```python\n",
    "def test_screen_off(screen1):\n",
    "    off = niimpy.preprocess.screen_off(screen1)\n",
    "    assert pd.Timestamp(60,  unit='s', tz=TZ) in off.index\n",
    "```\n",
    "\n",
    "`assert` statemnts run the tested functions - when there are errors `pytest` will provide much more useful error messages than you might expect.  You can have multiple asserts within a function, to test multiple things.\n",
    "\n",
    "\n",
    "You run tests with `pytest niimpy/` or `pytest niimpy/test_screen.py`.  You can limit to certain tests with `-k` and engage a debugger on errors with `--pdb`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Documentation notes\n",
    "- You can use Jupyter or ReST.  ReST is better for narritive documentation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}