Quantization with float 8 (#42)

* equations

* add notebook

* change alignment

* remove unsupported option

* update notebook

* use black on notebook

* requiremens

* change the trigger

---------

Co-authored-by: Xavier Dupré <xavier.dupre@gmail.com>

.github/workflows/check_urls.yml

@@ -1,6 +1,8 @@
 name: Check URLs
 
-on: [push]
+on:
+  schedule:
+    - cron: '0 0 0 * *'
 
 jobs:
   build:
@@ -15,8 +17,8 @@ jobs:
         subfolder: mlstatpy
         file_types: .md,.py,.rst,.ipynb
         print_all: false
-        timeout: 2
-        retry_count# : 2
+        timeout: 5
+        retry_count# : 3
         # exclude_urls: https://dumps.wikimedia.org/other/pageviews/%Y/%Y-%m/pageviews-%Y%m%d-%H0000.gz,https://dumps.wikimedia.org/frwiki/latest/latest-all-titles-in-ns0.gz
         exclude_patterns: https://dumps.wikimedia.org/
         # force_pass : true
@@ -27,8 +29,8 @@ jobs:
         subfolder: _doc
         file_types: .md,.py,.rst,.ipynb
         print_all: false
-        timeout: 2
-        retry_count# : 2
-        exclude_urls: https://hal.archives-ouvertes.fr/hal-00990252/document
-        exclude_patterns: https://www.data.gouv.fr/fr/datasets/r/e3d83ab3-dc52-4c99-abaf-8a38050cc68c
+        timeout: 5
+        retry_count# : 3
+        exclude_urls: https://hal.archives-ouvertes.fr/hal-00990252/document,https://github.com/onnx/models/raw/main/vision/classification/mobilenet/model/mobilenetv2-12.onnx
+        exclude_patterns: https://www.data.gouv.fr/fr/datasets/r/e3d83ab3-dc52-4c99-abaf-8a38050cc68c,https://github.com/onnx/models/raw/main/vision/classification/mobilenet/model/mobilenetv2-12.onnx
         # force_pass : true

.gitignore

@@ -1,17 +1,19 @@
+*.dot
+*.dylib
+*.prof
 *.pyc
 *.pyd
-*.dylib
 *.so
-coverage.html/*
-_cache/*
 .coverage
-dist/*
-build/*
 .eggs/*
+_cache/*
+build/*
+dist/*
 *egg-info/*
 onnxruntime_profile*
 prof
 temp_*
+.ipynb_checkpoints
 _doc/CHANGELOGS.rst
 _doc/LICENSE.txt
 _doc/auto_examples/*
@@ -27,3 +29,5 @@ _unittests/ut__main/*.png
 _unittests/ut__main/_cache/*
 _unittests/ut__main/*.html
 _unittests/.hypothesis/*
+_doc/notebooks/ml/*.onnx
+_doc/notebooks/dsgarden/*.onnx

CHANGELOGS.rst

@@ -4,4 +4,5 @@ Change Logs
 0.4.0
 +++++
 
+* :pr:`42`: quantization
 * :pr:`39`: refactoring, use black, better documentation

_doc/c_garden/quantization.rst

@@ -140,6 +140,7 @@ pour :math:`\lambda` et *z* en faisant tendre *h* vers 0.
 On commence par le plus simple, le cas float 8 pour lequel on impose :math:`z=0`.
 
 .. math::
+    :label: eq-qua-1
 
     f(B,\lambda,h) = \frac{1}{h} \sum_{k=1}^{n} \sum_{ij} \pa{b_{ij} - \lambda d_k - z }^2
     K\pa{\frac{b_{ij} - \lambda d_k - z}{h}}
@@ -154,3 +155,43 @@ ce calcul devient une somme d'espérence.
 
 Résolution
 ==========
+
+If :math:`K(u)=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}u^2}` then
+:math:`K'(u) = -u \frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}u^2} = -u K(x)`.
+Let's denote :math:`g(b,x) = (b-xd)^2 K\pa{\frac{b-xd}{h}}`. Then:
+
+.. math::
+
+    \begin{array}{rcl}
+    g(b,x) &=& \frac{1}{h} (b-xd)^2 K\pa{\frac{b-xd}{h}} \\
+    \frac{\partial g}{\partial x}(b,x) &=&
+    \frac{1}{h}\cro{ -2d(b-xd)K\pa{\frac{b-xd}{h}} -\frac{d}{h} (b-xd)^2 K'\pa{\frac{b-xd}{h}} } \\
+    &=& -\frac{d(b-xd)}{h}\cro{2 K\pa{\frac{b-xd}{h}} + \frac{b-xd}{h} K'\pa{\frac{b-xd}{h}} }
+    \end{array}
+
+Applied to :eq:`eq-qua-1`:
+
+.. math::
+
+    \begin{array}{rcl}
+    f(B,\lambda,h) &=& \frac{1}{h} \sum_{k=1}^{n} \sum_{ij} \pa{b_{ij} - \lambda d_k}^2
+    K\pa{\frac{b_{ij} - \lambda d_k}{h}} \\
+    &=& \sum_{k=1}^{n} \sum_{ij} g(b_{ij}, \lambda)
+    \end{array}
+
+Then:
+
+.. math::
+
+    \begin{array}{rcl}
+    \frac{\partial f}{\partial \lambda} &=& \sum_{k=1}^{n} \sum_{ij}
+    \frac{\partial g}{\partial \lambda}(b_{ij}, \lambda) 
+    \end{array}
+
+Notebooks
+=========
+
+.. toctree::
+    :maxdepth: 1
+
+    ../notebooks/dsgarden/quantization_f8

_doc/conf.py

@@ -75,6 +75,8 @@
     "title": project,
 }
 
+mathjax3_config = {"chtml": {"displayAlign": "left"}}
+
 intersphinx_mapping = {
     "onnx": ("https://onnx.ai/onnx/", None),
     "matplotlib": ("https://matplotlib.org/", None),