Skip to content

Commit 801dc40

Browse files
bruce1408bruce1408
authored
Merge pull request #2 from ddbourgin/master
update and forks
2 parents bf211c0 + 4f37707 commit 801dc40

File tree

110 files changed

+3560
-1889
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

110 files changed

+3560
-1889
lines changed

‎.github/ISSUE_TEMPLATE/a--bug-performance-issue.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,9 @@ labels: bugfix
1313

1414
**Describe the expected behavior**
1515

16-
**Code to reproduce the issue**
16+
**Code to reproduce the issue**
1717
<!-- Provide a reproducible test case that is the bare minimum necessary to generate the problem. -->
1818

19-
**Other info / logs**
20-
<!-- Include any logs or source code that would be helpful to diagnose the problem.
19+
**Other info / logs**
20+
<!-- Include any logs or source code that would be helpful to diagnose the problem.
2121
If including tracebacks, please include the full traceback. Large logs and files should be attached. -->

‎.readthedocs.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ sphinx:
1313
# configuration: mkdocs.yml
1414

1515
# Optionally build your docs in additional formats such as PDF and ePub
16-
formats:
16+
formats:
1717
- htmlzip
1818

1919
# Optionally set the version of Python and requirements required to build your docs

‎CONTRIBUTING.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Thank you for contributing to numpy-ml!
1010
2. For pull requests, please make sure all commits are [*atomic*](https://en.wikipedia.org/wiki/Atomic_commit) (i.e., one feature per commit)
1111
3. If you're submitting a new model / feature / module, **please include proper documentation and unit tests.**
1212
- See the `test.py` file in one of the existing modules for examples of unit tests.
13-
- Documentation is loosely based on the [NumPy docstring style](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html). When in doubt, refer to existing examples
13+
- Documentation is loosely based on the [NumPy docstring style](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html). When in doubt, refer to existing examples
1414
4. Please format your code using the [black](https://github.com/python/black) defaults. You can use this [online formatter](https://black.now.sh/).
1515

1616
### Specific guidelines

‎MANIFEST.in

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
include README.md
2+
include requirements*.txt
3+
include docs/*.rst
4+
include docs/img/*.png

‎README.md

+171-3
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,177 @@
11
# numpy-ml
22
Ever wish you had an inefficient but somewhat legible collection of machine
3-
learning algorithms implemented exclusively in numpy? No?
3+
learning algorithms implemented exclusively in NumPy? No?
4+
5+
## Installation
6+
7+
### For rapid experimentation
8+
To use this code as a starting point for ML prototyping / experimentation, just clone the repository, create a new [virtualenv](https://pypi.org/project/virtualenv/), and start hacking:
9+
10+
```sh
11+
$ git clone https://github.com/ddbourgin/numpy-ml.git
12+
$ cd numpy-ml && virtualenv npml && source npml/bin/activate
13+
$ pip3 install -r requirements-dev.txt
14+
```
15+
16+
### As a package
17+
If you don't plan to modify the source, you can also install numpy-ml as a
18+
Python package: `pip3 install -u numpy_ml`.
19+
20+
The reinforcement learning agents train on environments defined in the [OpenAI
21+
gym](https://github.com/openai/gym). To install these alongside numpy-ml, you
22+
can use `pip3 install -u 'numpy_ml[rl]'`.
423

524
## Documentation
6-
To see all of the available models, take a look at the [project documentation](https://numpy-ml.readthedocs.io/) or see [here](https://github.com/ddbourgin/numpy-ml/blob/master/numpy_ml/README.md).
25+
For more details on the available models, see the [project documentation](https://numpy-ml.readthedocs.io/).
26+
27+
## Available models
28+
1. **Gaussian mixture model**
29+
- EM training
30+
31+
2. **Hidden Markov model**
32+
- Viterbi decoding
33+
- Likelihood computation
34+
- MLE parameter estimation via Baum-Welch/forward-backward algorithm
35+
36+
3. **Latent Dirichlet allocation** (topic model)
37+
- Standard model with MLE parameter estimation via variational EM
38+
- Smoothed model with MAP parameter estimation via MCMC
39+
40+
4. **Neural networks**
41+
* Layers / Layer-wise ops
42+
- Add
43+
- Flatten
44+
- Multiply
45+
- Softmax
46+
- Fully-connected/Dense
47+
- Sparse evolutionary connections
48+
- LSTM
49+
- Elman-style RNN
50+
- Max + average pooling
51+
- Dot-product attention
52+
- Embedding layer
53+
- Restricted Boltzmann machine (w. CD-n training)
54+
- 2D deconvolution (w. padding and stride)
55+
- 2D convolution (w. padding, dilation, and stride)
56+
- 1D convolution (w. padding, dilation, stride, and causality)
57+
* Modules
58+
- Bidirectional LSTM
59+
- ResNet-style residual blocks (identity and convolution)
60+
- WaveNet-style residual blocks with dilated causal convolutions
61+
- Transformer-style multi-headed scaled dot product attention
62+
* Regularizers
63+
- Dropout
64+
* Normalization
65+
- Batch normalization (spatial and temporal)
66+
- Layer normalization (spatial and temporal)
67+
* Optimizers
68+
- SGD w/ momentum
69+
- AdaGrad
70+
- RMSProp
71+
- Adam
72+
* Learning Rate Schedulers
73+
- Constant
74+
- Exponential
75+
- Noam/Transformer
76+
- Dlib scheduler
77+
* Weight Initializers
78+
- Glorot/Xavier uniform and normal
79+
- He/Kaiming uniform and normal
80+
- Standard and truncated normal
81+
* Losses
82+
- Cross entropy
83+
- Squared error
84+
- Bernoulli VAE loss
85+
- Wasserstein loss with gradient penalty
86+
- Noise contrastive estimation loss
87+
* Activations
88+
- ReLU
89+
- Tanh
90+
- Affine
91+
- Sigmoid
92+
- Leaky ReLU
93+
- ELU
94+
- SELU
95+
- Exponential
96+
- Hard Sigmoid
97+
- Softplus
98+
* Models
99+
- Bernoulli variational autoencoder
100+
- Wasserstein GAN with gradient penalty
101+
- word2vec encoder with skip-gram and CBOW architectures
102+
* Utilities
103+
- `col2im` (MATLAB port)
104+
- `im2col` (MATLAB port)
105+
- `conv1D`
106+
- `conv2D`
107+
- `deconv2D`
108+
- `minibatch`
109+
110+
5. **Tree-based models**
111+
- Decision trees (CART)
112+
- [Bagging] Random forests
113+
- [Boosting] Gradient-boosted decision trees
114+
115+
6. **Linear models**
116+
- Ridge regression
117+
- Logistic regression
118+
- Ordinary least squares
119+
- Bayesian linear regression w/ conjugate priors
120+
- Unknown mean, known variance (Gaussian prior)
121+
- Unknown mean, unknown variance (Normal-Gamma / Normal-Inverse-Wishart prior)
122+
123+
7. **n-Gram sequence models**
124+
- Maximum likelihood scores
125+
- Additive/Lidstone smoothing
126+
- Simple Good-Turing smoothing
127+
128+
8. **Multi-armed bandit models**
129+
- UCB1
130+
- LinUCB
131+
- Epsilon-greedy
132+
- Thompson sampling w/ conjugate priors
133+
- Beta-Bernoulli sampler
134+
- LinUCB
135+
136+
8. **Reinforcement learning models**
137+
- Cross-entropy method agent
138+
- First visit on-policy Monte Carlo agent
139+
- Weighted incremental importance sampling Monte Carlo agent
140+
- Expected SARSA agent
141+
- TD-0 Q-learning agent
142+
- Dyna-Q / Dyna-Q+ with prioritized sweeping
143+
144+
9. **Nonparameteric models**
145+
- Nadaraya-Watson kernel regression
146+
- k-Nearest neighbors classification and regression
147+
- Gaussian process regression
148+
149+
10. **Matrix factorization**
150+
- Regularized alternating least-squares
151+
- Non-negative matrix factorization
152+
153+
11. **Preprocessing**
154+
- Discrete Fourier transform (1D signals)
155+
- Discrete cosine transform (type-II) (1D signals)
156+
- Bilinear interpolation (2D signals)
157+
- Nearest neighbor interpolation (1D and 2D signals)
158+
- Autocorrelation (1D signals)
159+
- Signal windowing
160+
- Text tokenization
161+
- Feature hashing
162+
- Feature standardization
163+
- One-hot encoding / decoding
164+
- Huffman coding / decoding
165+
- Term frequency-inverse document frequency (TF-IDF) encoding
166+
- MFCC encoding
167+
168+
12. **Utilities**
169+
- Similarity kernels
170+
- Distance metrics
171+
- Priority queue
172+
- Ball tree
173+
- Discrete sampler
174+
- Graph processing and generators
7175

8176
## Contributing
9177

@@ -14,4 +182,4 @@ library](https://docs.python.org/3/library/) and [NumPy](https://www.numpy.org/)
14182
[SciPy library](https://scipy.github.io/devdocs/) is also permitted under special
15183
circumstances ;)
16184

17-
See full contributing guidelines [here](./CONTRIBUTING.md).
185+
See full contributing guidelines [here](./CONTRIBUTING.md).

‎docs/Makefile

+1-1
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,4 @@ help:
1616
# Catch-all target: route all unknown targets to Sphinx using the new
1717
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
1818
%: Makefile
19-
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
19+
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

‎docs/conf.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
# -- Project information -----------------------------------------------------
2525

2626
project = "numpy-ml"
27-
copyright = "2019, David Bourgin"
27+
copyright = "2020, David Bourgin"
2828
author = "David Bourgin"
2929

3030
# The short X.Y version

‎docs/index.rst

+7-5
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Welcome to numpy-ml
22
===================
33
`numpy-ml`_ is a growing collection of machine learning models, algorithms, and
4-
tools written exclusively in `NumPy`_ and the Python `standard library`_.
4+
tools written exclusively in `NumPy`_ and the Python `standard library`_.
55

66
The purpose of the project is to provide reference implementations of common
77
machine learning components for rapid prototyping and experimentation. With
@@ -15,7 +15,7 @@ that in mind, don't just read the docs -- read the source!
1515

1616
We're working to expand our coverage. During this time there are likely to
1717
be typos, bugs, and poorly-worded sections. If you encounter any of the
18-
above, please file an `issue`_ or submit a `pull request`_!
18+
above, please file an `issue`_ or submit a `pull request`_!
1919

2020
.. _issue: https://github.com/ddbourgin/numpy-ml/issues
2121
.. _pull request: https://github.com/ddbourgin/numpy-ml/pulls
@@ -38,6 +38,8 @@ that in mind, don't just read the docs -- read the source!
3838

3939
numpy_ml.nonparametric
4040

41+
numpy_ml.factorization
42+
4143
numpy_ml.trees
4244

4345
numpy_ml.neural_nets
@@ -49,8 +51,8 @@ that in mind, don't just read the docs -- read the source!
4951
numpy_ml.utils
5052

5153
##########
52-
Disclaimer
53-
##########
54-
54+
Disclaimer
55+
##########
56+
5557
This software is provided as-is: there are no guarantees that it fits your
5658
purposes or that it is bug-free. Use it at your own risk!

‎docs/numpy_ml.bandits.bandits.rst

+23-9
Original file line numberDiff line numberDiff line change
@@ -9,31 +9,45 @@ Bandit Environments
99
:inherited-members:
1010

1111

12-
``MABMultinomialPayoff``
12+
``MultinomialBandit``
1313
-------------------------
14-
.. autoclass:: numpy_ml.bandits.MABMultinomialPayoff
14+
.. autoclass:: numpy_ml.bandits.MultinomialBandit
1515
:members:
1616
:undoc-members:
1717
:show-inheritance:
1818

19-
``MABBernoulliPayoff``
19+
``BernoulliBandit``
2020
-----------------------
21-
.. autoclass:: numpy_ml.bandits.MABBernoulliPayoff
21+
.. autoclass:: numpy_ml.bandits.BernoulliBandit
2222
:members:
2323
:undoc-members:
2424
:show-inheritance:
2525

2626

27-
``MABGaussianPayoff``
27+
``GaussianBandit``
2828
----------------------
29-
.. autoclass:: numpy_ml.bandits.MABGaussianPayoff
29+
.. autoclass:: numpy_ml.bandits.GaussianBandit
3030
:members:
3131
:undoc-members:
3232
:show-inheritance:
3333

34-
``MABShortestPath``
35-
--------------------
36-
.. autoclass:: numpy_ml.bandits.MABShortestPath
34+
``ShortestPathBandit``
35+
-----------------------
36+
.. autoclass:: numpy_ml.bandits.ShortestPathBandit
37+
:members:
38+
:undoc-members:
39+
:show-inheritance:
40+
41+
``ContextualBernoulliBandit``
42+
------------------------------
43+
.. autoclass:: numpy_ml.bandits.ContextualBernoulliBandit
44+
:members:
45+
:undoc-members:
46+
:show-inheritance:
47+
48+
``ContextualLinearBandit``
49+
------------------------------
50+
.. autoclass:: numpy_ml.bandits.ContextualLinearBandit
3751
:members:
3852
:undoc-members:
3953
:show-inheritance:

‎docs/numpy_ml.bandits.policies.rst

+7
Original file line numberDiff line numberDiff line change
@@ -28,3 +28,10 @@ Policies
2828
:members:
2929
:undoc-members:
3030
:show-inheritance:
31+
32+
``LinUCB``
33+
--------------------------------
34+
.. autoclass:: numpy_ml.bandits.policies.LinUCB
35+
:members:
36+
:undoc-members:
37+
:show-inheritance:

‎docs/numpy_ml.bandits.rst

-1
Original file line numberDiff line numberDiff line change
@@ -9,4 +9,3 @@ Multi-armed bandits
99
numpy_ml.bandits.policies
1010

1111
numpy_ml.bandits.trainer
12-

‎docs/numpy_ml.bandits.trainer.rst

+3-3
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
Trainer
22
=======
33

4-
``MABTrainer``
5-
--------------
6-
.. autoclass:: numpy_ml.bandits.trainer.MABTrainer
4+
``BanditTrainer``
5+
------------------
6+
.. autoclass:: numpy_ml.bandits.trainer.BanditTrainer
77
:members:
88
:undoc-members:
99
:inherited-members:

‎docs/numpy_ml.factorization.factors.rst

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
``VanillaALS``
2+
--------------
3+
.. autoclass:: numpy_ml.factorization.VanillaALS
4+
:members:
5+
:undoc-members:
6+
7+
``NMF``
8+
--------
9+
.. autoclass:: numpy_ml.factorization.NMF
10+
:members:
11+
:undoc-members:

‎docs/numpy_ml.factorization.rst

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Matrix factorization
2+
####################
3+
4+
.. toctree::
5+
:maxdepth: 3
6+
7+
numpy_ml.factorization.factors

0 commit comments

Comments
 (0)