Shinnosuke : Deep Learning Framework

Descriptions

Shinnosuke is a high-level neural network which has almost the same API to Keras with slightly differences. It was written by Python only, and dedicated to realize experimentations quickly.

Here are some features of Shinnosuke:

1. Based on Numpy (CPU version) and native to Python. GPU Version
2. Without any other 3rd-party deep learning library.
3. Keras-like API, several basic AI Examples are provided, easy to get start.
4. Support commonly used models such as: Dense, Conv2D, MaxPooling2D, LSTM, SimpleRNN, etc.
5. Sequential model (for most sequence network combinations ) and Functional model (for resnet, etc) are implemented.
6. Training is conducted on forward graph and backward graph.
7. Autograd is supported .

Shinnosuke is compatible with: Python 3.x (3.6 is recommended)

###################################### \^^shinnosuke documents^^/ ######################################

Jupyter Notebook

Markdown

---

Getting started

The core networks of Shinnosuke is a model, which provide a way to combine layers. There are two model types: Sequential (a linear stack of layers) and Functional (build a graph for layers).

Here is a example of Sequential model:

from shinnosuke.models import Sequential

m = Sequential()

Using .add() to connect layers:

from shinnosuke.layers import Dense

m.add(Dense(n_out=500, activation='relu', n_in=784))  #must be specify n_in if current layer is the first layer of model
m.add(Dense(n_out=10, activation='softmax'))  #no need to specify n_in as shinnosuke will automatic calculate the input and output shape

Here are some differences with Keras, n_out and n_in are named units and input_dim in Keras respectively.

Once you have constructed your model, you should configure it with .compile() before training:

m.compile(loss='sparse_categorical_crossentropy', optimizer='sgd')

If you apply softmax to multi-classify task and your labels are one-hot encoded vectors/matrix, you shall specify loss as sparse_categorical_crossentropy, otherwise use categorical_crossentropy. (While in Keras categorical_crossentropy supports for one-hot encoded labels). And Shinnosuke model only supports one metrics--accuracy, which no need to specify in compile.

Use print() function to see details of model:

print(m)
# you shall see 
***************************************************************************
Layer(type)          Output Shape         Param        Connected to   
###########################################################################
Dense                (None, 500)          392500       
              
---------------------------------------------------------------------------
Dense                (None, 10)           5010         Dense          
---------------------------------------------------------------------------
***************************************************************************
Total params: 397510
Trainable params: 397510
Non-trainable params: 0

Having finished compile, you can start training your data in batches:

#trainX and trainy are Numpy arrays --for 2 dimension data, trainX's shape should be (training_nums,input_dim)
m.fit(trainX, trainy, batch_size = 128, epochs = 5, validation_ratio = 0., draw_acc_loss = True)

By specify validation_ratio = (0.0,1.0], shinnosuke will split validation data from training data according to validation_ratio, otherwise validation_ratio=0. means no validation data. Alternatively you can feed validation_data manually:

m.fit(trainX, trainy, batch_size = 128, epochs = 5, validation_data = (validX, validy), draw_acc_loss = True)

If draw_acc_loss = True, a dynamic updating figure will be shown in the training process, like below:

Once completing training your model, you can save or load your model by save() / load(), respectively.

m.save(save_path)
m.load(model_path)

Evaluate your model performance by .evaluate():

acc, loss = m.evaluate(testX, testy, batch_size=128)

Or obtain predictions on new data:

y_hat = m.predict(x_test)

For Functional model, first instantiate an Input layer:

from shinnosuke.layers import Input

X_input = Input(shape = (None, 1, 28, 28))   #(batch_size,channels,height,width)

You need to specify the input shape, notice that for Convolutional networks,data's channels must be in the axis 1 instead of -1, and you should state batch_size as None which is unnecessary in Keras.

Then Combine your layers by functional API:

from shinnosuke.models import Model
from shinnosuke.layers import Conv2D,MaxPooling2D
from shinnosuke.layers import Activation
from shinnosuke.layers import BatchNormalization
from shinnosuke.layers import Flatten,Dense

X = Conv2D(8, (2, 2), padding = 'VALID', initializer = 'normal', activation = 'relu')(X_input)
X = MaxPooling2D((2, 2))(X)
X = Flatten()(X)
X = Dense(10, initializer = 'normal', activation = 'softmax')(X)
model = Model(inputs = X_input, outputs = X)  
model.compile(optimizer = 'sgd', loss = 'sparse_categorical_cross_entropy')
model.fit(trainX, trainy, batch_size = 256, epochs = 80, validation_ratio = 0.)

Pass inputs and outputs layer to Model(), and then compile and fit model like Sequentialmodel.

Building an image classification model, a question answering system or any other model is just as convenient and fast~

In the Examples folder of this repository, you can find more advanced models.

---

Both dynamic and static graph features

As you will see soon in below, Shinnosuke has two basic classes - Layer and Node. For Layer, operations between layers can be described like this (here gives an example of '+' ):

from shinnosuke.layers import Input,Add
from shinnosuke.layers import Dense

X = Input(shape = (3, 5))
X_shortcut = X
X = Dense(5)(X)  #Dense will output a (3,5) tensor
X = Add()([X_shortcut, X])

Meanwhile Shinnosuke will construct a graph as below:

While Node Operations have both dynamic graph and static graph features:

from shinnosuke.layers.Base import Variable

x = Variable(3)
y = Variable(5)
z = x + y  
print(z.get_value())

You suppose get value 8, at same time shinnosuke construct a graph as below:

Autograd

What is autograd? In a word, It means automatically calculate the network's gradients without any prepared backward codes for users, Shinnosuke's autograd supports for several operators, such as +, -, *, /, etc... Here gives an example:

For a simple fully connected neural network, you can use Dense() to construct it:

from shinnosuke.models import Sequential
from shinnosuke.layers import Dense
import numpy as np

#announce a Dense layer
fullyconnected = Dense(4, n_in = 5)
m = Sequential()
m.add(fullyconnected)
m.compile(optimizer = 'sgd', loss = 'mse')  #don't mean to train it, use compile to initialize parameters
#initialize inputs
np.random.seed(0)
X = np.random.rand(3, 5)
#feed X as fullyconnected's inputs
fullyconnected.feed(X, 'inputs')
#forward
fullyconnected.forward()
out1 = fullyconnected.get_value()
print(out1.get_value())
#feed gradient to fullyconnected
fullyconnected.feed(np.ones_like(out1), 'grads')
#backward
fullyconnected.backward()
W, b = fullyconnected.variables
print(W.grads)

We can also construct the same layer by using following codes:

from shinnosuke.layers import Variable

a = Variable(X) # the same as X in previous fullyconnected
c = Variable(W.get_value())  # the same value as W in previous fullyconnected
d = Variable(b.get_value())  # the same value as b in previous fullyconnected
out2 = a @ c + d   # @ represents for matmul
print(out2.get_value())
out2.grads = np.ones_like(out2.get_value())   #specify gradients
# by using grad(),shinnosuke will automatically calculate the gradient from out2 to c
c.grad()
print(c.grads)

Guess what? out1 has the same value of out2, and so did W and c's grads. This is the magic autograd of shinnosuke. By using this feature, users can implement other networks as wishes without writing any backward codes.

See autograd example in Here!

Installation

Before installing Shinnosuke, please install the following dependencies:

• Numpy = 1.15.0 (recommend)
• matplotlib = 3.0.3 (recommend)

Then you can install Shinnosuke by using pip:

$ pip install shinnosuke

Installation from Github source will be supported in the future.

---

Supports

Two basic class:

- Layer:

• Dense
• Flatten
• Conv2D
• MaxPooling2D
• MeanPooling2D
• Activation
• Input
• Dropout
• Batch Normalization
• Layer Normalization
• Group Normalization
• TimeDistributed
• SimpleRNN
• LSTM
• Embedding
• GRU (waiting for implemented)
• ZeroPadding2D
• Add
• Minus
• Multiply
• Matmul

- Node:

• Variable
• Constant

Optimizers

• StochasticGradientDescent
• Momentum
• RMSprop
• AdaGrad
• AdaDelta
• Adam

Waiting for implemented more

Objectives

• MeanSquaredError
• MeanAbsoluteError
• BinaryCrossEntropy
• SparseCategoricalCrossEntropy
• CategoricalCrossEntropy

Activations

• Relu
• Linear
• Sigmoid
• Tanh
• Softmax

Initializations

• Zeros
• Ones
• Uniform
• LecunUniform
• GlorotUniform
• HeUniform
• Normal
• LecunNormal
• GlorotNormal
• HeNormal
• Orthogonal

Regularizes

waiting for implement.

Utils

• get_batches (generate mini-batch)
• to_categorical (convert inputs to one-hot vector/matrix)
• concatenate (concatenate Nodes that have the same shape in specify axis)
• pad_sequences (pad sequences to the same length)

Contact

• Email: eleven_1111@outlook.com