Repository navigation
Advanced‐Tutorials
Build · Experiment · Validate · Optimize
A practical learning pathway for advanced model development, numerical verification, training workflows, acceleration, and interoperability.
Home · Getting Started · API Reference · Examples and Tutorials
Repository: krishanth7/EveryO
Maintainer: @krishanth7
Audience: Intermediate and advanced Python developers, ML engineers, and researchers
Document Status: Tutorial guidance
Important
Before You Begin
EveryO is an experimental, NumPy-based neural computing framework intended for learning, experimentation, and numerical verification.
Advanced features may require optional dependencies, specific hardware, or supported input shapes.
The examples below illustrate documented APIs and engineering patterns. Validate them against your installed EveryO version before using them in larger projects.
- Introduction
- Prerequisites
- Development Environment
- Advanced Learning Path
- Tensor Computation
- Computational Graphs
- Gradient Verification
- Custom Neural Network Modules
- Multilayer Perceptrons
- Convolutional Networks
- Normalization and Regularization
- Recurrent Neural Networks
- LSTM and GRU Architectures
- Attention Mechanisms
- Transformer Encoders
- Advanced Training Pipelines
- Gradient Clipping
- Learning Rate Scheduling
- Early Stopping
- Checkpointing and Serialization
- Mixed Precision
- CUDA Acceleration
- Distributed Training
- ONNX Export
- Quantization
- Profiling and Optimization
- Reproducible Experiments
- Debugging Advanced Models
- Advanced Exercises
- Learning Roadmap
- Official Resources
The Advanced Tutorials section helps developers move beyond introductory tensor operations and build more sophisticated machine learning experiments.
By completing this guide, readers should understand how to:
- Compose neural network architectures.
- Inspect computational graphs.
- Verify analytical gradients.
- Work with convolutional and sequence models.
- Design reliable training workflows.
- Validate model serialization.
- Evaluate optional acceleration.
- Export supported models to ONNX.
- Profile computational bottlenecks.
TENSOR FOUNDATIONS
|
v
AUTOGRAD VALIDATION
|
v
CUSTOM ARCHITECTURES
|
v
ADVANCED TRAINING
|
v
SEQUENCE AND ATTENTION
|
v
MODEL SERIALIZATION
|
v
OPTIONAL ACCELERATION
|
v
INTEROPERABILITY
|
v
REPRODUCIBLE RESULTS
Before proceeding, readers should understand:
- Python programming.
- NumPy arrays.
- Matrix multiplication.
- Basic calculus.
- Neural network fundamentals.
- Gradient descent.
- Training and validation datasets.
- Getting Started
- Tensor Engine
- Automatic Differentiation
- Neural Networks
- Model Training
- Numerical Accuracy and Validation
git clone https://github.com/krishanth7/EveryO.git
cd EveryOpython -m venv .venvLinux/macOS:
source .venv/bin/activateWindows PowerShell:
.\.venv\Scripts\Activate.ps1python -m pip install -e .python -m pip install -e ".[dev]"everyo info
everyo doctor| Level | Topic | Expected Outcome |
|---|---|---|
| 1 | Tensor operations | Understand shapes and numerical behavior |
| 2 | Automatic differentiation | Verify computational gradients |
| 3 | Custom modules | Build reusable architectures |
| 4 | CNNs | Process spatial data |
| 5 | Recurrent networks | Model sequences |
| 6 | Attention | Understand contextual representations |
| 7 | Training pipelines | Manage optimization experiments |
| 8 | CUDA | Evaluate optional acceleration |
| 9 | ONNX | Validate interoperability |
| 10 | Profiling | Identify computational bottlenecks |
import everyo as eo
a = eo.tensor([
[1.0, 2.0],
[3.0, 4.0],
])
b = eo.tensor([
[5.0, 6.0],
[7.0, 8.0],
])
c = eo.matmul(a, b)
print(c)
print(c.shape)Expected numerical output:
[[19. 22.]
[43. 50.]]
Expected shape:
(2, 2)
[ C_{ij}=\sum_k A_{ik}B_{kj} ]
Modify the input matrices to use:
- A 3 × 4 matrix.
- A 4 × 2 matrix.
Verify that the result has shape (3, 2).
Automatic differentiation tracks operations required to compute gradients.
import everyo as eo
x = eo.tensor(
[2.0],
requires_grad=True,
)
a = x * x
b = a * x
y = b + x
y.backward()
print(x.grad)[ y=x^3+x ]
Therefore:
[ \frac{dy}{dx}=3x^2+1 ]
At (x=2):
[ \frac{dy}{dx}=13 ]
Expected gradient:
[13.]
x
|
+---+---+
| |
v |
x*x |
| |
v |
*x |
| |
+---+---+
|
v
Add
|
v
y
|
v
Backward
Gradient checking is essential when developing custom mathematical operations.
[ g_{\text{numeric}}= \frac{f(x+h)-f(x-h)}{2h} ]
def f(x):
return x ** 3 + x
x = 2.0
h = 1e-5
numeric_gradient = (
f(x + h) - f(x - h)
) / (2 * h)
analytic_gradient = 3 * x ** 2 + 1
print("Numeric:", numeric_gradient)
print("Analytic:", analytic_gradient)
print(
"Absolute error:",
abs(numeric_gradient - analytic_gradient),
)Use a justified combination of absolute and relative tolerances.
Implement gradient checks for:
- Squaring.
- Multiplication.
- Matrix multiplication.
- Mean reduction.
EveryO includes Module and Parameter abstractions.
Custom modules help organize:
- Reusable layers.
- Model components.
- Trainable parameters.
- Forward computations.
A custom module should:
- Define its parameters clearly.
- Implement a predictable forward computation.
- Preserve gradient tracking.
- Handle invalid shapes appropriately.
- Support the relevant model lifecycle.
CUSTOM MODULE
|
+------+------+
| |
v v
PARAMETERS FORWARD
| |
+------+------+
|
v
OUTPUT
|
v
AUTOGRAD GRAPH
Custom modules intended for .evo serialization should follow the documented registration mechanism, including @register_module where applicable.
Implement a custom residual block using documented module operations.
Verify its output shape and gradients before using it in a larger model.
import everyo as eo
model = eo.Sequential(
eo.Linear(8, 32),
eo.ReLU(),
eo.Linear(32, 16),
eo.ReLU(),
eo.Linear(16, 3),
)x = eo.tensor([
[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0],
])
output = model(x)
print(output.shape)Expected shape:
(1, 3)
INPUT (8)
|
v
LINEAR (32)
|
v
RELU
|
v
LINEAR (16)
|
v
RELU
|
v
LINEAR (3)
|
v
OUTPUT
Compare models with different hidden layer widths while keeping the dataset and optimizer configuration constant.
EveryO provides convolutional and pooling layers.
EveryO's documented convolution input format is NHWC:
(batch, height, width, channels)
import numpy as np
import everyo as eo
images = eo.tensor(
np.random.default_rng(42).random(
(4, 28, 28, 1)
)
)
print(images.shape)Expected shape:
(4, 28, 28, 1)
IMAGE
|
v
CONVOLUTION
|
v
RELU
|
v
POOLING
|
v
FLATTEN
|
v
LINEAR
|
v
CLASSIFICATION
python examples/convolutional_network.pyInvestigate how kernel size, stride, and padding affect output dimensions.
EveryO includes:
- BatchNorm1D.
- BatchNorm2D.
- LayerNorm.
- Dropout.
[ \hat{x}= \frac{x-\mu}{\sqrt{\sigma^2+\epsilon}} ]
Dropout randomly disables selected activations during training.
Layers such as dropout and batch normalization may behave differently depending on model mode.
Compare a baseline model with a model using regularization.
Report both training and validation metrics.
Recurrent neural networks process sequential information.
[ h_t=f(W_xx_t+W_hh_{t-1}+b) ]
A sequence input typically includes:
- Batch dimension.
- Time dimension.
- Feature dimension.
Verify the precise shape contract of the chosen EveryO recurrent layer.
python examples/sequence_models.py- Verify input shapes.
- Check hidden-state dimensions.
- Test different sequence lengths.
- Validate gradients through time.
Long Short-Term Memory networks use gates to regulate information flow.
Typical components include:
- Input gate.
- Forget gate.
- Output gate.
- Cell state.
Gated Recurrent Units use a smaller gating structure.
Typical components include:
- Update gate.
- Reset gate.
- Hidden state.
| Architecture | Characteristic |
|---|---|
| Basic RNN | Simple recurrence |
| LSTM | Separate cell and hidden states |
| GRU | Gated recurrence with fewer gate types |
Compare RNN, LSTM, and GRU models on the same synthetic sequence prediction task.
Record convergence and computational cost.
Attention allows a model to compute context-dependent representations.
[ Attention(Q,K,V)= softmax\left( \frac{QK^T}{\sqrt{d_k}} \right)V ]
where:
- (Q): Queries.
- (K): Keys.
- (V): Values.
- (d_k): Key dimension.
QUERY KEY
| |
+----+----+
|
v
Q × Kᵀ
|
v
SCALE SCORES
|
v
SOFTMAX
|
v
WEIGHTED VALUES
|
v
OUTPUT
Multiple attention heads learn different representation subspaces.
Verify:
- Tensor shapes.
- Attention score normalization.
- Output dimensions.
- Gradient propagation.
EveryO includes Transformer encoder components.
INPUT
|
v
POSITION ENCODING
|
v
MULTI-HEAD ATTENTION
|
v
RESIDUAL PATH
|
v
NORMALIZATION
|
v
FEED-FORWARD
|
v
RESIDUAL PATH
|
v
NORMALIZATION
|
v
OUTPUT
This diagram illustrates a common post-normalization architecture; verify the actual ordering in the installed EveryO implementation.
- Shape consistency.
- Attention masking where supported.
- Positional encoding behavior.
- Forward correctness.
- Gradient correctness.
Build a small sequence encoder using documented EveryO components and compare its output dimensions across sequence lengths.
EveryO provides a Trainer abstraction.
import everyo as eo
model = eo.Sequential(
eo.Linear(4, 16),
eo.ReLU(),
eo.Linear(16, 2),
)
optimizer = eo.Adam(
model.parameters(),
lr=0.001,
)
loss_fn = eo.CrossEntropyLoss()
trainer = eo.Trainer(
model,
optimizer,
loss_fn,
metrics=["accuracy"],
)
history = trainer.fit(
train_loader,
epochs=10,
validation_loader=validation_loader,
)The training and validation loaders must be created before running this example.
DATASET
|
v
DATALOADER
|
v
MODEL
|
v
FORWARD
|
v
LOSS
|
v
BACKWARD
|
v
OPTIMIZER STEP
|
v
VALIDATION
|
v
CHECKPOINT
Track:
- Training loss.
- Validation loss.
- Accuracy or other relevant metrics.
- Learning rate.
- Gradient stability.
Gradient clipping helps control excessively large gradient updates.
For a gradient vector (g):
[ g_{\text{clipped}}= g\cdot \min\left(1,\frac{c}{|g|}\right) ]
where (c) is the maximum norm.
- Recurrent networks.
- Deep architectures.
- Training with unstable gradients.
Measure gradient norms before choosing clipping thresholds.
Compare training with and without clipping on a deliberately challenging sequence task.
Learning rate scheduling changes the learning rate during training.
- Step decay.
- Exponential decay.
- Plateau-based reduction.
- Cosine scheduling.
These are general strategies; verify which are supported by the installed EveryO version.
Exponential decay:
[ \eta_t=\eta_0\gamma^t ]
where:
- (\eta_0): Initial learning rate.
- (\gamma): Decay factor.
- (t): Schedule step.
Compare a constant learning rate against a supported scheduling strategy.
Early stopping can reduce unnecessary training after validation performance stops improving.
- Monitored metric.
- Improvement direction.
- Patience.
- Minimum improvement.
TRAIN EPOCH
|
v
VALIDATE MODEL
|
v
METRIC IMPROVED?
/ \
YES NO
| |
v v
SAVE BEST INCREMENT
STATE PATIENCE
|
v
LIMIT REACHED?
/ \
NO YES
| |
v v
CONTINUE STOP
Use a held-out validation set and document the stopping criterion.
EveryO supports .evo model archives.
import everyo as eo
eo.save(model, "advanced_model.evo")restored_model = eo.load(
"advanced_model.evo"
)Compare outputs from the original and restored models on identical inputs.
The documented .evo archive uses:
model.evo
|
+-- manifest.json
|
+-- parameters.npz
The format avoids Python pickle execution, but untrusted archives still require careful handling.
EveryO provides mixed-precision utilities.
Mixed precision combines lower-precision operations with higher-precision state where appropriate.
Potential benefits:
- Reduced memory usage.
- Faster supported operations on suitable hardware.
Potential risks:
- Underflow.
- Overflow.
- Numerical differences.
- No CPU speed improvement.
python examples/mixed_precision.pyCompare float32 and mixed-precision execution on a supported model.
Report both runtime and numerical error.
CUDA is optional in EveryO.
import everyo as eo
print(eo.cuda.is_available())import everyo as eo
device = eo.device(
"cuda",
strict=True,
)Strict mode requires CUDA to be available.
EveryO documents GPU-resident tensor utilities under eo.cuda.
GPU-resident operations support selected functionality and should not be assumed to provide general autograd integration.
Benchmark a supported operation on CPU and CUDA.
Include host-device transfer time in an end-to-end comparison.
EveryO includes experimental process-based distributed training functionality.
python examples/distributed_training.py- Worker process.
- Process rank.
- World size.
- Gradient synchronization.
- Communication overhead.
Compare distributed results against a single-process reference.
The documented TCP communication implementation assumes a trusted network.
Successful same-host testing does not establish reliable cross-machine behavior.
EveryO provides ONNX interoperability for supported models.
python -m pip install -e ".[onnx]"import everyo as eo
model = eo.Sequential(
eo.Linear(4, 8),
eo.ReLU(),
eo.Linear(8, 2),
)
model.eval()
eo.export_onnx(
model,
"advanced_model.onnx",
input_shape=(1, 4),
)python examples/onnx_export.pyRun the exported model through ONNX Runtime and compare outputs.
Traced models may encode fixed control-flow or sequence-length assumptions.
EveryO includes dynamic quantization utilities.
Quantization reduces the precision of stored numerical values.
For an idealized weight-only comparison:
float32 weight = 4 bytes
int8 weight = 1 byte
This suggests a potential reduction in raw weight storage, excluding metadata and other model components.
EveryO does not document specialized faster int8 execution kernels for its dynamic quantization implementation.
Compare original and quantized model outputs and serialized sizes.
EveryO provides profiling utilities.
import everyo as eo
with eo.profile() as profiler:
output = model(input_tensor)
print(profiler.summary())The model and input tensor must be defined beforehand.
Identify:
- Expensive operations.
- Repeated computations.
- Memory-intensive stages.
- Unexpected bottlenecks.
PROFILE
|
v
FIND BOTTLENECK
|
v
VERIFY CAUSE
|
v
APPLY CHANGE
|
v
TEST CORRECTNESS
|
v
BENCHMARK
|
v
DOCUMENT RESULT
Do not optimize based solely on assumptions.
python -m pip freeze > requirements-lock.txtgit rev-parse HEADexperiment:
name: advanced-classifier
framework: EveryO
model:
architecture: multilayer-perceptron
input_features: 8
hidden_features: 32
output_features: 3
training:
optimizer: adam
learning_rate: 0.001
epochs: 10
batch_size: 32
evaluation:
metric: accuracy
environment:
device: cpu
seed: 42This is a suggested record format, not a required EveryO configuration schema.
Store:
- Configuration.
- Metrics.
- Model checkpoints.
- Software versions.
- Relevant logs.
| Problem | Possible Cause | Recommended Action |
|---|---|---|
| Shape mismatch | Incorrect dimensions | Inspect tensor shapes |
| Missing gradients | Broken gradient path | Verify autograd |
| NaN loss | Numerical instability | Inspect inputs and learning rate |
| Slow training | Computational bottleneck | Profile execution |
| CUDA unavailable | Missing hardware or extension | Run diagnostics |
| ONNX mismatch | Unsupported or shape-sensitive export | Compare outputs |
| Serialization error | Unsupported module or invalid archive | Verify registration and file |
| Overfitting | Excessive model complexity | Validate regularization |
everyo doctor
python -m pytest -q
ruff check .Reduce a failing model to the smallest architecture that reproduces the issue.
Build a small computation graph and compare analytical gradients with finite differences.
Deliverable: Numerical error report.
Build a reusable neural network module using documented EveryO abstractions.
Deliverable: Module implementation and tests.
Run the repository convolutional network example.
Deliverable: Input/output shape analysis and training results.
Compare RNN, LSTM, and GRU architectures.
Deliverable: Evaluation table and reproducible configuration.
Inspect attention scores and verify their normalization.
Deliverable: Mathematical explanation and numerical tests.
Export a supported model and compare EveryO and ONNX Runtime outputs.
Deliverable: Absolute and relative error report.
Profile a model and identify its most expensive operations.
Deliverable: Profiling summary and optimization proposal.
Save and reload a supported model.
Deliverable: Parameter and prediction equivalence test.
Focus: Tensor operations and gradients.
Completion criteria:
- Understand tensor shapes.
- Validate matrix multiplication.
- Derive simple gradients.
- Implement finite-difference checks.
Focus: Model architectures.
Completion criteria:
- Build an MLP.
- Run a CNN example.
- Understand normalization.
- Validate model outputs.
Focus: Advanced architectures.
Completion criteria:
- Run recurrent model examples.
- Explain LSTM and GRU.
- Understand scaled dot-product attention.
- Validate sequence shapes.
Focus: Reproducible experiments.
Completion criteria:
- Configure a training pipeline.
- Monitor validation metrics.
- Save checkpoints.
- Verify restored outputs.
Focus: Optional advanced capabilities.
Completion criteria:
- Understand CUDA availability.
- Evaluate mixed precision.
- Export a supported ONNX model.
- Compare numerical outputs.
- Profile performance.
| Resource | Link |
|---|---|
| EveryO Repository | GitHub |
| README | README.md |
| Example Programs | examples/ |
| API Reference | API Reference Wiki |
| Tensor Engine | Tensor Engine Wiki |
| Automatic Differentiation | Autograd Wiki |
| Neural Networks | Neural Networks Wiki |
| Model Training | Model Training Wiki |
| Benchmarking | Benchmarking Guide |
| Numerical Validation | Numerical Accuracy and Validation |
| Contributing | CONTRIBUTING.md |
| Support | SUPPORT.md |
| NumPy | numpy.org |
| ONNX | onnx.ai |
Next Recommended Page: Custom Modules and Layers →
Home · Examples and Tutorials · API Reference
EveryO — Open Machine Learning. Built for Developers.
Project Maintainer: @krishanth7
Open Source · MIT Licensed · Community Driven
© 2026 EveryO Contributors