Repository navigation
Glossary
Learn the Concepts · Understand the Mathematics · Build with Confidence
A comprehensive reference of the terminology, mathematical concepts, computing technologies, and engineering principles used throughout the EveryO ecosystem.
Home · Getting Started · Framework Architecture · API Reference
Project: EveryO
Maintainer: @krishanth7
Note
About This Glossary
This glossary contains both EveryO-specific terminology and general machine learning concepts.
The inclusion of a technical term does not necessarily mean EveryO implements that feature.
For confirmed functionality, consult the API Reference and the repository's README.
- How to Use This Glossary
- Alphabetical Index
- A — Automatic Differentiation and Architecture
- B — Backpropagation and Batching
- C — Computing and Convolution
- D — Data and Derivatives
- E — Evaluation and Execution
- F — Functions and Floating Point
- G — Gradients and Graphs
- H — Hardware and Hyperparameters
- I — Inference and Initialization
- J–K — Jacobians and Kernels
- L — Layers and Losses
- M — Models and Matrices
- N — Neural Networks and Normalization
- O — Optimization and Operations
- P — Parameters and Precision
- Q–R — Quantization and Reproducibility
- S — Serialization and Scaling
- T — Tensors and Training
- U–V — Utilities and Validation
- W–Z — Weights and Zero Gradients
- Mathematical Symbols
- EveryO API Terminology
- Common Abbreviations
- Frequently Confused Concepts
- Learning Path
- Official References
The EveryO glossary is organized alphabetically to make technical concepts easy to locate.
Each definition explains what a term means and, where appropriate, how it relates to neural computing.
| Category | Description |
|---|---|
| Mathematics | Numerical and mathematical foundations |
| Machine Learning | Model development and learning concepts |
| Neural Networks | Layers, activations, and architectures |
| Computing | CPU, GPU, and hardware concepts |
| Software Engineering | APIs, testing, and development |
| EveryO | Framework-specific classes and features |
For beginners:
- Tensor.
- Neural Network.
- Forward Pass.
- Loss Function.
- Gradient.
- Backpropagation.
- Optimizer.
- Training Loop.
For advanced readers:
- Computational Graph.
- Reverse-Mode Automatic Differentiation.
- Jacobian.
- Numerical Stability.
- Mixed Precision.
- CUDA.
- Distributed Training.
- ONNX.
| Letter | Topics |
|---|---|
| A | Activation, Adam, Autograd, Architecture |
| B | Backpropagation, Batch, Broadcasting |
| C | CUDA, CNN, Computational Graph |
| D | Dataset, DataLoader, Derivative |
| E | Epoch, Evaluation, Execution |
| F | Forward Pass, Float32, Finite Difference |
| G | Gradient, GPU, Gradient Clipping |
| H | Hyperparameter, Hidden Layer |
| I | Inference, Initialization |
| J | Jacobian |
| K | Kernel |
| L | Layer, Learning Rate, Loss |
| M | Matrix, Model, Momentum |
| N | Neural Network, NumPy, Normalization |
| O | Optimizer, ONNX, Operation |
| P | Parameter, Precision, Pooling |
| Q | Quantization |
| R | ReLU, RNN, Reproducibility |
| S | SGD, Serialization, Softmax |
| T | Tensor, Training, Transformer |
| U | Underfitting |
| V | Validation, Vector |
| W | Weight, Weight Decay |
| X | Xavier Initialization |
| Y | YAML |
| Z | Zero Gradient |
A mathematical function applied to a neural network layer's output to introduce nonlinearity.
Examples include:
- ReLU.
- Sigmoid.
- Tanh.
- Softmax.
Adaptive Moment Estimation.
An optimization algorithm that maintains moving averages of gradients and squared gradients.
A simplified parameter update is:
[ \theta_{t+1} = \theta_t - \eta \frac{\hat m_t}{\sqrt{\hat v_t}+\epsilon} ]
Where:
- (\theta): model parameters.
- (\eta): learning rate.
- (\hat m_t): bias-corrected first moment.
- (\hat v_t): bias-corrected second moment.
- (\epsilon): numerical stability constant.
EveryO: Implemented.
Application Programming Interface.
A defined set of functions, classes, and methods used to interact with software.
The structural design of a neural network or software system.
For neural networks, architecture describes the arrangement of layers and connections.
A collection of numerical values organized into one or more dimensions.
A neural network mechanism that computes relationships between elements of an input sequence.
A common formulation is:
\operatorname{softmax} \left( \frac{QK^T}{\sqrt{d_k}} \right)V ]
A technique for computing derivatives by applying the chain rule to a sequence of mathematical operations.
EveryO uses reverse-mode automatic differentiation.
Automatic gradient computation.
EveryO's autograd engine records differentiable operations and propagates gradients backward through the resulting graph.
The process of propagating gradients backward through a computational graph.
It applies the chain rule to calculate how model parameters affect a loss.
A group of samples processed together.
Example:
Batch size = 32
Samples processed together = 32
A technique that normalizes intermediate activations using batch statistics during training.
It may improve optimization behavior in some architectures.
The number of samples processed in one training iteration.
A trainable additive parameter in a neural network layer.
A linear transformation is:
[ y = xW + b ]
A mechanism that allows compatible arrays of different shapes to participate in element-wise operations.
The numerical execution system responsible for carrying out operations.
EveryO uses NumPy for its core CPU implementation and supports optional acceleration and integrations.
A mathematical rule for differentiating composite functions.
\frac{dy}{du} \frac{du}{dx} ]
The chain rule is fundamental to automatic differentiation.
A saved representation of model state or training progress.
A machine learning task that assigns inputs to discrete categories.
A directed representation of mathematical operations and their dependencies.
Example:
x
|
v
Multiply
|
v
y
|
v
Loss
An operation that applies learned filters across an input.
It is commonly used in image-processing neural networks.
Convolutional Neural Network.
A neural network architecture built around convolution operations.
Central Processing Unit.
The primary processor responsible for general-purpose computation.
Compute Unified Device Architecture.
NVIDIA's platform for GPU computing.
EveryO provides an optional native CUDA extension.
A loss function commonly used for classification.
For a target distribution (y) and predicted probabilities (p):
[ L=-\sum_i y_i\log(p_i) ]
A utility that produces batches from a dataset.
EveryO provides a DataLoader abstraction.
A collection of examples used for training, validation, or testing.
A mathematical quantity describing how a function changes with respect to an input.
The hardware execution location of a tensor.
Examples:
- CPU.
- CUDA GPU.
An axis of a tensor.
For example, a matrix has two dimensions.
Training that coordinates computation across multiple processes or devices.
A regularization technique that randomly disables selected activations during training.
Data type.
It describes how tensor values are represented, such as float32 or float64.
One complete pass through the training dataset.
Measuring model performance on data without performing parameter updates.
A model state used for inference or evaluation.
It changes the behavior of certain layers, such as dropout and batch normalization.
The system that performs numerical operations.
A training problem where gradient magnitudes become excessively large.
The ability to repeat an experiment under controlled conditions and obtain comparable results.
A numerical approximation of a derivative.
The central finite-difference formula is:
[ f'(x)\approx \frac{f(x+h)-f(x-h)}{2h} ]
EveryO uses finite-difference checks to validate gradients.
A 16-bit floating-point representation.
It uses less memory than float32 but provides a narrower precision and numerical range.
A 32-bit floating-point representation widely used in machine learning.
A 64-bit floating-point representation with greater precision than float32.
The process of computing model outputs from inputs.
A reusable software foundation that provides common components and abstractions.
A mathematical or programmatic mapping from inputs to outputs.
A vector of partial derivatives describing how a scalar function changes with respect to its inputs.
[ \nabla f(x)= \begin{bmatrix} \frac{\partial f}{\partial x_1}\ \frac{\partial f}{\partial x_2}\ \vdots\ \frac{\partial f}{\partial x_n} \end{bmatrix} ]
Combining gradients from multiple backward computations before updating parameters.
A technique that limits gradient magnitude to improve training stability.
An optimization method that updates parameters in the negative gradient direction.
\theta_t-\eta\nabla L(\theta_t) ]
Graphics Processing Unit.
A processor capable of highly parallel numerical computation.
Visiting nodes in a computational graph in an order suitable for executing operations or propagating gradients.
Using specialized hardware to speed up computation.
A weight initialization technique designed for networks using ReLU-like activations.
Hidden Layer
A neural network layer between the input and output layers.
A configuration value selected before or during training rather than learned through gradient updates.
Examples:
- Learning rate.
- Batch size.
- Number of epochs.
- Hidden layer width.
The process of selecting hyperparameters to improve model performance.
Using a trained model to generate predictions.
Assigning starting values to model parameters.
A tensor supplied to a model or mathematical operation.
A single training step, typically involving a batch of data.
The output of an internal neural network layer.
A matrix of partial derivatives for a vector-valued function.
[ J_{ij}= \frac{\partial y_i}{\partial x_j} ]
A term with different meanings depending on context.
Convolution kernel: A filter applied to an input.
CUDA kernel: A function executed on a GPU.
The act of starting a GPU kernel execution.
A model evaluation method that divides data into multiple subsets for repeated training and validation.
This is a general machine learning concept; its inclusion does not imply a dedicated EveryO implementation.
A computational component within a neural network.
A normalization technique that computes statistics across selected feature dimensions.
A hyperparameter controlling the size of optimizer updates.
A trainable affine transformation.
[ y=xW+b ]
Raw model scores before conversion to probabilities.
A function measuring disagreement between predictions and targets.
Long Short-Term Memory.
A recurrent neural network architecture designed to model sequential dependencies.
A two-dimensional array of values.
A mathematical operation that combines compatible matrices.
For matrices (A) and (B):
[ C_{ij}=\sum_k A_{ik}B_{kj} ]
A regression loss function.
[ MSE= \frac{1}{N} \sum_{i=1}^{N}(y_i-\hat y_i)^2 ]
A subset of training examples processed together.
Using multiple floating-point precisions during computation.
A parameterized mathematical system that maps inputs to outputs.
An optimization technique that incorporates previous update directions.
A reusable neural network component containing operations, parameters, or child modules.
An attention architecture that computes multiple attention projections in parallel.
A parameterized computational model composed of interconnected operations or layers.
A conceptual computational unit that combines inputs and applies a transformation.
A tensor layout representing:
N = Batch
H = Height
W = Width
C = Channels
EveryO's convolutional operations use NHWC layout.
An alternative image tensor layout:
N = Batch
C = Channels
H = Height
W = Width
Transforming values to improve numerical or optimization behavior.
A Python library for numerical array computation.
NumPy is the foundation of EveryO's CPU tensor implementation.
The ability of an algorithm to avoid excessive numerical error under floating-point arithmetic.
Open Neural Network Exchange.
An interoperable format for representing machine learning models.
A runtime used to execute supported ONNX models.
A mathematical transformation applied to one or more tensors.
An algorithm that updates trainable parameters using gradients.
A situation where a model learns training data patterns too specifically and performs poorly on unseen data.
A tensor produced by an operation or model.
A trainable value used by a model.
Examples include weights and biases.
A change to model parameters performed by an optimizer.
An operation that reduces spatial dimensions by summarizing local regions.
The numerical resolution of a floating-point representation.
A model-generated output.
A tool that measures execution behavior, such as operation timing.
EveryO includes profiling utilities.
The programming language used to implement EveryO's primary framework.
Representing numerical values using lower-precision formats.
EveryO includes dynamic int8 weight quantization.
The number of dimensions in a tensor.
In distributed computing, rank also identifies a participating process.
Rectified Linear Unit.
[ ReLU(x)=\max(0,x) ]
A technique intended to improve generalization or constrain model complexity.
The ability to repeat an experiment under controlled conditions.
A derivative computation technique that propagates gradients backward from outputs toward inputs.
Recurrent Neural Network.
A model architecture that processes sequential information using recurrent state.
A single numerical value.
Stochastic Gradient Descent.
An optimization algorithm that updates parameters using gradients estimated from individual samples or batches.
Converting model information into a format that can be stored and loaded.
EveryO uses .evo model archives.
The dimensions of a tensor.
Example:
(32, 128)
An activation function:
[ \sigma(x)=\frac{1}{1+e^{-x}} ]
A function that converts scores into normalized probabilities.
[ softmax(x_i)= \frac{e^{x_i}}{\sum_j e^{x_j}} ]
A mapping of parameter names to stored values.
The step size used when moving a convolution or pooling window across an input.
Training using examples with associated targets.
A multidimensional numerical data structure.
The subsystem responsible for tensor operations and associated numerical behavior.
A machine learning framework used optionally for comparison and interoperability with EveryO.
Data reserved for evaluating model performance after training decisions have been made.
The process of adjusting model parameters to reduce a loss.
The repeated sequence of:
Forward Pass
|
v
Loss Calculation
|
v
Backward Pass
|
v
Optimizer Update
|
v
Next Batch
A neural network architecture based on attention mechanisms.
A record of executed operations.
A model exporter that constructs a graph from operations executed during an example forward pass.
A situation where a model cannot adequately represent the relationships in its training data.
A test that verifies an individual software component.
Data used to evaluate model behavior during development and model selection.
The loss measured on validation data.
A one-dimensional numerical array.
Expressing computation through array operations rather than explicit element-by-element Python loops.
A system for tracking changes to source code.
EveryO uses Git and GitHub.
A trainable model parameter controlling the influence of an input or intermediate feature.
A regularization technique that penalizes or reduces parameter magnitude during optimization.
A weight initialization method designed to maintain appropriate activation variance through neural network layers.
A human-readable data serialization format often used for configuration files.
EveryO provides optional YAML-related dependencies.
Clearing stored gradients before a new optimization step.
This prevents unintended accumulation when gradients are stored across backward passes.
| Symbol | Meaning |
|---|---|
| (x) | Input variable or tensor |
| (y) | Output or target |
| (\hat y) | Predicted output |
| (W) | Weight matrix |
| (b) | Bias |
| (\theta) | Model parameters |
| (\eta) | Learning rate |
| (L) | Loss function |
| (\nabla) | Gradient operator |
| (\partial) | Partial derivative |
| (\sum) | Summation |
| (\mu) | Mean |
| (\sigma) | Standard deviation or sigmoid notation |
| (\epsilon) | Small numerical constant |
| (N) | Number of samples |
| (B) | Batch size |
| (T) | Sequence length |
| (d) | Feature dimension |
| (J) | Jacobian matrix |
| (Q) | Attention query |
| (K) | Attention key |
| (V) | Attention value |
This section maps common concepts to EveryO's public API.
| Concept | EveryO API | Purpose |
|---|---|---|
| Tensor | Tensor |
Numerical data and gradient tracking |
| Tensor constructor | tensor |
Create tensors |
| Module | Module |
Base class for neural network components |
| Trainable parameter | Parameter |
Model parameter |
| Linear layer | Linear |
Affine transformation |
| Sequential model | Sequential |
Compose modules |
| ReLU | ReLU |
Nonlinear activation |
| Optimizer | SGD |
Gradient-based updates |
| Optimizer | Adam |
Adaptive optimization |
| Regression loss | MSELoss |
Mean squared error |
| Classification loss | CrossEntropyLoss |
Cross-entropy objective |
| Dataset | ArrayDataset |
Array-backed dataset |
| Data loading | DataLoader |
Batch generation |
| Training | Trainer |
Training workflow |
| Save | save |
Serialize model |
| Load | load |
Restore model |
| Profiling | profile |
Collect execution measurements |
import everyo as eo
x = eo.tensor(
[1.0, 2.0, 3.0],
requires_grad=True,
)import everyo as eo
model = eo.Sequential(
eo.Linear(4, 16),
eo.ReLU(),
eo.Linear(16, 2),
)optimizer = eo.Adam(
model.parameters(),
lr=0.001,
)For detailed signatures and supported parameters, consult the API Reference.
| Abbreviation | Full Form |
|---|---|
| AI | Artificial Intelligence |
| ML | Machine Learning |
| DL | Deep Learning |
| NN | Neural Network |
| ANN | Artificial Neural Network |
| CNN | Convolutional Neural Network |
| RNN | Recurrent Neural Network |
| LSTM | Long Short-Term Memory |
| GRU | Gated Recurrent Unit |
| MHA | Multi-Head Attention |
| SGD | Stochastic Gradient Descent |
| MSE | Mean Squared Error |
| MAE | Mean Absolute Error |
| CE | Cross-Entropy |
| CPU | Central Processing Unit |
| GPU | Graphics Processing Unit |
| CUDA | Compute Unified Device Architecture |
| API | Application Programming Interface |
| CLI | Command-Line Interface |
| CI | Continuous Integration |
| CD | Continuous Delivery / Deployment |
| ONNX | Open Neural Network Exchange |
| NHWC | Batch, Height, Width, Channels |
| NCHW | Batch, Channels, Height, Width |
| FP16 | 16-bit Floating Point |
| FP32 | 32-bit Floating Point |
| FP64 | 64-bit Floating Point |
| INT8 | 8-bit Integer |
| JSON | JavaScript Object Notation |
| YAML | YAML Ain't Markup Language |
| PR | Pull Request |
| OSS | Open-Source Software |
| Tensor | NumPy Array |
|---|---|
| EveryO abstraction | NumPy data structure |
| Can track gradients | Does not natively build an autograd graph |
| Supports EveryO operations | Supports NumPy operations |
| Can carry device metadata | Primarily CPU array storage |
| Parameter | Hyperparameter |
|---|---|
| Learned during training | Selected as configuration |
| Example: weights | Example: learning rate |
| Updated by optimizer | Changed through tuning or configuration |
| Training | Inference |
|---|---|
| Computes gradients | Usually disables gradients |
| Updates parameters | Uses learned parameters |
| Uses training data | Uses input data for predictions |
| CPU | GPU |
|---|---|
| General-purpose processor | Highly parallel processor |
| Works without CUDA | CUDA requires compatible NVIDIA hardware |
| Suitable for smaller experiments | Can benefit larger parallel workloads |
A derivative describes the rate of change of a function with respect to a variable.
A gradient is a collection of partial derivatives for a scalar function with multiple inputs.
Batch: A subset of samples processed in one step.
Epoch: One complete pass through the training dataset.
Serialization: Preserves model information in a framework-specific format.
ONNX export: Converts a supported model into an interoperable computational graph.
Implemented: Code for a feature exists.
Validated: The feature has been tested against defined acceptance criteria.
A feature may be implemented without having been verified in every environment.
Start with:
- Python.
- NumPy.
- Tensors.
- Matrices.
- Neural networks.
- Activation functions.
- Loss functions.
- Gradient descent.
Recommended pages:
Study:
- Computational graphs.
- Automatic differentiation.
- Backpropagation.
- Optimizers.
- Training loops.
- Data loading.
- Model serialization.
Recommended pages:
Explore:
- Numerical gradient verification.
- CUDA.
- Distributed training.
- Mixed precision.
- Quantization.
- ONNX tracing.
- Profiling.
- Framework architecture.
Recommended pages:
| Resource | Link |
|---|---|
| EveryO Repository | GitHub |
| README | README.md |
| Architecture | Architecture Documentation |
| Automatic Differentiation | Autograd Documentation |
| API Reference | API Documentation |
| CUDA | CUDA Documentation |
| Scaling | Scaling Documentation |
| Contributing | CONTRIBUTING.md |
| Support | SUPPORT.md |
| Security | SECURITY.md |
| NumPy | numpy.org |
| Python | python.org |
| ONNX | onnx.ai |
Next Recommended Page: Community Guidelines →
Home · API Reference · Frequently Asked Questions
EveryO — Open Machine Learning. Built for Developers.
Project Maintainer: @krishanth7
Open Source · MIT Licensed · Community Driven
© 2026 EveryO Contributors