-
Notifications
You must be signed in to change notification settings - Fork 0
C# API
Complete C# API documentation - only shows publicly exposed classes, functions, interfaces, and properties.
Also refer to the Mathematical Definitions Documentation for the exact formulas used by the API's functions.
Classes for handling the training of neural networks.
Provides standard supervised training functionality using predefined training datasets and optional testing datasets.
using NNNCSharp.Components.Trainers;
- public Model Model { get; private set; } - stores the neural network being trained by the instance - updated to the best-performing version after every Train() call
public Trainer(Model model, Optimizer optimizer, Cost cost, float maxGradNorm = 1.0f);
- Model model - neural network to train
- Optimizer optimizer - optimizer instance to use for parameter updates
- Cost cost - cost function instance to use during loss calculation
- float maxGradNorm - maximum gradient norm to clip gradients to
Initializes a new Trainer instance to train the given neural network model.
public void Train(BatchBuffer batchBuffer, int batchSize, int epochs, bool batchAllInputs = true,
Func<Model, int, bool>? testFunc = null, bool decayLR = true, float minLRFraction = 0.1f,
int testEvery = 100, int testLength = 1000, string? saveTo = null);
- BatchBuffer batchBuffer - BatchBuffer instance to sample training batches from
- int batchSize - number of input-target pairs to use per training batch
- int epochs - total number of epochs to train for
- bool batchAllInputs - whether to train on the maximum possible number of training batches every epoch - does not repeat any input-target pairs in a single epoch
- Func<Model, int, bool>? testFunc - optional fitness test function to use for performance evaluation - should receive the training neural network and test index as arguments and return whether the model successfully past the performance test - performance tests will be skipped if left null
- bool decayLR - whether to decay the learning rate over the course of training - uses a single-cycle Cosine Annealing Learning Rate Decay function
- float minLRFraction - minimum fraction of its initial value to which the learning rate is allowed to decay - will be reached on the final epoch
- int testEvery - number of epochs to run between performance evaluations
- int testLength - number of times to run the fitness test function for each performance evaluation
- string? saveTo - optional file name to which intermediate models will be saved to during training
Trains the model contained in the instance for a given number of epochs. If provided with a fitness function (testFunc), will cache the best-performing model at each performance evaluation and will store the overall best-performing version in the instance's Model property after the final training epoch. The trained model should always be read from the instance's Model property after calling Train().
public void StopTraining();
Terminates the current training session early. If a fitness function was provided, stores the best-performing cached model in the instance's Model property.
Provides Deep Q-Learning (DQN) training functionality for environments with discrete action spaces. Supports both solo and self-play environments.
using NNNCSharp.Components.Trainers;
- public Model Agent { get; private set; } - stores the neural network agent being trained by the instance - updated to the best-performing version after every Train() call
public DQNTrainer(Model agent, DQNEnvironment environment, Optimizer optimizer, Cost cost,
int trainEvery = 4, float discount = 0.995f, float exploration = 1.0f, float explorationDecay = 0.99f,
float minExploration = 0.01f, int replayBufferSize = 10000, int batchSize = 64, int agentBufferSize = 5,
int opponentCopyRate = 100, int minRandomOpponentEpisodes = 200, float tau = 0.005f,
float maxGradNorm = 1.0f, int minExperiences = 1000);
- Model agent - neural network to train
- DQNEnvironment environment - discrete environment instance to train the agent in
- Optimizer optimizer - optimizer instance to use for parameter updates
- Cost cost - cost function instance to use during loss calculation
- int trainEvery - number of environment steps between agent training steps
- float discount - discount factor to apply during Q-Value estimation
- float exploration - initial exploration rate to use
- float explorationDecay - factor by which to decay the learning rate after every training episode
- float minExploration - minimum exploration rate to decay to
- int replayBufferSize - number of experiences to store for replay during training
- int batchSize - number of experiences to include in each training batch
- int agentBufferSize - number of frozen past agent versions to store - used as opponents in self-play environments
- int opponentCopyRate - number of training episodes between agent versions being frozen to use as opponents - only applicable to self-play environments
- int minRandomOpponentEpisodes - minimum number of episodes which must be played against a random opponent - only applicable to self-play environments
- float tau - tau value to use during smooth target model parameter updates
- float maxGradNorm - maximum gradient norm to clip gradients to
- int minExperiences - minimum number of stored experiences required prior to starting training
Initializes a new DQNTrainer instance to train the given neural network.
public void Train(ref FIFOBuffer<Episode>? episodeBuffer, int episodes = 1000, int testEvery = 100,
int testEpisodes = 5000, string? saveTo = null);
- ref FIFOBuffer? episodeBuffer - optional buffer to store recent training episodes in - can be used to view previous episodes after training is completed
- int episodes - number of episodes to train for
- int testEvery - number of episodes between performance evaluations
- int testEpisodes - number of episodes to run during each performance evaluation
- string? saveTo - optional file name to which intermediate agents will be saved to during training
Trains the agent contained in the instance for a given number of episodes. Will use the environment's fitness function to cache the best-performing agent at each performance evaluation and will store the overall best-performing version in the instance's Model property after the final training episode. The trained agent should always be read from the instance's Model property after calling Train().
public void StopTraining();
Terminates the current training session early. Stores the best-performing cached agent in the instance's Agent property.
Classes for handling neural network parameter updates.
Base class defining general optimizer functionality and signatures. Can be inherited from to define custom optimizer functions.
using NNNCSharp.Components.Optimizers;
- public float LR { get; set; } - learning rate factor applied during parameter updates
public Optimizer(float learningRate);
- float learningRate - learning rate factor to apply during parameter updates
Serves as the default constructor for all optimizers.
public abstract void Step(Tensor parameter, int iterations);
- Tensor parameter - parameter tensor to update values of
- int iterations - number of times the neural network's parameters have been updated - used as part of certain optimizer step functions
Abstract function declaration used by all optimizer parameter update functions.
Stochastic Gradient Descent optimizer. Inherits from Optimizer.
using NNNCSharp.Components.Optimizers;
- public float LR { get; set; } - learning rate factor applied during parameter updates
public SGD(float learningRate) : base(learningRate);
- float learningRate - learning rate factor to apply during parameter updates
Creates a new Stochastic Gradient Descent optimizer instance.
public override void Step(Tensor parameter, int iterations);
- Tensor parameter - parameter tensor to update values of
- int iterations - number of times the neural network's parameters have been updated - not used for SGD optimization
Updates the parameter tensor's values based on the current stored gradients using the Stochastic Gradient Descent optimizer function.
Adaptive Moment Estimation optimizer. Inherits from Optimizer.
using NNNCSharp.Components.Optimizers;
- public float LR { get; set; } - learning rate factor applied during parameter updates
public Adam(float learningRate, float beta1 = 0.9f, float beta2 = 0.999f, float epsilon = 1e-8f,
float weightDecay = 0.0f) : base(learningRate);
- float learningRate - learning rate factor to apply during parameter updates
- float beta1 - exponential decay rate of first moment estimates
- float beta2 - exponential decay rate of second moment estimates
- float epsilon - epsilon value to use in calculations
- float weightDecay - AdamW-style weight decay rate to use
Creates a new Adaptive Moment Estimation optimizer instance.
public override void Step(Tensor parameter, int iterations);
- Tensor parameter - parameter tensor to update values of
- int iterations - number of times the neural network's parameters have been updated - used to correct bias in moment estimates
Updates the parameter tensor's values based on the current stored gradients using the Adaptive Moment Estimation optimizer function with AdamW-style weight decay applied.
Classes for calculating loss values.
Base class defining general cost functionality and signatures. Can be inherited from to define custom cost functions.
using NNNCSharp.Components.Costs;
public Cost();
Serves as the default constructor for all costs.
public abstract Tensor CalculateCost(Tensor predictions, Tensor targets);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
Tensor containing a single value representing the average loss of all predictions in the batch.
Abstract function declaration used by all cost calculation functions. Expects both predictions and targets to have a leading batch dimension.
public abstract Tensor CalculatePerSampleCost(Tensor predictions, Tensor targets);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
Tensor containing the individual loss values for each prediction in the batch.
Abstract function declaration used by all cost calculation functions. Expects both predictions and targets to have a leading batch dimension.
public virtual CostResult CalculateCostWithPriority(Tensor predictions, Tensor targets, double[]? weights = null);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
- doublep[]? weights - Prioritized Experience Replay (PER) sampling weights for each experience in the batch
CostResult instance containing the per-prediction losses tensor and corresponding Prioritized Experience Replay (PER) sampling priorities.
Calculates the per-prediction losses of the given experience batch, scales each loss by the corresponding Prioritized Experience Replay (PER) sampling weight (if weights are passed), and computes the corresponding PER sampling priorities of each experience. Expects both predictions and targets to have a leading batch dimension. Used for DQN training.
Stores the per-prediction losses and corresponding Prioritized Experience Replay (PER) sampling priorities computed by a Cost instance.
using NNNCSharp.Components.Costs;
- public Tensor Losses { get; init; } - per-prediction losses tensor
- public double[] Priorities { get; init; } - corresponding per-experience Prioritized Experience Replay (PER) sampling priorities
public CostResult(Tensor Losses, double[] Priorities);
- Tensor Losses - per-prediction losses tensor to store
- double[] Priorities - corresponding per-experience Prioritized Experience Replay (PER) sampling priorities to store
Creates a new CostResult record instance storing the given losses and Prioritized Experience Replay (PER) sampling priorities.
Mean Squared Error cost. Inherits from Cost.
using NNNCSharp.Components.Costs;
public MSE();
Creates a new Mean Squared Error cost instance.
public override Tensor CalculateCost(Tensor predictions, Tensor targets);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
Tensor containing a single value representing the average loss of all predictions in the batch.
Calculates the average loss of all predictions in a batch using the Mean Squared Error cost function. Expects both predictions and targets to have a leading batch dimension.
public override Tensor CalculatePerSampleCost(Tensor predictions, Tensor targets);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
Tensor containing the individual loss values for each prediction in the batch.
Calculates the loss of each prediction in a batch using the Mean Squared Error cost function. Expects both predictions and targets to have a leading batch dimension.
public virtual CostResult CalculateCostWithPriority(Tensor predictions, Tensor targets, double[]? weights = null);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
- doublep[]? weights - Prioritized Experience Replay (PER) sampling weights for each experience in the batch
CostResult instance containing the per-prediction losses tensor and corresponding Prioritized Experience Replay (PER) sampling priorities.
Calculates the per-prediction losses of the given experience batch using the Mean Squared Error cost function, scales each loss by the corresponding Prioritized Experience Replay (PER) sampling weight (if weights are passed), and computes the corresponding PER sampling priorities of each experience. Expects both predictions and targets to have a leading batch dimension. Used for DQN training.
Pseudo-Huber cost. Inherits from Cost.
using NNNCSharp.Components.Costs;
public Huber(float delta = 1.0f);
- float delta - delta value at which to transition from Mean Squared Error to Mean Absolute Error
Creates a new pseudo-Huber cost instance.
public override Tensor CalculateCost(Tensor predictions, Tensor targets);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
Tensor containing a single value representing the average loss of all predictions in the batch.
Calculates the average loss of all predictions in a batch using the pseudo-Huber cost function. Expects both predictions and targets to have a leading batch dimension.
public override Tensor CalculatePerSampleCost(Tensor predictions, Tensor targets);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
Tensor containing the individual loss values for each prediction in the batch.
Calculates the loss of each prediction in a batch using the pseudo-Huber cost function. Expects both predictions and targets to have a leading batch dimension.
public virtual CostResult CalculateCostWithPriority(Tensor predictions, Tensor targets, double[]? weights = null);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
- doublep[]? weights - Prioritized Experience Replay (PER) sampling weights for each experience in the batch
CostResult instance containing the per-prediction losses tensor and corresponding Prioritized Experience Replay (PER) sampling priorities.
Calculates the per-prediction losses of the given experience batch using the pseudo-Huber cost function, scales each loss by the corresponding Prioritized Experience Replay (PER) sampling weight (if weights are passed), and computes the corresponding PER sampling priorities of each experience. Expects both predictions and targets to have a leading batch dimension. Used for DQN training.
Softmax Cross-Entropy cost. Inherits from Cost.
using NNNCSharp.Components.Costs;
public SoftmaxCrossEntropy();
Creates a Softmax Cross-Entropy cost instance.
public override Tensor CalculateCost(Tensor predictions, Tensor targets);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
Tensor containing a single value representing the average loss of all predictions in the batch.
Calculates the average loss of all predictions in a batch using the Softmax Cross-Entropy cost function. Expects both predictions and targets to have a leading batch dimension.
public override Tensor CalculatePerSampleCost(Tensor predictions, Tensor targets);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
Tensor containing the individual loss values for each prediction in the batch.
Calculates the loss of each prediction in a batch using the Softmax Cross-Entropy cost function. Expects both predictions and targets to have a leading batch dimension.
public virtual CostResult CalculateCostWithPriority(Tensor predictions, Tensor targets, double[]? weights = null);
- Tensor predictions - predictions generated by the neural network for an input batch
- Tensor targets - target outputs corresponding to the input batch
- doublep[]? weights - Prioritized Experience Replay (PER) sampling weights for each experience in the batch
CostResult instance containing the per-prediction losses tensor and corresponding Prioritized Experience Replay (PER) sampling priorities.
Calculates the per-prediction losses of the given experience batch using the Softmax Cross-Entropy cost function, scales each loss by the corresponding Prioritized Experience Replay (PER) sampling weight (if weights are passed), and computes the corresponding PER sampling priorities of each experience. Expects both predictions and targets to have a leading batch dimension. Used for DQN training.
Neural Network container classes.
Represents a complete neural network comprised of a sequence of layers. Implements IDisposable.
using NNNCSharp.Components.Models;
- public Layer[] Layers { get; private set; } - sequence of layers used by the neural network
- public List<Tensor> Parameters { get; } - list of all parameter tensors across all of the neural network's layers
- public int ParameterCount { get; } - number of parameter tensors across all of the neural network's layers
public Model(Layer[] layers, Tensor inputFormat);
- Layer[] layers - sequence of layers the neural network will contain
- Tensor inputFormat - general format of the input the neural network will receive - should have the dimensions [batch, input dimensions] where the batch dimension can simply equal 1
Creates a new sequential Model instance with the given sequence of layers and initializes all of its layers to receive the given input format. Used to initialize a completely new neural network.
public Model(Layer[] layers);
- Layer[] layers - sequence of layers the neural network will contain
Creates a new sequential Model instance with the given sequence of layers. Does not initialize its layers. Used to initialize a new neural network containing layers with already-initialized / trained parameters.
public Tensor Predict(Tensor input);
- Tensor input - input for the neural network to process - must match the input format dimensions provided during Model initialization - requires a batch dimension of 1 if processing a single input (consider using Tensor.WrapBatch())
Tensor containing the outputs predicted by the model for the given input.
Gets the neural network's predictions for a given input. Does not construct an Autograd graph for the forward pass. More performance and memory efficient for getting predictions when not training.
public Tensor Forward(Tensor input);
- Tensor input - input for the neural network to process - must match the input format dimensions provided during Model initialization - requires a batch dimension of 1 if processing a single input (consider using Tensor.WrapBatch())
Tensor containing the outputs predicted by the model for the given input.
Gets the neural network's predictions for a given input. Constructs a complete Autograd graph for the forward pass. Allows for gradients to be subsequently calculated during training.
public void Dispose();
Frees all memory used by the neural network, its layers, and their parameter tensors, effectively deleting the instance. Should be used whenever a Model instance is no longer being used, such as when creating a copy and discarding the original. Failure to use may lead to memory leaks.
public Model Copy();
Deep-copy of the neural network containing identical parameters.
Can be used to create a new (deep-copy) Model instance with identical parameters to an existing instance. Recommended to be used in conjunction with Dispose().
public IEnumerable<Tensor> GetParameters();
IEnumerable instance containing all of the neural network's parameter tensors.
Can be used to iterate through all of the parameter tensors of a neural network directly. Functionally equivalent to the Parameters property.
public ulong GetTotalParameterSize();
Unsigned 64-bit integer representing the total number of individual values across all of the neural network's parameter tensors.
Can be used to determine the overall size of a model. Primarily used to provide additional data in .nnn files.
Classes representing layers of neurons in a neural network.
General class defining neural network layer functionality. Can be inherited from to create custom neuron layers. Implements IDisposable.
using NNNCSharp.Components.Models.Layers;
- public Tensor Biases { get; protected set; } - bias parameter tensor of the layer
- public Activation Activation { get; protected set; } - activation function used by the layer
- public Tensor OutputFormat { get; protected set; } - format of the tensor the layer will output - will have a leading batch dimension of 1
- public float Dropout { get; protected set; } - parameter dropout rate of the layer
public Layer(Activation activation, float dropout = 0.0f);
- Activation activation - activation function to be used by the layer
- float dropout - parameter dropout rate to be used by the layer
Serves as the primary constructor to be extended by layer subclasses.
public Layer();
Used when loading layers from .nnn files.
public abstract void SetUpLayer(Tensor inputFormat);
- Tensor inputFormat - general format of the input the layer will receive - should have the dimensions [batch, input dimensions] where the batch dimension can simply equal 1
Abstract function for layer subclasses to initialize their parameter tensors to accept the given input format. Must also update the layer's OutputFormat property.
public abstract Tensor Forward(Tensor input);
- Tensor input - input to be processed by the layer - must match the input format dimensions provided during Model initialization - requires a batch dimension of 1 if processing a single input (consider using Tensor.WrapBatch())
Tensor containing the result of passing the input through the layer.
Abstract function for layer subclasses to process inputs. Using the Forward and Predict functions of the Model class is recommended over directly calling Forward on layers.
public virtual void Dispose();
Frees all memory used by the layer, and its parameter tensors, effectively deleting the instance. Should be used whenever a Layer instance is no longer being used, such as when creating a copy and discarding the original. Failure to use may lead to memory leaks. Extended by layer subclasses to dispose any additional parameter tensors they introduce.
public abstract Layer Copy();
Deep-copy of the layer containing identical parameters.
Abstract function for layer subclasses to implement deep-copy functionality. Recommended to be used in conjunction with Dispose().
public abstract IEnumerable<Tensor> GetParameters();
IEnumerable instance containing all of the layer's parameter tensors.
Abstract function for layer subclasses to expose their parameter tensors in an iterable format.
public void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Loads all of the layer's parameters from the given file stream starting at the stream's current position. Used when loading layers from .nnn files.
public string PrintLayer(FileStream stream);
- FileStream stream - file stream from which to read data
String containing a human-readable summary of the layer data contained in the file stream starting at the stream's current position.
Parses the layer data contained in the file stream starting at the stream's current position. Provides a human-readable summary of the data found.
Represents a fully connected neural network layer. Inherits from Layer and implements IDisposable.
using NNNCSharp.Components.Models.Layers;
- public int NeuronCount { get; private set; } - number of neurons contained in the layer
- public bool Flatten { get; private set; } - whether the layer flattens its input prior to processing
- public Tensor Weights { get; private set; } - weights parameter tensor of the layer
- public Tensor Biases { get; protected set; } - bias parameter tensor of the layer
- public Activation Activation { get; protected set; } - activation function used by the layer
- public Tensor OutputFormat { get; protected set; } - format of the tensor the layer will output - will have a leading batch dimension of 1
- public float Dropout { get; protected set; } - parameter dropout rate of the layer
public Dense(int neuronCount, Activation activation, bool flatten = false, float dropout = 0.0f)
: base(activation, dropout);
- int neuronCount - number of neurons the layer will contain
- Activation activation - activation function to be used by the layer
- bool flatten - whether the layer will flatten its input prior to processing
- float dropout - parameter dropout rate to be used by the layer
Serves as the default constructor for Dense layers. Creates a new Dense layer instance with the given parameters. Does not automatically initialize parameter tensors.
public Dense(int neuronCount, Tensor weights, Tensor biases, Activation activation, bool flatten, float dropout)
: base(activation, dropout);
- int neuronCount - number of neurons the layer will contain
- Tensor weights - weights parameter tensor to be used by the layer
- Tensor biases - bias parameter tensor to be used by the layer
- Activation activation - activation function to be used by the layer
- bool flatten - whether the layer will flatten its input prior to processing
- float dropout - parameter dropout rate to be used by the layer
Creates a new Dense layer instance with the given complete parameters. All parameter tensors must be correctly initialized prior to the constructor being used. Used to create deep-copies of Dense layers.
public Dense();
Used when loading Dense layers from .nnn files.
public override void SetUpLayer(Tensor inputFormat);
- Tensor inputFormat - general format of the input the layer will receive - should have the dimensions [batch, input dimensions] where the batch dimension can simply equal 1
Initializes the layer's parameter tensors based on the given input format. Also initializes the layer's OutputFormat property.
public override Tensor Forward(Tensor input);
- Tensor input - input to be processed by the layer - must match the input format dimensions provided during Model initialization - requires a batch dimension of 1 if processing a single input (consider using Tensor.WrapBatch())
Tensor containing the result of passing the input through the layer.
Performs a matrix multiplication between the input and the layer's weights parameter tensor, and adds the layer's bias parameter tensor. Using the Forward and Predict functions of the Model class is recommended over directly calling Forward on layers.
public override void Dispose();
Frees all memory used by the layer, and its parameter tensors, effectively deleting the instance. Should be used whenever a Layer instance is no longer being used, such as when creating a copy and discarding the original. Failure to use may lead to memory leaks.
public override Layer Copy();
Deep-copy of the layer containing identical parameters.
Can be used to create a new (deep-copy) layer instance with identical parameters to an existing instance. Recommended to be used in conjunction with Dispose().
public override IEnumerable<Tensor> GetParameters();
IEnumerable instance containing all of the layer's parameter tensors.
Can be used to iterate through all of the parameter tensors of a Dense layer.
public void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Loads all of the layer's parameters from the given file stream starting at the stream's current position. Used when loading layers from .nnn files.
public string PrintLayer(FileStream stream);
- FileStream stream - file stream from which to read data
String containing a human-readable summary of the layer data contained in the file stream starting at the stream's current position.
Parses the layer data contained in the file stream starting at the stream's current position. Provides a human-readable summary of the data found.
Represents a convolutional neural network layer. Inherits from Layer and implements IDisposable.
- public int FilterCount { get; private set; } - number of filters in the layer
- public int[] KernelDims { get; private set; } - dimensions of the layer's kernels
- public Tensor Kernels { get; private set; } - kernels parameter tensor of the layer
- public Tensor Biases { get; protected set; } - bias parameter tensor of the layer
- public Activation Activation { get; protected set; } - activation function used by the layer
- public Tensor OutputFormat { get; protected set; } - format of the tensor the layer will output - will have a leading batch dimension of 1
- public float Dropout { get; protected set; } - spatial parameter dropout rate used by the layer
using NNNCSharp.Components.Models.Layers;
public Conv(int filterCount, int[] kernelDims, Activation activation, Padding padding, float dropout = 0.0f)
: base(activation, dropout);
- int filterCount - number of filters the layer will contain
- int[] kernelDims - kernel dimensions to be used by the layer
- Activation activation - activation function to be used by the layer
- Padding padding - type of input padding to be used by the layer
- float dropout - spatial parameter dropout rate to be used by the layer
Serves as the default constructor for Conv layers. Creates a new Conv layer instance with the given parameters. Does not automatically initialize parameter tensors.
public Conv(int filterCount, int[] kernelDims, Tensor kernels, Tensor biases, Activation activation, Padding padding,
int[] paddingDims, float dropout)
: base(activation, dropout);
- int filterCount - number of filters the layer will contain
- int[] kernelDims - kernel dimensions to be used by the layer
- Tensor kernels - kernels parameter tensor to be used by the layer
- Tensor biases - bias parameter tensor to be used by the layer
- Activation activation - activation function to be used by the layer
- Padding padding - type of input padding to be used by the layer
- int[] paddingDims - dimensions of input padding applied by the layer
- float dropout - spatial parameter dropout rate to be used by the layer
Creates a new Conv layer instance with the given complete parameters. All parameter tensors must be correctly initialized prior to the constructor being used. Used to create deep-copies of Conv layers.
public Conv();
Used when loading Conv layers from .nnn files.
public override void SetUpLayer(Tensor inputFormat);
- Tensor inputFormat - general format of the input the layer will receive - should have the dimensions [batch, input dimensions] where the batch dimension can simply equal 1
Initializes the layer's parameter tensors based on the given input format. Also initializes the layer's OutputFormat property.
public override Tensor Forward(Tensor input);
- Tensor input - input to be processed by the layer - must match the input format dimensions provided during Model initialization - requires a batch dimension of 1 if processing a single input (consider using Tensor.WrapBatch())
Tensor containing the result of passing the input through the layer.
Performs a convolution between the input and the layer's kernels parameter tensor, and adds the layer's bias parameter tensor. Using the Forward and Predict functions of the Model class is recommended over directly calling Forward on layers.
public override void Dispose();
Frees all memory used by the layer, and its parameter tensors, effectively deleting the instance. Should be used whenever a Layer instance is no longer being used, such as when creating a copy and discarding the original. Failure to use may lead to memory leaks.
public override Layer Copy();
Deep-copy of the layer containing identical parameters.
Can be used to create a new (deep-copy) layer instance with identical parameters to an existing instance. Recommended to be used in conjunction with Dispose().
public override IEnumerable<Tensor> GetParameters();
IEnumerable instance containing all of the layer's parameter tensors.
Can be used to iterate through all of the parameter tensors of a Conv layer.
public void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Loads all of the layer's parameters from the given file stream starting at the stream's current position. Used when loading layers from .nnn files.
public string PrintLayer(FileStream stream);
- FileStream stream - file stream from which to read data
String containing a human-readable summary of the layer data contained in the file stream starting at the stream's current position.
Parses the layer data contained in the file stream starting at the stream's current position. Provides a human-readable summary of the data found.
Enumeration of available padding types for Conv layers. Contained inside the Conv class.
using NNNCSharp.Components.Models.Layers;
or
using static NNNCSharp.Components.Models.Layers.Conv;
- Valid - standard 'valid' padding type - equivalent to no padding
- Same - padding type which ensures the output of a convolution has the same dimensions as the input
Classes providing neural network activation functions.
Base class defining activation function functionality and signatures. Can be inherited from to create custom activation functions.
using NNNCSharp.Components.Activations;
public Activation();
Serves as the default constructor for all activation functions.
public abstract Tensor Forward(Tensor input);
- Tensor input - input tensor to apply activation function to
Tensor containing the result of applying the activation function to the input.
Abstract function for activation subclasses to implement their activation function logic.
public abstract Activation Copy();
Deep-copy of the activation function.
Can be used to create a new (deep-copy) instance of an activation function with identical parameters.
public virtual void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Loads all of the activation's parameters from the given file stream starting at the stream's current position. Used when loading activations from .nnn files. Has no effect unless the specific activation function subclass requires parameters to be loaded from the file.
public virtual string PrintActivation(FileStream stream);
- FileStream stream - file stream from which to read data
String containing a human-readable summary of the activation data contained in the file stream starting at the stream's current position.
Parses the activation data contained in the file stream starting at the stream's current position. Provides a human-readable summary of the data found. Returns an empty string unless the specific activation function subclass includes parameters.
Represents the linear activation function - equivalent to not applying any activation function. Inherits from Activation.
using NNNCSharp.Components.Activations;
public Linear();
Creates a new Linear activation function instance.
public override Tensor Forward(Tensor input);
- Tensor input - input tensor to apply the linear activation function to
Tensor containing the result of applying the linear activation function to the input.
Used to apply the linear activation function to an input while retaining autograd graph connections.
public override Activation Copy();
Deep-copy of the Linear activation function instance.
Can be used to create a new (deep-copy) instance of a Linear activation function.
public virtual void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Inherited from parent Activation class. Has no effect due to the Linear activation function not including parameters.
public virtual string PrintActivation(FileStream stream);
An empty string due to the Linear activation function not including parameters.
Inherited from parent Activation class. Returns an empty string due to the Linear activation function not including parameters.
Represents the sigmoid activation function. Inherits from Activation.
using NNNCSharp.Components.Activations;
public Sigmoid();
Creates a new Sigmoid activation function instance.
public override Tensor Forward(Tensor input);
- Tensor input - input tensor to apply the sigmoid activation function to
Tensor containing the result of applying the sigmoid activation function to the input.
Used to apply the sigmoid activation function to an input.
public override Activation Copy();
Deep-copy of the Sigmoid activation function instance.
Can be used to create a new (deep-copy) instance of a Sigmoid activation function.
public virtual void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Inherited from parent Activation class. Has no effect due to the Sigmoid activation function not including parameters.
public virtual string PrintActivation(FileStream stream);
An empty string due to the Sigmoid activation function not including parameters.
Inherited from parent Activation class. Returns an empty string due to the Sigmoid activation function not including parameters.
Represents the hyperbolic tangent activation function. Inherits from Activation.
using NNNCSharp.Components.Activations;
public Tanh();
Creates a new Tanh activation function instance.
public override Tensor Forward(Tensor input);
- Tensor input - input tensor to apply the hyperbolic tangent activation function to
Tensor containing the result of applying the hyperbolic tangent activation function to the input.
Used to apply the hyperbolic tangent activation function to an input.
public override Activation Copy();
Deep-copy of the Tanh activation function instance.
Can be used to create a new (deep-copy) instance of a Tanh activation function.
public virtual void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Inherited from parent Activation class. Has no effect due to the Tanh activation function not including parameters.
public virtual string PrintActivation(FileStream stream);
An empty string due to the Tanh activation function not including parameters.
Inherited from parent Activation class. Returns an empty string due to the Tanh activation function not including parameters.
Represents the Rectified Linear Unit activation function. Inherits from Activation.
using NNNCSharp.Components.Activations;
public ReLU();
Creates a new ReLU activation function instance.
public override Tensor Forward(Tensor input);
- Tensor input - input tensor to apply the Rectified Linear Unit activation function to
Tensor containing the result of applying the Rectified Linear Unit activation function to the input.
Used to apply the Rectified Linear Unit activation function to an input.
public override Activation Copy();
Deep-copy of the ReLU activation function instance.
Can be used to create a new (deep-copy) instance of a ReLU activation function.
public virtual void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Inherited from parent Activation class. Has no effect due to the ReLU activation function not including parameters.
public virtual string PrintActivation(FileStream stream);
An empty string due to the ReLU activation function not including parameters.
Inherited from parent Activation class. Returns an empty string due to the ReLU activation function not including parameters.
Represents the Leaky Rectified Linear Unit activation function. Inherits from Activation.
public LeakyReLU(float tau = 0.05f);
- float tau - tau (factor) parameter to be used by the Leaky Rectified Linear Unit activation function
Creates a new LeakyReLU instance with its tau parameter initialized.
public LeakyReLU();
Used when loading LeakyReLU activation functions from .nnn files.
public override Tensor Forward(Tensor input);
- Tensor input - input tensor to apply the Leaky Rectified Linear Unit activation function to
Tensor containing the result of applying the Leaky Rectified Linear Unit activation function to the input.
Used to apply the Leaky Rectified Linear Unit activation function to an input.
public override Activation Copy();
Deep-copy of the LeakyReLU activation function instance.
Can be used to create a new (deep-copy) instance of a LeakyReLU activation function.
public override void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Loads all of the activation's parameters from the given file stream starting at the stream's current position. Used when loading activations from .nnn files.
public override string PrintActivation(FileStream stream);
- FileStream stream - file stream from which to read data
String containing a human-readable summary of the activation data contained in the file stream starting at the stream's current position.
Parses the activation data contained in the file stream starting at the stream's current position. Provides a human-readable summary of the data found.
Represents the softmax activation function. Inherits from Activation.
using NNNCSharp.Components.Activations;
public Softmax();
Creates a new Softmax activation function instance.
public override Tensor Forward(Tensor input);
- Tensor input - input tensor to apply the softmax activation function to
Tensor containing the result of applying the softmax activation function to the input.
Used to apply the softmax activation function to an input.
public override Activation Copy();
Deep-copy of the Softmax activation function instance.
Can be used to create a new (deep-copy) instance of a Softmax activation function.
public virtual void BuildFromData(FileStream stream);
- FileStream stream - file stream from which to read data
Inherited from parent Activation class. Has no effect due to the Softmax activation function not including parameters.
public virtual string PrintActivation(FileStream stream);
An empty string due to the Softmax activation function not including parameters.
Inherited from parent Activation class. Returns an empty string due to the Softmax activation function not including parameters.
Tensor data structure and functionality, and autograd engine functionality.
Class representing a tensor data structure and implementing all of the related functionality. Wraps a native C++ tensor instance. Implements IDisposable.
using NNNCSharp.Components.Autodiff;
- public int Rank { get; } - number of dimensions of the tensor
- public Span Dimensions { get; } - span over the dimensions array of the tensor
- public Span Strides { get; } - span over the spatial strides array of the tensor
- public int ElementCount { get; } - total number of elements contained in the tensor
- public int GradCount { get; } - total number of gradient values contained in the tensor
- public Span Data { get; } - underlying linear data array of the tensor (row-major)
- public Span Grad { get; } - underlying linear array of gradient values of the tensor (row-major)
- public bool RequiresGrad { get; set; } - whether the tensor must be included in gradient caculations
- public static bool LogDebug { set; } - whether the native C++ tensor implementation should write logs to the console - reserved for debugging purposes
- public static bool Inference { get; set; } - whether the native C++ autograd engine is in inference mode - skips autograd graph construction when true
public Tensor(int[] dims, bool requiresGrad = false);
- int[] dims - array representing the dimensions the new tensor will have
- bool requiresGrad - whether the new tensor must be included in gradient calculations
Creates a new zero-initialized C++ tensor instance and corresponding C# wrapper instance.
public Tensor();
Creates a new completely empty C++ tensor instance and corresponding C# wrapper instance. Used to allocate lightweight default values. Should not be used in any operations or to store data.
public Tensor(float value, int[] dims, bool requiresGrad = false);
- float value - scalar value to initialize all elements in the tensor to
- int[] dims - array representing the dimensions the new tensor will have
- bool requiresGrad - whether the new tensor must be included in gradient calculations
Creates a new C++ tensor instance with all elements initialized to the given value, along with a corresponding C# wrapper instance.
public static Tensor Scalar(float value, int[] dims, bool requiresGrad = false);
- float value - scalar value to initialize all elements in the tensor to
- int[] dims - array representing the dimensions the new tensor will have
- bool requiresGrad - whether the new tensor must be included in gradient calculations
New C# wrapper around a newly created C++ tensor instance with all elements initialized to the given scalar value.
Functionally equivalent to the scalar constructor.
public static Tensor InitWeights(int inputCount, int neuronCount);
- int inputCount - number of inputs the weights will be applied to (excluding batches)
- int neuronCount - number of neurons represented by the weights
New C# wrapper around a newly created C++ tensor instance representing the weights parameter tensor of a fully-connected neural network layer.
Initializes a new weights parameter tensor for a fully-connected neural network layer using He-Initialization.
public static Tensor InitBiases(int neuronCount);
- int neuronCount - number of bias values to include
New C# wrapper around a newly created C++ tensor instance representing the bias parameter tensor of a neural network layer.
Initializes a new bias parameter tensor for a neural network layer. All bias values are initialized to 0.01.
public void Dispose();
Releases the native C++ memory used by the C# wrapper instance. The native memory will continue to be kept alive as long as required by the native C++ functionality. Should be used whenever a tensor instance is no longer being used by the C# code. Failure to use will likely lead to excessive memory usage and/or memory leaks.
public Tensor Copy();
Deep-copy of the C# wrapper instance and underlying native C++ tensor instance with identical data.
Can be used to create a new (deep-copy) native C++ tensor instance and corresponding C# wrapper instance with data identical to an existing instance.
public float this[params int[] indices] { get; set; }
- params int[] indices - spatial indices to index at
Value at the given spatial indices in the tensor.
Can be used to get and/or set the value at the given spatial indices in a tensor.
public float this[int index] { get; set; }
- int index - linear index (row-major) to index at
Value at the given linear index (row-major) in the tensor.
Can be used to get and/or set the value at the given linear index (row-major) in a tensor.
public int LinearIndex(params int[] indices);
- params int[] indices - spatial indices to convert into a linear index
Linear index (row-major) corresponding to the given spatial indices in the tensor.
Can be used to convert spatial indices into a linear index (row-major) in a tensor.
public int[] GetFullIndices(int index);
public void GetFullIndices(int index, Span<int> indices);
- int index - linear index (row-major) to convert to spatial indices
- Span indices - span to write spatial indices into
Spatial indices array corresponding to the given linear index in the tensor (if no span to write to is provided).
Can be used to convert a linear index (row-major) to spatial indices in a tensor, either returned as a new array or written into a provided span.
public void ClearGraph();
Clears all autograd graph connections held by the tensor instance.
public void Backward();
Computes the gradients of all tensors in the current autograd graph, starting at the tensor on which Backward is called.
public static Tensor operator +(Tensor a, Tensor b);
public static Tensor operator +(Tensor a, float b);
public static Tensor operator +(float a, Tensor b);
- Tensor/float a - first tensor/scalar to add
- Tensor/float b - second tensor/scalar to add
Tensor containing the result of adding the two arguments.
Either computes the element-wise sum of two tensors or adds a scalar value to all elements in a tensor.
public static Tensor operator -(Tensor a, Tensor b);
public static Tensor operator -(Tensor a, float b);
public static Tensor operator -(float a, Tensor b);
- Tensor/float a - first tensor/scalar to subtract from
- Tensor/float b - second tensor/scalar to subtract
Tensor containing the result of subtracting the second argument from the first argument.
Either computes the element-wise difference of two tensors, subtracts a scalar value from every element in a tensor, or subtracts every element in a tensor from a scalar value.
public static Tensor operator *(Tensor a, Tensor b);
public static Tensor operator *(Tensor a, float b);
public static Tensor operator *(float a, Tensor b);
- Tensor/float a - first tensor/scalar to multiply
- Tensor/float b - second tensor/scalar to multiply
Tensor containg the result of multiplying the two arguments.
Either computes the element-wise product of two tensors or multiplies all elements in a tensor by a scalar value.
public static Tensor operator /(Tensor a, Tensor b);
public static Tensor operator /(Tensor a, float b);
public static Tensor operator /(float a, Tensor b);
- Tensor/float a - first argument to divide
- Tensor/float b - second argument to divide by
Tensor containing the result of dividing the first argument by the second argument.
Either computes the element-wise quotient of two tensors, divides every element in a tensor by a scalar value, or divides a scalar value by every element in a tensor.
public static Tensor Pow(Tensor a, Tensor exp);
public static Tensor Pow(Tensor a, float exp);
public static Tensor Pow(float a, Tensor exp);
- Tensor/float a - base tensor/scalar to exponentiate
- Tensor/float exp - exponent tensor/scalar to raise to
Tensor containing the result of raising the base 'a' to the given exponent 'exp'.
Either computes the element-wise power of a base 'a' tensor raised to an exponent 'exp' tensor, raises every element in a base 'a' tensor to an exponent 'exp' scalar value, or raises a base 'a' scalar value to the power of every element in an exponent 'exp' tensor.
public static Tensor Exp(Tensor t);
- Tensor t - tensor to raise to the power of
Tensor containing the result of raising 'e' to the power of the given tensor.
Raises 'e' to the power of every element in a tensor.
public static Tensor Log(Tensor arg, Tensor logBase);
public static Tensor Log(Tensor arg, float logBase);
public static Tensor Log(float arg, Tensor logBase);
- Tensor/float arg - tensor/scalar to use as the argument of the logarithm
- Tensor/float logBase - tensor/scalar to use as the base of the logarithm
Tensor containing the result of the logarithm with base 'logBase' of the argument 'arg'.
Computes either the element-wise logarithm with base tensor 'logBase' and argument tensor 'arg', or the logarithm with base scalar 'logBase' and argument tensor 'arg', or the logarithm with base tensor 'logBase' and argument scalar 'arg'.
public static Tensor Ln(Tensor t);
- Tensor t - tensor to compute natural logarithm of
Tensor containing the result of the natural logarithm of the given tensor.
Computes the element-wise natural logarithm of a tensor.
public static Tensor operator ^(Tensor a, Tensor b);
- Tensor a - first argument to matrix multiply
- Tensor b - second argument to matrix multiply
Tensor containing the result of the matrix multiplication of the first (left) and second (right) tensors.
Computes the matrix multiplication product of two tensors. Uses the first argument as the left tensor and the second argument as the right tensor. Does not have higher precedence than element-wise tensor operators.
public static Tensor Convolve(Tensor input, Tensor kernels);
- Tensor input - input tensor to convolve
- Tensor kernels - kernels tensor to convolve with
Tensor containing the result of the convolution of the given input tensor with the given kernels tensor.
Computes the convolution of an input tensor with a kernels tensor.
public static int ArgMax(Tensor t);
- Tensor t - tensor to compute argument maximum of
Linear index (row-major) of the highest value element in the given tensor.
Finds the linear index (row-major) of the highest value in a tensor.
public static Tensor Sum(Tensor t);
- Tensor t - tensor to compute sum of
Tensor containing a single element representing the sum of all elements in the given tensor.
Computes the sum of all elements in a tensor while retaining autograd graph connections.
public static Tensor Mean(Tensor t);
- Tensor t - tensor to compute mean of
Tensor containing a single element representing the mean of all elements in the given tensor.
Computes the mean of all elements in a tensor while retaining autograd graph connections.
public static Tensor Transpose(Tensor t);
public static Tensor Transpose(Tensor t, int[] axes);
- Tensor t - tensor to transpose
- int[] axes - specifies the permutation order to transpose with - defaults to reversing all axes
Tensor representing the transpose of the given tensor using the given permutation order.
Transposes a tensor based on a specific permutation order, or defaulting to reversing all axes, while retaining autograd graph connections.
public static Tensor Broadcast(Tensor t, int[] targetDims);
- Tensor t - tensor to broadcast
- int[] targetDims - dimensions to broadcast to - must end with the dimensions of the input tensor 't'
Tensor representing the broadcast of the given tensor to the given target dimensions.
Broadcasts a tensor to a new set of dimensions while retaining autograd graph connections.
public static Tensor Reshape(Tensor t, int[] newDims);
- Tensor t - tensor to reshape
- int[] newDims - dimensions to reshape to
Tensor representing the given tensor reshaped to the given new dimensions.
Reshapes a tensor to a new set of dimensions, without modifying the linear ordering of the underlying data, while retaining autograd graph connections.
public static Tensor Flatten(Tensor t, int startAxis = 0);
- Tensor t - tensor to flatten
- int startAxis - axis to flatten from
Tensor representing the given tensor flattened from the given axis.
Flattens a tensor starting at a specific axis, without modifying the linear ordering of the underlying data, while retaining autograd graph connections.
public static Tensor WrapBatch(Tensor t);
- Tensor t - tensor to wrap
Tensor representing the given tensor wrapped as a new batch.
Wraps a tensor into a new batch by adding a leading dimension with a length of 1 to represent the batch dimension.
public static Tensor MaskActions(Tensor qValues, int[] actions);
public static Tensor MaskActions(Tensor qValues, List<Experience> batch);
- Tensor qValues - Q-Values to mask
- int[] actions - action indices to mask by
- List<Experience> batch - list of experiences corresponding to the given Q-Values - masks by the action indices of the corresponding experiences
Tensor containing the given Q-Values masked based on the given action indices with all other Q-Values being replaced with 0's.
Masks a set of Q-Values based on corresponding action indices or experiences while retaining autograd graph connections.
public static Tensor Clip(Tensor t, float min, float max);
- Tensor t - tensor to clip values of
- float min - minimum value to clip to
- float max - maximum value to clip to
Tensor containing the elements of the given tensor clipped to the specified range.
Clips all elements of a tensor to a specified range while retaining autograd graph connections.
public static Tensor GetDenseDropoutMask(Tensor t, float dropout);
- Tensor t - tensor to which the mask will be applied
- float dropout - dropout rate to use
Tensor representing the standard dropout mask for the given tensor.
Generates a standard dropout mask for a tensor using a specific dropout rate. Does not apply the dropout mask to the given tensor.
public static Tensor GetSpatialDropoutMask(Tensor t, float dropout);
- Tensor t - tensor to which the mask will be applied
- float dropout - spatial dropout rate to use
Tensor representing the spatial dropout mask for the given tensor.
Generates a spatial dropout mask for a tensor using a specific spatial dropout rate. Does not apply the dropout mask to the given tensor.
public static Tensor Linear(Tensor t);
- Tensor t - tensor to apply the linear activation function to
Tensor containing the result of applying the linear activation function to the given tensor.
Applies the linear activation function to a tensor while retaining autograd graph connections.
public static Tensor Sigmoid(Tensor t);
- Tensor t - tensor to apply the sigmoid activation function to
Tensor containing the result of applying the sigmoid activation function to the given tensor.
Applies the sigmoid activation function to a tensor while retaining autograd graph connections.
public static Tensor Tanh(Tensor t);
- Tensor t - tensor to apply the hyperbolic tangent activation function to
Tensor containing the result of applying the hyperbolic tangent activation function to the given tensor.
Applies the hyperbolic tangent activation function to a tensor while retaining autograd graph connections.
public static Tensor ReLU(Tensor t);
- Tensor t - tensor to apply the Rectified Linear Unit activation function to
Tensor containing the result of applying the Rectified Linear Unit activation function to the given tensor.
Applies the Rectified Linear Unit activation function to a tensor while retaining autograd graph connections.
public static Tensor LeakyReLU(Tensor t, float tau);
- Tensor t - tensor to apply the Leaky Rectified Linear Unit activation function to
- float tau - tau coefficient to use
Tensor containing the result of applying the Leaky Rectified Linear Unit activation function to the given tensor.
Applies the Leaky Rectified Linear Unit activation function to a tensor while retaining autograd graph connections.
public static Tensor Softmax(Tensor t);
- Tensor t - tensor to apply the softmax activation function to
Tensor containing the result of applying the softmax activation function to the given tensor.
Applies the softmax activation function to a tensor while retaining autograd graph connections.
public static Tensor MSE(Tensor t, Tensor target);
- Tensor t - tensor to apply the Mean Squared Error cost function to
- Tensor target - tensor to use as target for Mean Squared Error cost
Tensor containing the result applying the Mean Squared Error cost function to the given tensor based on the given target tensor.
Applies the Mean Squared Error cost function to a tensor based on a specific target tensor while retaining autograd graph connections.
public static Tensor Huber(Tensor t, Tensor target, float delta);
- Tensor t - tensor to apply the pseudo-Huber cost function to
- Tensor target - tensor to use as target for pseudo-Huber cost
- float delta - delta value to use
Tensor containing the result applying the pseudo-Huber cost function to the given tensor based on the given target tensor.
Applies the pseudo-Huber cost function to a tensor based on a specific target tensor while retaining autograd graph connections.
public static Tensor SoftmaxCrossEntropy(Tensor t, Tensor target);
- Tensor t - tensor to apply the Softmax Cross-Entropy cost function to
- Tensor target - tensor to use as target for Softmax Cross-Entropy cost
Tensor containing the result applying the Softmax Cross-Entropy cost function to the given tensor based on the given target tensor.
Applies the Softmax Cross-Entropy cost function to a tensor based on a specific target tensor while retaining autograd graph connections.
public static bool DimensionsMatch(Span<int> a, Span<int> b);
- Span a - span over first dimensions array to compare
- Span b - span over second dimensions array to compare
Whether the two given dimensions spans are exactly identical.
Determines whether two dimensions spans are exactly identical.
Classes representing reinforcement learning (RL) training environments.
Base class defining Deep Q-Learning (DQN) training environment functionality and sigantures. Can be inherited from to create custom DQN training environments.
using NNNCSharp.Components.DQNEnvironments;
- public int StateSize { get; } - total number of elements in the environment's state representation
- public Tensor StateFormat { get; } - tensor representing the format of the environment's state tensor - will have a leading dimension of 1 to represent the batch dimension
- public int ActionCount { get; } - number of discrete actions which can be taken in the environment
- public string EnvironmentName { get; } - display name of the environment
public DQNEnvironment();
Serves as the default constructor for all DQN environments.
public abstract Tensor GetState();
Tensor containing the current state of the environment.
Abstract function for DQN environment subclasses to return their current state.
public abstract Tensor GetNormalizedState();
Tensor containing the current normalized state of the environment.
Abstract function for DQN environment subclasses to return their current state normalized for processing by neural network DQN agents.
public abstract void Reset();
Abstract function for DQN environment subclasses to implement logic for resetting their internal state.
public abstract (float reward, Tensor nextState, bool done) Step(int action, int steps);
- int action - index of the action to take
- int steps - number of steps which have been taken in the current episode
Reward incrued by the given action, Tensor containing the normalized state after the action was taken, and whether the current DQn training episode has ended.
Abstract function for DQN environment subclasses to implement their step logic.
public abstract bool ValidAction(int action, Tensor? state = null);
- int action - index of the action to check validity of
- Tensor? state - optional state in which to verify action validity - defaults to environment's current state
Whether the given action is valid in the given state.
Abstract function for DQN environment subclasses to implement their action validation logic.
public abstract int PickAgentAction(Tensor qValues, Tensor? state = null);
- Tensor qValues - Q-Values predicted by the agent
- Tensor? state - optional state for which to select agent action - defaults to environment's current state
Index of the valid action with the highest Q-Value.
Abstract function for DQN environment subclasses to implement their agent action selection logic.
public abstract int PickRandomAction();
Index of a random valid action for the environment's current state.
Abstract function for DQN environment subclasses to implement their random action selection logic.
public abstract float TestTrainingProgress(Model agent, int testEpisodes);
- Model agent - agent to test performance of
- int testEpisodes - number of test episodes to run
Average performance of the agent across all test episodes.
Abstract function for DQN environment subclasses to implement their performance evaluation logic.
public virtual void Render(Episode episode, int step);
- Episode episode - episode from which to render state
- int step - index of the step to render from the given episode
Can be used to render steps in a past episode generated by the DQN environment. Allows DQN environment subclasses with simple to represent states to render past states in the logging output. Does nothing unless overridden.
public virtual void PlayDemo();
Plays the DQN environment's dedicated demonstration. Allows DQN environment subclasses to implement fully encapsulated demonstrations. Not required for training functionality. Throws a NotImplementedException unless overridden.
Interface for Deep Q-Learning (DQN) training environments which require the agent to play against itself during training.
using NNNCSharp.Components.DQNEnvironments;
- public bool Won { get; set; } - whether the most recent step resulted in a win
- public bool AgentTurn { get; set; } - whether it is the training agent's turn to act
- public int OpponentCount { get; set; } - number of opponent agents currently stored
- public int OpponentIndex { get; set; } - index of the currently selected opponent agent - values >= OpponentCount indicate a randomly-acting opponent
- public Random Random { get; init; } - Random instance used by the environment instance
public int GetAgentAction(Model agent, Tensor? state = null);
- Model agent - agent to select action with
- Tensor? state - optional state for which to select action - defaults to environment's current state
Index of the valid action with the highest Q-Value predicted by the agent.
Interface function which all self-play DQN environments are required to implement.
public int PickAgentAction(Tensor qValues, Tensor? state = null);
- Tensor qValues - Q-Values predicted by the agent
- Tensor? state - optional state for which to select action - defaults to environment's current state
Index of the valid action with the highest Q-Value.
Interface function which all self-play DQN environments are required to implement. Also part of the standard DQNEnvironment abstract class.
public int PickRandomAction();
Index of a random valid action for the environment's current state.
Interface function which all self-play DQN environments are required to implement. Also part of the standard DQNEnvironment abstract class.
Class representing a 2D movement grid Deep Q-Learning (DQN) environment. Inherits from DQNEnvironment.
using NNNCSharp.Components.DQNEnvironments;
- public int StateSize { get; } - total number of elements in the environment's state representation
- public Tensor StateFormat { get; } - tensor representing the format of the environment's state tensor - will have a leading dimension of 1 to represent the batch dimension - includes 4 inputs: agent x-position, agent y-position, target x-position, target y-position
- public int ActionCount { get; } - number of discrete actions which can be taken in the environment - includes 4 actions: left, up, right, down
- public string EnvironmentName { get; } - display name of the environment
public MovementGrid2D(int xMin, int xMax, int yMin, int yMax, int maxSteps = 50);
- int xMin - lowest X-coordinate for the grid to have
- int xMax - highest X-coordinate for the grid to have
- int yMin - lowest Y-coordinate for the grid to have
- int yMax - highest Y-coordinate for the grid to have
- int maxSteps - maximum number of steps the agent will be allowed to take before timing out
Creates a new MovementGrid2D DQN environment instance and prepares it for the first training episode.
public override Tensor GetState();
Tensor containing the current state of the environment.
Can be used to get the current state of the movement grid DQN environment.
public override Tensor GetNormalizedState();
Tensor containing the current normalized state of the environment.
Can be used to get the current state of the movement grid DQN environment normalized based on grid size for processing by neural network agents.
public override void Reset();
Resets the movement grid DQN environment and initializes random agent and target positions.
public override (float reward, Tensor nextState, bool done) Step(int action, int steps);
- int action - index of the action to take
- int steps - number of steps which have been taken in the current episode
Reward incrued by the given action, Tensor containing the normalized state after the action was taken, and whether the current DQn training episode has ended.
Moves the agent's x- and y-position coordinates based on the given action.
public override bool ValidAction(int action, Tensor? state = null);
- int action - index of the action to check validity of
- Tensor? state - optional state in which to verify action validity - defaults to environment's current state
Whether the given action is valid in the given state.
Always returns true due to the movement grid DQN environment action space never having invalid actions.
public override int PickAgentAction(Tensor qValues, Tensor? state = null);
- Tensor qValues - Q-Values predicted by the agent
- Tensor? state - optional state for which to select agent action - defaults to environment's current state
Index of the valid action with the highest Q-Value.
Always returns the action index with the highest Q-Value due to the movement grid DQN environment action space never having invalid actions.
public override int PickRandomAction();
Index of a random valid action for the environment's current state.
Returns a fully random action due to the movement grid DQN environment action space never having invalid actions.
public override float TestTrainingProgress(Model agent, int testEpisodes);
- Model agent - agent to test performance of
- int testEpisodes - number of test episodes to run
Average performance of the agent across all test episodes.
Measures the percentage of episodes in which the agent successfully reaches the target over the course of the given number of randomly initialized test episodes.
public override void Render(Episode episode, int step);
- Episode episode - episode from which to render state
- int step - index of the step to render from the given episode
Can be used to render steps in a past episode generated by the DQN environment.
public override void PlayDemo();
Plays the DQN environment's dedicated demonstration.
public bool Run(Model agent);
- Model agent - agent to use
Whether the agent successfully reached the target before timing out.
Runs a single episode with the given agent.
Class representing a Tic-Tac-Toe Deep Q-Learning (DQN) training environment. Inherits from DQNEnvironment and implements ISelfPlay.
using NNNCSharp.Components.DQNEnvironments;
- public int StateSize { get; } - total number of elements in the environment's state representation
- public Tensor StateFormat { get; } - tensor representing the format of the environment's state tensor - will have a leading dimension of 1 to represent the batch dimension - 9 board positions ordered linearly (row-major), whether X or O is to move
- public int ActionCount { get; } - number of discrete actions which can be taken in the environment - 1 for each board position
- public string EnvironmentName { get; } - display name of the environment
- public bool Won { get; set; } - whether the most recent step resulted in a win
- public bool AgentTurn { get; set; } - whether it is the training agent's turn to act
- public int OpponentCount { get; set; } - number of opponent agents currently stored
- public int OpponentIndex { get; set; } - index of the currently selected opponent agent - values >= OpponentCount indicate a randomly-acting opponent
- public Random Random { get; init; } - Rnadom instance used by the environment instance
public TicTacToe();
Creates a new TicTacToe DQN environment instance.
public override Tensor GetState();
Tensor containing the current state of the environment.
Can be used to get the current state of the TicTacToe DQN environment.
public override Tensor GetNormalizedState();
Tensor containing the current normalized state of the environment.
Can be used to get the current state of the TicTacToe DQN environment normalized for processing by neural network agents.
public override void Reset();
Resets the TicTacToe DQN environment and prepares a new game.
public override (float reward, Tensor nextState, bool done) Step(int action, int steps);
- int action - index of the action to take
- int steps - number of steps which have been taken in the current episode
Reward incrued by the given action, Tensor containing the normalized state after the action was taken, and whether the current DQn training episode has ended.
Places the corresponding X/O in the position indicated by the action.
public override bool ValidAction(int action, Tensor? state = null);
- int action - index of the action to check validity of
- Tensor? state - optional state in which to verify action validity - defaults to environment's current state
Whether the given action is valid in the given state.
Checks whether the position corresponding to the action is empty.
public int GetAgentAction(Model agent, Tensor? state = null);
- Model agent - agent to select action with
- Tensor? state - optional state for which to select action - defaults to environment's current state
Index of the valid action with the highest Q-Value predicted by the agent.
Uses the agent to select the highest Q-Value action index corresponding to an empty position.
public override int PickAgentAction(Tensor qValues, Tensor? state = null);
- Tensor qValues - Q-Values predicted by the agent
- Tensor? state - optional state for which to select agent action - defaults to environment's current state
Index of the valid action with the highest Q-Value.
Returns the highest Q-Value action index corresponding to an empty position.
public override int PickRandomAction();
Index of a random valid action for the environment's current state.
Returns the idnex of a random action corresponding to an empty position.
public override float TestTrainingProgress(Model agent, int testEpisodes);
- Model agent - agent to test performance of
- int testEpisodes - number of test episodes to run
Average performance of the agent across all test episodes.
Measures the percentage of episodes in which the agent either won or tied against a randomly acting opponent over the course of the given number of test episodes.
public override void Render(Episode episode, int step);
- Episode episode - episode from which to render state
- int step - index of the step to render from the given episode
Can be used to render steps in a past episode generated by the DQN environment.
public override void PlayDemo();
Plays the DQN environment's dedicated demonstration.
public void Play(Model agent);
- Model agent - agent to play with
Plays a game of Tic-Tac-Toe between the user and the given agent.
Class representing a Snake (game) Deep Q-Learning (DQN) training environment. Inherits from DQNEnvironment.
using NNNCSharp.Components.DQNEnvironments;
- public int StateSize { get; } - total number of elements in the environment's state representation
- public Tensor StateFormat { get; } - tensor representing the format of the environment's state tensor - will have a leading dimension of 1 to represent the batch dimension - 2D image representing the game board with 8 channels for one-hot encoding snake pieces / apples and the direction each element will move
- public int ActionCount { get; } - number of discrete actions which can be taken in the environment - left, forward, right
- public string EnvironmentName { get; } - display name of the environment
public Snake(int width = 20, int height = 20);
- int width - width of the game board
- int height - height of the game board
Creates a new Snake DQN environment instance.
public override Tensor GetState();
Tensor containing the current state of the environment.
Can be used to get the current state of the snake DQN environment.
public override Tensor GetNormalizedState();
Tensor containing the current normalized state of the environment.
Can be used to get the current state of the snake DQN environment normalized for processing by neural network agents.
public override void Reset();
Resets the snake DQN environment and initializes the snake and first apple.
public override (float reward, Tensor nextState, bool done) Step(int action, int steps);
- int action - index of the action to take
- int steps - number of steps which have been taken in the current episode
Reward incrued by the given action, Tensor containing the normalized state after the action was taken, and whether the current DQn training episode has ended.
Moves the snake based on the given action and handles necessary game logic.
public override bool ValidAction(int action, Tensor? state = null);
- int action - index of the action to check validity of
- Tensor? state - optional state in which to verify action validity - defaults to environment's current state
Whether the given action is valid in the given state.
Always returns true due to the snake DQN environment action space never having invalid actions.
public override int PickAgentAction(Tensor qValues, Tensor? state = null);
- Tensor qValues - Q-Values predicted by the agent
- Tensor? state - optional state for which to select agent action - defaults to environment's current state
Index of the valid action with the highest Q-Value.
Always returns the action index with the highest Q-Value due to the snake DQN environment action space never having invalid actions.
public override int PickRandomAction();
Index of a random valid action for the environment's current state.
Returns a fully random action due to the snake DQN environment action space never having invalid actions.
public override float TestTrainingProgress(Model agent, int testEpisodes);
- Model agent - agent to test performance of
- int testEpisodes - number of test episodes to run
Average performance of the agent across all test episodes.
Measures the average snake length reached by the agent across the given number of test episodes.
public override void Render(Episode episode, int step);
- Episode episode - episode from which to render state
- int step - index of the step to render from the given episode
Can be used to render steps in a past episode generated by the DQN environment.
public override void PlayDemo();
Plays the DQN environment's dedicated demonstration.
public void Play(Model agent);
- Model agent - agent to have play
Plays a game of Snake using the given agent.
Data structures for storing and managing training data.
Data structure for storing and batching standard supervised training input-target pairs. Automatically handles releasing of native C++ memory used by the tensors in the dataset.
using NNNCSharp.Components.Buffers;
public ReplayBuffer(Tensor[] data, Tensor[] targets);
- Tensor[] data - array of all input tensors in the training dataset
- Tensor[] targets - array of all target tensors in the training dataset - must be in the same order as the data array
Creates a new BatchBuffer instance containing the given dataset of input and target tensors.
public (Tensor batchInputs, Tensor batchTargets) GetBatch(int batchSize);
- int batchSize - number of input-target pairs to include in the batch - must be > 0 and <= total number of input-target pairs
Tensors containing the batched inputs and corresponding batched targets.
Batches inputs and targets from the dataset by randomly sampling input-target pairs without any repeats.
public (Tensor[] batchInputs, Tensor[] batchTargets) GetBatches(int batchSize);
- int batchSize - number of input-target pairs to include in each batch
Tensor arrays containing batched tensors of inputs and the corresponding targets.
Randomly samples all of the input-target pairs in the dataset into training batches of the given size. Will likely include a tail batch with a smaller size if dataset size is not perfectly divisible by the given batch size.
Data structure implementing standard First-In First-Out functionality. Automatically disposes stored elements when applicable.
using NNNCSharp.Components.Buffers;
- public int MaxSize { get; init; } - maximum number of elements the buffer can hold
- public int Count { get; } - number of elements currently stored in the buffer
public FIFOBuffer(int maxSize);
- int maxSize - maximum number of elements the buffer will be able to hold
Creates a new FIFOBuffer instance with the given maximum size.
public void Add(T item);
- T item - new item to add to the buffer
Adds the given item to the buffer using standard First-In First-Out rules.
public T this[int index] { get; }
- int index - index to index at
Element at the given index in the buffer.
Can be used to access the element at a specific index in the buffer, with index 0 always corresponding to the oldest element.
Data structure implementing Deep Q-Learning (DQN) replay buffer functionality with Prioritized Experience Replay (PER) sampling using an underlying sum tree. Automatically releases native C++ memory used by experience instances when applicable.
using NNNCSharp.Components.Buffers;
- public int Count { get; } - number of experiences currently stored in the buffer
public ReplayBuffer(int capacity, double alpha = 0.6);
- int capacity - maximum number of experiences the buffer will be able to hold
- double alpha - alpha value to use during PER sampling
Creates a new ReplayBuffer instance with the given capacity.
public void Add(Experience experience);
- Experience experience - experience to add to the buffer
Adds a new experience to the buffer using standard First-In First-Out rules and updates the underlying PER sampling sum tree.
public (List<Experience> batch, int[] indices, double[] weights) GetBatch(int batchSize);
- int batchSize - number of experiences to include in the batch
Experience array containing the sampled experiences, array of corresponding indices of the sampled experiences in the buffer, and array of the corresponding PER sampling weights.
Samples a batch with the given number of experiences using Prioritized Experience Replay (PER) sampling.
public void UpdatePriorities(int[] indices, double[] priorities);
- int[] indices - indices of the experiences to update priorities of
- double[] priorities - new priorities to set for the experiences at the given indices
Used to update Prioritized Experience Replay (PER) sampling priorities of experiences following a training step.
Data structure implementing standard sum tree functionality for Deep Q-Learning (DQN) Prioritized Experience Replay (PER) sampling. Automatically disposes stored elements when applicable.
using NNNCSharp.Components.Buffers;
- public int Count { get; private set; } - number of elements currently stored in the sum tree
- public double TotalPriority { get; } - total priority of all elements currently stored in the sum tree
public SumTree(int capacity);
- int capacity - maximum number of elements the sum tree will be able to hold
Creates a new SumTree instance with the given capacity.
public void Add(T item, double priority);
- T item - item to add to the sum tree
- double priority - priority to assign to the given item
Adds a new item to the sum tree using standard First-In First-Out rules and assigns it the given priority.
public void Update(int treeIndex, double priority);
- int treeIndex - index of the item to update the priority of
- double priority - value to update the priority to
Assigns a new priority to an item in the sum tree and updates the other nodes in the tree correspondingly.
public (int treeIndex, double priority, T item) Get(double value);
- double value - value to use for PER sampling
Index, priority, and item sampled using PER sampling.
Samples an item from the sum tree using Prioritized Experience Replay (PER) sampling with the given sampling value.
Data structure storing a single Deep Q-Learning (DQN) training episode. Implements IDisposable.
using NNNCSharp.Components.Episodes;
- public List Experiences { get; init; }
public Episode(List<Experience> experiences);
- List<Experience> experiences - experiences encompassed by the episode
Creates a new Episode instance wrapping the given set of experiences.
public void Dispose();
Releases all native C++ memory used by the Episode instance and the experiences contained in it.
Data structure representing a single Deep Q-Learning (DQN) training experience. Implements IDisposable.
using NNNCSharp.Components.Episodes;
- public Tensor State { get; init; } - environment state at the beginning of the experience
- public int Action { get; init; } - action which the agent took
- public float Reward { get; init; } - reward accrued by the agent
- public Tensor NextState { get; init; } - environment state following the action being taken
- public bool Done { get; init; } - whether the experience terminated the training episode
- public float Priority { get; set; } - PER sampling priority of the experience
public Experience(Tensor state, int action, float reward, Tensor nextState, bool done, float priority = 1.0f);
- Tensor state - environment state at the beginning of the experience
- int action - action which the agent took
- float reward - reward accrued by the agent
- Tensor nextState - environment state following the action being taken
- bool done - whether the experience terminated the training episode
- float priority - PER sampling priority of the experience
Creates a new Experience instance with the given data.
public void Dispose();
Releases all native C++ memory used by the instance and the contained state tensors.
Classes providing functionality for reading and writing .nnn file. Refer to the File Format Documentation for the specifications of the .nnn file format used by Neural Network Notions.
Provides functionality for saving and loading neural network models to and from .nnn files.
using NNNCSharp.Components.Utilities.SaveSystem;
- public static string DirectoryPath { get; set; } - full directory path to save and load from
- public const int MagicNumber - identifiying number used by .nnn files
public static void SaveModel(Model model, string fileName, string desc = "");
- Model model - neural network model to save (without file extension)
- string fileName - file name to save model to
- string desc - optional description to include in the file
Saves a neural network model to a file using the .nnn file format.
public static Model LoadModel(string fileName);
- string fileName - name of the file to load neural network model from (without file extension)
Model contained in the given .nnn file.
Loads a neural network model from a .nnn file.
public static bool FileExists(string fileName);
- string fileName - name of the file to find (without file extension)
Whether a .nnn file with the given name was found.
Searches for a .nnn file with a given name and returns whether it was found. Only searches the directory specified by the DirectoryPath property.
public static string GetFullPath(string fileName);
- string fileName - name of the file to get the full path of (without file extension)
Full path of the .nnn file with the given name.
Generates the full path of a .nnn file with a given name. Assumes the file is in the directory specified by the DirectoryPath property.
Provides functionality for reading and writing data to and from .nnn files.
using NNNCSharp.Components.Utilities.SaveSystem;
public static void WriteBool(FileStream stream, bool data);
- FileStream stream - file stream to write to
- bool data - boolean to write
Writes the given boolean to the given file stream using .nnn encoding.
public static void WriteFloat(FileStream stream, float data);
- FileStream stream - file stream to write to
- float data - float to write
Writes the given float to the given file stream using .nnn encoding.
public static void WriteFloatArray(FileStream stream, float[] data);
- FileStream stream - file stream to write to
- float[] data - float array to write
Writes the given array of floats to the given file stream using .nnn encoding.
public static void WriteInt32(FileStream stream, int data);
- FileStream stream - file stream to write to
- int data - 32-bit integer to write
Writes the given 32-bit (signed) integer to the given file stream using .nnn encoding.
public static void WriteInt32Array(FileStream stream, int[] data);
- FileStream stream - file stream to write to
- int[] data - 32-bit integer array to write
Writes the given 32-bit (signed) integer array to the given file stream using .nnn encoding.
public static void WriteLayer(FileStream stream, Layer layer);
- FileStream stream - file stream to write to
- Layer layer - neural network layer to write
Writes the given neural network layer to the given file stream using .nnn encoding.
public static void WriteModel(FileStream stream, Model model);
- FileStream stream - file stream to write to
- Model model - neural network model to write
Writes the given neural network model to the given file stream using .nnn encoding.
public static void WriteString(FileStream stream, string data);
- FileStream stream - file stream to write to
- string data - string to write
Writes the given string to the given file stream using .nnn encoding.
public static void WriteTensor(FileStream stream, Tensor data);
- FileStream stream - file stream to write to
- Tensor data - tensor to write
Writes the given tensor to the given file stream using .nnn encoding.
public static void WriteUInt64(FileStream stream, ulong data);
- FileStream stream - file stream to write to
- ulong data - 64-bit (unsigned) integer to write
Writes the given 64-bit (unsigned) integer to the given file stream using .nnn encoding.
public static bool ReadBool(FileStream stream);
- FileStream stream - file stream to read from
Boolean read from the file stream.
Reads the boolean stored at the given file stream's current position using .nnn encoding.
public static float ReadFloat(FileStream stream);
- FileStream stream - file stream to read from
Float read from the file stream.
Reads the float stored at the given file stream's current position using .nnn encoding.
public static float[] ReadFloatArray(FileStream stream);
- FileStream stream - file stream to read from
Float array read from the file stream.
Reads the array of floats stored at the given file stream's current position using .nnn encoding.
public static int ReadInt32(FileStream stream);
- FileStream stream - file stream to read from
32-bit (signed) integer read from the file stream.
Reads the 32-bit (signed) integer stored at the given file stream's current position using .nnn encoding.
public static int[] ReadInt32Array(FileStream stream);
- FileStream stream - file stream to read from
32-bit (signed) integer array read from the file stream.
Reads the array of 32-bit (signed) integers stored at the given file stream's current position using .nnn encoding.
public static Layer ReadLayer(FileStream stream);
- FileStream stream - file stream to read from
Neural network Layer read from the file stream.
Reads the neural network layer stored at the given file stream's current position using .nnn encoding.
public static Model ReadModel(FileStream stream);
- FileStream stream - file stream to read from
Neural network Model read from the file stream.
Reads the neural network model stored at the given file stream's current position using .nnn encoding.
public static string ReadString(FileStream stream);
- FileStream stream - file stream to read from
String read from the file stream.
Reads the string stored at the given file stream's current position using .nnn encoding.
public static Tensor ReadTensor(FileStream stream);
- FileStream stream - file stream to read from
Tensor read from the file stream.
Reads the tensor stored at the given file stream's current position using .nnn encoding.
public static ulong ReadUInt64(FileStream stream);
- FileStream stream - file stream to read from
64-bit (unsigned) integer read from the file stream.
Reads the 64-bit (unsigned) integer stored at the given file stream's current position using .nnn encoding.
public static string PrintLayer(FileStream stream);
- FileStream stream - file stream to read from
Human-readable summary of the neural network layer stored in the file stream.
Parses the neural network layer stored at the given file stream's current position into a human-readable string using .nnn encoding.
public static string PrintTensor(FileStream stream);
- FileStream stream - file stream to read from
Human-readable summary of the tensor stored in the file stream.
Parses the tensor stored at the given file stream's current position into a human-readable string using .nnn encoding.