Custom CNN with Automatic Differentiation in Julia

This project implements a custom reverse-mode automatic differentiation (AD) engine and a simple neural network library in Julia. It demonstrates the training of a Convolutional Neural Network (CNN) for sentiment analysis on the IMDb dataset using this custom stack. The project also includes benchmarks against Flux.jl and PyTorch.

Project Structure

• SimpleAutoDiff.jl: Core automatic differentiation engine.
• SimpleNN.jl: Basic neural network layers (Dense, Embedding, etc.) and MLP model structure.
• CNNLayers.jl: CNN-specific layers (Conv1DLayer, MaxPool1DLayer).
• LossFunctions.jl: Loss functions (e.g., binary cross-entropy).
• Optimizers.jl: Optimizers (e.g., Adam, SGD).
• data_prep.jl: Script to process the raw IMDb dataset into a usable format.
• train_cnn.jl: Main script to train the custom CNN and (optionally) generate detailed performance profiles.
• comprehensive_benchmark.jl: Script to benchmark the custom CNN against an equivalent Flux.jl model and provides a placeholder for PyTorch comparison.
• pytorch_model.py: PyTorch CNN model definition (used by benchmark_pytorch_cnn.py).
• benchmark_pytorch_cnn.py: Python script to benchmark the PyTorch CNN.
• save_data_for_pytorch.jl: Julia script to convert and save data into .npy format for the PyTorch benchmark.
• test_autodiff_correctness.jl: Script to run correctness tests for the SimpleAutoDiff.jl module.
• data/: Directory for datasets.
  • imdb_dataset.jld2: (Required) Raw IMDb dataset.
  • imdb_dataset_prepared.jld2: Generated by data_prep.jl.
  • X_train_pytorch.npy, y_train_pytorch.npy, X_test_pytorch.npy, y_test_pytorch.npy, embeddings_pytorch.npy: Generated by save_data_for_pytorch.jl.

Prerequisites

• Julia: Version 1.6 or later (developed with 1.11).
• Python 3: With NumPy, PyTorch, and psutil (for PyTorch benchmark).

  pip install numpy torch psutil

Julia Setup and Running the Project

All Julia scripts should be run from the project's root directory.

1. Install Julia Packages:

# In Julia REPL
using Pkg
Pkg.add(["JLD2", "TextAnalysis", "Languages", "Random", "Statistics", "Printf", "LinearAlgebra", "InteractiveUtils", "Flux", "Optimisers", "Profile", "StatProfilerHTML", "NPZ"])

To start programm
julia -t auto train_cnn.jl