Skip to content

Update methods for creating and fitting neural networks #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 18, 2024
Merged

Update methods for creating and fitting neural networks #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5a378ec
install Eval method for parametrized morphisms
kamalsaleh Jul 16, 2024
2507cf5
no comments are printed while constructing a neural network
kamalsaleh Jul 18, 2024
40a581d
if the number of epochs is n then display the loss
kamalsaleh Jul 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion PackageInfo.g
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ SetPackageInfo( rec(

PackageName := "MachineLearningForCAP",
Subtitle := "Exploring categorical machine learning in CAP",
Version := "2024.07-15",
Version := "2024.07-17",
Date := (function ( ) if IsBound( GAPInfo.SystemEnvironment.GAP_PKG_RELEASE_DATE ) then return GAPInfo.SystemEnvironment.GAP_PKG_RELEASE_DATE; else return Concatenation( ~.Version{[ 1 .. 4 ]}, "-", ~.Version{[ 6, 7 ]}, "-01" ); fi; end)( ),
License := "GPL-2.0-or-later",

Expand Down
10 changes: 10 additions & 0 deletions gap/CategoryOfParametrisedMorphisms.gi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,16 @@ InstallMethod( ReparametriseMorphism,

end );

##
InstallOtherMethod( Eval,
[ IsMorphismInCategoryOfParametrisedMorphisms, IsDenseList ],

function( f, pair )

return Eval( UnderlyingMorphism( f ), Concatenation( pair ) );

end );

##
InstallMethod( NaturalEmbeddingIntoCategoryOfParametrisedMorphisms,
[ IsCapCategory, IsCategoryOfParametrisedMorphisms ],
Expand Down
8 changes: 5 additions & 3 deletions gap/FitParameters.gi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,20 +139,22 @@ InstallMethod( Fit,

l_n := Length( String( n ) );

for i in [ 0 .. n ] do
Print( "Epoch ", JoinStringsWithSeparator( ListWithIdenticalEntries( l_n - 1, " " ), "" ), "0/", String( n ), " - loss = ", String( get( w )[1] ), "\n" );

for i in [ 1 .. n ] do

str_i := String( i );

l_i := Length( str_i );

spaces := JoinStringsWithSeparator( ListWithIdenticalEntries( l_n - l_i, " " ), "" );

w := put( w );

loss := get( w );

Print( "Epoch ", spaces, String( i ), "/", String( n ), " - loss = ", String( loss[1] ), "\n" );

w := put( w );

#Display( w );

od;
Expand Down
6 changes: 0 additions & 6 deletions gap/NeuralNetworks.gi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@ InstallMethod( LogitsMorphismOfNeuralNetwork,

N := Length( dims );

Print( "The total number of layers is ", String( N ), "\n\n" );

L := [ ];

for i in [ 1 .. N - 1 ] do
Expand All @@ -25,8 +23,6 @@ InstallMethod( LogitsMorphismOfNeuralNetwork,

P_i := UnderlyingObject( l_i );

Print( "Creating a morphism from layer ", String( i ), " to ", String( i + 1 ), " with ", String( RankOfObject( P_i ) ), " parameters\n" );

Add( L, l_i );

if i <> N - 1 then
Expand All @@ -37,8 +33,6 @@ InstallMethod( LogitsMorphismOfNeuralNetwork,

od;

Print( "\n" );

return PreComposeList( Para, L );

end );
Expand Down
9 changes: 3 additions & 6 deletions tst/functor.tst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,10 @@ gap> Smooth := SkeletalSmoothMaps;;
gap> Lenses := CategoryOfLenses( Smooth );;
gap> Para := CategoryOfParametrisedMorphisms( Smooth );;
gap> Para_Lenses := CategoryOfParametrisedMorphisms( Lenses );;
gap> f := LossMorphismOfNeuralNetwork( Para, 2, [], 1, "IdFunc" );;
The total number of layers is 2

Creating a morphism from layer 1 to 2 with 3 parameters
gap> ell := LossMorphismOfNeuralNetwork( Para, 2, [], 1, "IdFunc" );;
gap> dummy_input := ConvertToExpressions( [ "w1", "w2", "b1", "x1", "x2", "y" ] );
[ w1, w2, b1, x1, x2, y ]
gap> Display( f : dummy_input := dummy_input );
gap> Display( ell : dummy_input := dummy_input );
ℝ^3 -> ℝ^1 defined by:

Parameter Object:
Expand All @@ -24,7 +21,7 @@ Parametrised Morphism:
‣ (w1 * x1 + w2 * x2 + b1 - y) ^ 2 / 1
gap> R := EmbeddingIntoCategoryOfParametrisedMorphisms( Para, Para_Lenses );
Embedding into category of parametrised morphisms
gap> Rf := ApplyFunctor( R, f );
gap> Rf := ApplyFunctor( R, ell );
(ℝ^3, ℝ^3) -> (ℝ^1, ℝ^1) defined by:

Parameter Object:
Expand Down
5 changes: 1 addition & 4 deletions tst/neural-network-1.tst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@ gap> Smooth := SkeletalSmoothMaps;;
gap> Lenses := CategoryOfLenses( Smooth );;
gap> Para := CategoryOfParametrisedMorphisms( Smooth );;
gap> f := LossMorphismOfNeuralNetwork( Para, 2, [], 1, "IdFunc" );;
The total number of layers is 2

Creating a morphism from layer 1 to 2 with 3 parameters
gap> optimizer := Lenses.AdamOptimizer( );;
gap> training_examples_path := SelectBasedOnCondition( IsExistingFile( "data-1.txt" ), "data-1.txt", "tst/data-1.txt" );;
gap> batch_size := 5;;
Expand Down Expand Up @@ -63,4 +60,4 @@ Epoch 47/50 - loss = 0.00092586526390837627
Epoch 48/50 - loss = 0.00091733056032563621
Epoch 49/50 - loss = 0.00091003319324738866
Epoch 50/50 - loss = 0.00090379645850528724
[ 1021, -0.00236067, -0.00633157, 0.000258869, 0.0135747, 0.0500079, 0.0310695, 2.00197, -2.99162, 0.997524 ]
[ 1001, -0.0024867, -0.00684601, 0.000293851, 0.0138455, 0.051012, 0.0316896, 2.00229, -2.99088, 0.997374 ]
5 changes: 0 additions & 5 deletions tst/neural-network-2.tst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,6 @@ gap> Smooth := SkeletalSmoothMaps;;
gap> Lenses := CategoryOfLenses( Smooth );;
gap> Para := CategoryOfParametrisedMorphisms( Smooth );;
gap> f := LossMorphismOfNeuralNetwork( Para, 2, [ 5, 5 ], 4, "Softmax" );;
The total number of layers is 4

Creating a morphism from layer 1 to 2 with 15 parameters
Creating a morphism from layer 2 to 3 with 30 parameters
Creating a morphism from layer 3 to 4 with 24 parameters
gap> optimizer := Lenses.GradientDescentOptimizer( : learning_rate := 0.01 );;
gap> training_examples_path := SelectBasedOnCondition( IsExistingFile( "data-2.txt" ), "data-2.txt", "tst/data-2.txt" );;
gap> batch_size := 1;;
Expand Down