lenet on mnist examples #209
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@MikeInnes @oxinabox julia> include("lenet_mnist.jl")
julia> train(loadpath="mtest.bson")
[ Info: Dataset MNIST: 60000 train and 10000 test examples
[ Info: Loaded model: 44426 trainable params
[ Info: Start Training
ERROR: MethodError: no method matching (::BSON.__deserialized_types__.var"##451"{Tuple{Int64,Int64,Int64}})(::Array{Float32,4})
The applicable method may be too new: running in world age 27011, while current world is 27013.
Closest candidates are:
#8(::Any) at /home/carlo/Git/model-zoo/vision/lenet_mnist/lenet_mnist.jl:22 (method too new to be called from this world context.)
Stacktrace:
[1] applychain(::Tuple{BSON.__deserialized_types__.var"##451"{Tuple{Int64,Int64,Int64}},Conv{2,4,typeof(relu),Array{Float32,4},Array{Float32,1}},MaxPool{2,4},Conv{2,4,typeof(relu),Array{Float32,4},Array{Float32,1}},MaxPool{2,4},BSON.__deserialized_types__.var"##452",Dense{typeof(relu),Array{Float32,2},Array{Float32,1}},Dense{typeof(relu),Array{Float32,2},Array{Float32,1}},Dense{typeof(identity),Array{Float32,2},Array{Float32,1}}}, ::Array{Float32,4}) at /home/carlo/.julia/packages/Flux/CjjeX/src/layers/basic.jl:30
[2] (::Chain{Tuple{BSON.__deserialized_types__.var"##451"{Tuple{Int64,Int64,Int64}},Conv{2,4,typeof(relu),Array{Float32,4},Array{Float32,1}},MaxPool{2,4},Conv{2,4,typeof(relu),Array{Float32,4},Array{Float32,1}},MaxPool{2,4},BSON.__deserialized_types__.var"##452",Dense{typeof(relu),Array{Float32,2},Array{Float32,1}},Dense{typeof(relu),Array{Float32,2},Array{Float32,1}},Dense{typeof(identity),Array{Float32,2},Array{Float32,1}}}})(::Array{Float32,4}) at /home/carlo/.julia/packages/Flux/CjjeX/src/layers/basic.jl:32
[3] eval_loss_accuracy(::DataLoader, ::Chain{Tuple{BSON.__deserialized_types__.var"##451"{Tuple{Int64,Int64,Int64}},Conv{2,4,typeof(relu),Array{Float32,4},Array{Float32,1}},MaxPool{2,4},Conv{2,4,typeof(relu),Array{Float32,4},Array{Float32,1}},MaxPool{2,4},BSON.__deserialized_types__.var"##452",Dense{typeof(relu),Array{Float32,2},Array{Float32,1}},Dense{typeof(relu),Array{Float32,2},Array{Float32,1}},Dense{typeof(identity),Array{Float32,2},Array{Float32,1}}}}, ::Function) at /home/carlo/Git/model-zoo/vision/lenet_mnist/lenet_mnist.jl:58
[4] (::var"#report#20"{Args,typeof(gpu),DataLoader,DataLoader})(::Int64) at /home/carlo/Git/model-zoo/vision/lenet_mnist/lenet_mnist.jl:124
[5] #train#17(::Base.Iterators.Pairs{Symbol,String,Tuple{Symbol},NamedTuple{(:loadpath,),Tuple{String}}}, ::typeof(train)) at /home/carlo/Git/model-zoo/vision/lenet_mnist/lenet_mnist.jl:139
[6] (::var"#kw##train")(::NamedTuple{(:loadpath,),Tuple{String}}, ::typeof(train)) at ./none:0
[7] top-level scope at REPL[24]:1When loading and using in REPL instead I don't see the issue julia> include("lenet_mnist.jl")
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
train (generic function with 1 method)
julia> mtest = BSON.load("mtest.bson")[:model]
Chain(#8, Conv((5, 5), 1=>6, relu), MaxPool((2, 2), pad = (0, 0, 0, 0), stride = (2, 2)), Conv((5, 5), 6=>16, relu), MaxPool((2, 2), pad = (0, 0, 0, 0), stride = (2, 2)), #9, Dense(256, 120, relu), Dense(120, 84, relu), Dense(84, 10))
julia> dtrain, dtest = get_data(Args());
julia> eval_loss_accuracy(dtest, mtest, cpu)
(loss = 2.2903f0, acc = 8.31) |
|
I have never seen this |
|
I reduced the issue to julia> import BSON
julia> function train()
model = x -> x
BSON.@save "model.bson" model
model = BSON.load("model.bson")[:model]
ŷ = model(1)
end
train (generic function with 1 method)
julia> train()
ERROR: MethodError: no method matching (::BSON.__deserialized_types__.var"#94#95")(::Int64)
The applicable method may be too new: running in world age 27203, while current world is 27204.
Closest candidates are:
#94(::Any) at REPL[73]:2 (method too new to be called from this world context.)
Stacktrace:
[1] train() at ./REPL[73]:5
[2] top-level scope at REPL[74]:1ref. JuliaIO/BSON.jl#69 |
|
I think this is ready to go. @dhairyagandhi96 @matsueushi it would be nice to have some feedback from you. On average, I expect the other scripts in this repo to be slimmer than this, but I wanted this particular one to showcase patterns and packages in the ecosystem |
matsueushi
reviewed
Mar 7, 2020
vision/lenet_mnist/lenet_mnist.jl
Outdated
|
|
||
| ## utility functions | ||
|
|
||
| nobs(loader) = sum(size(x)[end] for (x, y) in loader) # == size(loader.data[1])[end] but more generic |
There was a problem hiding this comment.
Do we really need to define nobs in this file?
julia> sum(size(x)[end] for (x, y) in train_loader)
60000
julia> train_loader.nobs
60000
julia> sum(size(x)[end] for (x, y) in test_loader)
10000
julia> test_loader.nobs
10000
vision/lenet_mnist/lenet_mnist.jl
Outdated
| # The model can be adapted to any image size | ||
| # and number of output classes. | ||
| function LeNet5(; imgsize=(28,28,1), nclasses=10) | ||
| out_conv_size = (imgsize[1]÷4 - 3, imgsize[2]÷4 - 3, 16) |
There was a problem hiding this comment.
Suggested change
| out_conv_size = (imgsize[1]÷4 - 3, imgsize[2]÷4 - 3, 16) | |
| out_conv_size = (imgsize[1]÷4 - 3, imgsize[2]÷4 - 3, 16) |
vision/lenet_mnist/lenet_mnist.jl
Outdated
| return Chain( | ||
| x -> reshape(x, imgsize..., :), | ||
| Conv((5, 5), imgsize[end]=>6, relu), | ||
| MaxPool((2,2)), |
There was a problem hiding this comment.
Suggested change
| MaxPool((2,2)), | |
| MaxPool((2, 2)), |
vision/lenet_mnist/lenet_mnist.jl
Outdated
| Conv((5, 5), imgsize[end]=>6, relu), | ||
| MaxPool((2,2)), | ||
| Conv((5, 5), 6=>16, relu), | ||
| MaxPool((2,2)), |
There was a problem hiding this comment.
Suggested change
| MaxPool((2,2)), | |
| MaxPool((2, 2)), |
.vscode/settings.json
Outdated
Comment on lines
1
to
3
| { | ||
| "julia.environmentPath": "/home/carlo/Git/model-zoo/vision/lenet_mnist" | ||
| } |
There was a problem hiding this comment.
I don't think settings.json for vscode should be included to this repository.
vision/lenet_mnist/lenet_mnist.jl
Outdated
|
|
||
| nobs(loader) = sum(size(x)[end] for (x, y) in loader) # == size(loader.data[1])[end] but more generic | ||
|
|
||
| num_params(model) = sum(length(p) for p in Flux.params(model)) |
There was a problem hiding this comment.
Suggested change
| num_params(model) = sum(length(p) for p in Flux.params(model)) | |
| num_params(model) = sum(length.(Flux.params(model))) |
It might be my preference...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Also showing progress bars, tensorboard logging and the new DataLoader.
I'm not sure here what the right balance between features and simplicity would be for an entry point script like "lenet + mnist" is supposed to be. Maybe we can have a much more minimalistic "mlp + mnist"
todo: