Template limitation `convertStructType` (Mixtures) #668

dehann · 2020-09-25T01:44:22Z

EDIT: Updated Post

MWE:

# existing works
julia> DFG.convertStructType(PackedLinearRelative)
LinearRelative

# but this new does not
julia> DFG.convertStructType(PackedMixtureRelative)
ERROR: type UnionAll has no field name
Stacktrace:
 [1] getproperty(::Type{T} where T, ::Symbol) at ./Base.jl:28
 [2] convertStructType(::Type{PackedMixtureRelative}) at /home/dehann/.julia/dev/DistributedFactorGraphs/src/Common.jl:8
 [3] top-level scope at REPL[5]:1

The difference is that is actually a template type PackedMixtureRelative{T}, contrasted against PackedLinearRelate which is a concrete type. So we need a better solution for chomping away the Packed portion:

DistributedFactorGraphs.jl/src/Common.jl

Line 8 in 10bf5c3

    
           convertStructType(::Type{PT}) where {PT <: PackedInferenceType} = getfield(PT.name.module, Symbol(string(PT.name.name)[7:end]))

julia> typeof(PackedLinearRelative)
DataType

# NOTE this used to be `struct PackedMixtureRelative{T} ... end`, but templating was later dropped

julia> typeof(PackedMixtureRelative)
UnionAll

Original Post

MWE:

julia> typeof(DFG.getSolverData(f1).fnc.usrfnc!)
MixtureRelative{2,LinearRelative{1,MvNormal{Float64,PDMats.PDMat{Float64,Array{Float64,2}},Array{Float64,1}}},(:c1, :c2),Tuple{Normal{Float64},Normal{Float64}}}

But what currently happens is:

julia> typeof(DFG.getSolverData(f1).fnc.usrfnc!).name.name
:MixtureRelative

Which is not enough to reconstruct the type properly when unpacking. Need a lot of that structure to rebuild MixtureRelative properly.

DistributedFactorGraphs.jl/src/services/Serialization.jl

Line 252 in 419a31d

props["fnctype"] = String(_getname(fnctype))

DistributedFactorGraphs.jl/src/Common.jl

Line 4 in 419a31d

_getname(t::T) where T = T.name.name

Example

DOWNLOAD FileDFG: test3.tar.gz

using IncrementalInference

julia> fg = loadDFG("/tmp/test3.tar.gz")
sfolder = split(dstname, '.') = SubString{String}["/tmp/test3", "tar", "gz"]
[ Info: loadDFG! detected a gzip /tmp/test3.tar.gz -- unpacking via /tmp/caesar/random/bff326e0 now...
[ Info: Loaded 2 variables - [:x0, :x1]
[ Info: Inserting variables into graph...
ERROR: Error while unpacking 'x0x1f1' as 'MixtureRelative', please check the unpacking/packing converters for this factor - 
type UnionAll has no field name
Stacktrace:
 [1] getproperty(::Type{T} where T, ::Symbol) at ./Base.jl:28
 [2] convertStructType(::Type{PackedMixtureRelative}) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/Common.jl:8
 [3] decodePackedType(::Type{CommonConvWrapper}, ::DistributedFactorGraphs.GenericFunctionNodeData{PackedMixtureRelative}) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/services/Serialization.jl:262
 [4] unpackFactor(::LightDFG{SolverParams,DFGVariable,DFGFactor}, ::Dict{String,Any}) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/services/Serialization.jl:298
 [5] (::DistributedFactorGraphs.var"#132#138"{LightDFG{SolverParams,DFGVariable,DFGFactor},Array{DFGFactor,1}})(::IOStream) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/FileDFG/services/FileDFG.jl:146
 [6] open(::DistributedFactorGraphs.var"#132#138"{LightDFG{SolverParams,DFGVariable,DFGFactor},Array{DFGFactor,1}}, ::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:325
 [7] open(::Function, ::String) at ./io.jl:323
 [8] loadDFG!(::LightDFG{SolverParams,DFGVariable,DFGFactor}, ::String) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/FileDFG/services/FileDFG.jl:144
 [9] loadDFG(::String) at /home/dehann/.julia/dev/IncrementalInference/src/FGOSUtils.jl:332
 [10] top-level scope at REPL[2]:1
 [11] eval(::Module, ::Any) at ./boot.jl:331
 [12] eval_user_input(::Any, ::REPL.REPLBackend) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:134
 [13] repl_backend_loop(::REPL.REPLBackend) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:195
 [14] start_repl_backend(::REPL.REPLBackend, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:180
 [15] run_repl(::REPL.AbstractREPL, ::Any; backend_on_current_task::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:292
 [16] run_repl(::REPL.AbstractREPL, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
 [17] (::Base.var"#806#808"{Bool,Bool,Bool,Bool})(::Module) at ./client.jl:399
 [18] #invokelatest#1 at ./essentials.jl:710 [inlined]
 [19] invokelatest at ./essentials.jl:709 [inlined]
 [20] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:383
 [21] exec_options(::Base.JLOptions) at ./client.jl:313
 [22] _start() at ./client.jl:506
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] unpackFactor(::LightDFG{SolverParams,DFGVariable,DFGFactor}, ::Dict{String,Any}) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/services/Serialization.jl:304
 [3] (::DistributedFactorGraphs.var"#132#138"{LightDFG{SolverParams,DFGVariable,DFGFactor},Array{DFGFactor,1}})(::IOStream) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/FileDFG/services/FileDFG.jl:146
 [4] open(::DistributedFactorGraphs.var"#132#138"{LightDFG{SolverParams,DFGVariable,DFGFactor},Array{DFGFactor,1}}, ::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:325
 [5] open(::Function, ::String) at ./io.jl:323
 [6] loadDFG!(::LightDFG{SolverParams,DFGVariable,DFGFactor}, ::String) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/FileDFG/services/FileDFG.jl:144
 [7] loadDFG(::String) at /home/dehann/.julia/dev/IncrementalInference/src/FGOSUtils.jl:332
 [8] top-level scope at REPL[2]:1
caused by [exception 1]
type UnionAll has no field name
Stacktrace:
 [1] getproperty(::Type{T} where T, ::Symbol) at ./Base.jl:28
 [2] convertStructType(::Type{PackedMixtureRelative}) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/Common.jl:8
 [3] decodePackedType(::Type{CommonConvWrapper}, ::DistributedFactorGraphs.GenericFunctionNodeData{PackedMixtureRelative}) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/services/Serialization.jl:262
 [4] unpackFactor(::LightDFG{SolverParams,DFGVariable,DFGFactor}, ::Dict{String,Any}) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/services/Serialization.jl:298
 [5] (::DistributedFactorGraphs.var"#132#138"{LightDFG{SolverParams,DFGVariable,DFGFactor},Array{DFGFactor,1}})(::IOStream) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/FileDFG/services/FileDFG.jl:146
 [6] open(::DistributedFactorGraphs.var"#132#138"{LightDFG{SolverParams,DFGVariable,DFGFactor},Array{DFGFactor,1}}, ::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:325
 [7] open(::Function, ::String) at ./io.jl:323
 [8] loadDFG!(::LightDFG{SolverParams,DFGVariable,DFGFactor}, ::String) at /home/dehann/.julia/packages/DistributedFactorGraphs/PRVGr/src/FileDFG/services/FileDFG.jl:144
 [9] loadDFG(::String) at /home/dehann/.julia/dev/IncrementalInference/src/FGOSUtils.jl:332
 [10] top-level scope at REPL[2]:1

The text was updated successfully, but these errors were encountered:

dehann · 2020-09-26T02:24:46Z

HI @GearsAD , also see JuliaRobotics/IncrementalInference.jl#912 for the associated IIF code

EDIT, actually more specific:

https://github.com/JuliaRobotics/IncrementalInference.jl/blob/a70c00d795ee925890ed4350fb13395abc99c93e/src/Factors/MixtureRelative.jl#L84-L96

dehann · 2020-09-26T06:18:38Z

Just a note, all that stuff in IIF has now been merged -- so you should be able to get the latest LinearRelative and Mixture_ stuff on IIF master. Also note that future plans might consolidate MixturePrior{..} and MixtureRelative{..} into only Mixture{..}, but that is not critical at this time.

GearsAD · 2020-09-27T05:19:08Z

The error above looks to be because the Packed type is templated. I think it would fail further down once that was resolved, but the short term fix it to either encode the templating into the packed type (so the packed type saves the template information and resolves the full type using it), or to have a set of concrete types for each template possibility (this, I don't think, is practical, so the first option is preferable).

dehann · 2020-09-29T02:28:40Z

HI @GearsAD, okay thanks, let's go with the first option. I will build it in to PackedMixture__ then. Thanks for taking a look!

dehann · 2020-09-29T16:19:21Z

Ah, I learned some more. I will fix this issue and involves two stages:

See updated MWE for non-concrete Packed types Template limitation convertStructType (Mixtures) #668 (comment)
Expand PackedMixtureRelative to capture all the Static Parameters contained in MixtureRelative{N,F,S,T} Template limitation convertStructType (Mixtures) #668 (comment)

dehann added types serialization feature request limitation labels Sep 25, 2020

dehann added this to the v0.10.6 milestone Sep 25, 2020

dehann mentioned this issue Sep 26, 2020

upgrade mixture prior, needed downstream JuliaRobotics/IncrementalInference.jl#912

Merged

dehann changed the title ~~Factor serialization name may need to go further~~ Factor serialization name may need to go further (Mixtures) Sep 26, 2020

dehann closed this as completed Sep 29, 2020

dehann changed the title ~~Factor serialization name may need to go further (Mixtures)~~ Template limitation convertStructType (Mixtures) Sep 29, 2020

dehann reopened this Sep 29, 2020

dehann closed this as completed Sep 29, 2020

This was referenced Sep 29, 2020

Serialization and Marshaling (again) #590

Closed

convertStructType for ::UnionAll too #670

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Template limitation `convertStructType` (Mixtures) #668

Template limitation `convertStructType` (Mixtures) #668

dehann commented Sep 25, 2020 •

edited

Loading

dehann commented Sep 26, 2020 •

edited

Loading

dehann commented Sep 26, 2020 •

edited

Loading

GearsAD commented Sep 27, 2020

dehann commented Sep 29, 2020

dehann commented Sep 29, 2020 •

edited

Loading

Template limitation convertStructType (Mixtures) #668

Template limitation convertStructType (Mixtures) #668

Comments

dehann commented Sep 25, 2020 • edited Loading

EDIT: Updated Post

Original Post

Example

dehann commented Sep 26, 2020 • edited Loading

dehann commented Sep 26, 2020 • edited Loading

GearsAD commented Sep 27, 2020

dehann commented Sep 29, 2020

dehann commented Sep 29, 2020 • edited Loading

Template limitation `convertStructType` (Mixtures) #668

Template limitation `convertStructType` (Mixtures) #668

dehann commented Sep 25, 2020 •

edited

Loading

dehann commented Sep 26, 2020 •

edited

Loading

dehann commented Sep 26, 2020 •

edited

Loading

dehann commented Sep 29, 2020 •

edited

Loading