Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Unexpected max serialisation length" on variable length field inside variant #9

Closed
melg8 opened this issue Nov 24, 2020 · 6 comments
Closed

"Unexpected max serialisation length" on variable length field inside variant #9

melg8 opened this issue Nov 24, 2020 · 6 comments

Comments

@melg8
Copy link

melg8 commented Nov 24, 2020

Problem

When try to generate code for variant which have variable length field inside, it will generate code, that reports:

/home/jenkins/comms_error2/include/tutorial4/message/Msg2.h:523:29: error: static assertion failed: Unexpected max serialisation length
     static_assert(MsgMaxLen == 257U, "Unexpected max serialisation length");
                   ~~~~~~~~~~^~~~~~~
In file included from /home/jenkins/comms_error2/test/tutorial4_input_test.cpp:12:
/home/jenkins/comms_error2/build/install/include/comms/process.h:119:20: error: ‘comms::ErrorStatus comms::processSingleWithDispatch(TBufIter&, std::size_t, TFrame&&, TMsg&, THandler&, TExtraValues ...) [with TBufIter = char*; TFrame = tutorial4::frame::Frame<comms::Message<comms::option::app::ReadIterator<const char*>, comms::option::app::WriteIterator<char*>, comms::option::app::LengthInfoInterface, comms::option::app::Handler<{anonymous}::Handler>, comms::option::def::Endian<comms::util::traits::endian::Big>, comms::option::def::MsgIdType<tutorial4::MsgId> >, std::tuple<tutorial4::message::Msg1<comms::Message<comms::option::app::ReadIterator<const char*>, comms::option::app::WriteIterator<char*>, comms::option::app::LengthInfoInterface, comms::option::app::Handler<{anonymous}::Handler>, comms::option::def::Endian<comms::util::traits::endian::Big>, comms::option::def::MsgIdType<tutorial4::MsgId> >, tutorial4::options::DefaultOptions>, tutorial4::message::Msg2<comms::Message<comms::option::app::ReadIterator<const char*>, comms::option::app::WriteIterator<char*>, comms::option::app::LengthInfoInterface, comms::option::app::Handler<{anonymous}::Handler>, comms::option::def::Endian<comms::util::traits::endian::Big>, comms::option::def::MsgIdType<tutorial4::MsgId> >, tutorial4::options::DefaultOptions> >, tutorial4::options::DefaultOptions>&; TMsg = std::unique_ptr<comms::Message<comms::option::app::ReadIterator<const char*>, comms::option::app::WriteIterator<char*>, comms::option::app::LengthInfoInterface, comms::option::app::Handler<{anonymous}::Handler>, comms::option::def::Endian<comms::util::traits::endian::Big>, comms::option::def::MsgIdType<tutorial4::MsgId> >, std::default_delete<comms::Message<comms::option::app::ReadIterator<const char*>, comms::option::app::WriteIterator<char*>, comms::option::app::LengthInfoInterface, comms::option::app::Handler<{anonymous}::Handler>, comms::option::def::Endian<comms::util::traits::endian::Big>, comms::option::def::MsgIdType<tutorial4::MsgId> > > >; THandler = {anonymous}::Handler; TExtraValues = {}]’ used but never defined [-Werror]
 comms::ErrorStatus processSingleWithDispatch(

Schema

We use will use original schema from tutorial 4 with slight changes:

<?xml version="1.0" encoding="UTF-8"?>
<schema name="tutorial4" endian="big">
    <fields>
        <string name="Msg1Name" defaultValue="Message 1" />
        <string name="Msg2Name" defaultValue="Message 2" />
            
        <enum name="MsgId" type="uint8" semanticType="messageId">
            <validValue name="M1" val="1" displayName="^Msg1Name" />
            <validValue name="M2" val="2" displayName="^Msg2Name" />
        </enum>
        
        <enum name="PropKey" type="uint8">
            <validValue name="K1" val="0" />
            <validValue name="K2" val="2" />
            <validValue name="K3" val="5" />
            <validValue name="K4" val="10" />
            <validValue name="K5" val="15" />
            <validValue name="K6" val="25" />
        </enum>
        
        <int name="PropKeyCommon" type="uint8" failOnInvalid="true"/>
        
        <variant name="KeyValueProp">
            <bundle name="Prop1">
                <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K1" validValue="PropKey.K1" />
                <int name="Val" type="uint16" />
            </bundle>
            
            <bundle name="Prop2">
                <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K2" validValue="PropKey.K2" />
                <float name="Val" type="float" defaultValue="nan" />
            </bundle>            
            
            <bundle name="Prop3">
                <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K3" validValue="PropKey.K3" />
                <string name="Val">
                    <lengthPrefix>
                        <int name="Length" type="uint8" />
                    </lengthPrefix>
                </string>
            </bundle>    
        </variant>
        
        <variant name="TlvProp">
            <description>
                Type-Length-Value Property
            </description>
            <members>
                <bundle name="Prop4">
                    <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K4" validValue="PropKey.K4" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <int name="Val" type="uint32" />
                </bundle>
                
                <bundle name="Prop5">
                    <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K5" validValue="PropKey.K5" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <float name="Val" type="double" defaultValue="inf" />
                </bundle>            
                
                <bundle name="Prop6">
                    <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K6" validValue="PropKey.K6" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <string name="Val" />
                </bundle>   
                
                <bundle name="Any">
                    <int reuse="PropKeyCommon" name="Key" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <data name="Val" />
                </bundle>  
            </members>
        </variant>
        
    </fields>
    
    <frame name="Frame">
        <size name="Size">
            <int name="SizeField" type="uint16" />
        </size>
        <id name="ID" field="MsgId" />
        <payload name="Data" />
    </frame>
    
    <message name="Msg1" id="MsgId.M1" displayName="^Msg1Name">
        <list name="F1" element="KeyValueProp">
            <countPrefix>
                <int name="Count" type="uint8" />
            </countPrefix>
        </list>
    </message>
    
    <message name="Msg2" id="MsgId.M2" displayName="^Msg2Name">
        <variant reuse="KeyValueProp" />
    </message>      
</schema>

Steps to reproduce

#!/bin/bash

mkdir comms_error2
cd comms_error2
commsdsl2comms ../schema_mod.xml
mkdir build
cd build
cmake -DOPT_BUILD_TEST=ON -DCMAKE_INSTALL_PREFIX=$PWD/install ../
make -j$(nproc --all

Note

What is interesting, when using the same approach but with additional list wrapper around variant - compiles fine.
But whe need to have no additional "number of elements" in our message, just pure variant.

<?xml version="1.0" encoding="UTF-8"?>
<schema name="tutorial4" endian="big">
    <fields>
        <string name="Msg1Name" defaultValue="Message 1" />
        <string name="Msg2Name" defaultValue="Message 2" />
            
        <enum name="MsgId" type="uint8" semanticType="messageId">
            <validValue name="M1" val="1" displayName="^Msg1Name" />
            <validValue name="M2" val="2" displayName="^Msg2Name" />
        </enum>
        
        <enum name="PropKey" type="uint8">
            <validValue name="K1" val="0" />
            <validValue name="K2" val="2" />
            <validValue name="K3" val="5" />
            <validValue name="K4" val="10" />
            <validValue name="K5" val="15" />
            <validValue name="K6" val="25" />
        </enum>
        
        <int name="PropKeyCommon" type="uint8" failOnInvalid="true"/>
        
        <variant name="KeyValueProp">
            <bundle name="Prop1">
                <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K1" validValue="PropKey.K1" />
                <int name="Val" type="uint16" />
            </bundle>
            
            <bundle name="Prop2">
                <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K2" validValue="PropKey.K2" />
                <float name="Val" type="float" defaultValue="nan" />
            </bundle>            
            
            <bundle name="Prop3">
                <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K3" validValue="PropKey.K3" />
                <string name="Val">
                    <lengthPrefix>
                        <int name="Length" type="uint8" />
                    </lengthPrefix>
                </string>
            </bundle>    
        </variant>
        
        <variant name="TlvProp">
            <description>
                Type-Length-Value Property
            </description>
            <members>
                <bundle name="Prop4">
                    <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K4" validValue="PropKey.K4" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <int name="Val" type="uint32" />
                </bundle>
                
                <bundle name="Prop5">
                    <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K5" validValue="PropKey.K5" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <float name="Val" type="double" defaultValue="inf" />
                </bundle>            
                
                <bundle name="Prop6">
                    <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K6" validValue="PropKey.K6" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <string name="Val" />
                </bundle>   
                
                <bundle name="Any">
                    <int reuse="PropKeyCommon" name="Key" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <data name="Val" />
                </bundle>  
            </members>
        </variant>
        
    </fields>
    
    <frame name="Frame">
        <size name="Size">
            <int name="SizeField" type="uint16" />
        </size>
        <id name="ID" field="MsgId" />
        <payload name="Data" />
    </frame>
    
    <message name="Msg1" id="MsgId.M1" displayName="^Msg1Name">
        <list name="F1" element="KeyValueProp">
            <countPrefix>
                <int name="Count" type="uint8" />
            </countPrefix>
        </list>
    </message>
    
    <message name="Msg2" id="MsgId.M2" displayName="^Msg2Name">
        <list name="F2" element="KeyValueProp" />
    </message>      
</schema>
@melg8
Copy link
Author

melg8 commented Nov 24, 2020

Ping @arobenko for help.

@melg8
Copy link
Author

melg8 commented Nov 24, 2020

Additionally, if we add count to list, error reappears.

Schema

<?xml version="1.0" encoding="UTF-8"?>
<schema name="tutorial4" endian="big">
    <fields>
        <string name="Msg1Name" defaultValue="Message 1" />
        <string name="Msg2Name" defaultValue="Message 2" />
            
        <enum name="MsgId" type="uint8" semanticType="messageId">
            <validValue name="M1" val="1" displayName="^Msg1Name" />
            <validValue name="M2" val="2" displayName="^Msg2Name" />
        </enum>
        
        <enum name="PropKey" type="uint8">
            <validValue name="K1" val="0" />
            <validValue name="K2" val="2" />
            <validValue name="K3" val="5" />
            <validValue name="K4" val="10" />
            <validValue name="K5" val="15" />
            <validValue name="K6" val="25" />
        </enum>
        
        <int name="PropKeyCommon" type="uint8" failOnInvalid="true"/>
        
        <variant name="KeyValueProp">
            <bundle name="Prop1">
                <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K1" validValue="PropKey.K1" />
                <int name="Val" type="uint16" />
            </bundle>
            
            <bundle name="Prop2">
                <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K2" validValue="PropKey.K2" />
                <float name="Val" type="float" defaultValue="nan" />
            </bundle>            
            
            <bundle name="Prop3">
                <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K3" validValue="PropKey.K3" />
                <string name="Val">
                    <lengthPrefix>
                        <int name="Length" type="uint8" />
                    </lengthPrefix>
                </string>
            </bundle>    
        </variant>
        
        <variant name="TlvProp">
            <description>
                Type-Length-Value Property
            </description>
            <members>
                <bundle name="Prop4">
                    <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K4" validValue="PropKey.K4" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <int name="Val" type="uint32" />
                </bundle>
                
                <bundle name="Prop5">
                    <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K5" validValue="PropKey.K5" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <float name="Val" type="double" defaultValue="inf" />
                </bundle>            
                
                <bundle name="Prop6">
                    <int reuse="PropKeyCommon" name="Key" defaultValue="PropKey.K6" validValue="PropKey.K6" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <string name="Val" />
                </bundle>   
                
                <bundle name="Any">
                    <int reuse="PropKeyCommon" name="Key" />
                    <int name="Length" type="uint8" semanticType="length" />
                    <data name="Val" />
                </bundle>  
            </members>
        </variant>
        
    </fields>
    
    <frame name="Frame">
        <size name="Size">
            <int name="SizeField" type="uint16" />
        </size>
        <id name="ID" field="MsgId" />
        <payload name="Data" />
    </frame>
    
    <message name="Msg1" id="MsgId.M1" displayName="^Msg1Name">
        <list name="F1" element="KeyValueProp">
            <countPrefix>
                <int name="Count" type="uint8" />
            </countPrefix>
        </list>
    </message>
    
    <message name="Msg2" id="MsgId.M2" displayName="^Msg2Name">
        <list name="F2" element="KeyValueProp" count="1"/>
    </message>      
</schema>

@arobenko
Copy link
Member

arobenko commented Nov 24, 2020 via email

arobenko added a commit that referenced this issue Nov 24, 2020
@arobenko
Fixing message length calculation with variant as a member field.
6fb7f5a 
Reported by github #9.
@arobenko
Copy link
Member

Reproduced and fixed on "hotfix_github_9" branch. Please use it for a while.
I'm planning to do the next official release within next couple of weeks.
Also just keep in mind the following current limitations:

  • Current v3.1.2 release of the COMMS library supports up to 48 variant field members.
  • The "develop" branch of the COMMS library (will become v3.1.3 release soon) supports up to 128 variant members.
  • The current "develop" branch of the "commsdsl" (uses "develop" branch of COMMS library) counts on limit of 128 variant members and generates member access functions itself (instead of counting on the COMMS library) in case number of used members exceeds 128.

The bottom line, if your number of variant members is below 48, use "v3.1.2" release of the COMMS library, otherwise use "develop" branch of commsdsl (also contains the recent fix from hotfix_github_9 branch) as well as "develop" branch of the COMMS library.

@melg8
Copy link
Author

melg8 commented Nov 25, 2020

Thanks for fast response!

@arobenko
Copy link
Member

Master branch now points to official v3.6 release that has the issue fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@arobenko @melg8