Need Help #2

mathisloge · 2019-08-08T07:00:59Z

Hey sorry for interrupting you again, but i dont know how to proceed. I have already two custom protocols implemented but the third isnt working and i dont know why. Ive prepared a simple
demo:
Schema:

<?xml version="1.0" encoding="UTF-8"?>
<schema name="MyProtocol" endian="big">
	<fields>
        <enum name="MsgId" type="uint8" semanticType="messageId" >
            <validValue name="TestId" val="0" />
        </enum>
    </fields>

	<interface name="FrameInterface">
    </interface>
	<frame name="Frame">
        <sync name="Sync">
            <int name="Sync" type="uint16" defaultValue="0xabcd" validValue="0xabcd" failOnInvalid="true"/>
        </sync>
        <size name="Size">
            <int name="Size" type="uint16" serOffset="4" displayOffset="4"/>
        </size>
        <id name="Id" field="MsgId" />
        <payload name="Data" />
		<checksum name="Checksum" alg="sum " from="Size">
            <int name="Checksum" type="uint16" />
        </checksum>
    </frame>       
	
	<message name="TestPoll" id="MsgId.TestId">
		<int name="TestField" type="uint8"/>
    </message>
</schema>

Definitions:


using InMessage =
          MyProtocol::FrameInterface<
              comms::option::ReadIterator<const std::uint8_t*>,
              comms::option::Handler<Application>,
              comms::option::NameInterface
         >;

     using InPollTest= MyProtocol::message::TestPoll<InMessage>;
     using AllInMessages = std::tuple<InPollTest>;
     using Frame = MyProtocol::frame::Frame<InMessage, AllInMessages>;
     using OutputMsg =
             MyProtocol::FrameInterface<
                 comms::option::WriteIterator<std::back_insert_iterator<std::vector<std::uint8_t> > >,
                 comms::option::LengthInfoInterface,
                 comms::option::IdInfoInterface,
                 comms::option::NameInterface
             >;
     Frame frame;

Test code:

MyProtocol::message::TestPoll<OutputMsg> msg;
    std::vector<std::uint8_t> outputBuf;
    outputBuf.reserve(frame.length(msg));
    auto iter = std::back_inserter(outputBuf);
    auto es = frame.write(msg, iter, outputBuf.max_size());
    if (es == comms::ErrorStatus::UpdateRequired) {
        auto updateIter = &outputBuf[0];
        es = frame.update(updateIter, outputBuf.size());
    }

    if (es != comms::ErrorStatus::Success) {
        UARTprintf("\nUnexpected error\n");
        return;
    }
    UARTprintf("TX:\n");
    for(int i = 0; i < outputBuf.size(); i++) {
        UARTprintf("%x ", outputBuf[i]);
    }
    UARTprintf("\nCONSUME:\n");

        const auto consumed = comms::processAllWithDispatch(&outputBuf[0],
                                                            outputBuf.size(),
                                                            frame,
                                                            *this);
       UARTprintf("RX:");
       for(auto i = 0; i < outputBuf.size(); i++)
       {
           UARTprintf("%x ", outputBuf[i]);
       }
       UARTprintf("---%i\n", consumed);
       if(consumed != outputBuf.size()) {
          LOG_ERROR("Something is wrong...");
       }

The Error-Code is on "InvalidMsgId"

Ive inserted some DebugMessages but cant find the root cause:

DEBUG | MsgIdLayer.h | doReadInternal():681 | 5
DEBUG | MsgFactoryBase.h | createMsgInternal():427 | VIRTUAL
DEBUG | MsgFactoryBase.h | dispatchMsgTypeInternal():364 | StandardTag
DEBUG | dispatch.h | dispatchMsgType():490 | Complex
DEBUG | dispatch.h | dispatchMsgTypeInternal():578 | PolymorphicTag
DEBUG | DispatchMsgPolymorphicHelper.h | dispatch():1074 | offset 0 1
DEBUG | DispatchMsgPolymorphicHelper.h | dispatchInternal():1115 | DirectTag 0
DEBUG | DispatchMsgPolymorphicHelper.h | dispatchInternal():1102 | DirectTag
DEBUG | DispatchMsgPolymorphicHelper.h | dispatch():895 | regIdx 0
DEBUG | DispatchMsgPolymorphicHelper.h | dispatch():897 | regIdx 0
DEBUG | MsgFactoryBase.h | createMsgInternal():430 | Success: 0
DEBUG | MsgFactoryBase.h | createMsg():168 | 0
DEBUG | MsgIdLayer.h | doReadInternal():687 | 1, 0, 0
RX:ab cd 0 6 0 0 0 6 ---8

Where is my mistake? I just cant see it.

The text was updated successfully, but these errors were encountered:

arobenko · 2019-08-08T07:31:45Z

Sorry Mathis, I don't see anything wrong in this code (just by reading it), and to retest it myself I won't be able to do so until tomorrow (maybe even weekend).

Are you sure that the error code was InvalidMsgId? Maybe you forgot to write proper handle(InPollTest&) member function in your Application class and your message was created, but not handled properly.

To help in debugging you may try to use the code of comms::processAllWithDispatch (and other functions in the same file) as reference and write your own processing loop with tighter control.

Try just manual reading of your input buffer without loop and dispatching:

auto* readIter = &outputBuf[0]
MyProtocol::MsgId retrievedId = ... /* some invalid value */;
auto es = frame.read(readIter, outputBuf.size(), comms::protocol::msgId(retrievedId));
... // print es and retrievedId

The comms::protocol::msgId() function may be used to pass the reference to a variable,which is going to be populated with detected message ID.

Hope it helps

mathisloge · 2019-08-08T08:06:50Z

Hey, thanks for the quick response.
Ive checked my Handlers and they seem to be fine.
Im using now this code to read in the message.

 Frame::MsgPtr msgptr;
       auto* readIter = &outputBuf[0];
       auto iter = comms::readIteratorFor(msgptr, readIter);

       std::size_t retrievedId = 0x99;/* some invalid value */
       std::size_t len = outputBuf.size();
       auto es = frame.read(msgptr, iter, len, comms::protocol::msgId(retrievedId));
       LOG_DEBUG("Recv msgId: %i, es: %i", retrievedId, es);

And this is the output: DEBUG | Application.cpp | task():136 | Recv msgId: 0, es: 5
Es is 5 so it should map to ErrorStatus::InvalidMsgId.

Currently im using gcc7.2.1 with c++17

In the Evening i will try to use this with the Visual Studio 2019 compiler. Just to try it.

arobenko · 2019-08-08T08:13:27Z

I need to debug it myself, looks like a bug somewhere in the comms library

…

On Thu, 8 Aug 2019, 18:06 Mathis Logemann ***@***.***> wrote: Hey, thanks for the quick response. Ive checked my Handlers and they seem to be fine. Im using now this code to read in the message. ``` Frame::MsgPtr msgptr; auto* readIter = &outputBuf[0]; auto iter = comms::readIteratorFor(msgptr, readIter); std::size_t retrievedId = 0x99;/* some invalid value */ std::size_t len = outputBuf.size(); auto es = frame.read(msgptr, iter, len, comms::protocol::msgId(retrievedId)); LOG_DEBUG("Recv msgId: %i, es: %i", retrievedId, es); And this is the output: `DEBUG | Application.cpp | task():136 | Recv msgId: 0, es: 5` Es is 5 so it should map to ErrorStatus::InvalidMsgId. Currently im using gcc7.2.1. In the Evening i will try to use this with the Visual Studio 2019 compiler. Just to try it. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASJKGWMYY4ZZIEYJTJUIK3QDPHZVANCNFSM4IKHBMIA> .

arobenko · 2019-08-08T08:14:48Z

Id was recognized correctly, but the correct object hasn't been created

…

On Thu, 8 Aug 2019, 18:13 Alex Robenko ***@***.***> wrote: I need to debug it myself, looks like a bug somewhere in the comms library On Thu, 8 Aug 2019, 18:06 Mathis Logemann ***@***.***> wrote: > Hey, thanks for the quick response. > Ive checked my Handlers and they seem to be fine. > Im using now this code to read in the message. > ``` > Frame::MsgPtr msgptr; > auto* readIter = &outputBuf[0]; > auto iter = comms::readIteratorFor(msgptr, readIter); > > std::size_t retrievedId = 0x99;/* some invalid value */ > std::size_t len = outputBuf.size(); > auto es = frame.read(msgptr, iter, len, comms::protocol::msgId(retrievedId)); > LOG_DEBUG("Recv msgId: %i, es: %i", retrievedId, es); > > And this is the output: `DEBUG | Application.cpp | task():136 | Recv msgId: 0, es: 5` > Es is 5 so it should map to ErrorStatus::InvalidMsgId. > > Currently im using gcc7.2.1. > > In the Evening i will try to use this with the Visual Studio 2019 compiler. Just to try it. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#2>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AASJKGWMYY4ZZIEYJTJUIK3QDPHZVANCNFSM4IKHBMIA> > . >

arobenko · 2019-08-08T09:12:16Z

Just added test37 to the "develop" branch to test the reported problem, and it all works fine (the message object is successfully created and dispatched to a handler), the test reports success.

Could you please review it and see whether you see any significant difference with your code?

arobenko · 2019-08-08T11:02:50Z

To build the test37 do

$> make commsdsl2comms.test37Test

To run the test do (from within the build directory)

$> ./app/commsdsl2comms/test/commsdsl2comms.test37Test -v

The -v option means verbose and it allows prints to stdout.

Another way to help me to help you is to fork one of the demo protocols, create new example application that fails for you and send me the link, so I could clone and check what's going on.

arobenko · 2019-08-08T11:09:07Z

Just noticed that you mentioned C++17. Actually I've never tested my work with C++17 enabled (only 11 and 14), and never used VS2019 also (only 2015 and 2017). Could you please try downgrade to C++11 and/or 14 and see whether the code works as expected?

mathisloge · 2019-08-08T11:57:58Z

My test repo

So i tested c++11, 14 and 17.
But the handler wont fire.

For my test repo im using the Code Composer Studio with the gcc compiler.
Im testing this later at home with Visual studio.

mathisloge · 2019-08-08T12:03:44Z

I have to mention, that the ubx implemention runs smoothly and one custom protocol also.
Its really strange, because the specific constellation with Test37 just wont dispatch the Handler functions.

mathisloge · 2019-08-08T14:32:38Z

Just tested the same code from the test repo in VisualStudio 19, with c++ 11,14,17.
With this compiler the Handler functions are called.

So I´m guessing it is something with the gcc compiler and embedded system?

mathisloge · 2019-08-08T18:12:13Z

So i´ve now tried a couple of things:

used debug1 code
linked it as a static lib
downloaded newest embedded toolchain https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads

Everything results in a not called Handler function. :/

These are my buildflags:
-mcpu=cortex-m4 -march=armv7e-m -mthumb -mfloat-abi=hard -mfpu=fpv4-sp-d16 -fno-exceptions -DPART_TM4C1294NCPDT -Dgcc -I"C:/Users/mathi/workspace_v9/comtest" -I"C:/Program Files (x86)/GNU Tools ARM Embedded/8 2019-q3-update/arm-none-eabi/include" -I"C:/Users/mathi/workspace_v9/commtestlib" -Og -ffunction-sections -fdata-sections -g -gdwarf-4 -gstrict-dwarf -Wall

I really dont know what is going on, simply because of the fact, that two implementations are working fine, but this doesnt.

arobenko · 2019-08-09T00:21:36Z

In general, I'd expect some specific compiler to fail compiling the code rather than compiling it in a wrong way. Because this is your particular compiler I won't be able to reproduce it. However, I might be able to guide your to find the reason and/or problematic location and probably you'll be able to workaround the problem (the COMMS library was designed with great flexibility to perform various compile time configurations)

First of all, I think that your code bombs out on message object creation attempt here

To verify my theory, please try to manually invoke the createMsg() member function of your frame:

auto msgPtr = frame.createMsg(MyProtocol::MsgId_TestId); 
if (!msgPtr) {
    ... // Report that there is a problem in message creation
}

If I'm wrong and the message object was successfully created, then there is a need to better understand what's going on in the loop I mentioned before, maybe you can also introduce retrieval of the "index" information.

std::size_t idx = std::numeric_limits<std::size_t>::max();
auto es = frame.read(msgptr, iter, len, comms::protocol::msgId(retrievedId), comms::protocol::msgIndex(idx));
... /* print value of idx */

If the value of idx is greater than 0, than probably the further read has failed (after the message object was created properly), but I think this is not the case.

If the message object cannot be created (most likely), then try playing with MsgFactory, which is used by the comms::protocol::MsgIdLayer class and reproduce the failure with it:

using MyFactory = comms::MsgFactory<InMessage, AllInMessages>;
MyFactory factory;
auto msgPtr = factory.createMsg(MyProtocol::MsgId_TestId); 
if (!msgPtr) {
    ... // Report that there is a problem in message creation
}

By default, the message factory uses comms::dispatchMsgType() to dispatch message id into the appropriate type. The function does compile time evaluation of the tuple with supported message types and their IDs and tries to choose between "polymorphic" and "static-bin-search' way of dispatching. If number of messages is small and/or not many gaps between message IDs (no more than 10% holes between the IDs), then "polymorphic" may is chosen. You can try to force it into using "static-bin-search" by passing comms::option::app::ForceDispatchStaticBinSearch option to the factory definition and see whether the failure still happens:

using MyFactory = comms::MsgFactory<InMessage, AllInMessages, comms::option::app::ForceDispatchStaticBinSearch>;

If such creation works now, here is your work around, you'll need to pass comms::option::app::ForceDispatchStaticBinSearch option to your "Id" layer of the frame, (just replace usage of default MyProtocol::options::DefaultOptions, with your class, that modifies appropriate type.

For the sake of completion, you may also try to have multiple message IDs with their types (not only single "TestId" one) and see whether the message creation starts working. If it does, it may lead me to better understanding why wrong code has been generated and see how I can improve the code of the COMMS library by some refactoring and/or introducing more static_asserts to force compile time rather than runtime errors.

mathisloge · 2019-08-09T05:41:03Z

Hey Alex,
thank you so much for guiding me through this!

I can confirm, that it works, if I using this option:

If such creation works now, here is your work around, you'll need to pass comms::option::app::ForceDispatchStaticBinSearch option to your "Id" layer of the frame, (just replace usage of default MyProtocol::options::DefaultOptions, with your class, that modifies appropriate type.

Today i have to first move on with my work, but i will investigate this further on the weekend.

mathisloge · 2019-08-10T09:18:04Z

Since the same code wont compile under the TI compiler v18 i have opened there a ticket to address those issues. The armopt.exe results in a segmentation fault.

For tracking purposes here is the link : http://e2e.ti.com/support/tools/ccs/f/81/p/828521/3066091#3066091

mathisloge · 2019-08-12T12:21:39Z

Hey Alex, unfortunatly i didnt got so much time on the weekend.
But one thing I noticed is, that if i have these Messages:

    MsgId_A= 1,
    MsgId_B = 2, 
    MsgId_C= 3, 
    MsgId_D = 4,
    MsgId_E = 5, 
    MsgId_F = 6, 
    MsgId_G = 7, 
    MsgId_H = 8, 
    MsgId_I = 9, 
    MsgId_J = 16, 
    MsgId_K = 17

And take A,B and K as my InputMessages, it will always call the void handle(typename T::InMessage& msg) which is the default handler, if no other handler is found.

If im choosing all messages as InMessages, it will call the correct handler or if im only using e.g. A and B.
Ive tried it with the StaticBinSearch. The PolymorpicSearch does not work with all or only two or three messages.

arobenko · 2019-08-12T22:36:30Z

I'll do some extra testing myself.
Just to clarify, is it reproducible only with the arm compiler that you are using or also on your development PC? If you cannot reproduce the problem on your PC, there probably little I can help you with (it's probably the problem with your cross-compiler). However, if you can reproduce the problem with your development PC, it indicates a bug in the COMMS library, which I can definitely find and fix.

arobenko · 2019-08-13T07:37:42Z

Just added some extra unit testing for type dispatching and message creation in the COMMS library (develop branch) and everything works as expected.

$> make comms.DispatchTest
$> ./comms/test/comms.MsgFactoryTest
$> make comms.MsgFactoryTest
$> ./comms/test/comms.MsgFactoryTest

The results of Windows builds with testing should be here when complete and Linux builds with testing should be here.

If builds and testing not failing, the chances are your cross-compiler is a problem. Maybe you can create simple test application that works for PC and fails for cross compiler and report the issue to the compiler developers.

arobenko · 2019-08-30T04:45:07Z

Hi Mathis,
I haven't seen any activity from you for while. How is it going for you? Have you been able to properly define an integrate your protocol or you still experience some unresolved issues?

mathisloge · 2019-09-02T17:02:45Z

Hey Alex,
sorry for the late response. Currently im finalizing my bachelor thesis, so i will be back in a few weeks.

Just to give short update on this:
it is definitely a cross-compiler problem.

If such creation works now, here is your work around, you'll need to pass comms::option::app::ForceDispatchStaticBinSearch option to your "Id" layer of the frame, (just replace usage of default MyProtocol::options::DefaultOptions, with your class, that modifies appropriate type.

i can resolve the issue with this. Additionally im waiting for Texas Instruments to fix the segmentation fault issue. Who knows how long that will be :D

mathisloge · 2019-09-17T05:50:10Z

It looks like the TI-Team fixed the issue.
TI Bug Tracker with the Bug-Id CODEGEN-6581

So currently it isnt released but the bug tracker status is "fixed"

mathisloge · 2020-10-18T10:25:10Z

I've forgotten to close this. The bug was fixed by Texas Instruments. It is available in the ti-compiler version 20+

Link to ti bug tracker: https://sir.ext.ti.com/jira/browse/EXT_EP-9476?jql=text%20~%20%22CODEGEN-6581%22

arobenko added a commit that referenced this issue Aug 8, 2019

Extra test for reported issue #2.

2746dbf

arobenko added the question Further information is requested label Aug 10, 2019

mathisloge closed this as completed Oct 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Need Help #2

Need Help #2

mathisloge commented Aug 8, 2019

arobenko commented Aug 8, 2019 •

edited

Loading

mathisloge commented Aug 8, 2019 •

edited

Loading

arobenko commented Aug 8, 2019 via email

arobenko commented Aug 8, 2019 via email

arobenko commented Aug 8, 2019

arobenko commented Aug 8, 2019

arobenko commented Aug 8, 2019

mathisloge commented Aug 8, 2019

mathisloge commented Aug 8, 2019

mathisloge commented Aug 8, 2019 •

edited

Loading

mathisloge commented Aug 8, 2019 •

edited

Loading

arobenko commented Aug 9, 2019

mathisloge commented Aug 9, 2019

mathisloge commented Aug 10, 2019

mathisloge commented Aug 12, 2019 •

edited

Loading

arobenko commented Aug 12, 2019

arobenko commented Aug 13, 2019 •

edited

Loading

arobenko commented Aug 30, 2019

mathisloge commented Sep 2, 2019

mathisloge commented Sep 17, 2019

mathisloge commented Oct 18, 2020 •

edited

Loading

Need Help #2

Need Help #2

Comments

mathisloge commented Aug 8, 2019

arobenko commented Aug 8, 2019 • edited Loading

mathisloge commented Aug 8, 2019 • edited Loading

arobenko commented Aug 8, 2019 via email

arobenko commented Aug 8, 2019 via email

arobenko commented Aug 8, 2019

arobenko commented Aug 8, 2019

arobenko commented Aug 8, 2019

mathisloge commented Aug 8, 2019

mathisloge commented Aug 8, 2019

mathisloge commented Aug 8, 2019 • edited Loading

mathisloge commented Aug 8, 2019 • edited Loading

arobenko commented Aug 9, 2019

mathisloge commented Aug 9, 2019

mathisloge commented Aug 10, 2019

mathisloge commented Aug 12, 2019 • edited Loading

arobenko commented Aug 12, 2019

arobenko commented Aug 13, 2019 • edited Loading

arobenko commented Aug 30, 2019

mathisloge commented Sep 2, 2019

mathisloge commented Sep 17, 2019

mathisloge commented Oct 18, 2020 • edited Loading

arobenko commented Aug 8, 2019 •

edited

Loading

mathisloge commented Aug 8, 2019 •

edited

Loading

mathisloge commented Aug 8, 2019 •

edited

Loading

mathisloge commented Aug 8, 2019 •

edited

Loading

mathisloge commented Aug 12, 2019 •

edited

Loading

arobenko commented Aug 13, 2019 •

edited

Loading

mathisloge commented Oct 18, 2020 •

edited

Loading