Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using a pb_bytes_array_t of max size > 256 #510

Closed
ImaraSpeek opened this issue Apr 3, 2020 · 12 comments
Closed

Using a pb_bytes_array_t of max size > 256 #510

ImaraSpeek opened this issue Apr 3, 2020 · 12 comments

Comments

@ImaraSpeek
Copy link

Hi everyone!

I have defined a simple message that will contain a byte array with a Camera Capture.

message CameraCapture {
    bytes data = 1;     // max defined in options
}
# io.options
CameraCapture.data       max_size:4096

I want to statically assign a max size in the options file of 4096 bytes. We have a buffer available that can manage that type of statically allocated size. However when I compile the generated c code. I get errors saying

[3/17] Compiling C object higo_debug@exe/src_protobuf_generated_c_wrapper.pb.c.o.
FAILED: higo_debug@exe/src_protobuf_generated_c_wrapper.pb.c.o
arm-none-eabi-gcc @higo_debug@exe/src_protobuf_generated_c_wrapper.pb.c.o.rsp
In file included from ../../src/protobuf/generated_c/wrapper.pb.h:6,
                 from ../../src/protobuf/generated_c/wrapper.pb.c:4:
..\..\src\Third-Party/ProtobufNanopb/pb.h:155:57: error: size of array 'pb_static_assertion_FIELDINFO_DOES_NOT_FIT_width1_field14_9_12' is negative
  155 | #      define PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER) pb_static_assertion_##MSG##_##LINE##_##COUNTER
      |                                                         ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:154:56: note: in expansion of macro 'PB_STATIC_ASSERT_MSG_'
  154 | #      define PB_STATIC_ASSERT_MSG(MSG, LINE, COUNTER) PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER)
      |                                                        ^~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:153:55: note: in expansion of macro 'PB_STATIC_ASSERT_MSG'
  153 | #      define PB_STATIC_ASSERT(COND,MSG) typedef char PB_STATIC_ASSERT_MSG(MSG, __LINE__, __COUNTER__)[(COND)?1:-1];
      |                                                       ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:698:5: note: in expansion of macro 'PB_STATIC_ASSERT'
  698 |     PB_STATIC_ASSERT(PB_FITS(tag,6) && PB_FITS(data_offset,8) && PB_FITS(size_offset,4) && PB_FITS(data_size,4) && PB_FITS(array_size,1), FIELDINFO_DOES_NOT_FIT_width1_field ## tag)
      |     ^~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:526:5: note: in expansion of macro 'PB_FIELDINFO_ASSERT_1'
  526 |     PB_FIELDINFO_ASSERT_ ## width(tag, PB_ATYPE_ ## atype | PB_HTYPE_ ## htype | PB_LTYPE_MAP_ ## ltype, \
      |     ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:523:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT'
  523 |     PB_GEN_FIELD_INFO_ASSERT(width, structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:520:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO2'
  520 |     PB_GEN_FIELD_INFO_ASSERT_AUTO2(PB_FIELDINFO_WIDTH_AUTO(atype, htype, ltype), structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../src/protobuf/generated_c/wrapper.pb.h:85:1: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO'
   85 | X(a, STATIC,   SINGULAR, BOOL,     error,            14) \
      | ^
..\..\src\Third-Party/ProtobufNanopb/pb.h:478:5: note: in expansion of macro 'Wrapper_FIELDLIST'
  478 |     msgname ## _FIELDLIST(PB_GEN_FIELD_INFO_ASSERT_ ## width, structname)
      |     ^~~~~~~
../../src/protobuf/generated_c/wrapper.pb.c:9:1: note: in expansion of macro 'PB_BIND'
    9 | PB_BIND(Wrapper, Wrapper, AUTO)
      | ^~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:155:57: error: size of array 'pb_static_assertion_FIELDINFO_DOES_NOT_FIT_width1_field15_9_13' is negative
  155 | #      define PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER) pb_static_assertion_##MSG##_##LINE##_##COUNTER
      |                                                         ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:154:56: note: in expansion of macro 'PB_STATIC_ASSERT_MSG_'
  154 | #      define PB_STATIC_ASSERT_MSG(MSG, LINE, COUNTER) PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER)
      |                                                        ^~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:153:55: note: in expansion of macro 'PB_STATIC_ASSERT_MSG'
  153 | #      define PB_STATIC_ASSERT(COND,MSG) typedef char PB_STATIC_ASSERT_MSG(MSG, __LINE__, __COUNTER__)[(COND)?1:-1];
      |                                                       ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:698:5: note: in expansion of macro 'PB_STATIC_ASSERT'
  698 |     PB_STATIC_ASSERT(PB_FITS(tag,6) && PB_FITS(data_offset,8) && PB_FITS(size_offset,4) && PB_FITS(data_size,4) && PB_FITS(array_size,1), FIELDINFO_DOES_NOT_FIT_width1_field ## tag)
      |     ^~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:526:5: note: in expansion of macro 'PB_FIELDINFO_ASSERT_1'
  526 |     PB_FIELDINFO_ASSERT_ ## width(tag, PB_ATYPE_ ## atype | PB_HTYPE_ ## htype | PB_LTYPE_MAP_ ## ltype, \
      |     ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:523:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT'
  523 |     PB_GEN_FIELD_INFO_ASSERT(width, structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:520:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO2'
  520 |     PB_GEN_FIELD_INFO_ASSERT_AUTO2(PB_FIELDINFO_WIDTH_AUTO(atype, htype, ltype), structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../src/protobuf/generated_c/wrapper.pb.h:86:1: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO'
   86 | X(a, STATIC,   SINGULAR, INT32,    end,              15)
      | ^
..\..\src\Third-Party/ProtobufNanopb/pb.h:478:5: note: in expansion of macro 'Wrapper_FIELDLIST'
  478 |     msgname ## _FIELDLIST(PB_GEN_FIELD_INFO_ASSERT_ ## width, structname)
      |     ^~~~~~~
../../src/protobuf/generated_c/wrapper.pb.c:9:1: note: in expansion of macro 'PB_BIND'
    9 | PB_BIND(Wrapper, Wrapper, AUTO)
      | ^~~~~~~
[12/17] Compiling C object analysis_lib@sta/src_application_sercom_log.c.o.
ninja: build stopped: subcommand failed.
The terminal process terminated with exit code: 1

I thought that defining PB_FIELD_32_BIT in pb.h would be enough to support large byte arrays. The website mentions this for the PB_FIELD_32_BIT field: Add support for tag numbers > 65535 and fields larger than 65535 bytes or 65535 array entries. Increases code size 9 bytes per each field. Compiler error will tell if you need this.
I defined it in the pb.h field and did not change anything else, like such:

/* Add support for tag numbers > 65536 and fields larger than 65536 bytes. */
#define PB_FIELD_32BIT 1

Do I need to add the flag as well in the compiler command? Am I misinterpreting the use of this define? Is there any other way to work with a byte array of max 4096 bytes?

Thanks in advance!
Imara

@PetteriAimonen
Copy link
Member

Hmm, that's certainly a bug. With 0.4.x, it should really work up to 65535 without anything special.

As a workaround, try adding this line into your .options:

CameraCapture   descriptorsize:DS_4

@PetteriAimonen
Copy link
Member

Ah, now that I look more closely, the error appears to come from some "Wrapper" message. Try applying the descriptorsize option to that.

The issue is probably that the generator cannot find the CameraCapture message at generation time for some reason, and that causes it to allocate too little storage for its field descriptor.

@ImaraSpeek
Copy link
Author

Hey @PetteriAimonen. Thanks for the super quick reply.

Let me give a little more explanation about the setup. Because the messages are nested.

I have a wrapper:

message Wrapper {
  
   oneof payload {
      DebugCommand debug_command = 6;
      LogMessage log_message = 7;
      IO io = 8;
      Memory memory = 9;
      Command command = 10;
      Connectivity connectivity = 11;
   }
}

This message contains an IO message and an IO message can contain a CameraCapture like such:

message IO {

    oneof IO {
        SupplyRail supply_rail = 1;
        I2C i2c = 2;
        Charging charging = 3;
        DetectModule detect_module = 4;
        IMU imu = 5;
        RTCControl rtc_control = 6;
        Audio audio = 7;
        CameraMode camera_mode = 8;
        CameraCapture camera = 9;
        // LedBrightness led_brightness = 10;
        // IRSensor ir_sensor = 11;
    }
}

I don't think it's because of finding the CameraCapture message because if I change it's max_size in the options file to 200 instead of 4096, everything works perfectly.

I have now set the options file to:

CameraCapture.data       max_size:4096 descriptorsize:DS_4

And it is still giving me the same error. Any idea on what I'm doing wrong?

@ImaraSpeek
Copy link
Author

ImaraSpeek commented Apr 3, 2020

Actually, it will give me the error starting from a max_size > 206, not > 265 (seems a pretty random number to me, but perhaps it helps)

@PetteriAimonen
Copy link
Member

Try:

 Wrapper descriptorsize:DS_4

in the .options file for the wrapper message.

The problem is that the CameraCapture submessage inside Wrapper is growing beyond 256 bytes, but for some reason the generator doesn't notice it when generating the Wrapper message. Your compile errors all refer to the Wrapper message.

@ImaraSpeek
Copy link
Author

Ok. I did that.

# wrapper.options

Wrapper       descriptorsize:DS_4
# io.options

CameraCapture.data       max_size:4096 descriptorsize:DS_4

Still getting these errors

[10/22] Compiling C object higo_debug@exe/src_protobuf_generated_c_wrapper.pb.c.o.
FAILED: higo_debug@exe/src_protobuf_generated_c_wrapper.pb.c.o
arm-none-eabi-gcc @higo_debug@exe/src_protobuf_generated_c_wrapper.pb.c.o.rsp
In file included from ../../src/protobuf/generated_c/wrapper.pb.h:6,
                 from ../../src/protobuf/generated_c/wrapper.pb.c:4:
..\..\src\Third-Party/ProtobufNanopb/pb.h:160:57: error: size of array 'pb_static_assertion_FIELDINFO_DOES_NOT_FIT_width2_field8_9_9' is negative
  160 | #      define PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER) pb_static_assertion_##MSG##_##LINE##_##COUNTER
      |                                                         ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:159:56: note: in expansion of macro 'PB_STATIC_ASSERT_MSG_'
  159 | #      define PB_STATIC_ASSERT_MSG(MSG, LINE, COUNTER) PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER)
      |                                                        ^~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:158:55: note: in expansion of macro 'PB_STATIC_ASSERT_MSG'
  158 | #      define PB_STATIC_ASSERT(COND,MSG) typedef char PB_STATIC_ASSERT_MSG(MSG, __LINE__, __COUNTER__)[(COND)?1:-1];
      |                                                       ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:706:5: note: in expansion of macro 'PB_STATIC_ASSERT'
  706 |     PB_STATIC_ASSERT(PB_FITS(tag,10) && PB_FITS(data_offset,16) && PB_FITS(size_offset,4) && PB_FITS(data_size,12) && PB_FITS(array_size,12), FIELDINFO_DOES_NOT_FIT_width2_field ## tag)
      |     ^~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:531:5: note: in expansion of macro 'PB_FIELDINFO_ASSERT_2'
  531 |     PB_FIELDINFO_ASSERT_ ## width(tag, PB_ATYPE_ ## atype | PB_HTYPE_ ## htype | PB_LTYPE_MAP_ ## ltype, \
      |     ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:528:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT'
  528 |     PB_GEN_FIELD_INFO_ASSERT(width, structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:525:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO2'
  525 |     PB_GEN_FIELD_INFO_ASSERT_AUTO2(PB_FIELDINFO_WIDTH_AUTO(atype, htype, ltype), structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../src/protobuf/generated_c/wrapper.pb.h:84:1: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO'
   84 | X(a, STATIC,   ONEOF,    MSG_W_CB, (payload,io,payload.io),   8) \
      | ^
..\..\src\Third-Party/ProtobufNanopb/pb.h:483:5: note: in expansion of macro 'Wrapper_FIELDLIST'
  483 |     msgname ## _FIELDLIST(PB_GEN_FIELD_INFO_ASSERT_ ## width, structname)
      |     ^~~~~~~
../../src/protobuf/generated_c/wrapper.pb.c:9:1: note: in expansion of macro 'PB_BIND'
    9 | PB_BIND(Wrapper, Wrapper, AUTO)
      | ^~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:160:57: error: size of array 'pb_static_assertion_FIELDINFO_DOES_NOT_FIT_width2_field9_9_10' is negative
  160 | #      define PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER) pb_static_assertion_##MSG##_##LINE##_##COUNTER
      |                                                         ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:159:56: note: in expansion of macro 'PB_STATIC_ASSERT_MSG_'
  159 | #      define PB_STATIC_ASSERT_MSG(MSG, LINE, COUNTER) PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER)
      |                                                        ^~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:158:55: note: in expansion of macro 'PB_STATIC_ASSERT_MSG'
  158 | #      define PB_STATIC_ASSERT(COND,MSG) typedef char PB_STATIC_ASSERT_MSG(MSG, __LINE__, __COUNTER__)[(COND)?1:-1];
      |                                                       ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:706:5: note: in expansion of macro 'PB_STATIC_ASSERT'
  706 |     PB_STATIC_ASSERT(PB_FITS(tag,10) && PB_FITS(data_offset,16) && PB_FITS(size_offset,4) && PB_FITS(data_size,12) && PB_FITS(array_size,12), FIELDINFO_DOES_NOT_FIT_width2_field ## tag)
      |     ^~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:531:5: note: in expansion of macro 'PB_FIELDINFO_ASSERT_2'
  531 |     PB_FIELDINFO_ASSERT_ ## width(tag, PB_ATYPE_ ## atype | PB_HTYPE_ ## htype | PB_LTYPE_MAP_ ## ltype, \
      |     ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:528:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT'
  528 |     PB_GEN_FIELD_INFO_ASSERT(width, structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:525:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO2'
  525 |     PB_GEN_FIELD_INFO_ASSERT_AUTO2(PB_FIELDINFO_WIDTH_AUTO(atype, htype, ltype), structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../src/protobuf/generated_c/wrapper.pb.h:85:1: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO'
   85 | X(a, STATIC,   ONEOF,    MSG_W_CB, (payload,memory,payload.memory),   9) \
      | ^
..\..\src\Third-Party/ProtobufNanopb/pb.h:483:5: note: in expansion of macro 'Wrapper_FIELDLIST'
  483 |     msgname ## _FIELDLIST(PB_GEN_FIELD_INFO_ASSERT_ ## width, structname)
      |     ^~~~~~~
../../src/protobuf/generated_c/wrapper.pb.c:9:1: note: in expansion of macro 'PB_BIND'
    9 | PB_BIND(Wrapper, Wrapper, AUTO)
      | ^~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:160:57: error: size of array 'pb_static_assertion_FIELDINFO_DOES_NOT_FIT_width1_field14_9_13' is negative
  160 | #      define PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER) pb_static_assertion_##MSG##_##LINE##_##COUNTER
      |                                                         ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:159:56: note: in expansion of macro 'PB_STATIC_ASSERT_MSG_'
  159 | #      define PB_STATIC_ASSERT_MSG(MSG, LINE, COUNTER) PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER)
      |                                                        ^~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:158:55: note: in expansion of macro 'PB_STATIC_ASSERT_MSG'
  158 | #      define PB_STATIC_ASSERT(COND,MSG) typedef char PB_STATIC_ASSERT_MSG(MSG, __LINE__, __COUNTER__)[(COND)?1:-1];
      |                                                       ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:703:5: note: in expansion of macro 'PB_STATIC_ASSERT'
  703 |     PB_STATIC_ASSERT(PB_FITS(tag,6) && PB_FITS(data_offset,8) && PB_FITS(size_offset,4) && PB_FITS(data_size,4) && PB_FITS(array_size,1), FIELDINFO_DOES_NOT_FIT_width1_field ## tag)
      |     ^~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:531:5: note: in expansion of macro 'PB_FIELDINFO_ASSERT_1'
  531 |     PB_FIELDINFO_ASSERT_ ## width(tag, PB_ATYPE_ ## atype | PB_HTYPE_ ## htype | PB_LTYPE_MAP_ ## ltype, \
      |     ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:528:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT'
  528 |     PB_GEN_FIELD_INFO_ASSERT(width, structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:525:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO2'
  525 |     PB_GEN_FIELD_INFO_ASSERT_AUTO2(PB_FIELDINFO_WIDTH_AUTO(atype, htype, ltype), structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../src/protobuf/generated_c/wrapper.pb.h:88:1: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO'
   88 | X(a, STATIC,   SINGULAR, BOOL,     error,            14) \
      | ^
..\..\src\Third-Party/ProtobufNanopb/pb.h:483:5: note: in expansion of macro 'Wrapper_FIELDLIST'
  483 |     msgname ## _FIELDLIST(PB_GEN_FIELD_INFO_ASSERT_ ## width, structname)
      |     ^~~~~~~
../../src/protobuf/generated_c/wrapper.pb.c:9:1: note: in expansion of macro 'PB_BIND'
    9 | PB_BIND(Wrapper, Wrapper, AUTO)
      | ^~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:160:57: error: size of array 'pb_static_assertion_FIELDINFO_DOES_NOT_FIT_width1_field15_9_14' is negative
  160 | #      define PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER) pb_static_assertion_##MSG##_##LINE##_##COUNTER
      |                                                         ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:159:56: note: in expansion of macro 'PB_STATIC_ASSERT_MSG_'
  159 | #      define PB_STATIC_ASSERT_MSG(MSG, LINE, COUNTER) PB_STATIC_ASSERT_MSG_(MSG, LINE, COUNTER)
      |                                                        ^~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:158:55: note: in expansion of macro 'PB_STATIC_ASSERT_MSG'
  158 | #      define PB_STATIC_ASSERT(COND,MSG) typedef char PB_STATIC_ASSERT_MSG(MSG, __LINE__, __COUNTER__)[(COND)?1:-1];
      |                                                       ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:703:5: note: in expansion of macro 'PB_STATIC_ASSERT'
  703 |     PB_STATIC_ASSERT(PB_FITS(tag,6) && PB_FITS(data_offset,8) && PB_FITS(size_offset,4) && PB_FITS(data_size,4) && PB_FITS(array_size,1), FIELDINFO_DOES_NOT_FIT_width1_field ## tag)
      |     ^~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:531:5: note: in expansion of macro 'PB_FIELDINFO_ASSERT_1'
  531 |     PB_FIELDINFO_ASSERT_ ## width(tag, PB_ATYPE_ ## atype | PB_HTYPE_ ## htype | PB_LTYPE_MAP_ ## ltype, \
      |     ^~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:528:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT'
  528 |     PB_GEN_FIELD_INFO_ASSERT(width, structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~
..\..\src\Third-Party/ProtobufNanopb/pb.h:525:5: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO2'
  525 |     PB_GEN_FIELD_INFO_ASSERT_AUTO2(PB_FIELDINFO_WIDTH_AUTO(atype, htype, ltype), structname, atype, htype, ltype, fieldname, tag)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../src/protobuf/generated_c/wrapper.pb.h:89:1: note: in expansion of macro 'PB_GEN_FIELD_INFO_ASSERT_AUTO'
   89 | X(a, STATIC,   SINGULAR, INT32,    end,              15)
      | ^
..\..\src\Third-Party/ProtobufNanopb/pb.h:483:5: note: in expansion of macro 'Wrapper_FIELDLIST'
  483 |     msgname ## _FIELDLIST(PB_GEN_FIELD_INFO_ASSERT_ ## width, structname)
      |     ^~~~~~~
../../src/protobuf/generated_c/wrapper.pb.c:9:1: note: in expansion of macro 'PB_BIND'
    9 | PB_BIND(Wrapper, Wrapper, AUTO)
      | ^~~~~~~
[18/22] Compiling C object analysis_lib@sta/src_application_sercom.c.o.
ninja: build stopped: subcommand failed.
The terminal process terminated with exit code: 1

@PetteriAimonen
Copy link
Member

 PB_BIND(Wrapper, Wrapper, AUTO)

For some reason the option doesn't appear to have applied, did you regenerate wrapper.pb.c file?

@ImaraSpeek
Copy link
Author

Ah no! Apologies, I changed the compile command to generate the c file to an empty folder when adding the options file. I corrected it, and it has generated a correct C file and I'm not getting the error anymore. Thanks so much!

Do I need to add this option to every proto file that uses large byte arrays or just the wrapper? And do I still need to keep the define in pb.h?

@PetteriAimonen
Copy link
Member

The PB_FIELD_32BIT define is not needed unless you go over 64 kbytes.

Normally the generator should automatically detect the need, and that seems to be working correctly inside io.proto. The only reason why it is needed for wrapper.proto is that it includes the large message that the generator cannot find for one reason or other.

It may work directly without any options if you generate all your .proto files in one step, e.g:

 python3 ...../nanopb_generator.py io.proto wrapper.proto

That makes sure the generator has all necessary files available.

@ImaraSpeek
Copy link
Author

yeah, I feel like I'm doing that in a suboptimal way now. I made a .bat file that generated the output like so:

protoc ^
--proto_path=../src/protobuf ^
--proto_path=../src/Third-Party/ProtobufNanopb ^
--proto_path=../src/Third-Party/ProtobufNanopb/generator/proto ^
--python_out=../src/protobuf/generated_python ^
--nanopb_out="--options-file=../src/protobuf/wrapper.options":../src/protobuf/generated_c ^
wrapper.proto %*

protoc ^
--proto_path=../src/protobuf ^
--proto_path=../src/Third-Party/ProtobufNanopb ^
--proto_path=../src/Third-Party/ProtobufNanopb/generator/proto ^
--python_out=../src/protobuf/generated_python ^
--nanopb_out="--options-file=../src/protobuf/debug_commands.options":../src/protobuf/generated_c ^
debug_commands.proto %*

protoc ^
--proto_path=../src/protobuf ^
--proto_path=../src/Third-Party/ProtobufNanopb ^
--proto_path=../src/Third-Party/ProtobufNanopb/generator/proto ^
--python_out=../src/protobuf/generated_python ^
--nanopb_out="--options-file=../src/protobuf/log_messages.options":../src/protobuf/generated_c ^
log_messages.proto %*

protoc ^
--proto_path=../src/protobuf ^
--proto_path=../src/Third-Party/ProtobufNanopb ^
--proto_path=../src/Third-Party/ProtobufNanopb/generator/proto ^
--python_out=../src/protobuf/generated_python ^
--nanopb_out="--options-file=../src/protobuf/io.options":../src/protobuf/generated_c ^
io.proto %*

protoc ^
--proto_path=../src/protobuf ^
--proto_path=../src/Third-Party/ProtobufNanopb ^
--proto_path=../src/Third-Party/ProtobufNanopb/generator/proto ^
--python_out=../src/protobuf/generated_python ^
--nanopb_out="--options-file=../src/protobuf/memory.options":../src/protobuf/generated_c ^
memory.proto %*

protoc ^
--proto_path=../src/protobuf ^
--proto_path=../src/Third-Party/ProtobufNanopb ^
--proto_path=../src/Third-Party/ProtobufNanopb/generator/proto ^
--python_out=../src/protobuf/generated_python ^
--nanopb_out="--options-file=../src/protobuf/commands.options":../src/protobuf/generated_c ^
commands.proto %*

protoc ^
--proto_path=../src/protobuf ^
--proto_path=../src/Third-Party/ProtobufNanopb ^
--proto_path=../src/Third-Party/ProtobufNanopb/generator/proto ^
--python_out=../src/protobuf/generated_python ^
--nanopb_out="--options-file=../src/protobuf/connectivity.options":../src/protobuf/generated_c ^
connectivity.proto %*

Is there a way to generate all the proto files in 1 go ensuring that they use their options file? I couldn't manage to have the compiler find the correct option files without specifying them myself.

@PetteriAimonen
Copy link
Member

It is annoying indeed. Protoc doesn't pass the --proto-path to plugins, so finding the options file is a bit difficult.

You could try this: --nanopb_out="--options-path=../src/protobuf/":../src/protobuf/generated_c and then it wouldn't need the filename at least.

@ImaraSpeek
Copy link
Author

great! Well that's an optimization. Thanks for all the help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants