Crash when import a generated py file from .proto and contains extensions with repeated fields.

[owent]

What version of protobuf and what language are you using?
Version: v3.21.5, 3.21.3.
Language: Python
With pip package: 4.21.3 and 4.21.5

What operating system (Linux, Windows, ...) and version?

Linux, CentOS 7

What runtime / compiler are you using (e.g., python version or gcc version)

python 3.9
gcc 12

What did you do?

1. Generate py file of https://github.com/xresloader/xres-code-generator/blob/main/pb_extension/xrescode_extensions_v3.proto
2. Import that file

What did you expect to see

Importing success

What did you see instead?

Crash

#0  0x00007fd147b64b20 in upb_Array_Size () from /home/owent/workspace/tencent/pix/server/third_party/install/linux-x86_64-gnu/.modules/lib/python3.9/site-packages/google/_upb/_message.abi3.so
#1  0x00007fd147b51a79 in PyUpb_ExtensionDict_Contains () from /home/owent/workspace/tencent/pix/server/third_party/install/linux-x86_64-gnu/.modules/lib/python3.9/site-packages/google/_upb/_message.abi3.so
#2  0x00007fd14f3621d0 in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x7fd14753fa40, throwflag=<optimized out>) at Python/ceval.c:3036
#3  0x00007fd14f30df02 in _PyEval_EvalFrame (throwflag=0, f=0x7fd14753fa40, tstate=0x89e150) at ./Include/internal/pycore_ceval.h:40
#4  function_code_fastcall (tstate=0x89e150, co=<optimized out>, args=<optimized out>, nargs=4, globals=<optimized out>) at Objects/call.c:330
#5  0x00007fd14f3617d6 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0xca1e68, callable=0x7fd1473a8700, tstate=0x89e150) at ./Include/cpython/abstract.h:118
#6  PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0xca1e68, callable=0x7fd1473a8700) at ./Include/cpython/abstract.h:127
#7  call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x89e150) at Python/ceval.c:5077
#8  _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0xca1c80, throwflag=<optimized out>) at Python/ceval.c:3506
#9  0x00007fd14f3606cf in _PyEval_EvalFrame (throwflag=0, f=0xca1c80, tstate=0x89e150) at ./Include/internal/pycore_ceval.h:40
#10 _PyEval_EvalCode (tstate=tstate@entry=0x89e150, _co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=<optimized out>, kwnames=0x7fd147c1a058, kwargs=0x7fd147d564a8, kwcount=7, kwstep=1, 
    defs=0x7fd147b88e38, defcount=7, kwdefs=0x0, closure=0x0, name=0x7fd14f5ca530, qualname=0x7fd147c27e40) at Python/ceval.c:4329
#11 0x00007fd14f30dd69 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#12 0x00007fd14f30d27d in _PyObject_FastCallDictTstate (tstate=0x89e150, callable=0x7fd1473a88b0, args=<optimized out>, nargsf=<optimized out>, kwargs=<optimized out>) at Objects/call.c:129
#13 0x00007fd14f30e68b in _PyObject_Call_Prepend (tstate=tstate@entry=0x89e150, callable=callable@entry=0x7fd1473a88b0, obj=obj@entry=0x7fd147d0a4c0, args=args@entry=0x7fd147da0b20, kwargs=kwargs@entry=0x7fd147cdbac0) at Objects/call.c:489
#14 0x00007fd14f33f839 in slot_tp_init (self=0x7fd147d0a4c0, args=0x7fd147da0b20, kwds=0x7fd147cdbac0) at Objects/typeobject.c:6969
#15 0x00007fd14f33cd4b in type_call (type=<optimized out>, args=0x7fd147da0b20, kwds=0x7fd147cdbac0) at Objects/typeobject.c:1026
#16 0x00007fd14f30d553 in _PyObject_MakeTpCall (tstate=0x89e150, callable=0x91ad90, args=0x95ab48, nargs=<optimized out>, keywords=<optimized out>) at Objects/call.c:191
#17 0x00007fd14f365e20 in _PyObject_VectorcallTstate (kwnames=0x7fd147e45820, nargsf=<optimized out>, args=0x95ab48, callable=0x91ad90, tstate=<optimized out>) at ./Include/cpython/abstract.h:116
#18 _PyObject_VectorcallTstate (kwnames=0x7fd147e45820, nargsf=<optimized out>, args=0x95ab48, callable=0x91ad90, tstate=<optimized out>) at ./Include/cpython/abstract.h:103
#19 PyObject_Vectorcall (kwnames=0x7fd147e45820, nargsf=<optimized out>, args=0x95ab48, callable=0x91ad90) at ./Include/cpython/abstract.h:127
#20 call_function (kwnames=0x7fd147e45820, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=<optimized out>) at Python/ceval.c:5077
#21 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x95a910, throwflag=<optimized out>) at Python/ceval.c:3537
#22 0x00007fd14f3606cf in _PyEval_EvalFrame (throwflag=0, f=0x95a910, tstate=0x89e150) at ./Include/internal/pycore_ceval.h:40
#23 _PyEval_EvalCode (tstate=tstate@entry=0x89e150, _co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x8f3190, kwcount=0, kwstep=1, defs=0x0, 
    defcount=0, kwdefs=0x0, closure=0x0, name=0x7fd147f565f0, qualname=0x7fd147f565f0) at Python/ceval.c:4329
#24 0x00007fd14f30dd69 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#25 0x00007fd14f361550 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775808, args=0x8f3190, callable=0x7fd147ccfca0, tstate=0x89e150) at ./Include/cpython/abstract.h:118
#26 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775808, args=0x8f3190, callable=0x7fd147ccfca0) at ./Include/cpython/abstract.h:127
#27 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x89e150) at Python/ceval.c:5077
#28 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=0x8f3020, throwflag=<optimized out>) at Python/ceval.c:3520
#29 0x00007fd14f3606cf in _PyEval_EvalFrame (throwflag=0, f=0x8f3020, tstate=0x89e150) at ./Include/internal/pycore_ceval.h:40
#30 _PyEval_EvalCode (tstate=0x89e150, _co=<optimized out>, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0, kwargs=0x0, kwcount=0, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, 
    closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4329
#31 0x00007fd14f360382 in _PyEval_EvalCodeWithName (_co=<optimized out>, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=<optimized out>, kwargs=0x0, kwcount=0, kwstep=2, defs=0x0, 
    defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4361
#32 0x00007fd14f360323 in PyEval_EvalCodeEx (_co=<optimized out>, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0)
    at Python/ceval.c:4377
#33 0x00007fd14f3d20fb in PyEval_EvalCode (co=co@entry=0x7fd147db1500, globals=globals@entry=0x7fd147f50100, locals=locals@entry=0x7fd147f50100) at Python/ceval.c:828
#34 0x00007fd14f3e3123 in run_eval_code_obj (tstate=0x89e150, co=0x7fd147db1500, globals=0x7fd147f50100, locals=0x7fd147f50100) at Python/pythonrun.c:1221
#35 0x00007fd14f3e30aa in run_mod (mod=<optimized out>, filename=<optimized out>, globals=0x7fd147f50100, locals=0x7fd147f50100, flags=<optimized out>, arena=<optimized out>) at Python/pythonrun.c:1242
#36 0x00007fd14f2b77a1 in pyrun_file (fp=fp@entry=0x8d81f0, filename=filename@entry=0x7fd147d97b70, start=start@entry=257, globals=globals@entry=0x7fd147f50100, locals=locals@entry=0x7fd147f50100, closeit=closeit@entry=1, 
    flags=0x7ffe748d3e88) at Python/pythonrun.c:1140
#37 0x00007fd14f2b754d in pyrun_simple_file (flags=0x7ffe748d3e88, closeit=1, filename=0x7fd147d97b70, fp=0x8d81f0) at Python/pythonrun.c:450
#38 PyRun_SimpleFileExFlags (fp=fp@entry=0x8d81f0, filename=<optimized out>, closeit=closeit@entry=1, flags=flags@entry=0x7ffe748d3e88) at Python/pythonrun.c:483
#39 0x00007fd14f2b840d in PyRun_AnyFileExFlags (fp=fp@entry=0x8d81f0, filename=<optimized out>, closeit=closeit@entry=1, flags=flags@entry=0x7ffe748d3e88) at Python/pythonrun.c:92
#40 0x00007fd14f3eac02 in pymain_run_file (cf=0x7ffe748d3e88, config=0x89ac90) at Modules/main.c:373
#41 pymain_run_python (exitcode=0x7ffe748d3e80) at Modules/main.c:598
#42 Py_RunMain () at Modules/main.c:677
#43 0x00007fd14f3ea757 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:731
#44 0x00007fd14e4fe555 in __libc_start_main () from /lib64/libc.so.6
#45 0x000000000040108e in _start ()

Here is gdb information of corefile.

(gdb) f 0
#0  0x00007fd147b64b20 in upb_Array_Size () from /home/owent/workspace/tencent/pix/server/third_party/install/linux-x86_64-gnu/.modules/lib/python3.9/site-packages/google/_upb/_message.abi3.so
(gdb) info all-registers
rax            0x0                 0
rbx            0x7fd1473bb2f0      140536819987184
rcx            0x0                 0
rdx            0x7fd1477d77b0      140536824297392
rsi            0x9b8af0            10193648
rdi            0x0                 0
rbp            0x7fd14753fa40      0x7fd14753fa40
rsp            0x7ffe748d3288      0x7ffe748d3288
r8             0x243f6a8885a308d3  2611923443488327891
r9             0x3f                63
r10            0xfcabbfeccfd89b2a  -239887131313923286
r11            0x13198a2e03707344  1376283091369227076
r12            0x7fd147cdc06e      140536829558894
r13            0x7fd14753fbf0      140536821578736
r14            0xd03df0            13647344
r15            0xa40328            10748712
rip            0x7fd147b64b20      0x7fd147b64b20 <upb_Array_Size>

It seems PyUpb_ExtensionDict_Contains call upb_Array_Size(NULL) .( $rdi=0 )

[haberman]

Sorry for the slow reply.

I was not able to reproduce this on my machine. Are you still experiencing this crash? If so can you provide a repro, using Docker if necessary?

[owent]

Sorry， I can not reproduce this either with python 3.10. I will try python 3.9 tomorrow.

[github-actions]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[owent]

I can not reproduce it neither after upgrade python package.