Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protobuf 2.6.0 incompatibility issues with Python 3.x #7

Closed
KJaunay opened this issue Sep 2, 2014 · 18 comments
Closed

Protobuf 2.6.0 incompatibility issues with Python 3.x #7

KJaunay opened this issue Sep 2, 2014 · 18 comments

Comments

@KJaunay
Copy link

KJaunay commented Sep 2, 2014

I've run into a problem when using Python 3.4 and protobuf 2.6.0.
I get several errors including:
protobuf/python/google/protobuf/internal/decoder.py, line 167
_DecodeVarint = _VarintDecoder((1 << 64) - 1, long)
NameError: name 'long' is not defined

@xfxyjwf xfxyjwf added this to the Python 3 support milestone Sep 3, 2014
@gpshead
Copy link
Contributor

gpshead commented Sep 3, 2014

Blanket Python 3 support was unintentionally listed in these release notes. We need to fix that up for a 2.6.x release. The code itself is ready with some 2to3 use, the setup.py and deps are not done yet. Sorry about that!

Here's my summary:

  • Support for Python 3 is almost in place. Our setup.py does not yet do the right thing for 2to3 conversions with Python 3 but for the adventurous, manually run the following files through 2to3 and it should work in Python 3.3+:

protobuf/descriptor.py
protobuf/internal/cpp_message.py
protobuf/internal/decoder.py
protobuf/internal/python_message.py
protobuf/internal/type_checkers.py
protobuf/internal/message_cpp2_test.py
protobuf/internal/message_factory_test.py
protobuf/internal/message_factory_cpp2_test.py
protobuf/internal/message_factory_python_test.py
protobuf/internal/message_python_test.py
protobuf/internal/message_test.py
protobuf/internal/reflection_cpp2_generated_test.py
protobuf/internal/reflection_python_test.py
protobuf/internal/reflection_test.py
protobuf/internal/test_util.py
protobuf/internal/text_format_test.py
protobuf/message_factory.py
protobuf/text_encoding.py
protobuf/text_format.py

setup.py also specifies an external dep on google-apputils needed in order to run the unittests. We need to push out an updated version of that with Python 3 support.

@brettcannon
Copy link

Any specific reason for wanting to use 2to3 instead of something like https://github.com/python-modernize/python-modernize to do the transition once?

@mrovner
Copy link

mrovner commented Sep 6, 2014

It says
"Modernizes Python code for eventual Python 3 migration. Build on top of
2to3"

and it's not apparent why it's better ;)

Also 2to3 support is in setuptools already.

On Thu, Sep 4, 2014 at 6:59 AM, Brett Cannon notifications@github.com
wrote:

Any specific reason for wanting to use 2to3 instead of something like
https://github.com/python-modernize/python-modernize to do the transition
once?


Reply to this email directly or view it on GitHub
#7 (comment).

Thanks,
--Mike

@brettcannon
Copy link

By unifying the code base to work on both Python 2 and 3 it makes development easier and installation faster (unless you release separate Python 2 and 3 packages). By not having to wait for 2to3 you eliminate a compile step during development. As for installation, you can forgo that step entirely. Plus the coding practices used in converted source is more in line with modern best practices and syntax.

@Kentzo
Copy link

Kentzo commented Sep 26, 2014

I wonder if we can reuse code from my https://github.com/GreatFruitOmsk/protobuf-py3

@tseaver
Copy link
Contributor

tseaver commented Nov 18, 2014

I'm +1 on @Kentzo's approach, and would volunteer to help review #66. As @brettcannon says, avoiding 2to3 is pretty much the "widely accepted" best practice for libraries which need to support both Python2 and Python3: e.g., see Lennart Regebro's excellent outline of strategies.

@virtuald
Copy link

I would love to see a python 3 compatible official protobuf available by Christmas.

@virtuald
Copy link

FWIW, using the current trunk of protobuf, the only change I had to make to the package after installation was change a number of str("", "utf-8") references in descriptor_pb2.py. Oddly enough, the protobuf I have installed from homebrew (2.6.1) generates _pb2.py files with a function that looks like this:

_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))

And it uses that to encode strings instead like this:

default_value=_b("").decode('utf-8')

So.. not sure why the descriptor_pb2.py file is out of date. If someone could push those changes into that file, then I believe trunk would at least work on python3 -- but obviously #66 would be even better.

@tseaver
Copy link
Contributor

tseaver commented Nov 30, 2014

That function should not be encoding literals as latin under Py3k, especially since they are going to be immediately decoded as utf-8. Use of latin1 in anything but edgiest cases (decoding wonky / unknown data, or encoding for antique wire protocols) is a sign of papered-over trouble: "mojibake ahead", it could read.

@virtuald
Copy link

So, I stand corrected. When I install protobuf properly, it installs/runs from trunk without any noticable problems now. Python 3.4.2 on OSX. Now someone just needs to push an updated package to pypi... :)

It would be nice to get better python3 support in trunk to support the edge cases properly, as @tseaver mentions.

@Ivoz
Copy link
Contributor

Ivoz commented Dec 2, 2014

@gpshead would you be fine with only supporting python 2.6+? Current readme.txt mentions 2.4.

Single-source cross python support is, FTR, like 10x easier if you only support python 2.6+ and 3.3+.

@gpshead
Copy link
Contributor

gpshead commented Dec 14, 2014

Absolutely, 2.6 is a good minimum. Anything older is obsolete (I consider 2.6 obsolete as well, but it is easy enough to support).

One of the reasons we started with some files needing to be run through 2to3 is that we have a few internal things still running on the old App Engine 2.5 legacy runtime that needed proto2 code and using 2to3 made maintaining 2.5 compatibility easier. I believe we have dealt with those internally now but I need to double check. We should absolutely be able to make a single source 2.6, 2.7, >= 3.3+ compatible version in the future.

@tseaver
Copy link
Contributor

tseaver commented Jan 30, 2015

FWIW, #164 modernizes the python code to drop all the PY25 stuff.

@thanatos
Copy link

thanatos commented Jul 13, 2016

Why is this bug marked as Closed & obsolete? I'm currently trying to get a third-party package (heka_py) that uses protobuf to work, and near as I can tell, the latest version is not Python 3 compatible.

There's a package python3_protobuf on PyPI (I have no idea if this is official or not) but it is on Protobuf 2.5, and unfortunately Heka assumes 2.6; so that doesn't work. protobuf itself doesn't support 3, so that also doesn't work. Near as I can tell, protocol buffers and Python 3 don't mix.

@jleclanche
Copy link

@thanatos https://pypi.python.org/pypi/protobuf pip install protobuf==3.0.0b3 is python3-compatible.

@thanatos
Copy link

thanatos commented Jul 15, 2016

@thanatos https://pypi.python.org/pypi/protobuf pip install protobuf==3.0.0b3 is python3-compatible.

While technically true, my understanding is that the major number here is bumped because it isn't backwards-compatible with compiled protocol buffers from protobuf v2; empirically, this seems to be the case; e.g., <a Message derived class>.INTEGER doesn't exist in protobuf==3.0.0b3.

(Did it change to VALUE_INTEGER_FIELD_NUMBER? (Nope.) Perhaps if this is the only change that heka_py requries, I might be able to make a small patch to heka_py to make it protobuf==3.0.0b3 compatible? But the problem seems to still stand — if I pull in a library that relies on protobuf, that restricts me to legacy Python.)

@xfxyjwf
Copy link
Contributor

xfxyjwf commented Jul 15, 2016

On Fri, Jul 15, 2016 at 2:41 PM, Roy Wellington Ⅳ notifications@github.com
wrote:

@thanatos https://github.com/thanatos
https://pypi.python.org/pypi/protobuf pip install protobuf==3.0.0b3 is
python3-compatible.

While technically true, my understanding is that the major number here is
bumped because it isn't backwards-compatible with compiled protocol buffers
from protobuf v2; empirically, this seems to be the case; e.g., .INTEGER doesn't exist in protobuf==3.0.0b3.

We bumped the major version number because of the new proto3 syntax
support, not because it's incompatible. If you find incompatible issues
with old generated code, please help file a bug for it.


You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
#7 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AIJ6iiG4RJcNqEfQ0dyXn0benZXh40lIks5qV_6bgaJpZM4Cdsma
.

@ZubairKhan001
Copy link

i have compiled caffe on Ubuntu 16.04 LTS both with python 2 and python, it is really a hectic process for beginners, for caffe with python 2.7 make sure that your /.local/lib/python2.7/site-packages only contains a version same as you protoc version. protoc version can be checked by entering
protoc --version
for me i got problems with python 2.7 because by mistake i installed protbuf 3.0 alpha in python 2.7 site packages, so there was a protobuf error after removing it from python 2.7 site packages everything worked fine
also install python-dev version
apt-get install the python3.x-dev
and for python 3.5 make sure you have python 3.0 alpha, if you have both python 2.7 and python 3.x in the system use pip3 otherwise use pip

pip3 install protobuf==3.0.0-alpha-3

or another protobuf such as 3.5 version of protobuf.
also do this
pip3 install python-dateutil --upgrade
after that do
make clean
make all
make runtest
make pycaffe
after performing make all make runtest steps don't forget to do that
sudo ldconfig

hope everything works OK for you.

TeBoring pushed a commit to TeBoring/protobuf that referenced this issue Jan 19, 2019
JSON test, symbolic enum names in JSON, and a few improvements.
PierrickVoulet pushed a commit to PierrickVoulet/protobuf that referenced this issue May 28, 2020
PierrickVoulet pushed a commit to PierrickVoulet/protobuf that referenced this issue May 30, 2020
rschu1ze pushed a commit to ClickHouse/google-protobuf that referenced this issue Jun 1, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rinarakaki pushed a commit to rinarakaki/protobuf that referenced this issue Aug 30, 2023
This fixes an MSAN warning of the form:

```
WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7fc964f2a597 in arena_findroot third_party/upb/upb/mem/arena.c:64:3
    #1 0x7fc964f2a597 in upb_Arena_Free third_party/upb/upb/mem/arena.c:211:7
    protocolbuffers#2 0x7fc9d2af0028 in std::__msan::unique_ptr<upb_Arena, void (*)(upb_Arena*)>::~unique_ptr() third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__memory/unique_ptr.h:277:7
    protocolbuffers#3 0x7fc9d2aef7a2 in ~Arena third_party/upb/upb/upb.hpp:70:7
    protocolbuffers#4 0x7fc9d2aef7a2 in ~InlinedArena third_party/upb/upb/upb.hpp:97:7
    protocolbuffers#5 0x7fc9d2aef7a2 in Cpp_InlinedArena2_Test::TestBody() third_party/upb/upb/test/test_cpp.cc:187:1
    protocolbuffers#6 0x7fc97da78a57 in testing::Test::Run() third_party/googletest/googletest/src/gtest.cc:2695:5
    protocolbuffers#7 0x7fc97da7a3e8 in testing::TestInfo::Run() third_party/googletest/googletest/src/gtest.cc:2844:11
    protocolbuffers#8 0x7fc97da7b897 in testing::TestSuite::Run() third_party/googletest/googletest/src/gtest.cc:3003:30
    protocolbuffers#9 0x7fc97daa5136 in testing::internal::UnitTestImpl::RunAllTests() third_party/googletest/googletest/src/gtest.cc:5899:44
    protocolbuffers#10 0x7fc97daa455c in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> third_party/googletest/googletest/src/gtest.cc
    protocolbuffers#11 0x7fc97daa455c in testing::UnitTest::Run() third_party/googletest/googletest/src/gtest.cc:5464:10
    protocolbuffers#12 0x562a7fb876f0 in RUN_ALL_TESTS third_party/googletest/googletest/include/gtest/gtest.h:2329:73
    protocolbuffers#13 0x562a7fb876f0 in main testing/base/internal/gunit_main.cc:86:10
    protocolbuffers#14 0x7fc9ba9b7632 in __libc_start_main (/usr/grte/v5/lib64/libc.so.6+0x61632) (BuildId: 280088eab084c30a3992a9bce5c35b44)
    protocolbuffers#15 0x562a7fafdbe9 in _start /build/work/ab393f4ac612f9027aae6b1a7226027ba2a2/google3/blaze-out/k8-opt/bin/third_party/grte/v5_src/grte-scratch/BUILD/src/csu/../sysdeps/x86_64/start.S:120

  Member fields were destroyed
    #0 0x562a7fb0b13d in __sanitizer_dtor_callback_fields third_party/llvm/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:960:5
    #1 0x7fc9d2aef79a in ~InlinedArena third_party/upb/upb/upb.hpp:105:8
    protocolbuffers#2 0x7fc9d2aef79a in ~InlinedArena third_party/upb/upb/upb.hpp:97:7
    protocolbuffers#3 0x7fc9d2aef79a in Cpp_InlinedArena2_Test::TestBody() third_party/upb/upb/test/test_cpp.cc:187:1
    protocolbuffers#4 0x7fc97da78a57 in testing::Test::Run() third_party/googletest/googletest/src/gtest.cc:2695:5
    protocolbuffers#5 0x7fc97da7a3e8 in testing::TestInfo::Run() third_party/googletest/googletest/src/gtest.cc:2844:11
    protocolbuffers#6 0x7fc97da7b897 in testing::TestSuite::Run() third_party/googletest/googletest/src/gtest.cc:3003:30
    protocolbuffers#7 0x7fc97daa5136 in testing::internal::UnitTestImpl::RunAllTests() third_party/googletest/googletest/src/gtest.cc:5899:44
    protocolbuffers#8 0x7fc97daa455c in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> third_party/googletest/googletest/src/gtest.cc
    protocolbuffers#9 0x7fc97daa455c in testing::UnitTest::Run() third_party/googletest/googletest/src/gtest.cc:5464:10
    protocolbuffers#10 0x562a7fb876f0 in RUN_ALL_TESTS third_party/googletest/googletest/include/gtest/gtest.h:2329:73
    protocolbuffers#11 0x562a7fb876f0 in main testing/base/internal/gunit_main.cc:86:10
    protocolbuffers#12 0x7fc9ba9b7632 in __libc_start_main (/usr/grte/v5/lib64/libc.so.6+0x61632) (BuildId: 280088eab084c30a3992a9bce5c35b44)
    protocolbuffers#13 0x562a7fafdbe9 in _start /build/work/ab393f4ac612f9027aae6b1a7226027ba2a2/google3/blaze-out/k8-opt/bin/third_party/grte/v5_src/grte-scratch/BUILD/src/csu/../sysdeps/x86_64/start.S:120
```

PiperOrigin-RevId: 511849224
rschu1ze pushed a commit to ClickHouse/google-protobuf that referenced this issue Oct 24, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rschu1ze pushed a commit to ClickHouse/google-protobuf that referenced this issue Oct 24, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rschu1ze pushed a commit to ClickHouse/google-protobuf that referenced this issue Oct 25, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rschu1ze pushed a commit to ClickHouse/google-protobuf that referenced this issue Nov 12, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rschu1ze pushed a commit to ClickHouse/google-protobuf that referenced this issue Nov 15, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rschu1ze pushed a commit to ClickHouse/google-protobuf that referenced this issue Nov 16, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
rschu1ze pushed a commit to ClickHouse/google-protobuf that referenced this issue Nov 17, 2023
==500166==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xdd383e in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7
    #1 0xdd1eb2 in google::protobuf::compiler::DiskSourceTree::DiskFileToVirtualFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:431:51
    #2 0x540619 in google::protobuf::compiler::CommandLineInterface::MakeProtoProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1362:24
    #3 0x5314fb in google::protobuf::compiler::CommandLineInterface::MakeInputsBeProtoPathRelative(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1415:10
    #4 0x5314fb in google::protobuf::compiler::CommandLineInterface::InitializeDiskSourceTree(google::protobuf::compiler::DiskSourceTree*, google::protobuf::DescriptorDatabase*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1168:8
    #5 0x52541f in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) build_msan/./contrib/protobuf/src/google/protobuf/compiler/command_line_interface.cc:1026:10
    protocolbuffers#6 0x50f5d6 in google::protobuf::compiler::ProtobufMain(int, char**) build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:104:14
    protocolbuffers#7 0x50fe58 in main build_msan/./contrib/protobuf/src/google/protobuf/compiler/main.cc:112:10
    protocolbuffers#8 0x7f817143db24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
    protocolbuffers#9 0x48904d in _start (/mnt/ch/ClickHouse/build_msan/contrib/protobuf-cmake/protoc+0x48904d)

  Uninitialized value was created by an allocation of 'sb' in the stack frame of function '_ZN6google8protobuf8compiler14DiskSourceTree12OpenDiskFileERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE'
    #0 0xdd3320 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:492

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_msan/./contrib/protobuf/src/google/protobuf/compiler/importer.cc:504:7 in google::protobuf::compiler::DiskSourceTree::OpenDiskFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
Exiting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests