-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to build a simple S3 Client #14
Comments
I am using clang 3.4.2 on RHEL 7, if that matters.
|
Incomplete definition means there is a forward declaration and no formal You need to #include <aws/s3/model/GetObjectRequest.h> For our custom memory management api, you pass an allocation tag if you
|
Thanks @JonathanHenson! I should have spotted that. I am returning to C++ after a long time and I guess my C++ skills have dulled a bit. :-( Is the SDK documented anywhere? Even a beta version of the documentation would do. I can certainly dig around in the source code to figure things out, but documentation would make life a little easier. :-) Thanks again! |
No problem, the number of c++ experts in the world == the standards There is some documentation of core features in the readme. The We are considering adding doxygen generation in the near future.
|
That would be great! Thanks again. |
…ance-test-framework to develop # By Konrad Zemek (36) and Tomasz Lichon (11) # Via Konrad Zemek (5) and others * commit 'f83ff7e5c1b129cda5e5fc3a4981ac295ee7f4fd': (47 commits) Add doxygen package to the builder. VFS-1006 Update Docker images with new project dependencies. VFS-1010 Update docker templates. VFS-1000 Stop fetching deps on each make.py Merge commit 'a08ab98ad59134d72baab33cc80976a0f14c85b5' into feature/VFS-1010-acceptance-test-framework VFS-1010 Don't shed root privileges on non-Linux platforms. VFS-1000 Add newer protobuf to builder image. ct tests fix VFS-1010 Run scripts in make.py as a running user. reorganize gen_dev helper modules Merge commit 'c34bb9d937883c03028e770146f60d809c22db0d' into feature/VFS-1007-global-registry-gendev Merge commit 'c34bb9d937883c03028e770146f60d809c22db0d' into feature/VFS-1007-global-registry-gendev VFS-1000 Document python scripts. improve compilation Merge branch 'feature/VFS-998-refactoring-disp-wh-ccm-nm' into feature/VFS-1007-global-registry-gendev VFS-1010 Cleanup scripts associated with running distributed tests. VFS-1010 Cleanup scripts associated with running distributed tests. naming changes preety -> pretty remove long lines ...
efb6bc2 VFS-1172, merge with develop 675751c VFS-1172, merge with develop b0f8eff Merge commit 'a330d9e8c14c2f2792da596af6fce0b398f137a1' into feature/VFS-1172-move-provider-registration-logic a330d9e Squashed 'bamboos/' changes from 72b50f4..950ffd4 e8b48a1 Merge commit 'ad085dac366e145960e0ca1ff5d9d411a20396da' into feature/VFS-1172-move-provider-registration-logic ad085da Squashed 'bamboos/' changes from 85d7862..72b50f4 fab59d4 Merge pull request aws#14 in VFS/op-ccm from VFS-1164_2 to develop 39dcd5b VFS-1174, minor cover update a4a589e Merge commit '62a5d82d342c759e8ec8ea89d308064a53bcbe54' into feature/VFS-1172-move-provider-registration-logic 62a5d82 Squashed 'bamboos/' changes from b9e5c93..85d7862 9347456 Merge commit '36e4ea3b6dad8ff15109edb242024834f12e57d7' into feature/VFS-1172-move-provider-registration-logic 36e4ea3 Squashed 'bamboos/' changes from 725768f..b9e5c93 6a797aa Merge commit '22223d207b7f70910abb7099d0e1064902c90702' into feature/VFS-1172-move-provider-registration-logic 22223d2 Squashed 'bamboos/' changes from caac3db..725768f a3f12a3 Merge pull request aws#13 in VFS/op-ccm from VFS-1164 to develop c2d9104 Merge commit '30d8c21959fd047db17dbed5a97fbaa66a4ea44a' into feature/VFS-1172-move-provider-registration-logic 30d8c21 Squashed 'bamboos/' changes from e8250b3..caac3db e795cbc VFS-1164, stress tests deps update 2b794ee VFS-1174, cover deps update 20f4732 Merge remote-tracking branch 'origin/develop' into feature/VFS-1174-cover-erlang-improvments e896c05 VFS-1174, cover minor update 5a4511e Merge commit 'd3386a159c95a9841aeef9c2c4dafdcf1d703961' into feature/VFS-1172-move-provider-registration-logic d3386a1 Squashed 'bamboos/' changes from 4413c96..e8250b3 ea695dc Merge pull request aws#11 in VFS/op-ccm from feature/VFS-1145-ssl2-integrated-with-oneprovider to develop a714fcf VFS-1174, Cover analysis deps update 61181b1 VFS-1174, Cover analysis update git-subtree-dir: op_ccm git-subtree-split: efb6bc2ab8a981e9c3c7a50599754f1088a33c4f
Changes to support VS2015 builds; requires a code regen
Looks like clang does some strange optimizations after which this code becomes invalid under multithreaded access. Here what adders sanitizer says when trying to download a file: ``` ================================================================= ==2219==ERROR: AddressSanitizer: heap-use-after-free on address 0x606000010ae0 at pc 0x000007cfac57 bp 0x7f4f0b4f4290 sp 0x7f4f0b4f4288 READ of size 4 at 0x606000010ae0 thread T6 #0 0x7cfac56 in std::__y1::less<int>::operator()(int const&, int const&) const /contrib/libs/cxxsupp/libcxx/include/__functional_base:55:17 aws#1 0x7cfac56 in std::__y1::__map_value_compare<int, std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, std::__y1::less<int>, true>::operator()(std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > > const&, int const&) const /contrib/libs/cxxsupp/libcxx/include/map:516 aws#2 0x7cfac56 in std::__y1::__tree_iterator<std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, std::__y1::__tree_node<std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, void*>*, long> std::__y1::__tree<std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, std::__y1::__map_value_compare<int, std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, std::__y1::less<int>, true>, std::__y1::allocator<std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > > > >::__lower_bound<int>(int const&, std::__y1::__tree_node<std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, void*>*, std::__y1::__tree_end_node<std::__y1::__tree_node_base<void*>*>*) /contrib/libs/cxxsupp/libcxx/include/__tree:2676 aws#3 0x7cfac56 in std::__y1::__tree_iterator<std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, std::__y1::__tree_node<std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, void*>*, long> std::__y1::__tree<std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, std::__y1::__map_value_compare<int, std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > >, std::__y1::less<int>, true>, std::__y1::allocator<std::__y1::__value_type<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > > > >::find<int>(int const&) /contrib/libs/cxxsupp/libcxx/include/__tree:2605 aws#4 0x7cfac56 in std::__y1::map<int, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> >, std::__y1::less<int>, std::__y1::allocator<std::__y1::pair<int const, std::__y1::basic_string<char, std::__y1::char_traits<char>, std::__y1::allocator<char> > > > >::find(int const&) /contrib/libs/cxxsupp/libcxx/include/map:1378 aws#5 0x7cfac56 in Aws::Monitoring::GetHttpClientMetricNameByType(Aws::Monitoring::HttpClientMetricsType) /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/source/monitoring/HttpClientMetrics.cpp:72 aws#6 0x7cf33b0 in Aws::Http::CurlHttpClient::MakeRequestInternal(Aws::Http::HttpRequest&, std::__y1::shared_ptr<Aws::Http::Standard::StandardHttpResponse>&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/source/http/curl/CurlHttpClient.cpp:495:38 aws#7 0x7cf8792 in Aws::Http::CurlHttpClient::MakeRequest(std::__y1::shared_ptr<Aws::Http::HttpRequest> const&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/source/http/curl/CurlHttpClient.cpp:545:5 aws#8 0x7d61be2 in Aws::Client::AWSClient::AttemptOneRequest(std::__y1::shared_ptr<Aws::Http::HttpRequest> const&, Aws::AmazonWebServiceRequest const&, char const*) const /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/source/client/AWSClient.cpp:304:23 aws#9 0x7d5cf37 in Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*) const /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/source/client/AWSClient.cpp:183:19 aws#10 0x7d6b4c4 in Aws::Client::AWSClient::MakeRequestWithUnparsedResponse(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*) const /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/source/client/AWSClient.cpp:360:47 aws#11 0x797332b in Aws::S3::S3Client::GetObject(Aws::S3::Model::GetObjectRequest const&) const /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-s3/source/S3Client.cpp:1820:27 aws#12 0x7e6fc10 in Aws::Transfer::TransferManager::DoSinglePartDownload(std::__y1::shared_ptr<Aws::Transfer::TransferHandle> const&) /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-transfer/source/transfer/TransferManager.cpp:668:64 aws#13 0x7e7436c in Aws::Transfer::TransferManager::DoDownload(std::__y1::shared_ptr<Aws::Transfer::TransferHandle> const&) /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-transfer/source/transfer/TransferManager.cpp:777:17 aws#14 0x7d2ed80 in std::__y1::__function::__value_func<void ()>::operator()() const /contrib/libs/cxxsupp/libcxx/include/functional:1860:16 aws#15 0x7d2ed80 in std::__y1::function<void ()>::operator()() const /contrib/libs/cxxsupp/libcxx/include/functional:2426 aws#16 0x7d2ed80 in Aws::Utils::Threading::ThreadTask::MainTaskRunner() /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/source/utils/threading/ThreadTask.cpp:41 aws#17 0x7d2f287 in decltype(*(std::__y1::forward<Aws::Utils::Threading::ThreadTask*&>(fp0)).*fp()) std::__y1::__invoke<void (Aws::Utils::Threading::ThreadTask::*&)(), Aws::Utils::Threading::ThreadTask*&, void>(void (Aws::Utils::Threading::ThreadTask::*&)(), Aws::Utils::Threading::ThreadTask*&) /contrib/libs/cxxsupp/libcxx/include/type_traits:3559:1 aws#18 0x7d2f287 in std::__y1::__bind_return<void (Aws::Utils::Threading::ThreadTask::*)(), std::__y1::tuple<Aws::Utils::Threading::ThreadTask*>, std::__y1::tuple<>, __is_valid_bind_return<void (Aws::Utils::Threading::ThreadTask::*)(), std::__y1::tuple<Aws::Utils::Threading::ThreadTask*>, std::__y1::tuple<> >::value>::type std::__y1::__apply_functor<void (Aws::Utils::Threading::ThreadTask::*)(), std::__y1::tuple<Aws::Utils::Threading::ThreadTask*>, 0ul, std::__y1::tuple<> >(void (Aws::Utils::Threading::ThreadTask::*&)(), std::__y1::tuple<Aws::Utils::Threading::ThreadTask*>&, std::__y1::__tuple_indices<0ul>, std::__y1::tuple<>&&) /contrib/libs/cxxsupp/libcxx/include/functional:2732 aws#19 0x7d2f287 in std::__y1::__bind_return<void (Aws::Utils::Threading::ThreadTask::*)(), std::__y1::tuple<Aws::Utils::Threading::ThreadTask*>, std::__y1::tuple<>, __is_valid_bind_return<void (Aws::Utils::Threading::ThreadTask::*)(), std::__y1::tuple<Aws::Utils::Threading::ThreadTask*>, std::__y1::tuple<> >::value>::type std::__y1::__bind<void (Aws::Utils::Threading::ThreadTask::*)(), Aws::Utils::Threading::ThreadTask*>::operator()<>() /contrib/libs/cxxsupp/libcxx/include/functional:2770 aws#20 0x7d2f287 in decltype(std::__y1::forward<std::__y1::__bind<void (Aws::Utils::Threading::ThreadTask::*)(), Aws::Utils::Threading::ThreadTask*> >(fp)()) std::__y1::__invoke<std::__y1::__bind<void (Aws::Utils::Threading::ThreadTask::*)(), Aws::Utils::Threading::ThreadTask*> >(std::__y1::__bind<void (Aws::Utils::Threading::ThreadTask::*)(), Aws::Utils::Threading::ThreadTask*>&&) /contrib/libs/cxxsupp/libcxx/include/type_traits:3618 aws#21 0x7d2f287 in void std::__y1::__thread_execute<std::__y1::unique_ptr<std::__y1::__thread_struct, std::__y1::default_delete<std::__y1::__thread_struct> >, std::__y1::__bind<void (Aws::Utils::Threading::ThreadTask::*)(), Aws::Utils::Threading::ThreadTask*> >(std::__y1::tuple<std::__y1::unique_ptr<std::__y1::__thread_struct, std::__y1::default_delete<std::__y1::__thread_struct> >, std::__y1::__bind<void (Aws::Utils::Threading::ThreadTask::*)(), Aws::Utils::Threading::ThreadTask*> >&, std::__y1::__tuple_indices<>) /contrib/libs/cxxsupp/libcxx/include/thread:343 aws#22 0x7d2f287 in void* std::__y1::__thread_proxy<std::__y1::tuple<std::__y1::unique_ptr<std::__y1::__thread_struct, std::__y1::default_delete<std::__y1::__thread_struct> >, std::__y1::__bind<void (Aws::Utils::Threading::ThreadTask::*)(), Aws::Utils::Threading::ThreadTask*> > >(void*) /contrib/libs/cxxsupp/libcxx/include/thread:353 aws#23 0x7f4f117f4668 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9668) aws#24 0x7f4f1171c322 in clone /build/glibc-4WA41p/glibc-2.30/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 0x606000010ae0 is located 32 bytes inside of 64-byte region [0x606000010ac0,0x606000010b00) freed by thread T0 here: #0 0x247bd32 in operator delete(void*) /place/sandbox-data/tasks/5/1/433792615/build/last/clang/src/projects/compiler-rt/lib/asan/asan_new_delete.cc:167:3 aws#1 0x7f4f11643ba6 in __run_exit_handlers /build/glibc-4WA41p/glibc-2.30/stdlib/exit.c:108:8 previously allocated by th ... cxx/include/functional:1860:16 aws#21 0x28f9d72 in std::__y1::function<void ()>::operator()() const /contrib/libs/cxxsupp/libcxx/include/functional:2426 aws#22 0x28f9d72 in TColoredProcessor::Run(std::__y1::function<void ()>, TBasicString<char, TCharTraits<char> > const&, char const*, bool) /library/unittest/utmain.cpp:475 aws#23 0x28c18ce in NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, TCharTraits<char> >, char const*, bool) /library/unittest/registar.cpp:375:18 aws#24 0x24bad72 in maps::factory::processing::tests::NTestSuitecloud_optimize_tasks_should::TCurrentTest::Execute() /maps/factory/processing/cloud_optimize/tests/tests.cpp:25:1 aws#25 0x28c3747 in NUnitTest::TTestFactory::Execute() /library/unittest/registar.cpp:484:19 aws#26 0x28f18ec in NUnitTest::RunMain(int, char**) /library/unittest/utmain.cpp:755:44 aws#27 0x7f4f116211e2 in __libc_start_main /build/glibc-4WA41p/glibc-2.30/csu/../csu/libc-start.c:308:16 Thread T4 created by T0 here: #0 0x243478d in __interceptor_pthread_create /place/sandbox-data/tasks/5/1/433792615/build/last/clang/src/projects/compiler-rt/lib/asan/asan_interceptors.cc:210:3 aws#1 0x7d2f0df in std::__y1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /contrib/libs/cxxsupp/libcxx/include/__threading_support:331:10 aws#2 0x7d2f0df in std::__y1::thread::thread<std::__y1::__bind<void (Aws::Utils::Threading::ThreadTask::*)(), Aws::Utils::Threading::ThreadTask*>, void>(std::__y1::__bind<void (Aws::Utils::Threading::ThreadTask::*)(), Aws::Utils::Threading::ThreadTask*>&&) /contrib/libs/cxxsupp/libcxx/include/thread:369 aws#3 0x7d2ebd6 in Aws::Utils::Threading::ThreadTask::ThreadTask(Aws::Utils::Threading::PooledThreadExecutor&) /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/source/utils/threading/ThreadTask.cpp:22:98 aws#4 0x7d292d3 in Aws::Utils::Threading::ThreadTask* Aws::New<Aws::Utils::Threading::ThreadTask, Aws::Utils::Threading::PooledThreadExecutor&>(char const*, Aws::Utils::Threading::PooledThreadExecutor&) /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/include/aws/core/utils/memory/AWSMemory.h:72:48 aws#5 0x7d292d3 in Aws::Utils::Threading::PooledThreadExecutor::PooledThreadExecutor(unsigned long, Aws::Utils::Threading::OverflowPolicy) /contrib/libs/aws-sdk-cpp/aws-cpp-sdk-core/source/utils/threading/Executor.cpp:91 aws#6 0x7eaaf35 in std::__y1::__compressed_pair_elem<Aws::Utils::Threading::PooledThreadExecutor, 1, false>::__compressed_pair_elem<unsigned long&, 0ul>(std::__y1::piecewise_construct_t, std::__y1::tuple<unsigned long&>, std::__y1::__tuple_indices<0ul>) /contrib/libs/cxxsupp/libcxx/include/memory:2160:9 aws#7 0x7eaaf35 in std::__y1::__compressed_pair<std::__y1::allocator<Aws::Utils::Threading::PooledThreadExecutor>, Aws::Utils::Threading::PooledThreadExecutor>::__compressed_pair<std::__y1::allocator<Aws::Utils::Threading::PooledThreadExecutor>&, unsigned long&>(std::__y1::piecewise_construct_t, std::__y1::tuple<std::__y1::allocator<Aws::Utils::Threading::PooledThreadExecutor>&>, std::__y1::tuple<unsigned long&>) /contrib/libs/cxxsupp/libcxx/include/memory:2264 aws#8 0x7eaaf35 in std::__y1::__shared_ptr_emplace<Aws::Utils::Threading::PooledThreadExecutor, std::__y1::allocator<Aws::Utils::Threading::PooledThreadExecutor> >::__shared_ptr_emplace<unsigned long&>(std::__y1::allocator<Aws::Utils::Threading::PooledThreadExecutor>, unsigned long&) /contrib/libs/cxxsupp/libcxx/include/memory:3684 aws#9 0x7eaaf35 in std::__y1::shared_ptr<Aws::Utils::Threading::PooledThreadExecutor> std::__y1::shared_ptr<Aws::Utils::Threading::PooledThreadExecutor>::make_shared<unsigned long&>(unsigned long&) /contrib/libs/cxxsupp/libcxx/include/memory:4343 aws#10 0x7eaaf35 in std::__y1::enable_if<!(is_array<Aws::Utils::Threading::PooledThreadExecutor>::value), std::__y1::shared_ptr<Aws::Utils::Threading::PooledThreadExecutor> >::type std::__y1::make_shared<Aws::Utils::Threading::PooledThreadExecutor, unsigned long&>(unsigned long&) /contrib/libs/cxxsupp/libcxx/include/memory:4722 SUMMARY: AddressSanitizer: heap-use-after-free /contrib/libs/cxxsupp/libcxx/include/__functional_base:55:17 in std::__y1::less<int>::operator()(int const&, int const&) const Shadow bytes around the buggy address: 0x0c0c7fffa100: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa 0x0c0c7fffa110: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd 0x0c0c7fffa120: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd 0x0c0c7fffa130: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa 0x0c0c7fffa140: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd =>0x0c0c7fffa150: fd fd fd fd fa fa fa fa fd fd fd fd[fd]fd fd fd 0x0c0c7fffa160: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa 0x0c0c7fffa170: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd 0x0c0c7fffa180: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd 0x0c0c7fffa190: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa 0x0c0c7fffa1a0: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc ``` There is no errors after this fix.
While trying to build a simple S3 client similar to one presented at AWS Blog, I am getting the following error:
Here is the code snippet that is throwing the error:
Am I missing something here?
Additionally, some explanation of ALLOCATION_TAG would be really helpful.
Is there any documentation or examples that I can use to explore this further?
The text was updated successfully, but these errors were encountered: