Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Shared Libs (.dll) on Windows with MSVC #1101

Closed
wants to merge 39 commits into from
Closed

Build Shared Libs (.dll) on Windows with MSVC #1101

wants to merge 39 commits into from

Conversation

traversebitree
Copy link
Contributor

Dynamic library on Windows cannot be generated before. So I made some changes to do that.
Major changes:

  1. In every target's CMakeLists.txt, add properties WINDOWS_EXPORT_ALL_SYMBOLS, POSITION_INDEPENDENT_CODE for building .dll on Windows, and add macro PCPP_tgt_BUILD_DLL for determining if build shared libs (tgt is target's name). Following is an example.
if(WIN32 AND BUILD_SHARED_LIBS)
  set_property(TARGET Common++ PROPERTY WINDOWS_EXPORT_ALL_SYMBOLS TRUE)
  set_property(TARGET Common++ PROPERTY POSITION_INDEPENDENT_CODE TRUE)
  target_compile_definitions(Common++ PUBLIC "PCPP_COMMON_BUILD_DLL")
endif()
  1. In every target's header folder, add a header file named tgtExport.h (tgt is target's name). For example: CommonExport.h. For building .dll shared libs, we need to declare __declspec(dllexport) and __declspec(dllimport) at the beginning of the declarations that are being exported. And define the macro PCAPPP_tgt_API to it (When using linux or build static libs, will ignore the macro). And set tgtExport.h in public headers in CMakeLists.txt.
#ifndef PCAPPP_COMMON_EXPORT
#define PCAPPP_COMMON_EXPORT

#if defined(_WIN32) && defined(Common___EXPORTS) && defined(PCPP_COMMON_BUILD_DLL)
#define PCAPPP_COMMON_API __declspec(dllexport)
#elif defined(_WIN32) && defined(PCPP_COMMON_BUILD_DLL)
#define PCAPPP_COMMON_API __declspec(dllimport)
#else
#define PCAPPP_COMMON_API
#endif


#endif // PCAPPP_COMMON_EXPORT
  1. In nearly every header file, add #include "tgtExport.h" at begin, and add PCAPPP_tgt_API ahead of each static member of class, for example:static const IPv4Address Zero; change to PCAPPP_COMMON_API static const IPv4Address Zero;. refer to https://www.kitware.com//create-dlls-on-windows-without-declspec-using-new-cmake-export-all-feature/.

  2. Add "copy .dll to build directory and install .dll to install directory" for every example target. Only if BUILD_SHARED_LIBS and WIN32, will copy .dll to the targets folder. Following the example:

if(BUILD_SHARED_LIBS AND WIN32)
  add_custom_command(
    TARGET ArpSpoofing
    POST_BUILD
    COMMAND ${CMAKE_COMMAND} -E copy_if_different $<TARGET_RUNTIME_DLLS:ArpSpoofing> $<TARGET_FILE_DIR:ArpSpoofing>
    COMMAND_EXPAND_LISTS)
endif()

  if(BUILD_SHARED_LIBS AND WIN32)
    install(FILES $<TARGET_RUNTIME_DLLS:ArpSpoofing> DESTINATION ${PCAPPP_INSTALL_BINDIR})
  endif()

Minor changes:

  1. In .gitignore, add compile_commands.json and .cache for programmers who use CMake and Clangd.

Then we can build shared libs (.dll) on Windows using MSVC with option BUILD_SHARED_LIBS is ON.

@codecov
Copy link

codecov bot commented Mar 12, 2023

Codecov Report

Merging #1101 (d9a24c3) into dev (9b064a3) will decrease coverage by 0.41%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##              dev    #1101      +/-   ##
==========================================
- Coverage   81.40%   81.00%   -0.41%     
==========================================
  Files         148      148              
  Lines       19220    18702     -518     
  Branches     7233     6285     -948     
==========================================
- Hits        15647    15150     -497     
+ Misses       3157     3137      -20     
+ Partials      416      415       -1     
Flag Coverage Δ
alpine315 ?
centos7 ?
fedora34 ?
macos-11 ?
macos-12 59.40% <95.45%> (ø)
macos-ventura 59.25% <81.81%> (+<0.01%) ⬆️
mingw32 67.19% <81.81%> (-0.07%) ⬇️
mingw64 67.17% <81.81%> (-0.07%) ⬇️
npcap 81.16% <92.85%> (-0.51%) ⬇️
ubuntu1804 ?
ubuntu2004 67.60% <66.66%> (-0.96%) ⬇️
ubuntu2204 ?
ubuntu2204-icpx ?
unittest 81.00% <100.00%> (-0.41%) ⬇️
vs2017 ?
windows-2019 81.22% <92.85%> (-0.46%) ⬇️
windows-2022 81.22% <92.85%> (-0.46%) ⬇️
winpcap 81.19% <92.85%> (-0.46%) ⬇️
zstd 71.24% <81.81%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
Common++/header/IpAddress.h 100.00% <ø> (ø)
Common++/header/MacAddress.h 100.00% <ø> (ø)
Packet++/header/DhcpV6Layer.h 100.00% <ø> (ø)
Packet++/header/DnsLayer.h 100.00% <ø> (ø)
Packet++/header/EthDot3Layer.h 90.00% <ø> (ø)
Packet++/header/EthLayer.h 100.00% <ø> (ø)
Packet++/header/GreLayer.h 93.75% <ø> (ø)
Packet++/header/IPSecLayer.h 82.60% <ø> (-1.40%) ⬇️
Packet++/header/IPv4Layer.h 89.89% <ø> (-0.30%) ⬇️
Packet++/header/IPv6Extensions.h 92.72% <ø> (-1.82%) ⬇️
... and 35 more

... and 77 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@clementperon
Copy link
Collaborator

clementperon commented Mar 12, 2023

Cool thank your for the MR :)

I'm not in the Windows env and really don't know but would it not be easier to do something like this directly in the Cmake

if WIN32 and Compile_Shared and Export
ADD_DEFINITIONS("PCAPPP_EXPORT=__declspec(dllexport)")
elif WIN32 and Compile_Shared
else
ADD_DEFINITIONS("PCAPPP_EXPORT=__declspec(dllimport)")
endif

Leave the PACPP_EXPORT empty for everybody else

And use PACPP_EXPORT in the source code, this will avoid to introduce a CommonExport.h

@traversebitree
Copy link
Contributor Author

traversebitree commented Mar 12, 2023

@clementperon Hello, here is the reference https://stackoverflow.com/questions/27429732/cmake-adds-dlibname-exports-compile-definition#:~:text=cmake%20add%20%3Clibname%3E_EXPORTS%20macros%20only%20for%20shared%20libraries.%20It%27s%20useful%20when%20exporting%20API%27s%20in%20Windows%20DLL.. CMake will automatic add tgt_Export, and when building .dll, the tgt_Export automatically be true (to be __declspec(dllexport)), and when being linked (other targets link this .dll), the tgt_Export automatically be false (to be __declspec(dllimport)).

Here also be a good example: https://github.com/SFML/SFML/blob/master/include/SFML/Config.hpp#:~:text=export%20and%20import-,%23,(dllimport),-//

@clementperon
Copy link
Collaborator

clementperon commented Mar 12, 2023

Yes I agree you're code is correct, I'm just looking for something simpler.

Just found this while looking :)
https://cmake.org/cmake/help/latest/module/GenerateExportHeader.html

@traversebitree
Copy link
Contributor Author

@clementperon Yes, you are right. GENERATE_EXPORT_HEADER() is a simpler and cleaner way and I'll have a test on it.

.gitignore Show resolved Hide resolved
@clementperon
Copy link
Collaborator

@traversebitree also would be nice to add a windows job with DLL built in CI, thanks.

Don't hesitate if you need help !

@traversebitree
Copy link
Contributor Author

@clementperon OK, I'll have a try.

@traversebitree
Copy link
Contributor Author

traversebitree commented Mar 14, 2023

  • I tried to add build shared libs (.dll) jobs in the workflow. But these jobs will fail when at stage "TestPcapPlusPlus" -> python ci\run_tests\run_tests_windows.py --coverage.
    image
    image

  • I try to manually execute PcapPlusPlus\Tests\Packet++Test\Bin\Packet++Test.exe in HyperV. The error occurs here:
    image

  • Then I try to execute using static libs (.lib), no the above error:
    image

  • Maybe the error occurs on the test case EthPacketPointerCreation. I try to manually run the test case EthPacketPointerCreation but failed. Can you help to check this? I guess the problem happens at this point. @clementperon @egecetin .

  • Add:
    I manually execute Pcap++Test\Bin\Pcap++Test.exe and there is no error like in Packet++Test.exe.
    image

  • Add:
    I manually execute python ci\run_tests\run_tests_windows.py --coverage but failed.
    image

@traversebitree
Copy link
Contributor Author

traversebitree commented Mar 14, 2023

I guess MSVC+dll incompatible with MemPlumber? When set bool skipMemLeakCheck = true;. Packet++Test won't stuck at the Runtime Error.
I manually delete the test cases, and I found the test cases (in Packet++Test) EthPacketPointerCreation, TcpChecksumInvalidRead, InsertDataToPacket, CreatePacketFromBuffer, IcmpV6EditTest will be stuck at the Runtime Error.

temporary stop memplumber
@seladb
Copy link
Owner

seladb commented Mar 15, 2023

I guess MSVC+dll incompatible with MemPlumber? When set bool skipMemLeakCheck = true;. Packet++Test won't stuck at the Runtime Error.
I manually delete the test cases, and I found the test cases (in Packet++Test) EthPacketPointerCreation, TcpChecksumInvalidRead, InsertDataToPacket, CreatePacketFromBuffer, IcmpV6EditTest will be stuck at the Runtime Error.

@traversebitree are these the only tests which fail in Packet++Test and Pcap++Test?
Do they fail even if you set skipMemLeakCheck = false?

Test disable memleakcheck
@traversebitree
Copy link
Contributor Author

traversebitree commented Mar 15, 2023

@seladb I test to manually set skipMemLeakCheck = true. And MSVC+dll is still stuck at Test PcapPlusPlus stage, and the log is here, There appears many [ERROR] which not appear in MSVC+lib build :

Packet++Test.exe:

OUILookup : PASSED
EthPacketCreation : PASSED
EthPacketPointerCreation : PASSED
EthAndArpPacketParsing : PASSED
ArpPacketCreation : PASSED
EthDot3LayerParsingTest : PASSED
EthDot3LayerCreateEditTest : PASSED
VlanParseAndCreation : PASSED
QinQ802_1adParse : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\MplsLayer.cpp: pcpp::MplsLayer::setMplsLabel:78] MPLS label mustn't exceed 20 bits which is the value 0xffff. Got a parameter with the value 0xffffff
MplsLayerTest : PASSED
VxlanParsingAndCreationTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\Packet.cpp: pcpp::Packet::insertLayer:229] Layer is already allocated to another packet. Cannot use layer in more than one packet
IPv4PacketCreation : PASSED
IPv4PacketParsing : PASSED
IPv4FragmentationTest : PASSED
IPv4OptionsParsingTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IPv4Layer.cpp: pcpp::IPv4OptionBuilder::IPv4OptionBuilder:64] Cannot build timestamp option of type IPv4TimestampOptionValue::Unknown
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IPv4Layer.cpp: pcpp::IPv4OptionBuilder::IPv4OptionBuilder:78] Cannot build timestamp option of type IPv4TimestampOptionValue::TimestampAndIP because number of timestamps and IP addresses is not equal
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IPv4Layer.cpp: pcpp::IPv4Layer::addOptionAt:472] Cannot add option - adding this option will exceed IPv4 total option size which is 40
IPv4OptionsEditTest : PASSED
IPv4UdpChecksum : PASSED
IPv6UdpPacketParseAndCreate : PASSED
IPv6FragmentationTest : PASSED
IPv6ExtensionsTest : PASSED
TcpPacketNoOptionsParsing : PASSED
TcpPacketWithOptionsParsing : PASSED
TcpPacketWithOptionsParsing2 : PASSED
TcpPacketCreation : PASSED
TcpPacketCreation2 : PASSED
TcpMalformedPacketParsing : PASSED
TcpChecksumInvalidRead : PASSED
TcpChecksumMultiBuffer : PASSED
PacketUtilsHash5TupleUdp : PASSED
PacketUtilsHash5TupleTcp : PASSED
PacketUtilsHash5TupleIPv6 : PASSED
InsertDataToPacket : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\Packet.cpp: pcpp::Packet::insertLayer:244] With the new layer the packet will exceed the size of the pre-allocated buffer: 46 bytes
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\Packet.cpp: pcpp::Packet::extendLayer:556] With the layer extended size the packet will exceed the size of the pre-allocated buffer: 46 bytes
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IPv4Layer.cpp: pcpp::IPv4Layer::addOptionAt:479] Could not extend IPv4Layer in [4] bytes
CreatePacketFromBuffer : PASSED
InsertVlanToPacket : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\Packet.cpp: pcpp::Packet::removeLayer:339] Layer of the requested type was not found in packet
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\Packet.cpp: pcpp::Packet::removeLayer:339] Layer of the requested type was not found in packet
RemoveLayerTest : PASSED
CopyLayerAndPacketTest : PASSED
PacketLayerLookupTest : PASSED
RawPacketTimeStampSetterTest : PASSED
ParsePartialPacketTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\Packet.cpp: pcpp::Packet::insertLayer:235] Cannot insert layer after packet trailer
PacketTrailerTest : PASSED
ResizeLayerTest : PASSED
PrintPacketAndLayers : PASSED
HttpRequestParseMethodTest : PASSED
HttpRequestLayerParsingTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\TextBasedProtocol.cpp: pcpp::TextBasedProtocolMessage::removeField:277] Cannot find field 'kuku'
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\TextBasedProtocol.cpp: pcpp::TextBasedProtocolMessage::insertField:198] Cannot add a field after end of header
HttpRequestLayerCreationTest : PASSED
HttpRequestLayerEditTest : PASSED
HttpResponseParseStatusCodeTest : PASSED
HttpResponseParseVersionTest : PASSED
HttpResponseLayerParsingTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\HttpLayer.cpp: pcpp::HttpMessage::addField:24] Field 'Server' already exists!
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\TextBasedProtocol.cpp: pcpp::TextBasedProtocolMessage::insertField:198] Cannot add a field after end of header
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\TextBasedProtocol.cpp: pcpp::TextBasedProtocolMessage::removeField:277] Cannot find field 'kuku5'
HttpResponseLayerCreationTest : PASSED
HttpResponseLayerEditTest : PASSED
HttpMalformedResponseTest : PASSED
PPPoESessionLayerParsingTest : PASSED
PPPoESessionLayerCreationTest : PASSED
PPPoEDiscoveryLayerParsingTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\PPPoELayer.cpp: pcpp::PPPoEDiscoveryLayer::removeTag:386] Couldn't find tag
PPPoEDiscoveryLayerCreateTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsLayer.cpp: pcpp::DnsLayer::parseResources:157] DNS layer contains more than 300 resources, probably a bad packet. Skipping parsing DNS resources
DnsLayerParsingTest : PASSED
DnsLayerQueryCreationTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsResourceData.cpp: pcpp::IPv4DnsResourceData::toByteArr:71] Cannot convert IPv4 address to byte array because address is not valid
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsResource.cpp: pcpp::DnsResource::setData:411] Cannot convert DNS resource data to byte array, data is probably invalid
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsResource.cpp: pcpp::DnsResource::setData:362] DNS record is of type A but given data isn't of type IPv4DnsResourceData
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsResourceData.cpp: pcpp::IPv6DnsResourceData::toByteArr:95] Cannot convert IPv6 address to byte array because address is not valid
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsResource.cpp: pcpp::DnsResource::setData:411] Cannot convert DNS resource data to byte array, data is probably invalid
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\GeneralUtils.cpp: pcpp::hexStringToByteArray:46] Input string is in odd size
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsResourceData.cpp: pcpp::GenericDnsResourceData::toByteArr:221] Input data is null or illegal
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsResource.cpp: pcpp::DnsResource::setData:411] Cannot convert DNS resource data to byte array, data is probably invalid
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\GeneralUtils.cpp: pcpp::hexStringToByteArray:60] Input string has an illegal character
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsResourceData.cpp: pcpp::GenericDnsResourceData::toByteArr:221] Input data is null or illegal
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DnsResource.cpp: pcpp::DnsResource::setData:411] Cannot convert DNS resource data to byte array, data is probably invalid
DnsLayerResourceCreationTest : PASSED
DnsLayerEditTest : PASSED
DnsLayerRemoveResourceTest : PASSED
DnsOverTcpParsingTest : PASSED
DnsOverTcpCreationTest : PASSED
IcmpParsingTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IcmpLayer.cpp: pcpp::IcmpLayer::setIpAndL4Layers:104] Cannot set ICMP data that involves IP and L4 layers on a layer not attached to a packet. Please add the ICMP layer to a packet and try again
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IcmpLayer.cpp: pcpp::IcmpLayer::setIpAndL4Layers:104] Cannot set ICMP data that involves IP and L4 layers on a layer not attached to a packet. Please add the ICMP layer to a packet and try again
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IcmpLayer.cpp: pcpp::IcmpLayer::setIpAndL4Layers:104] Cannot set ICMP data that involves IP and L4 layers on a layer not attached to a packet. Please add the ICMP layer to a packet and try again
IcmpCreationTest : PASSED
IcmpEditTest : PASSED
GreParsingTest : PASSED
GreCreationTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\GreLayer.cpp: pcpp::GreLayer::unsetSequenceNumber:176] Couldn't unset sequence number as it's already unset
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\GreLayer.cpp: pcpp::GREv0Layer::unsetChecksum:331] Couldn't unset checksum as it's already unset
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\GreLayer.cpp: pcpp::GreLayer::unsetSequenceNumber:176] Couldn't unset sequence number as it's already unset
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\GreLayer.cpp: pcpp::GreLayer::unsetSequenceNumber:176] Couldn't unset sequence number as it's already unset
GreEditTest : PASSED
SSLClientHelloParsingTest : PASSED
SSLExtensionWithZeroSizeTest : PASSED
SSLAppDataParsingTest : PASSED
SSLAlertParsingTest : PASSED
SSLMultipleRecordParsingTest : PASSED
SSLMultipleRecordParsing2Test : PASSED
SSLMultipleRecordParsing3Test : PASSED
SSLMultipleRecordParsing4Test : PASSED
SSLMultipleRecordParsing5Test : PASSED
SSLPartialCertificateParseTest : PASSED
SSLNewSessionTicketParseTest : PASSED
SSLMalformedPacketParsing : PASSED
TLS1_3ParsingTest : PASSED
TLSCipherSuiteTest : PASSED
ClientHelloTLSFingerprintTest : PASSED
ServerHelloTLSFingerprintTest : PASSED
SllPacketParsingTest : PASSED
SllPacketCreationTest : PASSED
NullLoopbackTest : PASSED
NflogPacketParsingTest : PASSED
DhcpParsingTest : PASSED
DhcpCreationTest : PASSED
DhcpEditTest : PASSED
IgmpParsingTest : PASSED
IgmpCreateAndEditTest : PASSED
Igmpv3ParsingTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3QueryLayer::addSourceAddressAtIndex:285] Cannot add source address at index -1, index is out of bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3QueryLayer::addSourceAddressAtIndex:285] Cannot add source address at index 4, index is out of bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3QueryLayer::addSourceAddressAtIndex:292] Cannot add source address at index 4, index is out of packet bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3QueryLayer::removeSourceAddressAtIndex:315] Cannot remove source address at index 4, index is out of bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3QueryLayer::removeSourceAddressAtIndex:315] Cannot remove source address at index -1, index is out of bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3QueryLayer::removeSourceAddressAtIndex:322] Cannot remove source address at index 4, index is out of packet bounds
Igmpv3QueryCreateAndEditTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3ReportLayer::addGroupRecordAtIndex:449] Cannot add group record, index -1 out of bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3ReportLayer::addGroupRecordAtIndex:449] Cannot add group record, index 4 out of bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3ReportLayer::addGroupRecordAtIndex:449] Cannot add group record, index 100 out of bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3ReportLayer::removeGroupRecordAtIndex:477] Cannot remove group record, index 4 is out of bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3ReportLayer::removeGroupRecordAtIndex:477] Cannot remove group record, index -1 is out of bounds
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\IgmpLayer.cpp: pcpp::IgmpV3ReportLayer::removeGroupRecordAtIndex:477] Cannot remove group record, index 100 is out of bounds
Igmpv3ReportCreateAndEditTest : PASSED
SipRequestParseMethodTest : PASSED
SipRequestLayerParsingTest : PASSED
SipRequestLayerCreationTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\SipLayer.cpp: pcpp::SipRequestFirstLine::setUri:321] URI cannot be empty
SipRequestLayerEditTest : PASSED
SipResponseParseStatusCodeTest : PASSED
SipResponseParseVersionCodeTest : PASSED
SipResponseLayerParsingTest : PASSED
SipResponseLayerCreationTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\SipLayer.cpp: pcpp::SipResponseFirstLine::setStatusCode:792] Requested status code is SipStatusCodeUnknown
SipResponseLayerEditTest : PASSED
SdpLayerParsingTest : PASSED
SdpLayerCreationTest : PASSED
SdpLayerEditTest : PASSED
RadiusLayerParsingTest : PASSED
RadiusLayerCreationTest : PASSED
RadiusLayerEditTest : PASSED
GtpLayerParsingTest : PASSED
GtpLayerCreationTest : PASSED
GtpLayerEditTest : PASSED
BgpLayerParsingTest : PASSED
BgpLayerCreationTest : PASSED
BgpLayerEditTest : PASSED
SSHParsingTest : PASSED
SSHMalformedParsingTest : PASSED
IPSecParsingTest : PASSED
DhcpV6ParsingTest : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DhcpV6Layer.cpp: pcpp::DhcpV6Layer::addOptionAfter:216] Option type 8 doesn't exist in layer
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DhcpV6Layer.cpp: pcpp::DhcpV6Layer::addOptionBefore:231] Option type 8 doesn't exist in layer
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\GeneralUtils.cpp: pcpp::hexStringToByteArray:60] Input string has an illegal character
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Packet++\src\DhcpV6Layer.cpp: pcpp::DhcpV6Layer::addOptionAt:179] Cannot build new option
DhcpV6CreationTest : PASSED
DhcpV6EditTest : PASSED
NtpMethodsTests : PASSED
NtpParsingV3Tests : PASSED
NtpParsingV4Tests : PASSED
NtpCreationTests : PASSED
TelnetCommandParsingTests : PASSED
TelnetDataParsingTests : PASSED
IcmpV6ParsingTest : PASSED
IcmpV6CreationTest : PASSED
IcmpV6EditTest : PASSED
FtpParsingTests : PASSED
FtpCreationTests : PASSED
FtpEditTests : PASSED
LLCParsingTests : PASSED
LLCCreationTests : PASSED
StpConfigurationParsingTests : PASSED
StpConfigurationCreationTests : PASSED
StpConfigurationEditTests : PASSED
StpTopologyChangeParsingTests : PASSED
StpTopologyChangeCreationTests : PASSED
StpTopologyChangeEditTests : PASSED
RapidStpParsingTests : PASSED
RapidStpCreationTests : PASSED
RapidStpEditTests : PASSED
MultipleStpParsingTests : PASSED
MultipleStpCreationTests : PASSED
MultipleStpEditTests : PASSED
SomeIpPortTest : PASSED
SomeIpParsingTest : PASSED
SomeIpCreationTest : PASSED
SomeIpTpParsingTest : PASSED
SomeIpTpCreationTest : PASSED
SomeIpTpEditTest : PASSED
SomeIpSdParsingTest : PASSED
SomeIpSdCreationTest : PASSED
WakeOnLanParsingTests : PASSED
WakeOnLanCreationTests : PASSED
WakeOnLanEditTests : PASSED
ALL TESTS PASSED!!
Test cases: 161, Passed: 161, Failed: 0, Skipped: 0

Pcap++Test.exe (run stuck, and I stopped the action process):

PcapPlusPlus version: v22.11+ (non-official release)
Built: Mar 15 2023 07:33:23
Git info: Git branch '', commit ''
Using ip: 10.1.0.58
Debug mode: off
Start running tests...
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\IpAddress.cpp: pcpp::IPv4Address::matchSubnet:59] subnet prefix '' must be an integer
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\IpAddress.cpp: pcpp::IPv4Address::matchSubnet:59] subnet prefix '' must be an integer
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\IpAddress.cpp: pcpp::IPv4Address::matchSubnet:59] subnet prefix 'aa' must be an integer
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\IpAddress.cpp: pcpp::IPv4Address::matchSubnet:76] Subnet prefix '33' must be between 0 and 32
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\IpAddress.cpp: pcpp::IPv4Address::matchSubnet:87] Subnet '999.999.1.1/24' is in illegal format
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\IpAddress.cpp: pcpp::IPv6Address::matchSubnet:160] subnet prefixLength '0' illegal
TestIPAddress : PASSED
TestMacAddress : PASSED
TestLRUList : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\GeneralUtils.cpp: pcpp::hexStringToByteArray:46] Input string is in odd size
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Common++\src\GeneralUtils.cpp: pcpp::hexStringToByteArray:60] Input string has an illegal character
TestGeneralUtils : PASSED
TestGetMacAddress : PASSED
TestLogger : PASSED
TestLoggerMultiThread : PASSED
TestPcapFileReadWrite : PASSED
TestPcapSllFileReadWrite : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Pcap++\src\PcapFileDevice.cpp: pcpp::PcapFileWriterDevice::open:605] The only Raw IP link type supported in libpcap/WinPcap/Npcap is LINKTYPE_DLT_RAW1, please use that instead
TestPcapRawIPFileReadWrite : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Pcap++\src\PcapFileDevice.cpp: pcpp::PcapFileWriterDevice::open:720] Pcap file has a different link layer type than the one chosen in PcapFileWriterDevice c'tor, 1, 113
TestPcapFileAppend : PASSED
TestPcapNgFileReadWrite : PASSED
[ERROR: D:\a\PcapPlusPlus\PcapPlusPlus\Pcap++\src\PcapFileDevice.cpp: pcpp::PcapNgFileReaderDevice::getOS:437] Pcapng file device 'PcapExamples/pcapng-example.pcapng' not opened
SUCCESS: The process with PID 7104 (child process of PID 6296) has been terminated.
SUCCESS: The process with PID 6296 (child process of PID 1436) has been terminated.
Interface is \Device\NPF_{FA39CAFD-1C99-475D-9BAD-3A9F6F602251} and IP address is 10.1.0.58
^CTraceback (most recent call last):
File "ci\run_tests\run_tests_windows.py", line 153, in
main()
File "ci\run_tests\run_tests_windows.py", line 109, in main
completed_process = subprocess.run(
File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\subprocess.py", line 495, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\subprocess.py", line 1020, in communicate
self.wait()
File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\subprocess.py", line 1083, in wait
return self._wait(timeout=timeout)
File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\subprocess.py", line 1361, in _wait
result = _winapi.WaitForSingleObject(self._handle,
KeyboardInterrupt
Error: The operation was canceled.

@seladb
Copy link
Owner

seladb commented Mar 15, 2023

@traversebitree I think I tried to build shared libs for Windows many years ago and stumbled into a somewhat similar issue. If I remember correctly, there was an issue when a DLL and an external app share memory - e.g the DLL gives direct access to its memory to the external app, or the app gives direct access to its memory to the DLL.

This happens quite a lot in PcapPlusPlus.

For example here the DLL shares a pointer to its internal memory with the external app (in this case the test app):
https://github.com/seladb/PcapPlusPlus/blob/master/Tests/Packet++Test/Tests/IPv4Tests.cpp#L50-L57

I'm not at all sure these are the issues you see, I just remember them vaguely from many years ago.

If this is indeed the issue, it'd be very hard to solve because PcapPlusPlus uses this memory sharing in many places in its API.

@egecetin
Copy link
Collaborator

@seladb I'm not 100% sure but looks like similar with situation explained Memory allocated in Memory Manager A and freed in Memory Manager B section of this

Test add `MTd` CXX Flag
@traversebitree
Copy link
Contributor Author

@egecetin Yes, I also noticed this problem, so I added this set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreaded$<$<CONFIG:Debug>:Debug>" CACHE STRING "MTd") CMakeLists.txt. It changes the compile flag from -MDd to -MTd. But the results haven't changed, and the problem remains.
@seladb Indeed, this problem is really very difficult, it seems to take a long time to solve it.

@seladb
Copy link
Owner

seladb commented Apr 4, 2023

@traversebitree what should we do with this PR?

@traversebitree
Copy link
Contributor Author

traversebitree commented Apr 4, 2023

@seladb The fact is that building .dll is not compatible with MemPlumber. When the memory check is skipped, the building .dll is valid and passes the test. So, I don't know whether to go ahead and modify the MemPlumber or terminate this PR. I've been working on my paper recently, so I haven't been able to continue working on this PR lately, sorry about that.

@seladb
Copy link
Owner

seladb commented Apr 5, 2023

@seladb The fact is that building .dll is not compatible with MemPlumber. When the memory check is skipped, the building .dll is valid and passes the test. So, I don't know whether to go ahead and modify the MemPlumber or terminate this PR. I've been working on my paper recently, so I haven't been able to continue working on this PR lately, sorry about that.

@traversebitree don't we have this issue as well? #1101 (comment) that is unrelated to MemPlumber?

@traversebitree
Copy link
Contributor Author

I'm sorry, but after trying too many times, it still doesn't pass the memory leak test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants