Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bio_dgram uses recvmsg/sendmsg to retrieve destination and set origin address #5257

Open
wants to merge 25 commits into
base: master
Choose a base branch
from

Conversation

mcr
Copy link
Contributor

@mcr mcr commented Feb 4, 2018

This code and two test cases adjusts the bss_dgram read/write which is used in DTLS such that
the originating and destination addresses are retrieved upon read, and are set in write. This permits
a UDP "server" socket to be bound to :: (0.0.0.0) and receive traffic to any interface on the machine.

This code includes IP_PKTINFO (v4) for Linux, and IP_RECVDSTADDR/IP_SENDSRCADDR (v4) for *BSD, and if not falls back to recvfrom/sendto(). For IPv6, it is assumed that the stock IETF APIs
using pktinfo are available. The choice of PKTINFO or RECVDSTADDR is determined by the Configure scripts.
These patches include four additional BIO_crtl messages to set/get the origin/dest address from the BIO.
There was previous concern about CMSG_SPACE() being constant or not; but the specifications say that those macros are guaranteed to be compile time constants.

In addition, code to enable IP_PKTINFO and IPV6_PKTINFO on OSX is included. OSX has RECDSTADDR (like other BSDs), but does not have SENDSRCADDR. IP_PKTINFO was added in OSX 10.9, prior versions may need to disable both IP_PKTINFO and IP_RECVDSTADDR.

@mcr
Copy link
Contributor Author

mcr commented Feb 4, 2018

The *BSD patches were tested on a FreeBSD 8.4 machine at ISC.ORG.

@FdaSilvaYY
Copy link
Contributor

A lot of code style issue ...
Consider running 'utils\openssl-format-source' script to uncover its.

@mcr
Copy link
Contributor Author

mcr commented Feb 5, 2018

@FdaSilvaYY , code style issues aside, the regression tests seem to all fail without any details. Do you have any idea if there is something systematic wrong? "make tests" works for me.

@mcr
Copy link
Contributor Author

mcr commented Feb 5, 2018

@FdaSilvaYY each patch updated with a run through openssl-format-source. Neither bss_dgram.c nor bio.h were clean before I started, so I included a commit that just fixes them before I edit them.

@mattcaswell
Copy link
Member

Travis is failing with 3 different errors (in different builds):

../../_srcdist/test/recipes/81-test_bio_write.t ................ 
1..1
    # Subtest: ../../test/bio_write_test
    1..2
        # Subtest: test_bio_write_v4
        1..1
        ok 1 - iteration 1
    ok 1 - test_bio_write_v4
        # Subtest: test_bio_write_v6
        1..1
bind in6fd: Cannot assign requested address
../../util/shlib_wrap.sh ../../test/bio_write_test => 4
not ok 1 - running bio_write_test
ccache gcc  -I. -Icrypto/include -Iinclude -Wall -O3 -pthread -m64 -DDEBUG_UNUSED -DPEDANTIC -pedantic -Wno-long-long -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wswitch -Wsign-compare -Wmissing-prototypes -Wshadow -Wformat -Wtype-limits -Wundef -Werror -fPIC -DDSO_DLFCN -DHAVE_DLFCN_H -DHAVE_IP_PKTINFO -DNDEBUG -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\""  -c -MMD -MF crypto/bio/bss_dgram.d.tmp -MT crypto/bio/bss_dgram.o -c -o crypto/bio/bss_dgram.o crypto/bio/bss_dgram.c
crypto/bio/bss_dgram.c: In function 'dgram_read_unconnected_v6':
crypto/bio/bss_dgram.c:462:5: error: C++ style comments are not allowed in ISO C90 [-Werror]
     //unsigned int     chdrlen = CMSG_SPACE(sizeof(struct in6_pktinfo));
     ^
crypto/bio/bss_dgram.c:462:5: error: (this will be reported only once per input file) [-Werror]
cc1: all warnings being treated as errors
make[1]: *** [crypto/bio/bss_dgram.o] Error 1
make[1]: Leaving directory `/home/travis/build/openssl/openssl'
make: *** [all] Error 2
+/// MAKE FAILED
i686-w64-mingw32-gcc  -I. -Icrypto/include -Iinclude -m32 -Wall -O3 -fomit-frame-pointer -Wno-pedantic-ms-format  -DDSO_WIN32 -DOPENSSL_USE_APPLINK -DNDEBUG -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DL_ENDIAN -DWIN32_LEAN_AND_MEAN -DUNICODE -D_UNICODE -D_MT -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1_1\""  -c -MMD -MF crypto/bio/bss_dgram.d.tmp -MT crypto/bio/bss_dgram.o -c -o crypto/bio/bss_dgram.o crypto/bio/bss_dgram.c
crypto/bio/bss_dgram.c:17:23: fatal error: arpa/inet.h: No such file or directory
 #include <arpa/inet.h>
                       ^
compilation terminated.
make[1]: *** [crypto/bio/bss_dgram.o] Error 1
make[1]: Leaving directory `/home/travis/build/openssl/openssl'
make: *** [all] Error 2
+/// MAKE FAILED

The Appveyor builds are failing with a version of the last error.

@mattcaswell
Copy link
Member

@FdaSilvaYY each patch updated with a run through openssl-format-source. Neither bss_dgram.c nor bio.h were clean before I started, so I included a commit that just fixes them before I edit them.

Don't do that! There is too much "noise" in this patch from the openssl-format-source output. Please back that commit out. If openssl-format-source is not clean to start with then we'll have to handle the style issues manually. Reading the style guide is a good start:

https://www.openssl.org/policies/codingstyle.html

Copy link
Member

@mattcaswell mattcaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skimmed over some parts (there is a lot of noise due to the openssl-format-source). Also didn't look too closely at the tests yet. But there's probably enough comments here for now already!

#include <arpa/inet.h>
#include <netinet/in.h>
#include <netinet/ip6.h>
#include <netinet/icmp6.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will fail on non-Unix platforms. Probably this needs to be guarded by appropriate ifdef

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way (build-server) on which I can test compile against such a platform?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only generally available options we have are travis/appveyor integration via github.

crypto/bio/bss_dgram.c Show resolved Hide resolved
@@ -294,26 +308,262 @@ static void dgram_reset_rcv_timeout(BIO *b)
# endif
}

# if defined(HAVE_IP_PKTINFO)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than creating a custom define here, could we not just use __linux__? Adding new defines to Configure just for this seems quite heavyweight.

mhdr.msg_iov = &iov;
mhdr.msg_iovlen = 1;

if (dstaddr) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style: if (dstaddr != NULL)

mhdr.msg_controllen = sizeof(chdr);
}

pkt_info = NULL;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just do this in the declaration

struct sockaddr *sa = (struct sockaddr *)BIO_ADDR_sockaddr(&data->peer);
if (data->addr.sa.sa_family == 0) {
dgram_get_sockname(b);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't use {} for just a single line if.

ret = BIO_ADDR_sockaddr_size(&data->addr);
if (ret == 0) { /* never set, retrieve it */
ret = dgram_get_sockname(b);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove {}

}

/* FIXME: if num < ret, we will only return part of an address.
That should bee an error, no? */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs resolution

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not my code actually. Happy to fix this in separate patch.

#include <openssl/bio.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <unistd.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't work on windows!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I have ifdef the entire test. Not sure what is the best way to test these things on windows.

BIO *out;
int expected_count = PACKET_COUNT;

pid_t pid = fork();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fork() doesn't exist on Windows!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattcaswell, my run of the tests on travis seemed to just die in the middle without any obvious error. I guess AppVeyor builds for Windows?
I also now recall that travis has no IPv6. Not sure what to do about that yet.
I don't want to rewrite this test case with select(), because that will just be more complicated and won't work on Windows anyway. Also, re, //- comments, shouldn't the default flags warn on that if travis builds break on that?
Finally, 90% of the style issues you complain about were introduced by openssl-format-source. So I think that openssl-format-source does not comply to the style guide.
What I did was to run it without any material changes and then commit those as on their own so that my changes would be not obscured by indent changes. Why is it wrong to fix bss_dgram.c and bio.h's indented?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess AppVeyor builds for Windows?

Yes

I also now recall that travis has no IPv6. Not sure what to do about that yet.

Possibly if we can't create the sockets then we skip that bit of the test - but don't fail it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, re, //- comments, shouldn't the default flags warn on that if travis builds break on that?

The default flags are designed for the general user - not people developing patches. OpenSSL has to run on lots of different platforms with lots of different compilers. We don't run the default with too high warning options because that will introduce spurious build failures for the general user. If you are developing patches you should use --strict-warnings as mentioned in the CONTRIBUTING file.

Finally, 90% of the style issues you complain about were introduced by openssl-format-source. So I think that openssl-format-source does not comply to the style guide.

Probably it doesn't. The style has developed further since that was written. I wouldn't recommend using it myself. As I said above I would back out those changes.

Why is it wrong to fix bss_dgram.c and bio.h's indented?

It generates a lot of noise and makes it difficult to review the patch (difficult to distinguish your changes from auto-generated changes). If we want to fix large amounts of the formatting in files it should be done in a dedicated PR. Localised fixes are ok.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed the patch that just ran openssl-format-source on that code.

@mcr mcr force-pushed the bio_dgram_uses_recvmsg branch 7 times, most recently from 49cce69 to 2166e8c Compare March 8, 2018 18:54
@mcr
Copy link
Contributor Author

mcr commented Mar 8, 2018

https://travis-ci.org/openssl/openssl/jobs/350967905
@levitte this worked earlier this morning! Maybe before I rebased...

    
Operating system: x86_64-whatever-linux2
/usr/bin/env __CNF_CPPDEFINES='' __CNF_CPPINCLUDES='' __CNF_CPPFLAGS='' __CNF_CFLAGS='' __CNF_CXXFLAGS='' __CNF_LDFLAGS='' __CNF_LDLIBS='' /usr/bin/perl ./Configure linux-x86_64 'no-asm' '-Werror' '--debug' 'no-afalgeng' 'no-shared' 'enable-crypto-mdebug' 'enable-rc5' 'enable-md2'
***** Mixing env / make variables and additional compiler/linker flags as
***** configure command line option is not permitted.
***** Affected env / make variables: CFLAGS, CXXFLAGS


0.00s$ ./configdata.pm --dump
/home/travis/.travis/job_stages: line 57: ./configdata.pm: No such file or directory
The command "./configdata.pm --dump" failed and exited with 127 during .

@mattcaswell
Copy link
Member

this worked earlier this morning! Maybe before I rebased...

Rebase again to pick up fb174fa?

@mcr
Copy link
Contributor Author

mcr commented Mar 9, 2018

all green. review comments dealt with.

Copy link
Member

@mattcaswell mattcaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a general comment there are lots of indentation errors in this PR. I didn't call them out individually. Please always use 4 spaces for indentation. There are numerous cases of a different number of spaces being used, or tab characters being used.

#else
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/ip6.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need to be able handle non-windows and non-unix/linux targets.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This remains an issue. It's unclear to me whether all of these headers will be available on all platforms. At least in some files we wrap some of these in an "ifdef OPENSSL_SYS_UNIX". We have, as yet, never used netinet/ip6.h - so I am concerned it may not be universally available on all the platforms that we might need to support.

BIO_ADDR *dstaddr, BIO_ADDR *peer)
{
int len = 0;
unsigned char chdr[BIO_CMSG_ADDR_SIZE];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too many spaces

if(dstaddr != NULL) {
memset(chdr, 0, sizeof(chdr));
cmsg = (struct cmsghdr *) chdr;
mhdr.msg_control = (void *) cmsg;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style nit: no space between variable and cast in above two lines

val = 1;
if(setsockopt(b->num, IPPROTO_IP, IP_PKTINFO, &val, sizeof(val)) < 0) {
return -1;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style nit: No {} around a single line if

{
if (cmsg->cmsg_level != IPPROTO_IP)
continue;
switch(cmsg->cmsg_type) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switch seems an odd choice when there is only one case

mhdr.msg_iovlen = 1;

srcaddr = (struct sockaddr_in *)BIO_ADDR_sockaddr(&data->addr);
if(srcaddr && srcaddr->sin_addr.s_addr != 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use srcaddr != NULL

struct cmsghdr *cmsg;
struct iovec iov;
char chdr[BIO_CMSG_PKT6_SIZE];
bio_dgram_data *data = (bio_dgram_data *)b->ptr;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blank line between declarations and code

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment is still relevant

}
#if 0
printf("bound to port: %u\n", ntohs(localhost.sin_port));
#endif
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this


if(ret <= 0) {
test_printf_stderr("BIO_read returned %d\n", ret);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No {} around single line if

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment still relevant

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please use TEST_info for informational messages.

#include <openssl/bio.h>
#include <openssl/rand.h>
#include <openssl/bio.h>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should be using the test framework more fully, i.e. using the TEST_ macros as defined in testutil.h. Take a look at the other tests for examples.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is still relevant.

@mcr
Copy link
Contributor Author

mcr commented Mar 15, 2018 via email

@mattcaswell mattcaswell added this to the Post 1.1.1 milestone Mar 20, 2018
@t8m t8m modified the milestones: Post 1.1.1, Post 3.0.0 May 12, 2021
mcr added 21 commits March 29, 2022 13:44
Includes a test case to write packets using the interface.
…_PKTINFO, and

set this option generically on Linux
…his author. Make sure it compiles on Windows using recvfrom()/sendto() instead
…tainers have IPv6 disabled

(not even a loopback)
so no IPv6 tests will function, turn off IPv6 version of bio_write_test and bio_read_test
not have SENDDSTADDR for IPv4.
RFC 3542 needs to be explicitely enabled with a #define
clang does not believe that Darwin CMSG_SPACE is a constant and issues warning
Comment on lines 34 to 40
# if OPENSSL_USE_IPV6
# if OPENSSL_SYS_WINDOWS
# include "shared/netiodef.h>
# else
# include <netinet/ip6.h>
# endif
# endif
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Referring to your discussion with Michael Wojcik:

>    > #if defined OPENSSL_SYS_WINDOWS
>    > # include <shared/netiodef.h>
>    > #else
>    > # include <netinet/ip6.h>
>    > #endif
>
> But, don't all the OPENSSL_* macros expand to 0/1, anyway, so we actually
> just want #if OPENSSL_SYS_WINDOWS?

In principle, you might be right, but in practice a quick git grep OPENSSL_SYS will convince you that everywhere throughout the OpenSSL source code we use either #ifdef OPENSSL_SYS* or #if defined(OPENSSL_SYS*) and not #if OPENSSL_SYS*. So for consistency reasons I ask you to please change it.

(The OPENSSL_USE_IPV6 macro unfortunately has a different semantics (see a5a0f32), here you should keep the #if. So much for consistency :-/ )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch: master Merge to master branch severity: fips change The pull request changes FIPS provider sources triaged: feature The issue/pr requests/adds a feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants