Runtime detection of avilable flags

[WojciechMula]

I want to use your great library in our pretty big system. We already have the detection of CPU/platform features, and it won't interfere with the library's settings. There is a little problem - when I want to use BASE64_FORCE_XXX (based on our settings) and the "XXX" is not really compiled into the library, we got the dummy implementation, which does nothing.

How can I ask the library which BASE64_FORCE_XXX are meaningful? Of course I can alter the sources, but it's not the proper solution.

I use the stable 0.3.0 version.

[BurningEnlightenment]

Sadly there isn't any built in function like

int base64_codec_supported(int flags);

but you can indirectly check for codec support by calling base64_decode with the appropriate flags and build a little helper function on top of that as demonstrated in test/codec_supported.c. The relevant snippet:

int base64ex_codec_supported(int flags)
{
    // Check if given codec is supported by trying to decode a test string:
    char a[] = "aGVsbG8=";
    char b[10];
    size_t outlen;

    return (base64_decode(a, sizeof(a)-1, b, &outlen, flags) != -1);
}

use it like so:

if (base64ex_codec_supported(BASE64_FORCE_AVX2))
{
     /*...*/
} 

[WojciechMula]

It is a really good solution, thank you for help.

[aklomp]

You should probably not be using the FORCE flags, they're mainly for testing purposes. On x86, the library should auto-detect at runtime the best compiled-in codec for the current platform. That code should do the right thing; if not, please file a bug!

This library is built for performance and portability. The idea is that this library could in principle be distributed in compiled binary form as a Debian package or so. This means that the capabilities of the target machine are unknown at compile time. To be portable but still get the best performance, this library does a number of runtime checks to determine the SIMD support level of the architecture it's running on, and chooses the best available codec, once, at runtime. The checking code (in codec_choose.c) is built without architecture-specific compiler flags, so that it will run on any generic machine.

So the idea is that you should build with the maximum SIMD level your compiler will support, and rely on runtime feature detection to determine the actual codec that the machine will run. If you don't want to rely on our platform detection, you can bypass it with the FORCE flags, but then it's on you to do proper fallback on older machines.

To get a list of built-in codecs, @BurningEnlightenment's solution is currently the correct one. You basically request some work from a codec and see if it returns an error. Maybe this should be a proper API call, since the test harness already has a use for this functionality...

[WojciechMula]

@aklomp The solution suggested by @BurningEnlightenment seems perfect, but in the end I won't use FORCE flags.

BTW I would suggest to get rid of FORCE flags at all. Instead an extra procedure base64_setup_coded could be available to change codec in runtime. Such procedure could be even enabled by some compile-time definition, as it's intended for tests as you said.

[BurningEnlightenment]

So the idea is that you should build with the maximum SIMD level your compiler will support, and rely on runtime feature detection to determine the actual codec that the machine will run.

Is there actually an use case where you would want to exclude one of the codecs? If not I'm tempted to replace the user options in #7 with some detection logic...

[aklomp]

Well, the basic idea is that you compile for the best case, and fall back to what is available. Sorry if I didn't word this clearly in #7 or the README. (Pull requests welcome :)
You might want to exclude a codec if your compiler does not have proper support for it (e.g. GCC 4.6 miscompiled NEON64), but in that case you could just... not pass in the architecture flags. Also, this library does not do runtime detection on ARM (it's complicated), so on that platform, you do have to choose at compile time.

[WojciechMula]

@aklomp OK, thanks for the explanation. Please close the issue.

BTW, your library was included in a pretty big project, it replaced our in-house code.