Dynamically determine crypto constants #49

Closed
stouset opened this Issue Mar 15, 2013 · 47 comments

Projects

None yet

5 participants

@stouset

Right now, the values of a bunch of constants from libsodium (e.g., crypto_auth_hmacsha256_BYTES) are hardcoded in the RbNaCl source due to FFI not giving access to C headers.

Is there any interest in determining these dynamically instead? I have code already written for a C extension that copies them over into relevant modules.

void Init_constants(void) {
    VALUE mSodium       = rb_const_get(rb_cObject, rb_intern("Sodium"));
    VALUE mSodiumAuth   = rb_const_get(mSodium,    rb_intern("Auth"));
    VALUE mSodiumBox    = rb_const_get(mSodium,    rb_intern("Box"));

    VALUE mSodiumAuthHmacsha256 = rb_const_get(mSodiumAuth, rb_intern("HMACSHA256"));
    rb_define_const(mSodiumAuthHmacsha256, "PRIMITIVE", rb_str_new2("hmacsha256"));
    rb_define_const(mSodiumAuthHmacsha256, "VERSION",   rb_str_new2(crypto_auth_hmacsha256_VERSION));
    rb_define_const(mSodiumAuthHmacsha256, "BYTES",     INT2NUM(crypto_auth_hmacsha256_BYTES));
    ...
@tarcieri
Cryptosphere member

This is an FFI library. This is sort of a known issue with FFI. I can ask around if there are better alternatives, but C code is not really one of them.

@namelessjon

Yeah, extra C code doesn't really work with FFI, unfortunately. But apparently there is this:

http://rubydoc.info/github/ffi/ffi/FFI/ConstGenerator

Which does ... something ... to pull constants from the library.

@namelessjon

I missed this one the first pass, but seems like it could be used to generate all the constants by name

@tarcieri
Cryptosphere member

Wonder if that's using GCC-XML o_O

Perhaps there's a clang-based alternative?

@tarcieri
Cryptosphere member

Some "helpful" advice from tenderlove ;)

https://twitter.com/tenderlove/status/312356136336633857

@stouset

My approach works with FFI, I believe. All it does is load up existing modules/classes and adds constants to them.

@stouset

To clarify, I'm not asking that the FFI bits be rewritten to be C extensions. But why not add a C extension to cover the cases that FFI can't handle? Is there a down-side I'm not familiar with? It seems like this way we get the best of both worlds.

@tarcieri
Cryptosphere member

@stouset FFI has this functionality built-in. C extensions don't work on JRuby.

@namelessjon

@tarcieri Oh nothing so sophisticated as gcc-xml. It dynamically generates and compiles some C code and then reads the constants from stdout.

@stouset

Hm, I could have sworn that JRuby had deprecated FFI.

@tarcieri
Cryptosphere member

@namelessjon yeah just had a chat with @brixen about how it works. Sounds good

@stouset no, JRuby has absolutely not deprecated FFI

@stouset

There's also ffi-gen, apparently

@tarcieri
Cryptosphere member

I am told ffi-gen does not extract constants, although @whitequark offered to add that functionality

@stouset

Ah, it was the C extension support that was deprecated in 1.7. That makes more sense, then.

@stouset

FFI::ConstGenerator seems like the best approach to this. That is, if you guys think it's worthwhile.

@tarcieri
Cryptosphere member

Yep, seems good

@whitequark

@tarcieri if you're fine with starting gcc at each library load, that is.

http://rubydoc.info/github/ffi/ffi/FFI/ConstGenerator:calculate → show source

@namelessjon

Compiling some c code every time you require 'rbnacl' seems a little heavy

@tarcieri
Cryptosphere member

Err, I assumed this would work as a one-time thing... apparently not? o_O

Something that just populates a nacl_constants.rb file or something via a Rake job would be nice

@whitequark

@tarcieri well that's the idea behind ffi_gen. But I guess you could hack something over ffi::constgenerator in half a hour

@stouset

My initial thinking on this is that RbNaCl would dynamically adapt to whatever libsodium version is currently installed. Obviously, stuff like crypto_auth_hmacsha512256_BYTES isn't going to change, but crypto_auth_PRIMITIVE and the default implementations for crypto_auth_* might.

That would necessitate these being populated on gem install rather than on gem build.

@namelessjon

I guess the 'correct' fix is to get libsodium to export these #defines via extern's or whatever, so we can use FFI::Library.attach_variable

@stouset

That's a reasonable approach. I'd forgotten that we're not stuck with a frozen-in-stone version of NaCl handed down from on-high by DJB. :)

@tarcieri
Cryptosphere member

+1 for dynamically obtaining the sizes via APIs in libsodium. Someone want to open a libsodium ticket for that? /cc @jedisct1

Also, for the record, -1 for anything that requires a C compiler to be installed to use the library

@whitequark

@namelessjon @stouset @tarcieri AFAIK the common solution to this problem in the C world is export the constants via functions, not variables (which you could change and whatnot). In the meantime you could use rake-compiler to provide an FFI-loadable shim which does that.

@namelessjon

@whitequark That's true, re changing. Does making them const not work? I guess not once they are accessible through rubyland? Anyway, The basic idea was 'Access through API', what exact API, I care less about.

@whitequark

@namelessjon depends on whether your compiler will decide to put them in .rodata or not. Yeah.

@stouset

The most conservative route is to use a function. This being crypto, that's probably the best call.

@tarcieri
Cryptosphere member

Now that this is available in libsodium we should definitely take advantage of it

@namelessjon

👍

I have an idea of how to do this relatively easily.

@tarcieri
Cryptosphere member

@namelessjon poke 😉

@namelessjon

... WTF was I thinking 3 months ago? It seemed so clear then.

Might take a little longer than my optimistic comment suggests ;)

@stouset

"I have discovered a truly marvelous way to implement this, which this comment box is too narrow to contain."

@stouset

Sorry, couldn't resist the Fermat joke.

You can use my approach for inspiration if you'd like.

@tarcieri
Cryptosphere member

@stouset nice!

@namelessjon

Ha, nice comment.

Not sure I like your yaml file approach, but I'll figure something out. I did it before.

@tarcieri
Cryptosphere member

@namelessjon perhaps a Ruby "DSL" is in order

@stouset

I considered that for mine, but really a Ruby DSL and a YAML config file are fundamentally pretty much the same thing. Both require the same amount of metaprogramming at any rate. Potato, potah-to.

@tarcieri
Cryptosphere member

@stouset half of one, a dozen of another. Avoiding YAML if possible seems nice though ;)

@stouset

Yeah. I liked having it all in one place, but I can also totally see why it might be nice to have

class RbNaCl::Auth::Poly1305
  include RbNaCl::Delegate

  nacl_family     :crypto_auth
  nacl_constants  %w{ BYTES KEYBYTES }
  nacl_primitives %w{ HMACSHA512256 HMACSHA256 } 

  nacl_functions {
    :crypto_auth        => [ :pointer, :pointer, :ulong_long, :pointer, :int ],
    :crypto_auth_verify => [ :pointer, :pointer, :ulong_long, :pointer, :int ],
  }
end
@stouset

Actually, that doesn't look half bad...

@tarcieri
Cryptosphere member

Seems good bro ;)

@stouset
# Copyright (c) 2013, Stephen Touset

:)

@namelessjon

I'll probably end up with something like that, it does seem a reasonable interface. Hopefully I'll have time to sit down/code this weekend. Thanks for the suggestions/ideas @stouset

@namelessjon namelessjon was assigned Sep 11, 2013
@tarcieri
Cryptosphere member
@tarcieri
Cryptosphere member

@namelessjon if this is something you don't have time to do I can take a crack at it.

@namelessjon namelessjon added a commit that referenced this issue Oct 5, 2013
@namelessjon namelessjon Change how the libsodium binding work
Fixes #78
Fixes #49
Fixes #68

Instead of all functions being bound to one central module, a module is
provided which gives an easy API for binding the return values from
libsodium constant functions to rubyland constants.  It also allows
defining ruby functions which wrap the libsodium ones, accounting for
the return value.

Also, we avoid calling specific implementations.
4f52fc3
@tarcieri tarcieri closed this in #79 Oct 6, 2013
@namelessjon namelessjon was unassigned by tarcieri Mar 1, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment