Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows fixes #162

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

ulrikstrid
Copy link

@ulrikstrid ulrikstrid commented May 22, 2019

This is a PR that implements missing things for Windows.
I'm under no illusion that the code is good or that the linking is done in a idiomatic way but this made it work for me.

It adds a endian.h that is cross-platform compatible. Same as #161.
And a entropy implementation that works on Windows. I decided to try to just use the Windows entropy instead of the unix-specific one without the user choosing.

I'm opening this up for discussion around how to best support Windows going forward.

@avsm
Copy link

avsm commented May 23, 2019

Thanks for this! The PR that this conflicts with is #158, which switches to Dune (and should make Windows significantly easier). I didn't quite understand the reason for having a new Windows module for the entropy -- what didn't work with the existing Lwt one? A lot of the code looks shared.

@ulrikstrid
Copy link
Author

I copied the old code but as there is no /dev/random or /dev/urandom on Windows I implemented it with C ffi with bcrypt instead.
We also don't want to compile the Windows specific code on Unix as that will likely fail.

If #158 is going to be merged soon I would love to implement this against that branch instead as that would likely be easier/cleaner.

BCryptCloseAlgorithmProvider(phAlgorithm, 0);

return Val_int(32);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious about the following points:

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest this is more of a proof of concept, but I'm happy to clean it up to try to make a proper implementation.
I would love more pointers on how to do it as I'm still pretty new to this.

I tried the signature you suggest but had some issues with it, but that might be because I called it with the wrong data.
Is it be possible to pass a Cstruct.t directly and set the bytes in there?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB: I'm neither author nor maintainer of ocaml-nocrypto.

not sure what your goal is, and what "proof of concept" means to you. if your goal is only "nocrypto compiles on windows", you don't even have to mess around with C stubs (but can just use 0s). if you want to provide entropy for nocrypto's rng, you should ensure to retrieve random numbers from windows. in case you want others to use your code, you should be very careful to get proper entropy (i.e. check result codes from the windows API, run some tests to check that the call really modifies the buffer passed).

I have not programmed against windows APIs for years - the above comment is from searching for "/dev/urandom on windows" and following the API reference/guidelines from MSDN.

Is it be possible to pass a Cstruct.t directly and set the bytes in there?

I'm not entirely sure what this means, but you could read how other C functions are used in this library to figure out how it is done -- take a look into the native.ml module. Cstruct.t is just a wrapper (length, offset, bigarray), and to me it looks like the bigarray is passed to C code directly.

I've not looked into anything apart from the get_random_bytes implementation of this PR, and don't have a windows installation to test it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proof of concept as in I wanted to be able to build and run a httpaf/H2 based webserver on Windows (which I now can). The initial goal of this PR was to see if there is any interest in this and to prove that it's possible.

As there seems to be a positive response to this I'll happily keep working on it. But since this is a lot lower level than I'm used to I appreciate the feedback.

As I see it what needs to be done in this part is:
Handle the possible errors, either by using a result or a option depending on what's possible with FFI.
Change to use BCRYPT_RNG_ALGORITHM.
Make the signature bytes -> int -> int, but with error handling.
If there is a possibility to not go via bytes but just use the Cstruct.t directly that would be preferable to the current implementation.

@XVilka
Copy link

XVilka commented Sep 15, 2019

- + ocamlfind ocamlc -g -ccopt '-DACCELERATE -mssse3 -maes -mpclmul' -ccopt '-isystem/home/travis/.opam/ocaml-base-compiler.4.04.2/lib/pkgconfig/../../include/minios-xen -isystem/home/travis/.opam/ocaml-base-compiler.4.04.2/lib/pkgconfig/../../include/minios-xen/posix -m64 -mno-red-zone -fno-reorder-blocks -fno-stack-protector -fno-asynchronous-unwind-tables -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/4.8/include -U__linux__ -U__FreeBSD__ -U__sun__ -U__linux -D__MINIOS__ -I/home/travis/.opam/ocaml-base-compiler.4.04.2/lib/pkgconfig/../../include/mirage-xen-ocaml/include -I/home/travis/.opam/ocaml-base-compiler.4.04.2/lib/pkgconfig/../../include/mirage-xen-posix/include -I/home/travis/.opam/ocaml-base-compiler.4.04.2/include -I/home/travis/.opam/ocaml-base-compiler.4.04.2/include/openlibm' -ccopt '-D_DEFAULT_SOURCE --std=c99 -Wall -Wextra -O3' -package 'bytes ocplib-endian cstruct' -c X/mirage-xen/src/native/misc.c
- In file included from X/mirage-xen/src/native/nocrypto.h:5:0,
-                  from X/mirage-xen/src/native/misc.c:1:
- X/mirage-xen/src/native/endian.h:120:3: error: #error platform not supported
-  # error platform not supported
-    ^
- X/mirage-xen/src/native/misc.c: In function ‘_nc_count_8_be’:
- X/mirage-xen/src/native/misc.c:22:3: warning: implicit declaration of function ‘be64toh’ [-Wimplicit-function-declaration]
-    uint64_t qw = be64toh (*init);
-    ^
- X/mirage-xen/src/native/misc.c:23:3: warning: implicit declaration of function ‘htobe64’ [-Wimplicit-function-declaration]
-    while (blocks --) *(dst ++) = htobe64 (qw ++);
-    ^
- Command exited with code 2.
- pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
-      '-build-dir' '_build' '-plugin-tag' 'package(ocb-stubblr)' '-tag'
-      'accelerate' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md' 'README.md'
-      'src/dllnocrypto_stubs.so' 'src/libnocrypto_stubs.a' 'src/nocrypto.a'
-      'src/nocrypto.cmxs' 'src/nocrypto.cmxa' 'src/nocrypto.cma'
-      'src/nocrypto.cmx' 'src/nocrypto.cmi' 'src/nocrypto.mli'
-      'unix/nocrypto_unix.a' 'unix/nocrypto_unix.cmxs'
-      'unix/nocrypto_unix.cmxa' 'unix/nocrypto_unix.cma'
-      'unix/nocrypto_entropy_unix.cmx' 'unix/nocrypto_entropy_unix.cmi'
-      'unix/nocrypto_entropy_unix.mli'
-      'X/mirage-xen/src/libnocrypto_stubs+mirage-xen.a']: exited with 10
[ERROR] The compilation of nocrypto failed at
        "/home/travis/.opam/opam-init/hooks/sandbox.sh build ocaml pkg/pkg.ml
        build --pinned true --tests false --with-lwt false --xen true --mirage
        false --freestanding false".

#=== ERROR while compiling nocrypto.0.5.4-1 ===================================#
# context     2.0.3 | linux/x86_64 | ocaml-base-compiler.4.04.2 | pinned(file:///home/travis/build/mirleft/ocaml-nocrypto)
# path        ~/.opam/ocaml-base-compiler.4.04.2/.opam-switch/build/nocrypto.0.5.4-1
# command     ~/.opam/opam-init/hooks/sandbox.sh build ocaml pkg/pkg.ml build --pinned true --tests false --with-lwt false --xen true --mirage false --freestanding false
# exit-code   1
# env-file    ~/.opam/log/nocrypto-29161-58c514.env
# output-file ~/.opam/log/nocrypto-29161-58c514.out
### output ###
# [...]
# Command exited with code 2.
# pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
#      '-build-dir' '_build' '-plugin-tag' 'package(ocb-stubblr)' '-tag'
#      'accelerate' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md' 'README.md'
#      'src/dllnocrypto_stubs.so' 'src/libnocrypto_stubs.a' 'src/nocrypto.a'
#      'src/nocrypto.cmxs' 'src/nocrypto.cmxa' 'src/nocrypto.cma'
#      'src/nocrypto.cmx' 'src/nocrypto.cmi' 'src/nocrypto.mli'
#      'unix/nocrypto_unix.a' 'unix/nocrypto_unix.cmxs'
#      'unix/nocrypto_unix.cmxa' 'unix/nocrypto_unix.cma'
#      'unix/nocrypto_entropy_unix.cmx' 'unix/nocrypto_entropy_unix.cmi'
#      'unix/nocrypto_entropy_unix.mli'
#      'X/mirage-xen/src/libnocrypto_stubs+mirage-xen.a']: exited with 10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants