export sizeof(secp256k1_surjectionproof) as SECP256K1_SURJECTIONPROOF_RAW_SIZE #37
the rationale is explained in #36 - knowing the struct size used in particular version of the library enables to alloc the struct without access to
For example, with python:
2 times, most recently
Dec 24, 2018
If the library authors decide that this is better done with a function that returns the size, I will submit a PR for that. I am preparing to create a release of python-bitcointx with support for Elements sidechain, but having a requirement to apply the patch to the secp256k1-zkp source before using python-bitcointx for Elements is not very convenient. I would appreciate a comment from the authors about how they think this should be handled - I will submit the code according to that.
I ported the transaction blinding code from C++ in Elements (https://github.com/ElementsProject/elements/blob/7f6351782a1555696e69d5ffc5304005a07876cd/src/blind.cpp#L224)
In SurjectOutput(), the
By using the patched library, the patch is in this PR.
In this PR, this is done by exporting a special symbol that is statically assigned with
Here is the lines in python code that does the allocation using this symbol: https://github.com/Simplexum/python-bitcointx/blob/31f95df76ae163a5139ef0bf6a9ec556c9c09cfd/bitcointx/sidechain/elements.py#L1473
If I understand correctly @apoelstra is asking because #36 can be interpreted as you're not using the include file. But it seems more like you are using it but you're concerned that the include file might be inconsistent with the (dynamically loaded) library. If that's the case I think it makes more sense to ship it with your code, link it statically or otherwise ensure that include file and library are consistent (as is the case with other libraries not only libsecp). If the files are inconsistent you might have more problems than just a different surjectionproof size.
TL;DR: Having build-time-defined value only in header file forces projects to use static linking or bundle the lib source with a project. This makes sense when the library is in consensus-critical code path, but not in other uses.
It is normal to use dymanically loaded library without the header, especially if you are using the library through FFI of some higher-level language. Linux distos usually ship headers in a separate *-dev package or the like. If you have certain values defined in a header file (that are part of the API), you just take this constants and copy to your higher-level language library code. For example, secp256k1_generator struct in include/secp256k1_generator.h is guaranteed to be 64 bytes in size (as per the comments in the file), so I can use SECP256K1_GENERATOR_SIZE = 64 in python code.
If there is some value that should not be copied verbatim from the API header file, there is usually a function to get the needed value (if it is calculated at runtime), or exported symbol with the said value (if it is defined at build-time), or some other way, but the idea is that you do not need the header file at runtime. Otherwise, the purpose of dynamic library is lost for any languages beside C/C++ (or maybe a couple of other langs that can parse C headers natively) and for systems without a compiler installed. Everyone would need to include their copy of the library with known build-time values. This might be OK for commercial software distribution, where you probably shipping everything as a bundle, but if this is a dynamic library that is going to be included in distros, then the functions that are part of the API should be usable without headers, in my opinion. (and for example secp256k1 is inclued in many distros, with headers in separate secp256k1-dev package in debian)
Let's say I add C extension to python-bitcointx package with the only purpose of getting this value from the header. Then:
So the only options without having this value exposed at runtime seems to be:
These breakage problems are not unique to python module, but is also present for C/C++ projects that do not include the library along with the distribution or statically link with it. If the code is not compatible with dynamic library, dynamic loader should not be able to load the library into the executable (version mismatch or symbol not found etc). If dynamic linker is able to link two incompatible code chunks, this will happen sooner or later. So having such a (non-totally-constant) value only in header file forces projects to use static linking or bundle the lib source with a project. This makes sense when the library is in consensus-critical code path, but not in other uses.
Sorry for the late reply and thanks for making your case so clearly. There was some discussion about your approach in #secp256k1 on freenode and I suggest you bring up your proposal there because I'm not knowledgeable enough in this area to mediate.
Fwiw I've had to overcome a similar problem when writing a rust bindings for MuSig including the the MuSig session type. Instead of allocating as many bytes as the size of a session, in rust it's possible to recreate the struct with the same layout as in C (see https://github.com/jonasnick/rust-secp256k1-zkp/blob/dea9e50b41bd70aae6cd705892f06e722755976d/src/ffi.rs#L39, that's what
This is only marginally better than just measuring the size of the struct by
And then you have to make sure that old version of your code (which uses smaller size for the struct) is never used with new version of the library. Otherwise you will get memory corruption.
This approach works only if the exact binary struct layout is strictly defined, and is part of the protocol/API.
Changed the approach a bit: in the header, I do
while in secp256k1.c, I #undef that name, and then declare the const variable that would be exported:
This allows for the name
After conversation on #secp256k1 channel it was determined that better approach is to introduce create/destroy functions for this struct, and the create function would also call initialize function, so the caller won't need to do extra call to init the struct. The code that wants to allocate the struct on the stack would still be able to do so, and would need to call initialize function directly, as before.
this PR is therefore superseded by #55
I'm closing this PR.