-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Probably incorrect error message. #497
Comments
Yeah, that's something expected. See e.g. #280: the GMP library can't nicely workaround out of memory condition, it just die with some error messages, which might vary with the GMP version. (Please, next time specify versions of the gmpy2 and used libraries.) Probably we should at least add a warning in docs. Not sure if it worth, but as it was noted in the referenced issue: it's possible to implement big integers (only basic arithmetic, of course) on top of low-level GMP API, this will not suffer from this problem. |
Recent, of course.
Using only limb functions and structures? |
Yes, sec 8 of the manual - Low-level functions. |
Oh, I forgot. In fact I did docs update, you can see warnings on my fork, which displayed up to date docs: https://gmpy-skirpichev.readthedocs.io/en/latest/overview.html @DennisYurichev, how this looks? @casevh, https://gmpy2.readthedocs.io/ again out of sync, this now display pre-2.2.0 docs. Maybe we could really solve #358? After update, probably this issue should be closed. @casevh, new integer type (based on low-level GMP API) does make sense for you? |
Well, I'm not a gmpy expert, can't say what is the right solution. |
@DennisYurichev, my question was about docs. That's just matter of user experience. Is this a a right place? Will this prevent issues like this or wording in docs is too vague and lacks something? |
Being in shoes of manual's maintainer, I would write something like: "In case of variable overflow, may print 'Floating point error' message, this is a known quirk, to be fixed in future versions". |
Unfortunately, it's not something that could be fixed in the gmpy2 alone. That seems to be a quirk of some OS (M$ Windows, I guess, because message looks totally misleading: e.g. what kind of "Floating point error"?). On my system I got:
|
I haven't access to that huge RAM instance anymore, but I remember I installed gmpy via pip3. But of course, it could be older versions. |
GMP aborts when it encounters an error that it can't handle. Two such errors are out-of-memory and divide-by-zero. There may be others (mpz maximum size overflow which may be handled differently than out-of-memory). gmpy2 tries to catch all the divide-by-zero situations and raise the appropriate exception. To trap out-of-memory, gmpy2 would need to install custom memory handlers into GMP. This was done in the gmpy 1.10 to 1.13 era. Unfortunately, no consistent error message appears across all errors and all platforms. |
Indeed, GMP just call abort() in this case too, per default. But we can catch this:
But divide-by-zero is not one error, that raises SIGFPE. Other example is MPZ_OVERFLOW: this exception happens on reallocation errors e.g. in mpz/realloc.c. I would guess, this is something happened in the second OP example. And that also signals out-of-memory condition. In principle, that case could be caught just as above. But I worry that using longjmp here is unsafe just for same reasons as for custom allocation functions (see below).
Are you sure that this is doable with recent GMP? Unlike above scenario, here GMP just calls abort() in memory.c. You can replace these functions with custom allocation functions, but manual says: "There’s currently no defined way for the allocation functions to recover from an error such as out of memory, they must terminate program execution. A longjmp or throwing a C++ exception will have undefined results. This may change in the future." |
Apologies for not using precise terminology. I was using
The custom memory allocation functions can't prevent GMP from aborting, but by inspecting the return value, they can raise the MemoryError exception. The program still aborts but at least there usually is a consistent error message. I more interested in modifying the memory allocation function for debugging crashes (how much memory was actually requested) and rounding up the size of the memory request to decrease reallocs. I'm also interested in using mimalloc when it used by CPython. There are challenges with shared libraries. When I have time, I'll create an issue for custom memory allocators. |
Looking on the code, I think there are several kind of such errors:
They can, but I doubt it's safe.
I don't think this is a solution. Custom allocation functions can customize behaviour for 3) scenario. But for 1) and 2) - it seems, the correct approach is using a signal handler to test gmp_errno and issue an appropriate error message. (Like above example, just without longjmp/setjmp.) I'll provide such patch. Now docs seems up to date and I think this will address this issue. Hard part if to figure out how to test this for all cases.
Not sure about debugging, but this does make sense for me. |
I tried (unsuccessfully) to find a next term for https://oeis.org/A348651
Using GMP:
On an instance with 300G of RAM, it consumed up to ~80-100G and stopped with the error: "gmp: overflow in mpz type".
Ouch!
Also I tried gmpy:
It also consumed that much RAM and stopped with error "Floating point exception".
Maybe you should know about such behaviour.
The text was updated successfully, but these errors were encountered: