-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Debian testing issue on VM #7
Comments
Ping @jkeiser |
If the hardware was not supported, you would get "UNSUPPORTED_ARCHITECTURE". |
Right on. I recall scanning the C++ code and seeing a fallback for chipset detection. For completeness, I had sprayed a few prints in. With those I get
So I end up with a bad validation (only on that hardware platform so far; CRAN actually tests with Debian testing too) but the error printing help message comes back with the indicated lack of context / initialization. And, just as a reminder, I am running off the |
Can you make this VM accessible to the simdjson team, via ssh... if only temporarily? |
No matter what, it should not report "uninitialized" when it fails to parse... this is bad. We need to investigate. |
I doubt it. It is in a university data center in Europe and I just a guest myself so there is a fair amount of tape of a certain color ... |
But I can ask. It may take a few (work) days though. |
Yes, UNINITIALIZED indicates an internal error. I doubt it is directly related, but if you could add a |
Er. Checking: what simdjson are you using? I was sort of assuming master, because UNSUPPORTED_ARCHITECTURE and @lemire we do have potential for dueling error codes where sometimes |
That doesn't build -- I am using a current version of the two files in |
Oh! Let me update those files. We generally don't do that until there is a release. I don't know if latest master will fix your problem, but we should probably at least try to get there so we're all working off the same code. I know I did make changes to how error codes are passed around, and that will also let you do the active_implementation code to rule out unsupported architecture issues. |
OK, headers in master are up to date. |
Let me know if there are issues building against it, too: I've been changing up the interfaces a bit. I kept backwards compatibility shims to keep existing code building, but it's always possible there are use cases somewhere not covered by our tests :) |
That worked. (It needed
|
OK, that does help. Can you run |
|
(Just the fourth of four.) From what I recall from conversations with the data center, it is a somewhat dated box. Which, now that I think about it, makes perfect sense. It's probably a decade-old Xeon cpu. Now I feel sheepish for wasting everybody's time :-/ Shall we add a simple predicate testing that we don't find a 386 or alike? |
You are not wasting anybody's time. The machine in question is indeed quite old (going back to 2006). We do not support such old systems. However, we need to have better error reporting. |
(Portable) hardware abstraction is hard. I know there is a sub-library within OpenMPI that does that. Is there something 'cheaper' we can do? |
Yep. I think master now covers the case pretty well as long as the user reports the error message. Is the idea to make it not compile (at least by default) if you are on a machine that couldn't run it? That's not a bad idea at all. |
Or compile but have a true/false flag indicating what we're at -- like |
Also seeing it in the macOS machine(s) at CRAN: https://www.r-project.org/nosvn/R.check/r-release-osx-x86_64/RcppSimdJson-00check.html I guess my added check for C++17 wasn't enough. :-/ |
What about code like the one in |
@eddelbuettel Do you mean "compile" or "run"? You write compile. Let me first handle decisively the "compile" issue. You need C++17. If you have a 64-bit system and a compiler that supports C++17 and simdjson does not compile, then report a bug.
I am not 100% sure, but I think that the error that we are seeing is at runtime... correct? That is, it tries to parse file twitter.json and then fails. If you mean run, and I think you do, then this is entirely different from "compiing" because whether you can run the code or not is a runtime determination. You can fail to compile simdjson on a machine because it does not have the proper compiler, and yet you could be able to run a binary (compiled elsewhere) and vice versa. So it is not at the compilation that you should check support, it is at runtime. You are getting an "Unitialized" error. I expect that it is a runtime error (maybe the CPU is too old?). Unlike @jkeiser, I don't think this is covered in simdjson/master. I think that there is an issue if only a matter of documentation. I am investigating. |
I may have been sloppy -- I meant "compile" as in "install". I understand when something can't run, the C++17 requirements are clear to me. Yet the CRAN setup only allows me to exclude Windows (as it is currently hamstruck by an old g++-4.9.3, hopefully replace by g++-8.* in a few weeks if all goes well) but not the other. And if you go to https://cloud.r-project.org/web/checks/check_results_RcppSimdJson.html you see and I am essentially reasoning about ways to work around the limitations of the ERRORing systems. Testing for |
Back to the original issue. So as much as I'd like to fault @jkeiser, I cannot reproduce the issue with master. Here is the program that I test...
Now, running via the Intel emulator, I can "emulate" the old hardware you have access to:
It gives the error code 16 as expected. It is possible that you are testing with older code... possibly... where we have this bug, but as it is, I cannot seem to reproduce the issue. |
Ok. So for solaris, you have "C++17 standard requested but CXX17 is not defined". That seems straight-forward. I don't know who is using Solaris these days, but it is pretty much legacy at this point, is it not? Can you even buy a Solaris box these days? On the osx boxes, you get "Uninitialized". If it is a matter of the CPU being unsupported, you should be getting UNEXPECTED_ERROR. I am trying to reproduce the issue, but it is kind of difficult. |
@lemire he is now getting UNSUPPORTED_ARCHITECTURE, I'm fairly certain. This might clear it up (@eddelbuettel please confirm):
I assumed it was my fault at first, too :) In fact, it still may have been--it's possible some of my code before Feb 8 screwed this up. |
@jkeiser The error logs are dated from today... so I am going with the assumption that the error still remains...??? |
Oh. I see... Though @eddelbuettel ran private tests with updated code, I don't think he updated his repo. @eddelbuettel Can you update your code to what we have in master under github/.../simdjson ? My hope is that your uninitialized errors will go away. Note that you probably want to handle gracefully the error code UNSUPPORTED_ARCHITECTURE if the servers you are using a too old. That is, getting UNSUPPORTED_ARCHITECTURE is not a sign of failure per se... |
Let's slow down for moment, if we could. I have a pretty firm handle on the CRAN system and build approaches. That is what I asked here if you can help me test for particular hardware that is likely to fail. I understand that is hard(ish) problem so if we can't, we can't. (The Solaris issues of 'C++17 standard requested but CXX17 is not defined' is effectively 100% their problem. I say that I need C++17 (in file (The CRAN logs are indeed from $CALENDAR-DAILY but they use the last-released file, in out case package 0.0.2 and NOT what may be on GitHub.) (My repo is indeed behind; I had another project with an issue and parallel discussion so I never pushed that. Done now.) For a quick glance at how I protected the one C++ entry function from not-having-C++17, see (and uggh just noticed the smell about the For a similar glance at run-time / package-load-time reflecting a compile-time find for the R entry level, see |
So can we build a test predicate? A simple function returning a |
@eddelbuettel oh, if you want a runtime test, that's not too hard :) You can do this in main or wherever you want: if (simdjson::active_implementation->name() == "unsupported") {
printf("unsupported CPU\n");
abort();
} It's the compile-time test I'm unsure about. It feels like we should have EDIT: fixed, thanks to @lemire |
That is excellent. I add that to the arsenal. I still think (as an engineering exercise) I need to make the build better at/before compile time. Ie for the idiotic Solaris build, I can ask at 'configure' time if R knows a C++17 compiler. If so, set Leaves the two stoopid macOS machines with, per an earlier comment, so old a clang version. They compile cleanly, don't produce valid code but thanks to your enhancement to the diagnostics and the 'unsupported arch' I can now flag this. So I can protect the example code, and it should no longer error. That should do. Traveling later in the week so maybe I'll play with that fix then. Thanks for the feedback! |
Thanks for the (simulataneous) posting on the compile-time checks. That looks nifty and I should play with that too, time permitting... |
Well, I realized I was at least partly wrong and deleted my second post :) is_supported() is a runtime thing, I'm not sure how I'd make it a |
I think @jkeiser's code won't work due to a typo, try this instead...
|
I just added that simple in 7c1fb60 -- thanks for the tip! The "scheme" to test for what R is configured with and only turn on C++17 if we see it promptly failed at the Solaris box it was aimed. Not sure what is happening there but not a biggie either way. |
Looks good to me. |
And just checked on the VM in question:
Protected tests with the same predicate so the package no longer 'fails' checks, it simply runs with reduced^Hno functionality due to lacking hardware support -- which is fine as it is beyond our control. So closing this ticket now. |
While testing reverse dependencies for Rcpp, I noticed that RcppSimdJson is flagged as having an issue. And I can repeat that issue on the test box.
The machine is a VM running Debian testing, generally up-to-date, with g++ at version 9.2.1-28 (9.2.1 20200203).
What happens is that when I run
example(validateJSON)
which just invokesvalidateJSON()
with the included (unmodified)twitter.json
file just like the basicsimdjson
example does:We end up in the 'not valid' branch, but do not get useable error message. All we get is 'uninitialized'.
I verified that the same fate happens with the sligtly earlier RcppSimdJson 0.0.1 so it is not recent. Could this possibly be compiler settings we get from the distribution? Invoking
R CMD INSTALL
on the sources leads (on that machine, with its Debian settings) toIs there a possible interaction from
-fstack-protector-strong -Wformat -Werror=format-security
?Or is it a likely interaction from the fact that I am ... in a wimpy VM and
simdjson
wants to talk more directly to the underlying, but virtualized away, metal? This may be the more likely culprit in which case I will just whitelist the package for the reverse dependencies checks on this VM.@lemire Any hints from you would be most welcome, as always ;-)
The text was updated successfully, but these errors were encountered: