REBx not playing well with REBOUND Version 3.17

[Rmelikyan]

Hi Dan,

Following a conversation with @hannorein, I upgraded my local Rebound project to version 3.17. During my next build of Rebx (trying to better diagnose #62) I encountered a build error.

I found that in steppers.c that many functions that were previously defined in rebound.h such as reb_integrator_ias15_reset had been moved to either integrator_ias15.h or integrator_whfast.h. These changes to the rebound.h file occurred during the documentation updates in the commit by @hannorein titled:

'The documentation is now using MkDocs and the Material theme.'

For the C build, I was able to solve these errors imply by including these files at the top of steppers.c. When I continued however to build the local pip installation (i.e. pip install -e .) the build broke with a fatal file not found error, where the newly included files integrator_ias15.h and integrator_whfast.h where causing the issue. I guess this is solvable with an updated include path for the compilation, but I am not that experienced yet with such flags.

I believe that this issue should be present for any user who has upgraded to REBOUND v3.17

Let me know how I can help,

-Robert

[dtamayo]

For some reason I can't reproduce this issue using 3.17 and the travis tests are still passing. We have to do some hacks with the pip installation to make it find REBOUND, and not being able to reproduce the error makes it hard to fix it. I see what you mean though. Can you try making a new installation in a separate directory of both the latest REBOUND and the latest REBOUNDx? That might help isolate the issue.

[Rmelikyan]

Ok I did a little more testing...

It seems that, for whatever reason, I can't have both rebound & reboundx pip installed locally without an error.

I can run the local pip install on the most recent rebound code and that runs smoothly. If I proceed to locally install the most recent reboundx code, I get thrown errors regarding steppers.c

However, if I pip install rebound from the web (it still is pulling 3.17) then I can proceed to pip install -e . on my local reboundx project and that works!

What's the difference? I think this works ok for now as I plan to be mostly developing in Reboundx, but I'm definitely curious as to what is happening here.

[Rmelikyan]

And, back to the C side of things. Even on fresh downloads of both code bases, I get the following errors when I attempt to build the project:

steppers.c:61:5: error: implicit declaration of function 'reb_integrator_ias15_reset' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_ias15_reset(sim);
^
steppers.c:61:5: note: did you mean 'reb_integrator_reset'?
../../rebound/src/rebound.h:641:6: note: 'reb_integrator_reset' declared here
void reb_integrator_reset(struct reb_simulation* r);
^
steppers.c:67:9: error: implicit declaration of function 'reb_integrator_ias15_part2' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_ias15_part2(sim);
^
steppers.c:67:9: note: did you mean 'reb_integrator_ias15_reset'?
steppers.c:61:5: note: 'reb_integrator_ias15_reset' declared here
reb_integrator_ias15_reset(sim);
^
steppers.c:77:5: error: implicit declaration of function 'reb_integrator_whfast_init' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_whfast_init(sim);
^
steppers.c:78:5: error: implicit declaration of function 'reb_integrator_whfast_from_inertial' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_whfast_from_inertial(sim);
^
steppers.c:78:5: note: did you mean 'reb_integrator_whfast_init'?
steppers.c:77:5: note: 'reb_integrator_whfast_init' declared here
reb_integrator_whfast_init(sim);
^
steppers.c:79:5: error: implicit declaration of function 'reb_whfast_kepler_step' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_whfast_kepler_step(sim, dt);
^
steppers.c:79:5: note: did you mean 'rebx_kepler_step'?
steppers.c:76:6: note: 'rebx_kepler_step' declared here
void rebx_kepler_step(struct reb_simulation* const sim, struct rebx_operator* const operator, const double dt){
^
steppers.c:80:5: error: implicit declaration of function 'reb_whfast_com_step' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_whfast_com_step(sim, dt);
^
steppers.c:80:5: note: did you mean 'reb_whfast_kepler_step'?
steppers.c:79:5: note: 'reb_whfast_kepler_step' declared here
reb_whfast_kepler_step(sim, dt);
^
steppers.c:81:5: error: implicit declaration of function 'reb_integrator_whfast_to_inertial' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_whfast_to_inertial(sim);
^
steppers.c:81:5: note: did you mean 'reb_integrator_whfast_from_inertial'?
steppers.c:78:5: note: 'reb_integrator_whfast_from_inertial' declared here
reb_integrator_whfast_from_inertial(sim);
^
steppers.c:85:5: error: implicit declaration of function 'reb_integrator_whfast_init' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_whfast_init(sim);
^
steppers.c:86:5: error: implicit declaration of function 'reb_integrator_whfast_from_inertial' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_whfast_from_inertial(sim);
^
steppers.c:87:5: error: implicit declaration of function 'reb_whfast_jump_step' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_whfast_jump_step(sim, dt);
^
steppers.c:88:5: error: implicit declaration of function 'reb_integrator_whfast_to_inertial' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_whfast_to_inertial(sim);
^
steppers.c:92:5: error: implicit declaration of function 'reb_integrator_whfast_init' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_whfast_init(sim);
^
steppers.c:93:5: error: implicit declaration of function 'reb_integrator_whfast_from_inertial' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_whfast_from_inertial(sim);
^
steppers.c:95:5: error: implicit declaration of function 'reb_whfast_interaction_step' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_whfast_interaction_step(sim, dt);
^
steppers.c:95:5: note: did you mean 'rebx_interaction_step'?
steppers.c:91:6: note: 'rebx_interaction_step' declared here
void rebx_interaction_step(struct reb_simulation* const sim, struct rebx_operator* const operator, const double dt){
^
steppers.c:96:5: error: implicit declaration of function 'reb_integrator_whfast_to_inertial' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
reb_integrator_whfast_to_inertial(sim);
^
15 errors generated.
make[1]: *** [steppers.o] Error 1
make: *** [libreboundx.so] Error 2

I've located the functions that are being referenced here, and, as best as I can tell, the've been moved into new locations in the most recent rebound file (i.e. integrator_ias15.h and integrator_whfast.h)

Is it possible that some commits that have been added to Rebound since the v3.17 release have broken compatibility?

[dtamayo]

Thanks for testing this Robert. I think the issue is that my compiler was spitting out warnings rather than errors. I've just added the appropriate includes in steppers.c and this works fine. I pushed the change to the REBOUNDx repo. Does that resolve your issue?

[hannorein]

I've just pushed a new version (3.17.1) to pypi. I don't think this was the issue, but just in case...

[dtamayo]

well by adding those includes I was able to mess things up on travis :) The issue is that we had to do some hacks to help pip installed versions to find the REBOUND header files. I can't sort this out until next week, but this should work if you clone the REBOUND and REBOUNDx repos side by side into the same directory (I.e. mkdir workspace; cd workspace; git clone REBOUND; git clone REBOUNDx). The makefiles should then be able to find. Or you could use an environment variable like in the docs: https://reboundx.readthedocs.io/en/latest/c_quickstart.html#installation

[dtamayo]

and if you add the includes in steppers.c. I've reverted REBOUNDx to not include those so people pip installing don't get any surprises.

[hannorein]

@dtamayo , if it helps, I can move the function declaration that REBOUNDx needs back into the rebound.h file.

[dtamayo]

Thanks @hannorein . That would certainly fix things for now.

[hannorein]

Ok. I think I moved all of them, but I also can't reproduce the error locally, so I'm not sure if I've missed one.

[dtamayo]

Thanks @hannorein . @Rmelikyan can you check whether the latest versions fix your problem?

[Rmelikyan]

Local Pip install now works for both Rebound v3.17.2 and Rebx v3.3.0!

Thanks for helping with this!

[sabaronett]

@dtamayo, not sure what progress has been made since @Rmelikyan's last #63 (comment), but pip install reboundx from PyPI still fails on a fresh conda venv w/ python 3.9.5 and rebound 3.17.3 (see log.txt). But pip install -e . in a local clone of reboundx 3.3.0 still works.

[sabaronett]

Not sure if it's related, but I encounter errors when compiling problem.c for any of the C examples/*; I'm able to run ./rebound only after a second make in a row outputs the binary; see make_errors.txt.

[hannorein]

derivatives.o: file not recognized: File truncated

That's a strange error. And it only shows up at the linking stage. I do not know what that could mean. ...

[dtamayo]

Thanks all, this is very useful! I am on vacation with my family this week but will sort this out first thing next week

[sabaronett]

@dtamayo, no worries. Enjoy your vacation!

[dtamayo]

Thanks for your patience! The latest commit and latest version on PyPI (3.3.1) is a temporary fix that forces installation of REBOUND 3.16.0 so that things will install. Working on an actual patch now.

[dtamayo]

REBOUNDx 3.4.0 should fix things now. Could you make sure it works for you and, if so, close the issue? Thanks for the bug report!

[sabaronett]

No more issues installing rebound 3.17.3 and reboundx 3.4.0 in a fresh conda env; also pulled reboundx's latest origin/master and ran TidesConstantTimeLag.ipynb with the fresh ipykernel w/ no issues.

Still encountered compile errors for C examples, so I opened #67.

GH won't let me close this issue (permissions?), but maybe @Rmelikyan can.

[dtamayo]

Thanks Stanley. I'll close this one for now, feel free to reopen @Rmelikyan !