-
Notifications
You must be signed in to change notification settings - Fork 17
How to set up a development environment
This page was started from Issue 10. This guide is most helpful for setting up an eclipse environment on linux but if you have built a working development environment in any other IDE or OS, please add relevant information to this page.
Before doing anything else it is worth making a specific folder which will contain all the git repo's connected to this project; in a terminal navigate to where you want this folder to be located and run the command mkdir JyNI-Project, then cd JyNI-Project.
At this point you need to find the url's for all the relevant git repos. At a minimum you will need the JyNI repo, a jython jar and a pyconfig.h but it is recommended that you have the full jython project and full CPython project(v2.7.15) so that you can debug more effectively and make changes to jython where needed. If you wish to improve numpy support you should also have the numpy repo and the nose repo(for running tests). To clone all these git repositories, type the following commands:
git clone https://github.com/Stewori/JyNI.git
git clone https://github.com/jythontools/jython.git
git clone https://github.com/python/cpython.git --branch v2.7.15
git clone https://github.com/numpy/numpy.git
git clone https://github.com/nose-devs/nose.git
Note: Current instructions cover Linux only and no debug versions yet. There are further instructions on setting up builds in eclipse later.
Jython - ant build
JyNI - edit makefile then make
numpy - python setup.py build_ext --inplace
Python - ./configure then make
TODO explain importing Python project (This is dependant on building I think)
Open eclipse and set up a workspace, it can be helpful to put this inside JyNI-Project to keep everything together, then from the menu, click: Window->Show View->Other->Git->Git Repositories->Open. In the Git Repositories, click Add an existing local Git Repository to this view. Then select the JyNI-Project directory and tick the Look for nested Repositories box, now select all the repositories and click finish.
You should install the C/C++ and PyDev plugins before importing any of the projects, if the plugins are not installed already, do this now.
Now you should see all the repositories in the Git Repositories section of eclipse. Using the drop down arrows look into the repositories, then right click on each of the folders listed below and click import projects... Each of these projects will need to be configured with the correct nature for their language(given after the -).
- JyNI/Working Tree/DemoExtension - C
- JyNI/Working Tree/JyNI-C - C
- JyNI/Working Tree/JyNI-Demo - Python
- JyNI/Working Tree/JyNI-Java - Java
- JyNI/Working Tree/JyNI-Lib - Python
- JyNI/Working Tree/JyNI-Loader - C
- jython/Working Tree/src - Java
- numpy/Working Tree/numpy - Python
If you don't add the correct natures to the Python and C projects as you import them, the natures can be added afterwards by right clicking on the project and selecting the following:
C: New->Other->C/C++->Convert to a C/C++ Project (Adds C/C++ Nature)
Python: PyDev->Set as PyDev Project
First the jython interpreter must be set up, it can also be helpful to have a python interpreter available for comparing behaviour of tests. From the menu: Window->Preferences->PyDev->Interpreters->Jython Interpreter->New. Set the name to whatever you want and the executable to the jython jar in your jython build(JyNI-Project/jython/dist/jython.jar). Then add the following to the libraries tab:
- JyNI-Project/JyNI/build/JyNI.jar
- JyNI-Project/numpy
- JyNI-Project/nose-1.3.7
- JyNI-Project/Python-2.7/lib/python2.7/lib-dynload
- JyNI-Project/Python-2.7/lib/python2.7/lib-tk
Then click Apply and Close, you should now be able to right click on a python file and run as->Jython run.
Setting up a python interpreter is a little simpler. From the menu: Window->Preferences->PyDev->Interpreters->Python Interpreter->New. Set the name to whatever you want and the executable to the python executable you wish to use. If you have a local development build of python you should use the interpreter it creates. You should also add the following to the libraries tab:
- JyNI-Project/numpy
- JyNI-Project/nose-1.3.7
- JyNI-Project/Python-2.7/lib/python2.7/lib-dynload
- JyNI-Project/Python-2.7/lib/python2.7/lib-tk
Then click Apply and Close, you should now be able to right click on a python file and run as->Python run.
It can often be simpler to just have a terminal open in which you run the build commands for the project you changed before running it, however, it is possible to set up eclipse to build some things.
Right click on the project then in the menu Properties->C/C++ Build, in the Builder Settings tab: untick Use default build command, set Build command to make and set Build location to JyNI-Project/JyNI. Then in Properties->C/C++ Build in the Behavior tab, set the Make build target: for Build (Incremental build) to clean all. This cleans first so that if you change header files then it will actually be recompiled.
If you are on a branch with an updated makefile(currently written, but not in master), instead of clean all you can put clean-libJyNI debug-libJyNI, this will only clean and recompile the JyNI-C section of the codebase (and it will compile with debug symbols). If you want to compile the whole codebase with debug symbols you can put clean debug instead (assuming your branch's makefile supports it).
The instructions for this project are the same as for JyNI-C unless you are on a branch with an updated makefile, in which case, instead of clean-libJyNI debug-libJyNI you can put clean-libJyNI-Loader debug-libJyNI-Loader to only clean and recompile the JyNI-Loader section of the codebase (with debug symbols).
Right click on the project then in the menu Properties->C/C++ Build, in the Builder Settings tab, untick Use default build command and set Build command to python setup.py. Then in Properties->C/C++ Build in the Behavior tab, set the Make build target: for Build (Incremental build) to build. If you have an updated makefile, you can also follow the instructions for JyNI-C but put build-tests instead of clean-libJyNI debug-libJyNI
The only way I have found to do this is to first add the C/C++ nature to the project (the java nature will be preserved) and then follow the instructions for JyNI-Loader and JyNI-C. In this case it shouldn't matter what branch you are on, the Make build target: for Build (Incremental build) should be set to cleanJ JyNI.
TODO jython building in eclipse
The python debugger should just work with no additional configuration.
If you want to be able to attach a java debugger to follow the code execution into the java code, the following arguments will need to be added to the VM arguments box in Run->Debug Configurations->Jython run->New(the top left icon)->Arguments -Dpython.security.respectJavaAccessibility=False -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=1044. This new configuration can then have the project and main module selected in the Main tab to run the file you want it to.
To actually attach a java debugger to the process you then need to go to Run->Debug Configurations->Remote Java Application->New. set the Host to localhost and the port to 1044, then make sure the source tab has all the java source code you are likely to be interested in added to the source lookup path.
The compiled .so file needs to have the debug symbols added for gdb to be able to debug effectively. A debug target has been added to the makefile to enable this. Simply run the command make debug in the JyNI-Project/JyNI folder (where you ran make to build JyNI).
Go to Run->Debug Configurations->C/C++ Attach to Application->New. Set the project and application , if you are testing a specific C extension then set the C/C++ Application field to be the .so file for that extension, otherwise set it to the libJyNI.so file in JyNI-Project/JyNI/build. Make sure the source tab has all the .c source code files you are likely to be interested in added to the source lookup path.
Now switch to the debugger tab, it is recommended that the gdb debugger should be configured to be version 8.1.0, other versions sometimes have issues with either eclipse or java or both (newer versions may work but haven't been tested). The command file can be set to be wherever you like but it is recommended to either put it in your home folder or in JyNI-Project. The command file needs to have the following lines added so that it can debug native calls from the jvm:
handle SIGSEGV nostop noprint pass
handle SIGXCPU nostop noprint pass
handle SIGPWR nostop noprint pass
Non-stop mode is also a VERY helpful box to tick, it prevents gdb from pausing the jvm as soon as it attaches and means that gdb will only stop the jvm thread that has hit a breakpoint when one is triggered. This means that you can continue to add java breakpoints while the jvm has been stopped by gdb in some C code.
When gdb launches it will need to be told which process to attach to, a popup box will appear with process names and ID's. Fortunately when the pydev debugger starts, it print's the pid of the process to the console, this is the pid to select from the list (starting to type "java" will shorten the list of processes to only the java processes). To give time for you to do this it is useful to have a breakpoint in the python code that triggers before the C code you are interested in is called.
When debugging with gdb, it is possible to run expressions using the debugger console tab(not the normal console tab). If the last line of the console doesn't start with (gdb) then press return, once the line does start with (gdb), simply use the command print Expression or p Expression to print the result of evaluating "Expression". There is a little more info here but using the help or h command in the debugger console is probably more useful.
Another helpful feature of eclipse is the launch group feature, select Run->Debug Configurations->Launch Group->New. Simply add your jython run, remote java application and C/C++ Attach to application launch configurations to the launches tab. This allows you to launch the 3 debuggers with one debug launch.
It is worth noting that in debug mode there are additional errors that will occur due to timing being thrown off. There is an issue (2517) in jython which explains some of this.