-
Notifications
You must be signed in to change notification settings - Fork 146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question: compiling multiple modules incrementally #10
Comments
I noticed that the c2mir implementation does not expose an API at the moment so I guess the answer to my question is no? |
Currently to execute MIR code which resides in memory (you can create it there by API or by parsing MIR textual representation or reading MIR binary representation) you need:
You can load module containing a function with a name another loaded function has. In this case the new function will rewrite the old one. For example, we can generate different versions of function for a Ruby method using different speculative assumptions. There a lot of functionality I'd like to add:
As for your scenario: now you can create a context, create mir code, load and link it, optionally generate the binary code for it and interpret it or call the binary code. Then free the context and all allocated memory for this task (including binary code) will be freed. That is how libgccjit works, for example. But you are right there is no currently way to free memory for binary code and reuse it until the corresponding context is released. |
Hi @vnmakarov
Sorry I didn't explain clearly what I am looking to do. Not freeing compiled code is not an immediate issue. I will try to explain more fully. Basically in Ravi, there is automatic compilation and user requested compilation. Either way, I need to compile different modules at different times. All above modules / functions within them will be active. And so on. For now it is not a problem if memory is not released; but how can I do above? In libgccjit I used to only create 1 context. Same with LLVM. I hope this is a bit clearer. Thanks and Regards |
You can use one context for your case. For example,
Module B can refers for a code in module A as it was already loaded and linked. If module B refers for a function (or data) in module A and module A refers for a function in module B, you should link them together:
You can use also different contexts. All generated code currently retains and can be used until finishing the corresponding context. |
Cool, I will try this out over the next couple of weeks. |
Hi, My simple compile front-end looks like this:
The following demonstrates how I can use this:
I have made these changes in my fork of your library. |
I think that it would be useful if the c2mir was also treated as a library / API, which means exposing some api for clients to call. My suggestion is something like:
That is, given an optional buffer plus command line arguments, generate a module and return it. Couple of other observations:
Regards |
Hi @vnmakarov Regarding the two C header files that c2mir looks for at runtime:
Are these essential? Is it possible to change the way they are found? Right now they are expected to be somewhere below "./" - ideally they should be found through the -I include path option, or maybe an environment variable such as C2MIR_INCLUDE_PATH could be used if available. Even better would be to add these definitions in the code itself, rather than relying on header files. |
No they are not essential but defined macros and types in these files are essential for c2mir work. Using the files was a fast implementation of a lot of predefined macros. As you propose they could be hard coded in c2mir code. As I now seriously consider to implement C as an additional input to the JIT (that is how you started to use it for Ravi), it would be nice to get rid of these files. I put this work on my list. So thank you for your proposal. More general question about configuration and installation. The current makefile and file locations are only for my current development.. I did not started to work on packaging and even designing it (what it would be static/dynamic libraries, how many libraries and so on). I can only say that most probably I will use autoconf. I don't like cmake.
|
I guess you should go with your preference. |
Well looks like my code doesn't need those header files. So for now I can simply avoid treating their lack as a fatal error. |
Sorry, I was not accurate. What I meant compiling some C programs (at least from tests I use) need these files. So they are necessary in a general case. But you can write code (and this is your case) where they are not necessary. |
There is a personal preference but the preference is also from the fact that CRuby or CPython does not use cmake. They use autoconf. I hope to have some integration of this project with CRuby project and using cmake would be an issue. |
Hi isn't it the reverse? The general case doesn't need those headers, they are only needed to emulate GCC behaviour? |
I don't know about other environments but when you use GLIBC, you can not compile a lot of standard headers w/o some of these GCC macros. That is why Clang mimics GCC and pretends implementing all GCC features although it is not true, for example you can not still compile elfutils with clang because clang does not implement nested functions which are heavily used by elfutils. |
I guess if you have always worked with gcc and glibc that may seem like the default. But for sure on Windows none of that applies. I think there are two questions here: a) Does MIR need those header files to parse/generate code? My impression is no. a) wasn't clear to me because I assumed those headers contained some code layout data. But when I looked at them they were just #defines and typedefs. |
For the current C2MIR, in some cases, yes. For example, you include a C standard header file whose implementation uses a macro defining endianess of CPU. This macro is provided by GCC. For C2MIR it is taken from the mentioned include file. Another case is macro describing an architecture like x86_64 and aarch64. These macros can be used even in files compiled on Windows, especially in mingw environment. But again, these c2mir header files can be removed and the macro and definitions can be hard-coded in c2mir sources. And I am going to do this later.
Providing full GCC like environment is a huge work, probably bigger that the current C2MIR implementation. So for C2MIR What I do is that I am just trying to implement a minimum (mostly add some predefined GCC macros) to compile and pass the current tests. |
Hi @vnmakarov
In a JIT environment many modules need to be compiled incrementally, i.e. not at the same time. I am planning to use the C2MIR component in Ravi - and my question is: is this currently supported with c2mir?
I noticed that there is a compile / link process in c2mir test code; presumably this needs to be done for all modules as they are compiled. But compiled functions need to be held in memory.
The model I use with LLVM or OMR is this:
Create Context - this holds all compiled objects
Repeat as required:
Compile a new module - compiled functions from new module are saved in the context.
When shutting down, destroy the context, thereby destroying all the compiled objects.
The text was updated successfully, but these errors were encountered: