CyStub is a .pyi Extension Stub Compiler tool for Cython users.
This tool was Inspired by this pull request seen here: cython/cython#3818 Which didn't make it into production and It was inpried by llparse and a few other dsls...
It has now been set in motion and I'll be implementing everything from Cystub to this portion I'll try to get help as to were it should be added and I'll use the previous author's fork if I need help with coming up with the implementation. But basically my goal is to merge it my fork Feel free to hop in and add feedback as I go along we will add a new file called PyiWriter.py to the Compiler Folder if you're curious. I'm now re-writing the Cytstub Code to now put in the final draft.
My Current goals with this implementation are to do all of the following
- Prove that it is possible to create an actual compiler that cython could interact with in theory
- Predict Return Type Annotations when people are too lazy to do so. Which is the hardest part of the whole process. (I actually got part of this done today but it requires some of the steps from cython's own pipeline to be executed (also figured out that Pyrex Types is the best route to take for this but they can be very unfriendly to builtin types) Snippet of this code will be on this repository soon with the foldername (return-value-expermentation RVE for short it might also just end up being a gist, idk))
- This library is mainly an experimentation area but you can call it a lab if you want to The real goal will be to make a pull request to the cython repository when I am done.
Over the years people have had to overcome the hard challege of writing stubfiles manually and the incompatabilty with VS Code has thrown many people off including myself , some of which takes hours just to write manually down, the other issue is that the compiled versions of your modules have to be slightly reserved engineered if you want all the nitty gritty details needed when using your module...
Currently libraries that utilize cython such as PyImgui and PyDukTape2 can be extremely difficult to navigate when it's hoverlist is not avalible.
But now I bring a New Solution to the table ... We can now compile our own stubfiles in a more automatic fasion provided that we are using the already existing Cython Code Writer and it's Compilers for help.
I re-designed the original pyi compiler and made some modifications to it and then I wrapped the compiler around click as it's argument parser for now .
Knowing how exciting this could end up being as well as revolutionary , I wanted to give you a Demo of this possible upcomming feature that I wish to have added to Cython itself in the future by currently giving it it's own Temporary repository until I can have the time to move this functionality straight into cython itself....
Note that I'll stop maintaining this library when cython adds these features we all desperately need into it's own tools...
This is Currently Experimental But I plan to start with testing this on a larger scale with PyImgui and post my interesting results here especially like the way I had kept a dev-journal with winloop so that you can also keep track of my progress...
click and cython are both required to get this to work as this tool attempts to use/have direct acess to the cython compiler itself...
pip install click cython
to use cd to the src directory and run the Cystub.py module to start using it...
python Cystub.py --help
Compiling cython code. Here's an example if you need more help look into what I've written in Cystub...
python Cystub.py -o output yourmodule.pyx othermodule.pxd
pyduktape2 can now be compiled normally and here's it's ouput I'm very proud of what were starting to accomplish over here. Note that I made no changes to the stubfile and that this is actually real. And now typedefines get their own special python converter to help translate the data into something that python has acess to Note that spaces are not meant to be perfect. I wasn't expecting perfection more than I was with variables being accessed and within reach.
# Python stub file generated by Cython 0.29.35 And CyStub.py
import contextlib
import os
import threading
import traceback
DUK_TYPE_NONE : int
DUK_TYPE_UNDEFINED : int
DUK_TYPE_NULL : int
DUK_TYPE_BOOLEAN : int
DUK_TYPE_NUMBER : int
DUK_TYPE_STRING : int
DUK_TYPE_OBJECT : int
DUK_TYPE_BUFFER : int
DUK_TYPE_POINTER : int
DUK_TYPE_LIGHTFUNC : int
DUK_ENUM_OWN_PROPERTIES_ONLY : None
DUK_VARARGS : int
DUK_ERR_ERROR : int
class DuktapeError(Exception):
pass
class DuktapeThreadError(DuktapeError):
pass
class JSError(Exception):
pass
class DuktapeContext(object):
def _check_thread(self): ...
def set_globals(self, **kwargs): ...
def get_global(self, name): ...
def set_base_path(self, path): ...
def eval_js(self, src): ...
def eval_js_file(self, src_path): ...
def get_file_path(self, src_path): ...
def _eval_js(self, eval_function): ...
def make_jsref(self, index: int): ...
class JSRef(object):
def __init__(self, py_ctx: DuktapeContext, ref_index: int): ...
def to_js(self): ...
def __del__(self): ...
class JSProxy(object):
def __init__(self, ref: JSRef, bind_proxy): ...
def __setattr__(self, name, value): ...
def __getattr__(self, name): ...
def __getitem__(self, name): ...
def __repr__(self): ...
def __call__(self, *args): ...
def new(self, *args): ...
def __nonzero__(self): ...
def __len__(self): ...
def __iter__(self): ...
def to_js(self): ...
@contextlib.contextmanager
def wrap_python_exception(py_ctx: DuktapeContext): ...I'll make another pull request to those contributors in a little bit but for now enjoy it... I'll try to see about doing the same with pyimgui soon so please stay tuned for when I'll make that pull request for them...
This tool is not entirely ready to be used on a wide-scale yet but I Welcome Any Contributors with open arms...
There's still a few things I have left to add or change so you'll have to be patient with me but if your feeling egar to have these things or implemented/finished sooner, feel free to start contributing to what I have left on the table. When I get this tool to be stable I'm going to make a pull request to cython and start merging with the PyiCompiler...
I'm Currently working off of my own stubfiles that I am sort of reverse-engineering the definitions for the cython complier and all of it's own cython nodes as I go along https://gist.github.com/Vizonex/58f412067403975be3450bdd0b91298f I'm working with VS Code which maybe also be part of my problem even though I like to have completeness in my code in order to verify that what I'm doing is absolutely correct...
I'll be asking some folks I know from one of the python discord servers for feedback as well since I was able to get some guidence from them with winloop. You will not find my actual alias there but I'm planning to swap the users soon . I mainly had a different alias in there because it's an account I made for guarding a bunch of my experimental discord bots.
def make_tree_from_file(file:str,options:Optional[CompilationOptions]=None):
if not options:
options = CompilationOptions(defaults=default_options)
ctx = Context.from_options(options)
init_thread()
return make_compilation_tree(file,ctx) , ctxinit_thread was the issue so I'll make a pull request shortly...