New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add_newdocs should happen before PyType_Ready #10167

Open
mattip opened this Issue Dec 6, 2017 · 4 comments

Comments

Projects
None yet
4 participants
@mattip
Copy link
Member

mattip commented Dec 6, 2017

Change add_newdocs to e.g. generate a header file at compile time with the docs (as is done in scipy.special), rather than patching them at runtime after PyType_Ready. Requires some hunting for the correct places to specify the docstrings. The current way tp_doc is hacked (after PyTypeReady) not only leaks the char* string yanked out of the PyStringObject, it does not work on PyPy.

gleaned from #10157, plagarized @pv 's hints

@eric-wieser

This comment has been minimized.

Copy link
Member

eric-wieser commented Dec 6, 2017

The problem with generating the header file at compile time is that fixing a typo in one docstring now causes that header file to change, and now every single C file which includes that header has to be recompiled.

For testing whether a sphinx construct renders correctly, this is a huge burden.

Perhaps we need to generate a .h file for each function? So /doc_header/concatenate.h would contain a single #define of PyArray_docstring_concatenate, or similar?

Or at least, generate a header per C file that we currently have.

@njsmith

This comment has been minimized.

Copy link
Member

njsmith commented Dec 6, 2017

Would it be terrible to move the actual docstrings into .c files in the first place, like most C API functions work, and then we wouldn't need to generate anything at all?

IIUC the only reason they're in a .py file in the first place is that building numpy used to be much harder, and they were split out so contributors could edit the docstrings without having to go through a build cycle. This is much less of an issue these days.

@pv

This comment has been minimized.

Copy link
Member

pv commented Dec 6, 2017

Or, you could autogen the C files. Writing it in python files avoids dealing with C quoting and the "\n" everywhere...

@njsmith

This comment has been minimized.

Copy link
Member

njsmith commented Dec 6, 2017

I guess I don't care that much -- it's not like we have any shortage of autogen code already :-)

If we do this, then instead of generating .h files, I think we can generate a .c file, and link to it. So docstring.c would export a char* called Pyarray_concatenate_docstring or whatever. This would avoid the rebuild-the-world issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment