Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

from git://git.fedorahosted.org/gcc-python-plugin.git

branch: master

alignment fixes

latest commit f77982e513
Jason Mueller authored March 16, 2012
Octocat-spinner-32 docs cpychecker: get the checker working on C++ code March 07, 2012
Octocat-spinner-32 examples minor PEP8 fix in example code January 10, 2012
Octocat-spinner-32 libcpychecker alignment fixes March 16, 2012
Octocat-spinner-32 misc misc/fedora: update status of mass rebuild March 05, 2012
Octocat-spinner-32 tests cpychecker: get the checker working on C++ code March 07, 2012
Octocat-spinner-32 .gitignore skeleton of an html page March 15, 2012
Octocat-spinner-32 COPYING Add COPYING and COPYING.LESSER, for the LGPLv3 June 08, 2011
Octocat-spinner-32 Makefile try to support setting GCC in the environment to avoid hardcoding 'gc… March 06, 2012
Octocat-spinner-32 README.rst Add note about ccache July 27, 2011
Octocat-spinner-32 configbuilder.py try to support setting GCC in the environment to avoid hardcoding 'gc… March 06, 2012
Octocat-spinner-32 contributors.rst add David Narvaez to contributors.rst January 10, 2012
Octocat-spinner-32 cpybuilder.py add metaclass to PyGccWrapper to fix garbage-collection segfault on u… January 18, 2012
Octocat-spinner-32 cpychecker.py Add GPLv3 license headers to all source files June 20, 2011
Octocat-spinner-32 demo.c Add the example from my blog post ( http://dmalcolm.livejournal.com/6… June 25, 2011
Octocat-spinner-32 gcc-python-attribute.c add support for creating custom attributes from Python October 24, 2011
Octocat-spinner-32 gcc-python-callbacks.c add a configuration phase and use it to support PLUGIN_FINISH_DECL February 02, 2012
Octocat-spinner-32 gcc-python-callgraph.c integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-cfg.c integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-closure.c bug in gcc_python_closure_new_generic October 31, 2011
Octocat-spinner-32 gcc-python-closure.h track which event we're within when a callback fires October 25, 2011
Octocat-spinner-32 gcc-python-compat.h integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-diagnostics.c integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-docs Various pre-release cleanups June 21, 2011
Octocat-spinner-32 gcc-python-function.c integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-gimple.c fix 8 memory leaks within the plugin February 06, 2012
Octocat-spinner-32 gcc-python-location.c integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-option.c integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-parameter.c integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-pass.c integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-plugin.spec prepare for 0.9 release February 06, 2012
Octocat-spinner-32 gcc-python-pretty-printer.c Add GPLv3 license headers to all source files June 20, 2011
Octocat-spinner-32 gcc-python-rtl.c fix 8 memory leaks within the plugin February 06, 2012
Octocat-spinner-32 gcc-python-tree.c cpychecker: get the checker working on C++ code March 07, 2012
Octocat-spinner-32 gcc-python-variable.c integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gcc-python-version.c move version handling into its own file January 12, 2012
Octocat-spinner-32 gcc-python-wrapper.c tidy up insides of gcc_python_wrapper_meta_tp_new January 18, 2012
Octocat-spinner-32 gcc-python-wrappers.h cpychecker: get the checker working on C++ code March 07, 2012
Octocat-spinner-32 gcc-python.c cpychecker: get the checker working on C++ code March 07, 2012
Octocat-spinner-32 gcc-python.h add a configuration phase and use it to support PLUGIN_FINISH_DECL February 02, 2012
Octocat-spinner-32 gcc-with-cpychecker cpychecker: make gcc-with-cpychecker run the refcount-checking code October 21, 2011
Octocat-spinner-32 gcc-with-python Set up various things in sys, including sys.path June 24, 2011
Octocat-spinner-32 gccutils.py cpychecker: get the checker working on C++ code March 07, 2012
Octocat-spinner-32 generate-callgraph-c.py integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 generate-cfg-c.py integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 generate-config-h.py add a configuration phase and use it to support PLUGIN_FINISH_DECL February 02, 2012
Octocat-spinner-32 generate-function-c.py integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 generate-gimple-c.py add "string" attribute to gcc.GimpleAsm February 13, 2012
Octocat-spinner-32 generate-location-c.py integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 generate-option-c.py integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 generate-parameter-c.py integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 generate-pass-c.py integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 generate-passes-svg.py Add an autogenerated SVG diagram of GCC's passes July 11, 2011
Octocat-spinner-32 generate-pretty-printer-c.py Add GPLv3 license headers to all source files June 20, 2011
Octocat-spinner-32 generate-rtl-c.py integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 generate-tables-of-passes-rst.py Try to scale image July 11, 2011
Octocat-spinner-32 generate-tree-c.py cpychecker: get the checker working on C++ code March 07, 2012
Octocat-spinner-32 generate-variable-c.py integrate with GCC's garbage collector (ticket #1) January 13, 2012
Octocat-spinner-32 gimple-types.txt.in Generate appropriate gcc.Gimple subclasses for each statement; add so… April 25, 2011
Octocat-spinner-32 maketreetypes.py Initial wrapping of the Register Transfer Language July 28, 2011
Octocat-spinner-32 rtl-types.txt.in Initial wrapping of the Register Transfer Language July 28, 2011
Octocat-spinner-32 run-test-suite.py try to support setting GCC in the environment to avoid hardcoding 'gc… March 06, 2012
Octocat-spinner-32 test-builder.py try to support setting GCC in the environment to avoid hardcoding 'gc… March 06, 2012
Octocat-spinner-32 test.c Add GPLv3 license headers to all source files June 20, 2011
Octocat-spinner-32 test.py Add GPLv3 license headers to all source files June 20, 2011
Octocat-spinner-32 testcpybuilder.py try to support setting GCC in the environment to avoid hardcoding 'gc… March 06, 2012
Octocat-spinner-32 testcpychecker.py cpychecker: convert format string checker from permerror to warning January 03, 2012
Octocat-spinner-32 tree-types.txt.in Try a more flexible approach to autogenerating Tree wrapper subclasses April 20, 2011
Octocat-spinner-32 wrapperbuilder.py add metaclass to PyGccWrapper to fix garbage-collection segfault on u… January 18, 2012
README.rst

gcc-python

This is a plugin for GCC, which links against libpython, and (I hope) allows you to invoke arbitrary Python scripts from inside the compiler. The aim is to allow you to write GCC plugins in Python.

The plugin is Free Software, licensed under the GPLv3 (or later).

It's still at the "experimental proof-of-concept stage"; expect crashes and tracebacks (I'm new to insides of GCC, and I may have misunderstood things).

It's already possible to use this to add additional compiler errors/warnings, e.g. domain-specific checks, or static analysis. One of my goals for this is to "teach" GCC about the common mistakes people make when writing extensions for CPython, but it could be used e.g. to teach GCC about GTK's reference-counting semantics, or about locking in the Linux kernel, or about signal-safety in APIs.

Other ideas include visualizations of code structure. Given a gcc.CFG instance, gccutils.render_to_dot(cfg) and gccutils.invoke_dot(cfg) will use graphviz and eog to plot a handy visualization of a control flow graph, showing the source code interleaved with GCC's GIMPLE internal representation.

Requirements

  • GCC: 4.6 or later (it uses APIs that weren't exposed to plugins in 4.5)

  • Python: tested with 2.7 and 3.2; it may work with earlier versions

  • "six": The libcpychecker code uses the "six" Python compatibility library to smooth over Python 2 vs Python 3 differences, both at build-time and run-time:

    http://pypi.python.org/pypi/six/

Usage

I use:

make

to build the plugin and run the tests

You can also use:

make demo

to demonstrate the new compiler errors.

All of my coding so far has been on Fedora 15 x86_64, using:

gcc-plugin-devel-4.6.0-0.15.fc15.x86_64

and I don't know to what extent it will be compatible with other versions and architectures.

The code also makes some assumptions about the Python version you have installed (grep for "PyRuntime" in the .py files). I've been using:

python-devel-2.7.1-5.fc15.x86_64
python-debug-2.7.1-5.fc15.x86_64
python3-debug-3.2-0.9.rc1.fc15.x86_64
python3-devel-3.2-0.9.rc1.fc15.x86_64

but you may have to hack up the PyRuntime() invocations in the code to get it to build on other machines. Ultimately I want the plugin to be buildable against multiple python versions, so there could be a python27.so, python27-debug.so, python-32mu.so, python-32-dmu.so, etc (c.f. PEP-3149)

There isn't an installer yet. In theory you should be able to add these arguments to the gcc invocation:

gcc -fplugin=python.so -fplugin-arg-python-script=PATH_TO_SCRIPT.py OTHER_ARGS

and have it run your script as the plugin starts up.

The plugin automatically adds the absolute path to its own directory to the end of its sys.path, so that it can find support modules, such as gccutils.py and libcpychecker.

The exact API is still in flux; you can currently connect to events by registering callbacks e.g. to be called for each function in the source at different passes.

It exposes GCC's various types as Python objects, within a "gcc" module. You can see the API by running:

import gcc
help(gcc)

from within a script.

Overview of the code

This is currently three projects in one:

gcc-python-*: the plugin for GCC. The entrypoint (init_plugin) is in gcc-python.c

libcpychecker and cpychecker.py: a Python library (and a driver script), written for the plugin, in which I'm building new compiler warnings to help people find bugs in CPython extension code.

cpybuilder: a handy module for programatically generating C source code for CPython extensions. I use this both to generate parts of the GCC plugin, and also in the selftests for the cpychecker script. (I initially attempted to use Cython for the former, but wrapping the "tree" type hierarchy required more programatic control)

Coding style: Python and GCC each have their own coding style guide for C. I've chosen to follow Python's (PEP-7), as I prefer it (although my code is admittedly a mess in places).

You'll find API documentation within the "docs" directory, written in the reStructuredText format (as is this file, in fact). If you have Sphinx installed (http://sphinx.pocoo.org/), you can regenerate these docs using:

make html

within the docs directory. Sphinx is the python-sphinx package on a Fedora/RHEL box.

Debugging

gcc can launch subprocesses, so it can be fiddly to debug.

When debugging, I've generally been adding "-v" to the gcc command line (verbose), so that it outputs the commands that it's running. I can then use this to launch:

$ gdb --args PROGRAM WITH ARGS

This approach to obtaining a debuggable process doesn't seem to work in the presence of ccache, in that it writes to a temporary directory with a name that embeds the process ID each time, which then gets deleted. I've worked around this by uninstalling ccache, but apparently setting:

CCACHE_DISABLE=1

before invoking gcc -v ought to also work around this.

I've also been running into this error from gdb:

[Thread debugging using libthread_db enabled]
Cannot find new threads: generic error

Apparently this happens when debugging a process that uses dlopen to load a library that pulls in libpthread (as does gcc when loading in my plugin), and a workaround is to link cc1 with -lpthread

The workaround I've been using (to avoid the need to build my own gcc) is to use LD_PRELOAD, either like this:

LD_PRELOAD=libpthread.so.0 gdb --args ./cc1 ...

or this:

(gdb) set environment LD_PRELOAD libpthread.so.0

Handy tricks

Given a (PyGccTree *) named "self":

(gdb) call debug_tree(self->t)

will use GCC's prettyprinter to dump the embedded (tree*) and its descendants to stderr; it can help to put a breakpoint on that function too, to explore the insides of that type.

Enjoy! David Malcolm <dmalcolm@redhat.com>

Something went wrong with that request. Please try again.