Skip to content

Project List

Jump to bottom Edit New page
Daniel Shiffman edited this page Feb 25, 2014 · 46 revisions

This page documents a list of projects and how you can help contribute to the project.

For all projects, it's incredibly important that things are kept as simple and user-friendly as possible. The PDE is not for developers, it's for people who are less familiar with code, and/or just want to get things done. We're far less interested in features to make the environment more powerful for advanced users than we are in features that make it easier to handle tasks that are common for a wide range of our audience.

In addition to this list, we are also tracking specific bugs and enhancements for Processing in the github issues list. In particular, you might take a look through all of the issues tagged enhancement.

Welcome GSOC 2014!

For our fourth year, Processing is a mentoring organization for GSOC. If you have an idea for a proposal, let us know by engaging on the Processing forum and submitting to GSOC. Anything is fair game and we're especially excited to hear about ideas for libraries, tools, and modes. If you are looking for inspiration for your proposal, here is what we are thinking about.

p5.js

  • Over the past year, we have been developing p5.js, a JavaScript library that starts with the original goal of Processing, to make coding accessible for artists, designers, educators, beginners, and reinterpets this for today, for the web. The library is intended to introduce creative coding, introduce web development, and provide a tie between the two. It it important that while this is accessible for beginners, it’s not a sandbox environment and people develop real web development literacy, and the ability to extend and learn new things on their own. The library is not domain specific, it’s useful for general creative coding from drawing, to working with text, images, DOM, etc. An official launch is planned in late spring.
    • Implementing WebGL renderer / shader support (right now only canvas renderer is supported).
    • Creating an addon module for creating and interfacing with GUI / user input DOM elements (forms, buttons, etc).
    • Creating an addon module for interfacing with other HTML5 supported mobile features (accelerometer, geolocation, etc).
    • We are currently in the process of building an IDE, both browser-based and standalone versions. If you are interested in working on this, or dreaming up new features for it, we'd love to hear.

Libraries

Processing Development Environment (PDE)

  • Tool Chest: We've build a system that allows the PDE to be extended to enhance the action of programming. There's an extensive list of potential tools on the Processing Wiki that are waiting to be programmed. Instructions for building tools are here on the GitHub wiki.
  • Internationalization and localization: We need to do the i18n and l10n work for Processing and its GUI. We need something very, very simple on the i18n side so that it's easy to deal with in code, and easy for people to add new languages. Not terribly difficult work, but just needs to be handled in a way that doesn't change every instance of text in the source code into five lines of confusing mess.

Video

The video library for Processing 2.0 uses an engine built on top of GStreamer. The library is complete for 2.0 but there is always room for improvement:

  • We could use help with updating the GStreamer binaries bundled with the video library to the 1.0 version. The ones currently in use correspond to the 0.10 series, which is no longer maintained. Furthermore, GStreamer 1.0 introduced several improvements, like embedded processors support, video decoding/encoding using GPUs, and capture using the Rpi camera module. The GStreamer project has been releasing binary builds for Windows and OSX of the 1.x series, which could be useful in the update of the video library. However, in order to use GStreamer 1.x in Processing, we need to update the gstreamer-java bindings since they are currently compatible only with GStreamer 0.10. There is ongoing discussion about re-writing the gstreamer-java bindings in order to support 1.x, check the gstreamer-1.x-java repo and this thread on the gstreamer-java Google Group.
  • As a entirely different direction of work, we could look into alternative low-level libraries for video playback and capture, ideally lighter than GStreamer. Some options that appear to be actively developed right now: Webcam capture, jlibav, the FFmpeg components in JavaCV and its capture modules (libdc1394, FLYCapture, etc.)
  • The two previous items are quite ambitious in their scope (updating to gstreamer 1.0, replacing the underlying video toolkit altogether), but there are plenty of smaller tasks/project ideas that are still very important:
    • We'd also like to support for as many capture devices as possible, this requires substantial amount of testing and could lead to write/improve gstreamer plugins for video capture. The wiki entry for the Video Library has some relevant information for capture plugins on OSX, specifically related to the issue of listing available capture devices which might be problematic given some changes in the gstreamer API in the 1.0 release.
    • The property-probe interface in gstreamer 0.10 that allowed to retrieve the list of available capture devices has been removed in 1.0, but there is now a replacement that is already included in GStreamer 1.2. Making use of this new device discovery/listing API would require changes in the video library as well as in gstreamer-java.
    • Investigate ways of improving playback to allow for smoother scrubbing of a video. The Processing Forum contains some discussion about this issue.
    • From an old GSoC 2012 thread, we could consider the following (relatively) simpler tasks: better error handling (showing descriptive warnings when there are no cameras available, or the selected framerate is not supported), clean-up the GStreamer plugins that are bundled with the video library from those that only support esoteric codecs and functionality that are not needed in Processing, and thus making the library smaller.

OpenGL

  • The Shader API in Processing 2.0 is completed (as of release 2.1.1), so this could be a good time to start compiling shader examples and effects in a centralized location. Also, an advanced shader tutorial following the introductory one here is also needed.
  • Right now, editing of GLSL shaders is not supported by the PDE. An external tool allowing to code shaders side by side with the sketch code could be very useful.
  • Working in collaboration with the p5.js team to implement the shader API in a webGL renderer for p5, that ideally can share GLSL shaders seamlessly between Processing 2 and p5.js
  • Visual quality fine-tuning: although the OpenGL renderer primary aim is performance rather than quality, work has been done in Processing 2.0 to make the output of P2D and P3D as good as possible - within the constrains of OpenGL - and specially when comparing P2D with the default renderer (JAVA2D). There are still several bugs related to visual quality that could be taken care of, for example: #115, #1185, #1997, #2014, #2018, #2065.
  • There is a new core library begin developed called GLW that renders the sketch into a native OpenGL window. Besides improving performance notably on OSX, this should allow using the OpenGL renderers (P2D, P3D) on the raspberry pi (because OpenGL output through AWT is not possible), but needs some work.
  • A library to do calculations on the GPU using the OpenCL API. This library could work in coordination with the OpenGL renderers to allow handling large particles systems, very complex dynamic meshes, etc. A prototype OpenCL library for an early Processing 2.0 alpha release, which implemented traer.physics solvers with OpenCL kernels, is available here.

Processing for Android

  • Additional Libraries to extend support to mobile device APIs for networking, mapping, and reading sensor data. We'd like to start with simple examples for different features (as we've done in the most recent release), and as these are ironed out, make them into libraries. We are not looking for a monolithic library to support all types of sensors, but lots of small examples of how to use different features. These libraries are not intended to be comprehensive, but rather focus on simple ways of using devices.
  • Video/Media library for Processing Android? This would probably wrap the media engine and player already available in Android.
  • Interface/back-end: In release 0194, the run/stop/cancel interface took a step backwards, as changes were made to get things working under the new 'mode' system. This needs some care. It involves staring at some difficult multi-threaded code that handles the build and run cycle, launching an emulator, making sure that an AVD is setup and installed, and so on. In addition, on Windows in particular, we aren't getting very helpful errors out of the ant build scripts. Some tinkering should improve the situation. All this is very important for Android development.

Bugs

Smaller, Isolated Projects and Fixes

There are a number of smaller projects that could be handled by someone who doesn't necessarily want to dive all the way in to the code. These are isolated things that I've just never had the time to get around to. Any help is appreciated!

Most Important

  • The Auto Format needs some love. A motivated person with a couple free hours could take care of a number of issues, including: placement of else statements, extra indents, switch blocks, better maintenance of cursor position, and being able to select a portion of code to be formatted. Someday we'll replace the entire editor, and with it, the formatter, but for now these simple fixes would be huge.
  • Undo is not behaving properly (issue 707). Also related is how sketches are being set modified: sometimes too aggressively, because some key events are coming through as text being typed when they're just modifiers like the alt key. Oh the embarrassment.
  • We've encountered some difficulty supporting all the OS features for Apple retina displays. We could use some help in tracking problems related retina displays and providing fixes. For example, see issue 2116 and issue 2117.

Other Quickies

  • Prepare a Debian package for the project. Issue 114 covers the attempts so far.
  • Wrap Jonathan Feinberg's awesome Python version of Processing as a proper mode so that it can be run from inside the PDE instead of just the command line.

Projects in Progress

This section tracks projects that are undergoing development by members of the Processing community as well as students from Google Summer of Code 2012 and 2013. While these projects have active interest and dedicated developers, please feel free to get in touch if you'd like to help out.

PDE

  • PDE X (processing-experimental) is an experimental Processing Mode which aims to bring advanced IDE features to the PDE. Initial work was started in 2012 in the form of two GSoC '12 projects: XQMode(live error checker) and Debug Mode(Debugger). In GSoC 2013 these two projects were combined into a single mode and further work was done to introduce features such as intelligent code completion, refactoring, import suggestions and more. It is currently under development.

Video

  • While there are many contributed libraries for computer vision (including a few for OpenCV), we would like to investigate the feasibility of a better-supported OpenCV library to give Processing built-in features such as face detection, blob detection, etc. At the moment, getting OpenCV to work is a tricky proposition due to cross-platform differences, build instructions, and 32- vs 64-bit problems. It should be possible to make a single library that covers OpenCV and a simple Processing layer to it, but that can be contained in a single library (requiring no additional installation steps), the way we've done with gstreamer. This is now under development and we are currently investigating the pros and cons of using JavaCV vs the new OpenCV desktop bindings for Java. The new OpenCV-Processing library by Greg Borenstein is already usable and available through the Library Manager.

User interface

  • Someday, somewhere, Processing might include classes for user interface elements as a core library. Martin Leopold created a new Graphical User Interface Library for GSOC 2013 and we're interested in continuing this work and further contributions.
Add a custom footer
Add a custom sidebar

Clone this wiki locally