Project List

Lauren McCarthy edited this page Mar 20, 2018 · 216 revisions

Welcome! This page contains a list of ideas and how you can help contribute to the Processing Foundation's work on Processing, p5.js, Processing for Android, processing.py, and Processing for ARM.

For all of our projects, it's incredibly important that things are kept as simple and user-friendly as possible. Our work is not for developers, it's for people who are less familiar with code, and/or just want to get things done. We're far less interested in features to make the environments more powerful for advanced users than we are in features that make it easier to handle tasks that are common for a wide range of our audience.

In addition to this list, we track specific bugs and enhancements via github issues:

In particular, you might take a look through all of the issues tagged enhancement as well as those tagged help in any of our repos.

Summer 2018 (GSoC)

We are pleased to announce we've been accepted for GSoC 2018 -- find out more here.

The Processing Foundation works each summer with students who are interested in contributing to open source. From 2011 through 2015 we participated in Google Summer of Code. In 2016, we ran an internship program through ITP at New York University. Last year we participated in GSoC as well as Rails Girls Summer of Code.

Apply!

Previous year GSOC reports

To get a sense of the kinds of project we work on via GSOC, take a look at these reports.

p5.js

p5.js is a JavaScript library that starts with the original goal of Processing, to make coding accessible for artists, designers, educators, beginners, and reinterprets this for today's web. Using the original metaphor of a software sketchbook, p5.js has a full set of drawing functionality. However, you’re not limited to your drawing canvas, you can think of your whole browser page as your sketch! p5.js has addon libraries that make it easy to interact with other HTML5 objects, including text, input, video, webcam, and sound. p5.js is a new interpretation of Processing, not an emulation or port, and it is in active development.

+ Improving p5.js WebGL/3D functionality

This project would involve extending the 3D API by implementing missing primitives in p5.js WebGL mode, including a subset of point, arc, image, text, bezierVertex, curveVertex, and quadraticVertex. In addition to implementing the code functionality, a successful project would also address documentation, updated examples, and unit tests. You can learn more about the current WebGl architecture here and see all open issues here.

  • Expected Outcomes: Implementation of additional WebGL primitives, updated documentation and tests
  • Skills Required: JavaScript, experience with WebGL or GL programming, familiarity with Github or version control
  • Possible Mentors: Kate Hollenbach, Stalgia Grigg
  • Difficulty: intermediate, advanced

+ Testing and updating mobile functionality

We have a set of mobile functionality including use of touch interaction and accelerometer. These functions need to first be tested across mobile devices and platforms and issues opened for unsupported functionality. Then these issues can be addressed, along with some known ones, including: accelerationXYZ problems on Android, elements not responsive on some browsers, mousePressed and touchStarted broken in chrome mobile, and missing cross browser support for deviceMoved() and deviceTurned(). Reference documentation could be updated to reflect technical hardware and browsers limitations for this functionality as well. Finally, the examples in the reference and under "Mobile" on the examples page need testing and possible updating.

  • Expected Outcomes: Mobile functionality tested and improved, issues logged, documentation and examples updated
  • Skills Required: Familiarity with Github or version control, basic JavaScript knowledge, experience working with mobile development
  • Possible Mentors: Kate Hollenbach, Lauren McCarthy
  • Difficulty: beginner, intermediate, advanced

+ Improving file I/O methods

This project involves testing existing file input and output methods and creating issues for functionality that is not working correctly. There are already four known issues, including: loadXML failing in instance mode, loadTable on a very large data set causes a empty p5.Table to return and no errors, Chinese fonts failing with various errors , and pre-existing createFileInput not enabled. Once all existing issues are identified, they can then be fixed or the documentation can be updated to better align with functionality and limitations. Additionally, examples and tests might need to be added to ensure the functions remain usable and clear.

  • Expected Outcomes: File I/O issues logged, addressed, documentation improved.
  • Skills Required: JavaScript, HTML, Github or version control
  • Possible Mentors: Lauren McCarthy, Stalgia Grigg, Alice M. Chung
  • Difficulty: intermediate

+ Updated hello.p5js.org site

hello.p5js.org was created several years ago and is a bit outdated now. We would like to update the site's content and code. The code has a number of dependencies that are outdated and need to be updated or replaced with more recent technologies. We'd also like to add two features: captions and the ability to support translated captions. Finally, it'd be nice to record some new video segments to represent more people in the p5.js community.

  • Expected Outcomes: Revamped hello.p5js.org site.
  • Skills Required: JavaScript, HTML, Github or version control, outreach
  • Possible Mentors: Lauren McCarthy, Dan Shiffman
  • Difficulty: intermediate

+ New JavaScript console in p5.js web editor

The p5.js web editor includes a console to display output of console.log() as well as error messages. The console is meant to be simple and friendly, however some features found in more sophisticated browser developer tools would be useful here.

  • Expected Outcomes: Create simple test version of new proposed console outside of p5.js web editor. Integrate new console into web editor. Desired features include:
    • Interactive console
    • Folding / unfolding of console logged objects
    • Repeated console logs are not duplicated (but displayed with a number displaying times logged).
  • Skills Required: Familiarity with Github and version control, JavaScript knowledge, experience with React a plus.
  • Possible Mentors: Cassie Tarakajian, Daniel Shiffman
  • Difficulty: intermediate, advanced

+ Classroom management system in p5.js web editor

An initial prototype of a classroom management system was created by Zach Rispoli for last year's Google Summer of Code. We would like to have this project refined and integrated into the web editor this summer.

  • Expected Outcomes: A page for each class that includes the following:
    • Classroom name
    • Usernames of all instructors
    • Usernames of all students
    • Classroom description
    • Classroom image/thumbnail
    • List of assignments
    • List of sketches submitted to each assignment
  • Skills Required: Familiarity with Github and version control, web development with React.
  • Possible Mentors: Cassie Tarakajian, Daniel Shiffman
  • Difficulty: advanced

+ Utilize the Audio Worklet in p5.Sound library

Audio Worklet (now available in Chrome Beta) opens up many exciting possibilities for the Web Audio API. It allows us to process audio efficiently on the audio thread, and it will replace the deprecated ScriptProcessorNode.

  • Expected Outcomes:
    • Phase out use of the ScriptProcessorNode in favor of the Audio Worklet, with a fallback for unsupported browsers
    • Creation of new custom nodes that utilize the Audio Worklet
    • Library is more efficient and reliable
  • Skills Required: Audio / Digital Signal Processing, JavaScript
  • Possible Mentors: Jason Sigal
  • Difficulty: intermediate

+ Algorithmic Composition with p5.Sound

Build off of last summer’s GSoC work (development of p5 Synth, Effects, and p5.SoundLoop—the audio equivalent to p5’s draw loop) to create examples that apply concepts from Nature of Code and Processing to sound & music composition.

  • Expected Outcomes:
    • A collection of examples that can be used by educators and people with an interest in music + code
    • Improved documentation for the features utilized by these examples
    • Refinements to the p5.sound library that make such examples possible
  • Skills Required: Interest in music + algorithms, JavaScript
  • Possible Mentors: Jason Sigal
  • Difficulty: intermediate

+ Modularize p5.Sound library & improve infrastructure

There are several issues that a GSoC student could address, with a focus on utilizing p5.sound in modern web projects and decoupling the components of the library.

  • Expected Outcomes:
    • Our GitHub becomes focused on issues, rather than questions, because we have examples covering common usage scenarios
    • Create examples that show how to upload sound files to a server (#211)
    • Create examples of importing p5.sound in Webpack module bundler; use with frameworks like Angular and React Decouple p5.sound from p5.js
    • Examples of using p5.sound with other sound libraries, especially Tone.js (for interactive music) and Meyda (#228) (for audio visualization)
    • Increase unit testing coverage
    • p5.sound can be used with or without p5.js, and in combination with other Web Audio libraries like Tone.JS and Meyda
    • Resolve .value deprecation notice (#233)
  • Skills Required: JavaScript
  • Possible Mentors: Jason Sigal
  • Difficulty: intermediate

+ Addon Library Development

Is there something you think p5.js should do, but it currently doesn't? p5.js can be extended through add-ons and we're always looking for contributions. Past GSoC library projects include extending p5 to work with maps and interacting with other microprocessors from within the browser. More information on how to create addon libraries here

  • Expected Outcomes: A github repository with library code, examples, and documentation.
  • Skills Required: JavaScript, HTML, CSS, Github
  • Possible Mentors: Daniel Shiffman
  • Difficulty: intermediate

Processing

The desktop version of Processing has ~250,000 unique users. Daily, there are 25-30,000 people who make use of the PDE. We want to improve the experience those people have with the software, and need far more people than just the tiny team of contributors.

+ Analytical Visualization Tools (Priority)

Implement a better way to understand how people use Processing. We collect stats on the number of people using Processing, what version they're on, what OS, etc. We also collect stats for what people have installed so that we know where to put our efforts for Libraries, Modes, and Tools. At the moment, these are looked at manually when we're trying to make decisions (documentation of how to access the data). But it'd be great to have a better way to share these with the community in an ongoing basis (Scratch does a nice job with this), and also for us to keep an eye on how things are going.

  • Expected Outcomes: Series of scripts and web visualizations
  • Skills Required: JavaScript, HTML/CSS
  • Possible Mentors: Casey Reas
  • Difficulty: beginner tech / advanced design

+ Sound (High Priority)

The Sound Library need to be updated to work with recent changes to OS X and Windows 10. We feel the library has the correct API, but it needs core infrastructure work to operate better.

  • Expected Outcomes: Updated full release of the Sound library
  • Skills Required: Java, JNI
  • Possible Mentors: Wilm Thoben
  • Difficulty: intermediate

+ Video (Priority)

The video library for Processing uses an engine built on top of GStreamer. During the previous years there have been different efforts to update GStreamer to more contemporary (1.x) versions, and we currently have a 2.0 beta version of the video library that uses the most recent version of GStreamer 1.x (1.12.3) and the gst1-java-core Java-GStreamer bindings, based on this fork from Gottfried Haider. Even though this beta version of the library works on Linux, macOS, and Windows, it needs addressing some issues to become the default.

  • Expected Outcomes: Updated stable 2.0 release of the Video library
  • Skills Required: Java, GStreamer, JNA
  • Possible Mentors: Gottfried Haider, Andres Colubri
  • Difficulty: intermediate, advanced

+ New OpenGL Renderer (Priority)

JOGL's development has slowed down considerably during the last few years, so it may be a good idea to work on a LWJGL-based renderer, even as a contributed library at first. We have created a fork of processing-core that uses LWJGL, it runs but is is not yet completely functional, but could serve as a starting point.

  • Expected Outcomes: New library using LWJGL that can later be integrated into the core renderer
  • Skills Required: Java, OpenGL
  • Possible Mentors: Andres Colubri
  • Difficulty: advanced

+ Development Environment: Beginner/New User Experience Features

The initial experience of using Processing should be improved. We have 100s of examples, and a better way to get people started with the PDE would be to introduce more of them, along with the tutorials, as part of the initial experience with the PDE. This work might also include improving or expanding upon the tutorials as well.

  • Expected Outcomes: Variable; maybe a set of loading screens, documentation built-in to the editor, a set of new Tools to guide beginners
  • Skills Required: Java
  • Possible Mentors: Scott Murray, Daniel Shiffman
  • Difficulty: beginner, intermediate

+ Development Environment: Processing Language Server

The Processing Development Environment (PDE) is currently built in Java, using custom components that make use of Java's “Swing” library, which is even more outdated now than when it was first released. Long-term we are considering moving to a JavaScript-based solution that would 1) allow more people to contribute to its further development, 2) provide better support for building the UI, 3) have more visually consistent cross-platform results, and 4) potentially open up other ways to distribute the PDE. In order for this to happen, we would like to first create a Processing "Language Server" In addition to a new PDE itself, this would allow for more sophisticated integration with existing code editors like Visual Studio Code, Atom, and more.

  • Expected Outcomes: Working Processing/Java Language Server Working prototype of a JavaScript IDE for Processing
  • Skills Required: Java Development
  • Possible Mentors: Manindra Moharana, Casey Reas
  • Difficulty: advanced

+ Library Development

Is there something you think Processing should do, but it currently doesn't? Processing can be extended through libraries and we're always looking for contributions. Instructions for building libraries are here on the GitHub wiki. Past GSoC library projects include an Image Processing Library to Ease Differentiation of Colors for People with Colorblindness and Kinect v2 Processing library for Windows.

  • Expected Outcomes:
    • A github repository with library code, examples, and documentation.
    • Packaging and release of library via the Processing contributions manager
  • Skills Required: Java Development
  • Possible Mentors: Daniel Shiffman
  • Difficulty: intermediate

Processing for Android

+ Augmented Reality (Priority)

Google will be releasing the stable version of the ARCore API in the coming months, and more generally there is a great interest in the creative applications of AR. Processing for Android currently does not have form of support for AR, and so it would very important to implement a simplified AR API in Processing allowing users to quickly create AR apps in combination with the rest of the Processing core and Android API.

  • Expected Outcomes: New AR renderer, examples, and documentation
  • Skills Required: Java, Android, ARcore
  • Possible Mentors: Sara Di Bartolomeo, Andres Colubri
  • Difficulty: intermediate

+ Android mode translations (Priority)

Many users around the world will benefit from a translated Android mode into their native languages. The mode has a relatively small number of text messages that would need to be translated. But this can be solved in general by implementing a new mechanism by which translations can be added to any mode

  • Expected Outcomes: System to add translations of a Processing mode
  • Skills Required:
  • Possible Mentors:
  • Difficulty: intermediate

+ New build toolchain

The Android mode and core library build toolchain still uses Ant and Eclipse as the IDE, although Gradle build files are also provided as an alternative. Some steps (such as updating dependencies, creating Maven artifacts) need to be performed manually. Updating this legacy build chain would help maintenance and incorporating new contributors.

  • Expected Outcomes: Updated and streamlined build toolchain
  • Skills Required: Gradle, Java, Android library development
  • Possible Mentors: Andres Colubri
  • Difficulty: intermediate

+ Android Debugger

The integrated debugger is available only in the Java mode, but would be as useful in the Android mode as well.

  • Expected Outcomes: Integrated debugger for Android mode
  • Skills Required: Advanced Android development
  • Possible Mentors:
  • Difficulty: advanced

Processing Python

+ Native Python and Processing (Priority)

We have started work on a native (C Python) version of Processing. While it doesn't have the advantage of library support that we get from the current Jython-based implementation, it opens up other ways to extend Processing with Python's syntax, features (numpy, scipy), and considerable base of support. This work was started by Abhik Pal during Summer of Code 2017 as p5py. We hope to make additional progress in this exciting work in 2018.

  • Expected Outcomes: Add missing features to bring it on par with Processing(Java) api, performance improvements and optimizations especially on Mac/Linux, code examples and documentation.

    • Improve rendering performance for UNIX systems
    • Add support for live sketches via python REPL
    • Add/Improve API support for text
    • Add/Improve API support for images
    • Add/Improve API support for PShape/3D
    • Fix existing open issues
    • Port existing Processing examples for p5py
    • Add missing documentation
    • Any other new ideas welcome!
  • References:

  • Skills Required: Python

  • Possible Mentors: Manindra Moharana

  • Difficulty: advanced

Processing for ARM devices (such as Raspberry Pi)

+ Add learning resources (Priority)

Processing on the Raspberry Pi is still rather under-documented, and the project would benefit from accessible documentation that focuses specifically on the initial steps necessary to get going, and to lead the interested user to further resources and avenues to be explored.

  • Expected Outcomes: Design and build learning resources that make it more easy to get going with Processing on Raspberry Pi
  • Skills Required: Didactic, empathic, design, possibly video
  • Possible Mentors: Gottfried Haider, Dan Shiffman (tbc)
  • Difficulty: intermediate

+ Performance Improvements (Priority)

ARM devices have become significantly more performant in recent years, but they are still much more likely to suffer from poor performance or memory exhaustion compared to their desktop and laptop siblings. We should look holistically at Processing runtime performance (e.g. startup time, JAR decompression time, memory usage, P3D fill-rate) to see where we can do better, and implement those changes in Processing proper.

  • Expected Outcomes: Identify performance issues, implement improvements, and get them merged into the Processing tree
  • Skills Required: UNIX instrumentation, Java, C
  • Possible Mentors: Gottfried Haider
  • Difficulty: advanced

+ Implement UI for Pocket C.H.I.P.

The Pocket C.H.I.P. is an interesting low-cost hardware, with popular programming applications such as PICO-8. The 480x272 pixels display currently makes it very challenging to use the Processing PDE, so an interesting project could be to design and implement a purposely-built (text-based?) environment that lets the users write and execute Processing sketches. This editor need not be written in Java itself.

  • Expected Outcomes: Create an editing environment for Pocket C.H.I.P.
  • Skills Required: Knowledge of UI toolkits, programming
  • Possible Mentors: Gottfried Haider
  • Difficulty: intermediate

+ Add libraries relevant for embedded & mobile usage

Processing on ARM invites new types of experimentation, which could benefit from the development of supporting libraries. Examples could be e.g. interfacing with inexpensive Makey-Makey-like input devices (e.g. CAP1188), doing speech recognition (e.g. Sphinx-4), enabling the creation of natural language software agents (e.g. AIML), talking to other devices in the home through MQTT, etc.

  • Expected Outcomes: Find existing FLOSS software to build upon, design and create a Java library, related examples and documentation and publish it
  • Skills Required: Java, (often times) JNA or JNI
  • Possible Mentors: Gottfried Haider
  • Difficulty: intermediate

Ongoing Projects

p5.js Web Editor

  • mobile/responsive development
  • exploring/searching sketches feature
  • a few small existing tickets - i.e. public/private sketches, uploading assets from a URL, and so on
  • implementing design changes
  • make the code editor even more beginner friendly

Processing

OpenGL Ideas

  • Right now, editing of GLSL shaders is not supported by the PDE. An external tool allowing to code shaders side by side with the sketch code could be very useful and cool.
  • Central repository of shaders, ideally linked with the editor. We could also incorporate support for a shader interoperability spec, such as the Interactive Shader Format, which would allow Processing users to share shaders with other applications for creation of interactive graphics.
  • An advanced shader tutorial following the introductory one here is needed.
  • A library for advanced shaders (lighting, geometry, tessellation), that handles the low-level stuff. Right now it is possible to create geometry or tessellation shaders by using the GL3/GL4 API, but would be nice if that could be handled automatically by this hypothetical library. Also integration with the repository suggested above would be neat.
  • A library to do calculations on the GPU using the OpenCL API. This library could work in coordination with the OpenGL renderers to allow handling large particles systems, very complex dynamic meshes, etc. A prototype OpenCL library for an early Processing 2.0 alpha release, which implemented traer.physics solvers with OpenCL kernels, is available here.

Libraries

  • Is there something you think Processing should do, but it currently doesn't? Processing can be extended through Libraries and we're always looking for contributions. Instructions for building libraries are here on the GitHub wiki.
  • The libraries that allow Processing to communicate with the MS Kinect could use work and updating. See: includes openkinect.
  • The Toxiclibs has been updated for Processing 3, but there is work that could be done to fix bugs, improve documentation and examples, and keep aligned with toxiclibs.js.

Tools for the Processing Development Environment (PDE)

Smaller, Isolated Projects and Fixes

There are a number of smaller projects that could be handled by someone who doesn't necessarily want to dive all the way in to the code. Often these are isolated things that we've just never had the time to get around to. We track many of these with the help label. Please help!

Processing for Android

Android Ideas

  • Improve SDK updater, so it properly shows download and installation progress, and also it's able to apply patches.
  • Incorporate support for the kotlin language, which is very popular among Android developers.
  • Video/Media library for Processing Android? This would probably wrap the media engine and player already available in Android. A former GSoC student developed a video library for the Android mode, supports video playback and camera capture through OpenGL textures for good performance. Perhaps it can be continued/expanded.

+ Processing Android Community Outreach

Processing for Android tries to make the programming of Android apps easier to people who are not professional Android developers, and allow everyone, including seasoned developers, to experiment with new ideas in mobile graphics and audio, interaction, use of sensor data, etc. Because of those aim, it occupies a somewhat unusual place between learning to program for mobile, more technical applications, and creative experimentation, which may make it harder for the beginner Processing users to move beyond simple apps and eventually contribute to the project, and for advanced Android developers to become more interested and incorporate it into their toolkit, and contributing to the project as well. Reaching out both communities, and everyone in between, would require an effort in different directions: improving online presence, engaging more actively with different groups of users, and showcasing projects that demonstrate the possibilities that Processing can bring to Android development.

  • Expected Outcomes: better website and other online presence, events for users of different levels (workshops, meetups), curated list of projects
  • Skills Required: Communication (online and face-to-face), familiarity with Processing and Android communities, event organization
  • Difficulty: beginner technical, advanced community organization

Processing.py

+ Python Mode with Processing.py

Python Mode for Processing allows you to write Processing sketches using the Python programming language. We are looking for help with the implementation of Python Mode and its documentation.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.