Buildbot GSoC Projects 2019

Pierre Tardy edited this page Feb 4, 2019 · 16 revisions

Google Summer of Code 2019

Buildbot is inviting students to join in Buildbot's development by applying for Google Summer of Code 2019.

This is the fifth year Buildbot will be seeking to participate, the first year it will candidate under the Python Software Foundation (PSF) umbrella

Introduction

Buildbot is a continuous integration framework built with python and Twisted. Started in 2005, Buildbot is one of the earliest player in continuous integration field.

The codebase is pretty stable, and lots of efforts have been made lately to modernise it using python3 latest features.

Buildbot also has a UI written in coffeescript and angularJS. Some of our ideas are about modernising this as well.

See our documentation for more details.

The developer quicktarts describes how to setup your environment for developping with python or javascript.

Our issues are also on github, which include bug reports and 'wishlist' items and enhancement plans and ideas.

If you are interested in participating in GSoC as a student, the best approach is to become an active and engaged contributor to the project right away. You should take a look at some of the existing issues on GitHub and see if there are any you think you might be able to take a crack at.

Try submitting a pull request for something and start getting the hang of the process and interacting with the Buildbot code base and development community.

Guidelines and Prerequisites

Students should start by reading the guidelines for participation. Google also provides guidelines to help with writing a proposal as part of their GSoC Student Guide. It is a good idea to start on your proposal early, post a draft to the buildbot irc room and iterate based on the feedback you receive. This will not only improve the quality of your proposal, but also help you find a suitable mentor.

Please note that as a sub-organization of the PSF (and active members of the Python community), we ask that all mentors and students working with Buildbot abide by the Python Community Code of Conduct.

Project Ideas

Below are a listing of possible projects that students might consider. We also encourage students to propose their own projects, though several of the following topics are relatively high on our priority list. Our priority list is flexible, and it is important that the topic matches the interest and background of the student.

When considering the following projects, don't be put off by the knowledge prerequisites -- you don't need to be an expert, and there is some scope for research and learning within the GSoC period. However, familiarity with and interest in the subject area and involved technologies will be helpful!

 

Add support for running worker on non-python platforms

Buildbot users often want to run tests on wide variety of platforms. Right now communication between master and worker is tightly integrated with Python and thus the list of platforms that are supported by Buildbot is limited to what's supported by Python and Twisted. Even then, installation of Twisted and its dependencies is complicated on obsolete distributions where the pip tool no longer works. Ideally, Buildbot worker should not depend on anything and installing it should be as easy as dropping an executable to a device and running it.

To achieve the above, a new, language-independent protocol would need to be designed and an new worker implementation in a language with fewer runtime dependencies would need to be developed. C++ is preferred for the worker due to its ubiquity and mentoring resources.

For more details, see Github issue #4344.

Expected outcomes:

  • A language-independent protocol that supports the current master-worker communication functionality is designed
  • The protocol is implemented from the master side in Python
  • A new worker implementation with less dependencies is developed for the protocol
  • The new worker is built and tested on weird platforms (stretch goal)
  • Continuous integration is setup to test the new worker on weird platforms (stretch goal)

Skills: Python, C++

Difficulty Level: Intermediate to Hard

Potential Mentors: Povilas Kanapickas (@p12tic at GitHub), Robin Jarry (@rjarry at GitHub)

 

Subdivide builders into projects

A single Buildbot instance is often used to run tests for many unrelated projects. Currently Buildbot lists all builders of all projects in a single page, which makes navigation complicated in instances where multiple projects have many builders. This could be solved by adding a projects feature which would let builders to be assigned to a certain project and then that information would be used to make navigation in the UI more convenient.

For more details, see Github issue #4373.

Expected outcomes:

  • Buildbot config allows configuration of a project for each builder and incoming changes are appropriately tagged.
  • The www UI allows selection of an active project. Doing that would only show builders and builds that are related to the active project.
  • The www UI allows selection of an active branch for the active project. Doing that would only show the latest builds for each builder (stretch goal).

Skills: Python, CoffeeScript, SQL

Difficulty Level: Intermediate

Potential Mentors: Robin Jarry (@rjarry at GitHub), Povilas Kanapickas (@p12tic at GitHub)

 

Polish support for multi-master mode

The Buildbot master is bound by the performance of a single core due to being written in Python. This puts a hard upper bound of the size of Buildbot installations which has been hit in large real world setups. The solution for the problem is running Buildbot in multi-master configuration. Unfortunately support for multi-master is currently experimental and there's little validation that it actually works. The goal of the project would be to attempt to run Buildbot in multi-master configuration, document the findings, identify areas where test coverage is missing and write new tests and finally identify problems (if any) in the multi-master support and attempt to fix them.

Expected outcomes:

  • A realistic environment to run Buildbot in multi-master configuration is created, problematic areas are found and documented.
  • Areas with lack of test coverage are identified and tests written to verify the behavior of Buildbot master.
  • All bugs found during the project are fixed (stretch goal)

Skills: Python, SQL

Difficulty Level: Intermediate

Potential Mentors: Povilas Kanapickas (@p12tic at GitHub), Pierre Tardy (@tardyp at GitHub),

 

Transition Web UI's Data module from CoffeeScript to Typescript

There is a large project to switch our UI Codebase to Typescript. Coffeescript complicates the build, adds dependencies and makes contributing harder as it's not a popular language.

This is a huge work not suitable for a single student for GSoC

Buildbot Data Module is the core of Buildbot web UI API.

It has been written during the GSoC 2016.

The API is well defined, and it is well suitable for a first step to convert the UI to typescript.

The buildbot data module also has some design issues, which would need to be fixed as part of this conversion.

For more details, see Github issue #4373.

Skills: Javascript, Typescript, Websocket

Difficulty Level: Intermediate

Potential Mentors: Craig Rodrigues (@rodrigc at GitHub), Pierre Tardy (@tardyp at GitHub)

Expected outcomes:

  • 'buildbot-data' module is implemented in typescript and implements API as defined in the data module doc.
  • This module shall not depend on AngularJS, and use pure typescript, with minimal dependencies.
  • AngularJS stubb is available and make the typescript API available as an angularJS 'service'

Extra points:

  • local caching is implemented in a generic way.
  • Without changing the API, the data module is caching the data in the browser with indexdb.
  • for each request, a REST api and indexdb request are sent in parallel. The indexdb result is displayed first if available, and REST api results will allow to make the data more up to date. This should make the UI much faster over high latency connection.

Other

Buildot is an open source project and as such we invite contributions from any interested developer. If you have an idea for an enhancement for Buildbot please contact one of the developers to discuss the possibilities for the project in GSOC19.

Probably the preferred way is IRC.

  • Choose a proper nickname, so that we remember you.
  • Don't ask and leave. People are not always looking at their laptops, so please allow a few hours to have an answer.

Timeline

  • January 15 - February 4 - sub-organization applications due into PSF
  • February 26 - organizations announced
  • February 27 - March 20 students discuss applications with mentoring organizations
  • March 25 - April 9th Student application period
  • May 6 - Accepted student proposals announced
  • May 6 - May 27 - community bonding
  • May 27 - August 26 - coding
  • August September 3 results announced

Source: https://developers.google.com/open-source/gsoc/timeline

Student Application Template

Python Software Foundation's student application template.

credits

This idea page is heavily inspired upon PySals

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.