From late 2011 on, the development process iterated. Not optimal, it lacked accountability. Bottlenecks festered and deadlines passed. Each release from WordPress 3.4 on endured major development process change; it wasn’t until WordPress 3.8 that process experimentation slowed.
Experimentation started at the core team meetup in Tybee Island, Georgia, in December 2011. Lead developers, committers, and active core developers discussed the project’s issues and roadmap, informing the scope setting meeting for WordPress 3.4 on January 4, 2012.
The scope chat notes cover the issues discussed. The team acknowledged process problems. Jen listed the issues: “lack of good time estimation, resource bottlenecks, lack of accountability, unknown/variable time commitments/disappearing devs, overassignment of tasks to some people, reluctance to cut features to meet deadline.”
They split feature development into teams; two contributors would lead each team to reduce contributor overextension and project abandonment.
Ideally, a lead developer or a committer would lead each feature team. To help engage and mentor new contributors, one new contributor would pair with each lead or committer. Each team had to post regular updates and deliver their feature on time. They created a schedule with overlapping cycles that encompassed development, UX, and bug fixes to reduce bottlenecks.
The proposal for the 3.4 release process.
The team decided that releases would have a specific focus. WordPress 3.3 had been a cluster of disparate features, some of which had been pulled because they weren't ready. WordPress 3.4 was the first cycle to have an overarching focus -- appearance/switching themes. The development team worked on improving both front-end and admin features that adjust a site’s appearance.
Between WordPress 3.4 and 3.5, the project leadership approach evolved -- a change that had started back in WordPress 3.0 when Dion Hulse became a committer. Though Dion received commit access, he was not a lead developer. This was the first step toward decoupling the lead developer position from commit access. To reduce bottlenecks, the project needed committers, but not necessarily more lead developers. Separating the roles offered opportunities for strong coders and patch triagers, and for those who wanted to contribute, but not necessarily in a leadership role.
And yet, confusion reigned on what the lead developer title actually entailed. Core team members perceived the role differently; no one agreed on what the lead developer title meant. For Mark Jaquith, the lead developer role has changed organically over the years. To begin with, the role related to coding. Over time, it transitioned, particularly around the growth of WordCamps: "We started going around and talking to users about some of the philosophy behind WordPress and the direction we wanted to take. It became more of a general leadership role in practice." The role also evolved as lead developers worked with the community: often, lead developers responded to comment threads to address issues, or speak up on behalf of the project.
However, the role had never been formally clarified. While the lead developers handled many coding decisions, their additional responsibilities and authority were unclear. Did they have authority across the project? Should they make decisions in areas such as WordCamps, documentation, or theme review?
The lead developer role "didn't really have a set of responsibilities assigned with it or expectations,” says Matt. According to him, the lead developer leads development, not other areas of the project; he believes that conferring authority and responsibility to a development role makes it difficult for non-coders to achieve project authority and responsibility.
Matt had articulated as much, as early as 2005, in stating that commit access did not equate to community status. It may never have been Matt’s intention to automatically confer authority on people with commit access, though extending commit demonstrated that committers had earned trust, and, as such, people naturally looked to committers as community and code leaders. Since roles and responsibilities were undefined, people simply perceived commit access as a general leadership role. The only non-coder who had an official leadership role in the project -- other than the lead developers -- was Jen, but she was an integral part of the core development team and there was no clear path for anyone to follow in her footsteps.
In June 2012, the confusion around the role brought conflict. Since WordPress 3.0, a new generation of coders had driven development. Contributor changes were marked between the 2010 core meetup in Orlando, Florida, and the 2011 meetup in Tybee Island, Georgia. In 2010, just the small team of lead developers (Mark Jaquith, Andrew Ozz, Peter Westwood, Matt, and Ryan Boren along with Jen) met up. In 2011, the meetup had new faces including Dion Hulse, Jon Cave (duck_), Daryl Koopersmith, and Andrew Nacin.
From time to time, someone arrives and influences the project. In the early days, Ryan Boren drove WordPress' development, but post-WordPress 3.0, it was Andrew Nacin. "Nacin like Ryan is one of those guys that just has an ability to get in a flow, and just really crank and get focused intensely, and get through a ton of work," says Matt.
Nacin’s project influence grew, and between WordPress 3.4 and 3.5, Ryan Boren proposed that since Nacin drove releases strongly, he deserved recognition and should be made a lead developer.
Rather than appoint Nacin as a lead developer immediately, Matt proposed an organizational shift in the project. Matt argued that the lead developer title was historical and non-meritocratic, that those driving the project should hold leadership roles. Matt wanted opportunities for new developers to assume project leadership roles. He proposed that, instead of having a small group of lead developers, the project move to release leads, nominating Andrew Nacin and Daryl Koopersmith to assume the role in the next release. Matt's proposal offered clear authority to individuals for a given release; release leads would have final say, both over the lead developers, and over Matt. Some in the "historical and un-meritocratic" roles perceived this move as an attempt to remove the old guard to make way for the new. While a number of the lead developers aren’t active in core on a daily basis, they are in regular contact with those who work on core, providing advice on architecture and development.
On reflection, Matt says that there was a misunderstanding. He didn't mean to imply that the people holding lead developer roles were worthless or irrelevant, but that the roles did not reflect project reality. "A source of some of the conflict," says Matt, "is this idea that the lead developers sort of had the same role as me where they had sort of purview over everything across all parts of WordPress."
The lead developer role discussion raised dissatisfaction around project decision making. Many of those who ran day-to-day development felt that unilateral decisions were made without team consultation. Matt had taken a less active role in recent years, as Jen and Ryan, supported by the other lead developers, drove the project, yet decisions were made and announced without consultation. This was a culmination of other issues within the core development team and in the wider project: checking in code without consultation, providing feedback only toward the end of a release, and decisions around WordPress.org -- the site Matt owns, but that influences the entire community.
In response, some core team members -- including lead developers and other team members -- sent a group email to Matt to express their discontent with the project. They felt that Matt’s perspective on the project's organization, authority, and responsibility, didn’t reflect the project's realities. The group proposed an official project leadership team; they wanted to retain the lead developer title as a meritocratic title for core developers who aligned with WordPress' philosophies, demonstrated leadership, code expertise, and mentored new developers. While the group happily supported the release leads proposal, they felt that the decisions for architecture, code, and implementation should rest with the lead developers, and that decisions affecting the project should lie with a leadership team.
In response, those involved scheduled a Google Hangout to discuss the issues, air grievances, and find a way forward. As a result, things changed, and the first 3.5 release cycle post reflects some of those changes.
Andrew Nacin was promoted to lead developer, and core development adopted release leads with responsibility for an individual release cycle. They chose media as the scope for the 3.5 release; the development team had wanted to tackle it for some time, but with such a large scope, it hadn't been attempted. Daryl Koopersmith created a plugin, called Mosaic, which was the prototype for new media management. Much of it was written in JavaScript, where Daryl's expertise lay. But as a PHP free software project, there were few who could help him. As a result, Andrew and Daryl spent between 80 - 100 hours a week working on the release.
While the release itself was well received and media was overhauled, the actual development cycle wasn't such a success. It was a huge amount of work, involving lots of feedback, codebase iterations, and coding a whole new feature from scratch. There were four major iterations to the user interface, including one 72 hours before the release. This meant that the new "release leads approach" got off to a faltering start. The intent was to have release leads guide and lead a release, not necessarily spend hours carrying out heroic coding feats. Once again, the release cycle focused on a single feature; it shipped because two coders broke their backs getting it done. But the next release cycle -- 3.6 -- revealed that the release-specific feature development model was broken.