Skip to content. | Skip to navigation

Personal tools

Navigation

You are here: Home / Steve's Blog / Comparisons: Fair and Unfair

Comparisons: Fair and Unfair

Posted by Steve McMahon at Nov 18, 2013 07:25 PM |
Filed under: ,
In which Matthew and Steve subject a short tweet to lengthy analysis.

By Matthew Wilkes and Steve McMahon

Twitter isn't a thoughtful medium, and it may be unfair to critically examine a less-than-140-character snippet of an idea. But, here we go!

Recently, Matt Hamilton set off a conversation with the tweet:

Interesting to see the trend in both commits and contributors for #plone ohloh.net/p/plone and #drupalohloh.net/p/drupal

Let's take the comparison apart. We've got three key words here: "contributors", "commits", and "trend."

Contributors

Ohloh's 30-day contributor count for Drupal core is six. For Plone core, it's 60. The 12-month count is Drupal, nine; Plone, 227. That's quite a difference! But, is it a fair comparison?

It may be a fair comparison of the way the communities work, but it's not a fair representation of the number of contributors.

The Drupal core project funnels its commits through a very small group. Contributors submit code to the small group with commit rights. They review and commit the code. So, contributors submitting posts are invisible to a robot (Ohloh's) examining commits. (They are credited in commit messages.)

Plone does not rely on a small, powerful group of committers. Instead, everyone who signs Plone's contributor's agreement may, in principle, commit to core. In fact, many don't do so; instead they submit pull requests and the pull requests are reviewed by package managers before merging. That still makes for a lot of contributors: Plone has a lot of package managers, and others who are heavily involved in a module often commit directly rather than via pull request.

A few short years ago, though, this wasn't the case. Before Plone moved its source code to GitHub we relied on three monolithic subversion repositories: the core 'plone' repository, the 'collective' repository and the lesser-used 'archetypes' repository. The practice of contributing via a pull request was not possible on subversion, so developers were encouraged to commit directly to a project. Although some people branched packages in the Plone and the collective repositories there was no standard procedure for merging these back in to trunk and they would often languish for years.

When we moved to GitHub we wanted to keep the feeling of collective ownership of code that had evolved over many years, as did many other projects, leading to a collision between the large community SVN workflows and the GitHub fork and pull request workflow; it's no coincidence that the contributors most likely to use a pull request are the newer contributors. This problem was so pronounced that we developed tools to manage the permissions in GitHub for us, to most closely model the permissions architecture of SVN.

How does Plone keep up core quality with many contributors? First of all, there is code review after commits, and code not meeting standards is reverted. Second, Plone does a tremendous amount of testing, and has been doing so for many years. That huge body of tests makes for a lot of regression insurance.

Drupal made different choices on repositories, moving from a self-hosted SVN repository to a self-hosted Git repository, so they didn't have the social pressure of GitHub's pull request system. For people running a self-hosted Git repository the strongest pointer to a workflow is the Linux kernel, which also works by sending emails containing a patch file. This has the effect of flattening an entire branch into a single commit, often meaning that many people should be listed as the author. In addition, it means that the original commit messages and metadata is lost, leaving the committer responsible for merging to populate them.

Although both communities value contributions very highly, Plone focuses more on attribution artifacts such as individual pieces of documentation whereas Drupal emphasizes non-code contributions on a contributor profile. This makes for less visible code contributions but vastly more visible community contributions.

Is either strategy better? That's your decision. Certainly both communities are great places to work, but the differences for end users are minimal. Both communities take code quality seriously, so the final choice of CMS should really be made on the basis of whether or not you like the application's feature set and your particular requirements.

Commits

30-days commits: Drupal 197, Plone 902. One-year: Drupal 4,389, Plone 11,563. Again, a large difference. But a fair comparison?

Based on the "contributors" discussion, you've probably realized it isn't a good comparison of activity. Drupal's commit trail is flattened by their funnel strategy. Plone's is flattened some, but not nearly as much.

Again, this says more about project community differences than activity. Plone is developed in public. You'll see the warts of early commits that failed tests or code review and had to be fixed. For Drupal, they're comparatively invisible. On the flip side, it's easier to find all code related to a given feature in a Drupal repository.

Trends

Drupal's funnel methodology means that comparing trends in contributors is just as meaningless as comparing recent contributor counts.

Comparing trends in commits, though, is probably fair game. We don't have any reason to believe that either project's code-processing methodologies have fundamentally changed. So, commit trends may say something about whether a project is more or less active than it was in the past.

Let's take a look at Ohloh's trend charts for commits for Drupal:

Drupal Commits per Month

and Plone:

Plone Commits per Month

Again, we can't really compare the aggregate activity levels. Just the trends.

What might explain the changes for each project? In our minds, there's no question about Plone's changes. Activity, and those intangibles, excitement and energy, are all on a dramatic upswing.

What about Drupal? We're not as well acquainted with the project, but we do know that Drupal has turned to emphasizing that Drupal's core is a "platform" — not an application. Drupal's core, out-of-the-box, doesn't do much of what is typically needed for a complex site. But, there are well-known add-ons that do the job. Drupal integrators know how to put them together with the core to match their project requirements. As Drupal continues the shift from application to platform, one might expect a continuing drop in core activity. That said, there is a clear difference between the chart before 2008 and after, corresponding almost exactly to the release of Drupal 8. This implies that since the release of Drupal 8 the community has stabilized to the level of activity that can be sustained by the current size of the pre-commit review team.

Plone made the application vs. platform decision differently. Plone is definitely emphasizing the "application" aspect of the project. (This has disappointed some early adopters who saw it as a web-application development platform.)  Plone is continuing to add major "application" features to its core.

What about other projects?

If commit trends are a more fair comparison, what about other popular projects?

Joomla:

Joomla Commits per Month

Wordpress:

Wordpress Commits per Month

Typo3:

Typo3 Commits per Month

Perhaps there are similar explanations for these changes? Others will have to chip in with those.

Numbers and charts for this post were extracted from ohloh.net on 2013-11-18.

Matthew Wilkes is a principal in The Code Distillery, which provides specialist consulting services for Python applications. Steve McMahon is a Plone consultant and instructor.