Gerrit Code Review - Hackathon #5, London - 7th-9th May 2013

GerritForge is proud to have organised and hosted the Hackathon #5 (the first in Europe) of Gerrit Code Review in London (UK) from 7th to 9th of May 2013.
It has been a fantastic opportunity to have the core Gerrit developer Team hacking on Gerrit code side-by-side on some new exciting new features to be released under Apache 2.0 OpenSource community license.

Gerrit Hackathons log

Some of the major improvements in Gerrit architecture and functionality came out from past Hackathons: plugins, revamped security ACLs, project dashboards, RESTFul API, audit-trail and validation hooks !!

A big thanks to ITHR Consulting for hosting the Hackathon in London Paddington and providing Breakfast, Lunch and refreshments during the Hackathon.

Overview

From 7th to 9th of May 2013 the first ever European Gerrit Hackathon took place in London: despite the challenges of getting people from all over the world it has been a success: a significant part of Gerrit contributors attended the event, coming from West Coast of the USA, Sweden, Denmark, Germany, UK and even the the far Japan. Unfortunately Shawn Pearce was not able to come over but he attended a couple of days the event remotely in VC from his desk @Google CA.

Shawn Pearce in remote VC at the Gerrit London Hackathon 2013 - Photo by Luca Milanesio

Shawn Pearce in remote VC at the Gerrit London Hackathon 2013 - Photo by Luca Milanesio

GerritForge has organised the event thanks to the kindly hospitality of ITHR Consulting at Paddington in Central London which offered the location and refreshments during all days.

Statistics

Some numbers of the Hackathon:

We can clearly see the influence of the London Gerrit Hackathon activity on the Gerrit git log from January 2013 till now.

Gerrit log graph in 2013 with the Hackathon peak

Gerrit log graph in 2013 with the Hackathon peak

Attendees

Name Country Company Web site
Dariusz Luksza Germany CollabNet http://www.collab.net
Dave Borowitz USA Google http://www.google.com
David Ostrovsky Germany LibreOffice https://www.libreoffice.org
David Pursehouse Japan SONY Mobile http://www.sonymobile.com
Deniz Türkoglu Sweden Spotify http://www.spotify.com
Edwin Kempin Germany SAP http://www.sap.com
Emanuele Zattin Denmark Swich Gears http://switch-gears.dk
Fredrik Luthander Sweden Consultant http://se.linkedin.com/in/luthander
Gustaf Lundh Sweden SONY Mobile http://www.sonymobile.com
Luca Milanesio UK GerritForge http://www.gerritforge.com
Martin Fick USA Qualcomm IC http://www.qualcomm.com
Thomas Swindells UK Cisco http://www.cisco.com

Day-1: topics discussion

As the time was very limited (3 days) we needed to list and prioritise the topics to cover during the Hackathon: we started a whiteboard exercise and associated discussion to select the ones to be covered and "hacked" together.

Hackathon topics whiteboard selection - Photo by Dariusz Luksza

Hackathon topics whiteboard selection - Photo by Dariusz Luksza

The short-track of topics selected have been the following:

  1. In-line editing (by Martin, Edwin and DaveB). The ability to perform changes to the code under review directly from the Gerrit Web UI.

  2. UI-plugin API (by Dariusz, Emanuele, Luca and DavidO) Introduction of new extension points for allowing plugins to enrich the Gerrit Web UI.

  3. Multi-master replication (by Deniz and Luca) Continuation of the replication work across multiple Gerrit masters. Unfortunately the time available was not enough to have a complete discussion or reworking of the current code base.

  4. Documentation (by Fredrick, Edwin, Dariusz, DavidP, DaveB) Refactoring of Gerrit on-line documentation set.

  5. Statistics (by Edwin, DavidP, Gustaf, Emanuele) Ability to extract and archive statistics on Gerrit usage.

  6. Capabilities (by DavidO, DaveB) Ability to contribute additional core capabilities (i.e. Forge author) through plugins.

  7. Branch creation review (by DavidP)

Face to face discussion on the hackathon topics - Photo by Fredrik Luthander

Face to face discussion on the hackathon topics - Photo by Fredrik Luthander

Before putting our hands into the code, we needed to have an Open and direct face-to-face discussion on how to approach those topics. It was great to have everybody on the same place, same time-zone, same mental wavelength :-). First day was mostly spent with the planning the discussions very useful to avoid spending time in writing code in the wrong direction.

Day-2: hacking

The second day was more focused on coding and getting things work. Major achievements were made on most of the topics, whilst some of them remained only at discussion level. All the people were working hard till late night to get things done and demoed on the next day.

Hands on hacking at the London Gerrit Hackathon - Photo by Fredrik Luthander

Hands on hacking at the London Gerrit Hackathon - Photo by Fredrik Luthander

See below a short summary of the achievements on a per-topic basis.

In-line editing

It has been main topic and killer feature of this Hackathon was inline editing. Part of the feature has now been merged and available on the current master branch (2.8-SNAPSHOT) (see Change). Gerrit can now provide the content of changes with a RESTFul API of the form: GET on /changes/<change-id>/revisions/<revision-id>/files/<patch-id>/content

The actual UI part that uses the RESTFul API is still under review (see Change) and a lot more patches are now submitted and mostly available for review (see Change) but there may be still a few issues that need to be resolved.

CodeMirror is the JavaScript engine that will drive most of the UI part, but for now we concentrated more on the REST API side on how to store edits on the server side to retrieve the content of a file from a revision was merged (see Change).

This is going definitely to be one of the most exciting features of the forthcoming Gerrit 2.8 release !

UI-Plugin API

It has been mainly focused on defining the possible additional extension points that we want to add to the Gerrit UI and choosing some of them for being hacked and finalised for review.

  1. Top menu items Allow a plugin to make its new functionalities available through the top-level Gerrit menu Menu entries could be static or dynamic, dependant on the status of the user or the UI elements.

  2. Buttons / Links / Fields / Forms Inserted into existing Gerrit pages, tied to well-known classes/ids.

  3. Additional buttons and links with actions Ability to add elements (buttons, links) and customise the actions associated to them.

  4. New pages or divs in existing one Render a whole content page, with gerrit menu, top header and footer available as reusable assets.

  5. RESTFul API for AJAX-based interations Plugins should be able to extende the RESTful API, allowing the backend counterpart of JS extensions to the GUI.

Dariusz was focused on a PoC of the point 1 (see Change) which was then experimented and used by Luca with the GitBlit plugin (see Change).

DavidO was previously working on a similar extension (see Change) focused on adding an extra button for triggering an ad-hoc build on Jenkins for validating a Patch Set as alternative to the auto-fetch and validate process integrated through the Gerrit Trigger plugin for Jenkins. The feature however can be used for any purpose as it is pluggable and can be associated to any action.

Dariusz goal was to have native UI plugins in Gerrit either in JavaSript, GWT or ClojureScript (everything that compiles down to JavaScript). A new proposed event based JavaScript API has been then proposed a few hours later: currently at concept stage, looking for feedback. It allows at the moment to add rows into patch info table, need expressed by Martin.

Finally Dariusz started working on prototype of AngularJS based WEB UI for gerrit: looking promising and hopefully to appear soon as RFC Change for review. Not much to share right now, but implementing the project list page in AngularJS was lots of fun :-)

Documentation

There has been a very interesting discussion and exchange of ideas on how to refactor the Gerrit Documentation set. Overall everybody agreed that current structure is not easily accessible by a Gerrit novice.

Fredrik proposed to use a different structure, more similar to other Open Source projects, with navigation bar and more focused sections to individual readers profiles. Luca highlighted the need of a full text search, as many people are really used to search throughout documentation with a Google search style syntax.

Many small fixes have been made all throughout the Gerrit documentation, thanks to the cooperation of all the "Gerrit hackers" present in the room. DaveB and Fredrik had a look at a potential script for automatically generate a navbar to replace the index page with in the future. DaveB proposed in the future to take a different approach, more similar to the plugins documentation generation where the HTML text is dynamically generated from the code at runtime rather than produced at compile time.

Fredrik rebuilt the index page from a semi-structured thing to a very hierarchical thing with chapters and subchapters. Additionally he submitted a section in access controls that cover special and magic references. A draft uploaded to start work on the refs/meta/config file content documentation. At the hackathon asciidoctor came up, but we decided that in the short term there is much lower-hanging fruit in terms of documentation reorganization that doesn't involve changing the build (not counting Documentation/Makefile). We will definitely revisit it in the future.

Overall it seems still silly that we use markdown for plugin docs with auto-generation based on the plugin loaded at runtime whilst still rely on asciidoc for core docs, so JRuby+asciidoctor may be a good way forward once the build system dust settles.

Statistics

We didn't find time to touch this topic. Edwin uploaded a draft for retrieving user statistics via REST (see Change), but there are still a lot todos left. Internally some people at Sony Mobile have been working on a tool to get stats related to the changes submitted towards the Android Open Source Project.

Edwin added this item to the agenda because the data that tool needs was only available by running multiple queries against the server.

We initially thought we'd add some statistics generation and display on the Gerrit side, but then we realised that needs from different people may be different and it is going to be difficult to find the one size that fits all.

Instead we focused on making the existing queries (or rather the REST APIs) return more data that the user can use to generate their own stats. Gustaf uploaded two changes for this features.

Branch creation review

Another topic that I've discussed with David and Martin was how to make the branch creation reviewable

Related to this I also started work on adding support for a new ref: refs/meta/requests Commits pushed to this ref should have specific meta data in the commit message, in the "Key: Value" format already used for e.g. Change-Id. When the change is merged, the MergeOp will invoke the REST API (to be added by Edwin) to create the branch.

Capabilities

Most of Gerrit administrators and contributors doesn’t know what term ‘capability’ means in Gerrit environment. Capability is type of permission (e.g. forge author, label verified) supported by Gerrit core. Currently plugins cannot contribute their own specific capabilities, see replication plugin as an example of those limitations: it uses start-replicate capability defined in core (but not used there) to grant users permission to execute replication.

DavidO started working on this topic DaveB and have produced a Change for review (see Change). Unfortunately the solution was not approved and has now been abandoned. Hopefully it would have inspired other changes to be submitted and merged before the 2.8 Release. This would then allow other plugins can contribute theirs own capability and extend this way Gerrit access rights.

Misc

PatchSetInserter

Outside of the query-stuff, Gustaf worked on some code refactoring (PatchSetInserter) which will hopefully help plugins (and features) in Gerrit to add new Patch Sets to Changes correctly and consistently. The plan is to bring it into ReceiveCommits in the future, hopefully making maintenance a bit easier.

Cherry-pick from Gerrit UI

Gustaf worked as well on getting Cherry Picking from the UI merged, with the help of DaveB for getting changes validated and merged.

Reviewers-by-blame plugin

Deniz started the refactoring of the reviewers-by-blame to a plugin. It is not finished yet as it seems complicated to use postReviewers within the worker thread.

Day-2: meet-up in Notting Hill

End of the second day, to reward the effort of the long hacking day in the Paddington offices, a meet-up was organised in a very nice Italian restaurant at Osteria Basilico in Notting Hill.

Gerrit Hackathon meet-up at Osteria Basilico Notting Hill - Photo by Mario Romano

Gerrit Hackathon meet-up at Osteria Basilico Notting Hill - Photo by Mario Romano

It has been a fantastic evening, nice and mild ... unusual weather for London in summertime :-)

Day-3: Demos and decisions on Gerrit

On the last day we demoed all the work prepared, discussed and done on the previous two days. Additional it was the perfect moment to have everybody on a round table to discuss opened burning points recently emerged, such as the proposal, with some push backs received, of moving away from Maven and adopting BUCK as build system.

Final demo and discussion on Gerrit burning topics - Photo by Fredrik Luthander

Final demo and discussion on Gerrit burning topics - Photo by Fredrik Luthander

Road-map

It has been agreed that from now on, after R2.6, we will have the objective to reduce the lead time between major releases. Next forthcoming releases in the Gerrit road map are:

Moving to Buck build

It has been agreed to continue to pursue the adoption of an alternative build system, most probably it will be buck. The buck build will be then merged into the master and an observation period defined to see if the problems that may arise will be sorted out or work-arounded effectively.

Acceptance criteria have now been published to: https://gerrit-review.googlesource.com/Documentation/dev-buck.html

First feedback of using BUCK instead of Maven are encouraging:

... still big problem to be sorted out: Windows support ! Facebook committed to allocate a dedicated resource to get it sorted out by this Summer :-)