I have been recently asked how the Android team at Capitaine Train is working: “How often do you release new versions of the application?”, “What’s your versioning strategy”, “Do you beta test builds?”, etc. Let’s be clear: I’m not a processes/methodologies freak and prefer not to talk about them most of the time. I would rather consider myself as a product guy. However, no matter how great a product is, software development processes take part of its success. This article is an attempt to share and discuss about our “way of working” in the Capitaine Train for Android team.

Note: Prior deep diving into this article, let’s start with a quick disclaimer. Indeed, I think it is mandatory to mention the strategies, processes and other methodologies described below are far from being ideal. Just like there is no perfect answer in UX or development, there is no perfect way to deal with large projects. In other words, this article has no intentions to force you to switch to new methodologies and should be read as a simple feedback on how the team manages projects like the Capitaine Train for Android apps.

The Capitaine Train for Android context

Methodologies have no meaning when taken outside of their context of usage. Thus, I believe an introduction to the Android team at Capitaine Train is mandatory. The team was born on March 2013 when I joined Capitaine Train to lead Android applications. It took me quite some time to get used to the extremely complex train european eco-system. We rapidly decided to grow the team and I was joined in November 2013 by Mathieu Calba and more recently by Flavien Laurent in October 2014. If you are (really) good at counting you will have noticed the Android team at Capitaine Train is a group of 3! Being such a relatively small team is an important point to keep in mind to better understand our processes.

I’m really proud to be part of Capitaine Train team but I’m even prouder of the Capitaine Train for Android team I built. Mathieu and Flavien amaze me every single day and it’s always a pleasure to work with them. All of the members are talented developers extremely focused on the product, do not hesitate to fight for what they believe in and have a clear understanding of what a good UI/UX is (which is not really common with developers…). In other words, each member of the team is extremely independent when it comes to design a new feature from the ground-up. Most of their time is spent working on Android related features. It means we are all mostly working with the Android framework but may also spend some time on other platforms (Ruby on Rails for instance) to implement features that specifically relate to Android (Google Now is a great example).

From a product point of view, Android at Capitaine Train can be summed up into two different applications. The most important one, the handheld, is adapted to both tablets and phones. The smaller and most recent one targets wearable devices. The minimum supported version of Android is Android 4.0 (API 14) and the app bundles 4 different languages: English, French, German and Italian.

Capitaine Train targets worldwide customers. However, our offer is clearly focused on european trains. As a consequence, most of our audience is based in Europe and lives in 2 to 3 different timezones. Having to deal with such a relatively small span of timezones is quite helpful, especially when announcing new stuff as communication is synchronous (in a way a given moment in the day is approximatively the same moment of the day for all of our users).

Regarding download numbers, we usually don’t communicate on them. However, at the time of the writing, Google Play Store publicly indicates the application has been downloaded between 50.000 and 100.000 times.

The implementation process

Talking about how we deal with code at Capitaine Train could be the subject of an entire article. Instead of deep diving into our design, development and/or testing techniques I will only give a surface overview of our processes.

One developer, one reviewer

The entire Android code base is managed by Git. I don’t think it is necessary to present Git in this article as it is well known to be one of the best source control management system out there. We use it extensively at Capitaine Train as all of our review techniques are based on this amazing development tool.

We are also using the git flow model on top of Git. This model ensures a coherent and understandable commits tree. Put simply, development is done on the dev branch. When a teammate needs to work on a new feature, a new “feature branch” is created starting from dev. It’s up to the teammate in charge of the feature to create the branch. The “feature branch” is rebased on dev until it is finally merged. Releasing the application is synonymous with merging dev into master using the --no-ff option. This option ensures the merge is always represented by a commit. Finally, the commit is tagged with the application version code. In other terms, the master should only contain commits that refer to a public version of the application.

Prior being merged […] the code […] is always read and validated by at least two members of the team.

Because the Android team is quite small, all of the features are always managed by a single member of the crew (let’s say Bob). Bob is entirely responsible for the development of the feature: from design to release. Once the feature is considered mature and polished, it is submitted as a “Merge Request” to another member of the Android team (Alice). The review process is done thanks to a tool called GitLab which can be seen as a GitHub clone. Alice is responsible for reviewing the code. Feedbacks can be extremely heterogenous. For instance, it is common to have reviews like: “You should use this method instead” to “I would have used a different text color and text size” or “I won’t merge this if it makes the APK heavier than 5MB!”. Reviews with a set of alternatives are usually extremely appreciated compared to a simple “No!”.

One of the particularities of the Capitaine Train for Android team is each member is responsible for QA. Indeed, there is no QA team in the company¹ and both Bob and Alice have to make sure there is no regression and that the code works perfectly. It basically means the code review does not only consist on reading code. Alice also has to test the implemented feature in all possible conditions.

The review process ends once Bob and Alice are both okay with the feature in general (code, design, introduced changes, API, etc.). As a consequence, prior being merged into dev, the code from the Capitaine Train for Android applications is always read and validated by at least two members of the team. This is actually the case for all projects at Capitaine Train. Larger companies also rely on a similar process but require at least two “+1"s from reviewers. We clearly can’t afford to do that in such a small team.

Packaging builds

The entire project is built on top of Gradle. The main advantage of the new Android Gradle-based build system relies in the fact it fits both development and packaging purposes. When developing a feature, we all use Android Studio (which also uses Gradle under the hood) and when it comes to packaging, command-line line is used. Builds packaging is done thanks to our continuous integration environment based on Jenkins. Jenkins currently manages two different Android projects:

• android-dev which builds what’s currently on the dev branch
• android-master which corresponds to the master branch.

A new build of these projects gets triggered when a new commit is pushed in the main remote repository (origin). The main differences are each projects point to a different branch and have different compilation options. Indeed, contrary to android-dev, android-master is optimised and obfuscated thanks to Proguard.

Because we had to create a lot of screenshots (3 form factors, 4 languages, 6 screenshots: 3x4x6 = 72 screenshots) we recently added a new home-made tool to the packaging process: automatic screenshots (thanks Flavien). The new utility takes care of taking screenshots in all of the necessary configurations and clean them up by unifying the status bar. At the time of the writing this tool is not integrated right into our Jenkins build stream but this is definitely something that will be done in the future.

When it comes to publishing on the Google Play Store, everything is done manually. Obviously, the new Google Play Publishing APIs could be used but we prefer to keep control over the releases for now. Because of our “lengthy” release life cycles, I’m convinced this is not an issue with our current way of dealing with releases.

Application versioning

Versioning an Android application is a mandatory process. Indeed, the Google Play Store uses application version code in order to detect new versions of the application. The only requirement from the Google Play Store is to make sure the application version code is incremented monotonically.

Instead of […] trying to determine whether a version is major, minor or patch, each new release containing at least one new user-visible feature is considered major.

The Capitaine Train for Android application don’t use the traditional major.minor.patch versioning (aka semantic versioning). Indeed, because we wanted to have as less friction as possible we came up with a simpler versioning model based on two version numbers: major and minor. The application code is computed based on these numbers thanks to the following formula:

The main idea behind this versioning strategy is to have no or low friction in the release process. Instead of scratching our heads trying to determine whether a version is major, minor or patch, each new release containing at least one new user-visible feature is considered major. Bug fixes and patches are always considered as minor versions. This works particularly great in conjunction with the release schedule described below.

From an external user point of view, only the major version is important. The version name of the application is always the major version regardless of the minor version. The main reason behind this naming strategy is minor versions are supposed to be completely transparent to the user as they do not contain any user-facing features. Using only the major number of the version name makes it simpler to remember and more recognisable. If you really want to know the exact version code of the application, you can open the “Settings” screen in the Capitaine Train for Android app. It shows the version name (also displayed in the system Settings app) + version code:

Development is fun but you can make it even more fun. All of our public builds are actually named internally (such like regular Android releases). As a huge of fan of Stargate SG-1, I named each major release according to some characters of the show: ANUBIS (101), BRATAC (201), CARTER (301), etc. Minor releases are names using the _MR<x> suffix where <x> is the minor version number minus one: FRAISER_MR1 (602), GEORGES_MR1 (702).

The Android Wear app versions also follow the same versioning pattern and are named according to characters from Pixar movies: ANDY, BUZZ, COLETTE, etc.² Even though the wear app is tied to the handheld app from a release point of view (the wear APK is packaged inside the handheld APK and published simultaneously on the Play Store), we decided to use a distinct versioning.

Having fun internally is obviously not the only purpose of maintaining such a list of version codes. It helps us easily change the execution behaviour depending on the current version of the application. Let’s say you had a serious issue when storing some stuff on disk in a given version of the application, you probably want to check if the app version is greater than the data stored on disk to know if it’s time to update the data to the new format.

The complete release process

Capitaine Train users may have noticed, the team is extremely focused on the product. We don’t release new features until they are definitely ready for production. Our high quality standards prevent us from publicly releasing non polished features. This is one of the reason there is no strict deadlines on the project.

Instead of sticking to hard deadlines, the […] Android app follows the release train software release schedule.

Instead of sticking to hard deadlines, the Capitaine Train for Android app follows the release train software release schedule. The release trains are time based release schedules. It does not wait for either features, or bug fixes but is based (as purely as possible) on time. Put simply, each new version of the app can be considered as a train that leaves and arrives on time. If a feature is ready for the planned time of arrival it jumps into the release train. If it is not, the feature has to wait for the next release train. Release trains enforce discipline in introducing features, give predictability, and allow more regular releasing. And yes! Similarities between the naming of the methodology and the company name are just a coincidence :-).

Release train only applies to major releases of the Capitaine Train for Android app. Minor releases are done on a completely different timeline. Because minor versions are usually hot-fixes on blocking or crashing issues, they are released as soon as possible regardless of the release train schedule. This usually only happens during the beta testing phase of the app, as discussed later in this article.

The circle of life: release life cycles

New major versions of the Capitaine Train Android application are released following a recurring pattern. This pattern repeats itself every 6 weeks. Why 6 in particular? To be honest there is no complex maths behind this figure. It is only empirical and comes from my experience as an Android app designer and developer. Here is the rationale:

• By default, Android auto-updates applications. When doing so, a notification is displayed in the notification tray. Releasing your application too often may be annoying for users as they may consider it as spam.
• Publishing a new version of the application too often is synonymous with small feature updates. This makes the release less attractive and more importantly less marketable.
• On the other side, releasing using large spans of time can also be counter-productive. Your users will completely forget about your application. Moreover, it increases the potential of having serious issues in every new releases of the application.
• Regarding our development/product team and the pace at which the product is evolving, we are convinced we can introduce new user-facing features every 6 weeks. Our history shows us we managed to stick to that for each releases since version 1 (and not version 1.0 ^^).

To be honest, I don’t think there is a perfect release life cycle length. The 6 weeks pattern works particularly great at Capitaine Train because it is half way from both users and our own expectations.

The diagram above describes our release life cycle. As explained earlier, each version v_(n) is being prepared for 7 weeks. Because successive life cycles overlap, a new version is released every 6 weeks. The planning during a release life cycle is quite flexible and up to the engineer. However it usually reduces to:

• Week 1 starts with a meeting with the product manager where features are prioritised and evaluated (in term of requirements, implementation duration, etc.). The feature ranking/ordering phase is actually usually done in advance but often finalised during Week 1.
• Week 1 to Week 4 mostly consist on developing new features.
• Week 5 is half new features polishing and half bugs fixing. Because we want to make sure the reviewer has at least a week to do the code review, the code is considered “feature frozen” at the end of Week 5. In general, the amount of time left for the code review is flexible and depends on the feature overall complexity.
• During Week 6 engineers concentrate on bug fixing, code reviews and integration testing. The main purpose of Week 6 is to polish and prepare the product for a beta release at the end of the week.
• Because nothing is supposed to happen during Week 7 on v_(n) (at least from a development perspective, it is more a beta testing week), this week is also Week 1 for v_(n+1). If nothing critical is found during the beta phase, the build published in the beta channel is promoted to production at the end of Week 7. In case a critical bug is found during the beta, it is up to the member of the team who introduced the bug to fix it and handle the next build & publish process at the end of Week 7.

And the release day of the week is…

Another interesting point is the Capitaine Train for Android apps are always released the exact same day of the week: Tuesday. More precisely: Tuesday morning. Tuesday have several advantages over some other days in the week. Most of these reasons are common to most software projects but some other are rather personal:

• It is well known, Tuesday morning is generally considered the best moment to publish new stuff. This is mostly because Tuesday is one of the most busy day in the work week. Releasing the app on Tuesday is a great way to help your marketing team and ensure the release will have the maximum impact on your users.
• Tuesday is the second day of the week (it’s not a surprise, right? Well. Actually it depends on what day you consider to be the first day of the week. From my point of view, it’s Monday). In case the public release turns into a drama of bugs and crashes, the development/support teams will be able to respond quickly and will have up to 3 days and a half to fix bugs and crashes prior the weekend. It is no surprise overall awareness during worked days is higher than during weekends…
• Tuesday is generally a “calm” day for the Android team. My colleague - Mathieu - and I are both living in Lyon (500kms away from Paris), working 3 days a week at the office in Paris and 2 days a week from home. Because we generally work remotely on Thursdays and Fridays, we tend to accumulate meetings and discussions on Monday clearing up our minds in order to be ready for a smooth release the day after.

Beta channels, crash reports and staged rollouts are in a boat

I previously mentioned a beta phase in Capitaine Train for Android release life cycle. At Capitaine Train, beta testing is done via the beta channel from the Google Play Store. The beta is private but people can ask to join. We are pretty picky when it comes to add new beta testers to the pool. Indeed, a beta tester has to be both extremely active (users travelling by train at least once a week) and trustable (we don’t want him/her to communicate about features to be released soon). Beta tester have an entire week (Week 6) to test and report important bugs.

Users report crashes through the Google Play […] only once every 25 crashes

Because crash reporting on the Google Play Store requires users to approve sending the crash info, we added an additional crash reporter: Crashlytics. This is extremely important as users tend not to report crashes. For example, for the current production version, there is a 25x difference between Crashlytics and Google Play Services reports. In other words, users report crashes through the Google Play feedback dialog only once every 25 crashes. Crashlytics helps us to be notified about important fatal crashes. We can also prioritise crashes based on the number of occurrences and the type of devices.

Google Play Store provides a really great feature called “staged rollouts”. Staged rollouts consist on making new versions of your application available to only a subset of your entire user base. For instance, it allows you to release the new build to only 10% of your audience. This is particularly great to test new features or potentially reduce the load on your servers. We have been experiencing with this feature during the first versions of Capitaine Train for Android. Because we are quite confident with our release life cycles, we now use it only sparingly (in case the code contains some ground-breaking changes). As a consequence, 95% of our releases are done in a single full release (100%).

Conclusion

I hope I have explained in details how the Android team at Capitaine Train works. I tried to be as precise as possible but may have forgotten some important points. Do not hesitate to leave a comment below and I will try to answer questions regarding missing important points. Once again, do not forget this article describes methodologies that are dedicated and apply nicely to the Android team at Capitaine Train. Always keep in mind all contexts are different. Think about the context in which you are working prior changing and adapting the way you work on your projects.

Thanks to @Mathieu_Calba for proofreading this post.

---

• ¹: Unfortunately, this is true 99% of the time in France. From my point of view, QA teams are extremely important to make sure the product works as intended. It appears most companies don’t want or can’t afford having a dedicated QA team and prefer relying on the development team.
• ²: Spoiling alert: the next version of the Capitaine Train for Android apps to be released are JONAS and DORY…