Blog — Programming

Standardizing interfaces across projects with Makefiles

by Pierre de La Morinerie, posted 30 September 2016 | Add comment

If you are working on a non-trivial software solution, you probably have several projects. Maybe one for the frontend, and one for the backend? Maybe one for managing data, or secrets, or deployment? Or did you drink the micro-services kool-aid (please don’t), and have now dozen of projects, each one containing a small server?

In this case, the README file for these projects probably have a common structure (“How to setup”, “How to run”, “How to test”) – but all of them will contain different invocations. Sometime the project needs to be started with npm run, or node server.js, or ember server – and some other project with bundle exec rails server, or maybe ./bin/server.

What if we could instead express standard shortcuts for all these commands?

That’s what Unix project maintainers have been doing for ages – and also what we eventually did with Captain Train projects: we used Makefiles.

Read more »

Building our web-app on GitLab-CI

by Pierre de La Morinerie, posted 21 July 2016 | 3 comments

The railway world is a fast-moving environment. To bring you the latest improvements and fixes as quick as possible, Captain Train’s web-app is often updated, sometimes several times per day.

Did you always wonder how we manage building and deploying all of this without a jolt? Then read-on: here is a technical peek into our engineering process.

GitLab at Captain Train

From Jenkins to GitLab-CI

We used to build our web-app using Jenkins. A robust and proven solution—which was polling our repositories every minute, and built the appropriate integration and production branches.

However we recently switched to a new system for building our web-app. To host our source-code and perform merge-requests, we’re using a self-hosted instance of GitLab. It’s nice, open-source—and features an integrated build system: GitLab-CI.

See it like Travis, but integrated: just add a custom .gitlab-ci.yml file at the root of your repository, and GitLab will automatically start building your app in the way you specified.

Now what’s cool about this?

Reliable dockerized builds

Jenkins builds were all executed on a resource-constrained server—and this made builds slow and unreliable. For instance, we observed several times PhantomJS crashing randomly during tests: apparently it didn’t like several builds running on the same machine at the same time—and a single PhantomJS process crashing would bring all of the others down.

So the first step of our migration was to insulate builds into Docker containers. In this way:

  • Every build is isolated from the others, and processes don’t crash each other randomly.
  • Building the same project on different architectures is easy, and that’s good news, because we need this to support multiple Debian versions.
  • Project maintainers have greater control on the setup of their build environment: no need to bother an admin when upgrading an SDK on the shared build machine.

It scales.

GitLab-CI allows us to add more runners very easily. And now that builds are performed in Docker containers, we don’t have to configure the runners specifically with our build tools: any out-of-the-box server will do.

Once a new runner is declared, scaling is automatic: the most available runner will be picked to start every new build. It’s so simple that you can even add your own machine to build locally.

We’ve already reduced our build time by switching to a more powerful runner—a migration that would have been more difficult to do using Jenkins. Although we regularly optimize the run time of our test suite, sometimes you also need to just throw more CPU at it.

Easier to control

With Jenkins, the configuration of the build job is stored in an external admin-restricted tool. You need the right credentials to edit the build configuration, and it’s not obvious how to do it.

Using GitLab-CI, the build jobs are determined solely from the .gitlab-ci.yml file in the repository. This makes it really simple to edit, and you get all the niceties of your usual git work-flow: versioning, merge requests, and so on. You don’t need to ask permission to add CI to your project. Lowering the barrier to entry for CI is definitely a good thing for engineering quality and developer happiness.

Tests on merge requests

GitLab-CI makes it really easy to build and test the branch of a merge request (or a “Pull request” in GitHub slang). Just a few lines added to our .gitlab-ci.yml file, and we were running tests for every push to a merge request.

Still testing – but just hit the "Merge When Build Succeeds" button and move on

Still testing—but just hit the button and move on.

We get nice red-or-green-status, the quite useful “Merge automatically when the build succeeds” button—and, as branches are now tested before being merged, much less build breakage.

Ready to merge.

Ready to merge.

A slick UI

GitLab-CI provides “Pipelines”, an overview of all your build jobs. This points you quickly to a failing build, and the stage where the problem occurs. Plus it gets you this warm and fuzzy feeling of safeness when everything is green.

All Pipelines are green, ready to deploy.

All Pipelines are green, ready to deploy.

In a nutshell

We found the overall experience quite positive. Once the initial hurdle of making the build pass in a Docker container, integrating it into GitLab-CI was really easy. And it gave us tons of positive signals, new features and neat integrations. 10/10, would build again.👍

Our Android team also migrated their pipeline, and are now building the integration and production Android APK with GitLab-CI.

For further reading, you can find on the official website a nice overview of GitLab-CI features, and some examples of gitlab-ci.yml files.

Nuts and bolts: our routing algorithm

by Tristram Gräbener, posted 23 October 2015 | 9 comments

Captain Train allows to book train tickets across Europe by automatically combining multiple carriers. All you need is a few clicks on your computer or a few taps on your phone and we do the rest.

Let’s assume you’re traveling from London St. Pancras to Roma Termini. Here’s what your journey looks like on our website.

London-to-Roma-CaptainTrain

From London to Roma with three carriers

This journey is made up of three sections and two transfers. Each section of this journey must be booked with a different carrier (Eurostar, SNCF and Trenitalia). A previous blog post explains in greater detail and in French how we combine those carriers.

How do we find the best ticket for you then? In order to pick the best carrier combination, the first step is to compute multiple routes. A route is the way taken by a train in getting from London to Roma. Today we’ll focus on the technical details of our internal routing engine.

Read more »