ROS build farm

The ROS build farm is an important infrastructure to support the ROS ecosystem. It provides building of source and binary packages, continuous integration, testing, and analysis. It was designed to meet the needs of the public ROS community and there is a hosted instance for open source packages at http://build.ros.org/, for more information and policies see: build.ros.org

The ROS build farm is also designed to enable deploying a custom build farm. This can be useful to release closed-source packages, build for platforms and architectures not provided by the official ROS build farm, and/or customize any part of the process to specific needs.

The main coordination is done by a Jenkins instance. There are other people using the tools with BuildBot. The goal is that most of the infrastructure is modular and reuseable. But for documentation we will assume you are using Jenkins.

If you are going to use any of the provided infrastructure please consider signing up for the build farm discussion forum (http://discourse.ros.org/c/buildfarm) in order to receive notifications e.g. about any upcoming changes.

The setup of a ROS build farm involves two steps:

Design

For more information on the design please see /Design

Provision machines

A ROS build farm requires three different kinds of machines:

• a Jenkins master orchestrating the execution of various jobs
• multiple Jenkins slaves performing the actual builds
• a webserver providing file hosting for end result products

Please follow the instructions from https://github.com/ros-infrastructure/buildfarm_deployment to provision the machines for a ROS build farm. This will involve creating a **private** repository containing the custom configuration for the build farm deployment.

The result of running the provisioning instructions will be an empty Jenkins master, a number of slave, and a repo which are waiting to be utilized. The slaves as well as the repository host will automatically keep their configuration up-to-date based on the configuration.

The provisioned machines can be used for running jobs for multiple ROS distributions.

Generate Jenkins jobs

The job generation happens in two steps. First a set of administrative jobs needs to generated. Second the Jenkins master will run special jobs which will generate the actual jobs performing all the builds.

Please follow the instructions from https://github.com/ros-infrastructure/ros_buildfarm to generate a custom set of Jenkins jobs. This will involve creating a repository containing the custom configuration for the Jenkins jobs.

The result of running the job generation instructions will be a Jenkins master configured with a set of jobs. Those jobs will perform the desired build automatically. Whenever the ROS distro database changes Jenkins will automatically reconfigure the jobs with new / updated / removed repositories and packages.

Manually interaction is commonly only necessary to synchronize built packages into the main repository.

Most of the jobs use Docker containers for isolation. It is not necessary to use Docker, and it may be necessary to use full VMs as we extend to non-Linux platforms. But at the moment Docker provides faster performance than VMs.

Previous pbuilder-based build farm

The old contents of this page, documenting the first generation build farm, has been moved to this subpage: /Gen1Buildfarm