Distributing Free Software: Herding Cats

The basic challenge of distributing Free Software is compiling that awesome open source code into machine code for different Linux distributions so it's easily consumable for users. Sounds simple, but it isn't. If you look at the dependencies of a typical Free Software project you will find thousands of other projects in the stack. They build on top of each other, have functional dependencies, sometimes they are interdependent, they conflict with each other and what not. In short: Building Free Software is like herding cats. And rightfully users of the software expect a steady, well behaved, streamlined herd of cats! The Open Build Service (OBS) is the tool which makes this possible, it helps Free Software distributors to automatically and consistently build binary packages from free and open source code.

It goes like this: Engineers upload source code and build instructions to the OBS and the system compiles that into binary packages for different distribution versions and different CPU architectures. The build instructions say how to compile the source code into machine code, defines the dependencies and the conflicting capabilities the software has in relation to other software, itemizes which files are needed to run it, and a whole lot of other meta data. The job of the OBS is to interpret all this information, to provide a build environment matching the requirements, the execution of the build, and of course the reporting of the outcome.

The end result are multiple binary packages out of one single source. The twist is: If other software depends on the package in some way, the OBS will trigger a rebuild of the depending package afterward. This ensures that changes propagate through the whole stack and that the user gets a consistent product to install.

Updates: Nobody is perfect.

Sadly, software is sometimes defective and people make mistakes. Nobody is perfect. That's why the second basic service a Free Software distributor delivers to its users is the art of exchanging pieces without breaking the whole: Updates. Your distributor does not want to interrupt you in going about your business, and to be able to do this he needs to reproduce the software builds at any given time in the long lifetime of the software.

The OBS helps distributors to achieve this by tracking who has made changes, when changes are made, and what has changed. The OBS also helps by utilizing a clean, virtualized build environment. This is how it goes: When an engineer triggers a build in the OBS by changing the software in it, the system saves the current state, gathers all the dependencies and kicks off a virtualized environment to execute the build. As the information on how to assemble the build environment is contained in the software and as all dependent software gets rebuilt too it makes every build reproducible.

And if something is reproducible it is predictable and that is what distributors aim for. If you can predict how all of the software projects in your stack influence each other, you can make sure that a change to a single piece can be managed through all of its dependencies, ensuring that the whole system of software continues to work after a change. This is how the OBS helps you as a user because updates from the OBS don't get in the way of your business.

Collaboration: Unity is strength

Some rights reserved by opensourceway

Now the software isn't the only herd of cats that needs taming. Each of the thousands of software projects is maintained by one or more developers that collaborate with each other, that's a whole bunch of awesome people! And this is the third basic challenge a Free Software distributor solves for its users. It brings together people who collaborate on the integration of free and open source software.

The OBS formalizes this collaboration into work flows that all engineers utilize. Everybody is using the same way to submit new software, to update existing software to a new version, to submit bug fixes and features. Everyone is using the same means to branch, study, change, and improve the software.

OBS simply helps Free Software engineers to harness the power of the open source development model. And this also helps the users because they get a tightly integrated software solution.

There is more...

This is how Free Software distributors utilizes the Open Build Service for the benefit of the users: Build binary packages for a wide range of distributions and architectures from source code in an automatic, consistent, reproducible and collaborative way.

But there is more: OBS itself is free software. You can run, copy, distribute, study, change and improve it. The source code and the developers are on github. As Free Software, it is able to keep up with the ever-evolving ecosystem, which constantly produces new distributions, new package formats, new architectures, software, standards and tools. At the current time, OBS supports more than 20 different distributions, half a dozen architectures and three different package formats. It can cryptographically sign your software and products, different instances of OBS can connect to each other and OBS can be used in conjunction with source code revision systems like git/github in continuous integration work flows.

Do. Or do not. There is no try.

Some rights reserved by Dov Harrington

Enough talk, let's get practical! You have Free Software that you want to make available? Here is a nice video tutorial on how you can utilize the Open Build Service reference server which is freely available for developers to build packages for the most popular Linux distributions including openSUSE, Debian, Fedora, Ubuntu, Arch, Red Hat Enterprise Linux and SUSE Linux Enterprise.

That's how you can build and distribute binary packages for a wide range of distributions and architectures with the Open Build Service. Enjoy!

Want to learn the more about software packaging? The Linux distributions have great introductions to the RPM, DEB and PKGBUILD formats that you should review. The tutorial github repository also contains build instructions for all kinds of different distributions.

Feedback?

Any criticism, remarks or praise about this post? Get in touch, I'm looking forward to your input!