I recently found myself needing a machine to compile binaries on for a CentOS server. I first considered actually spinning up a CentOS system on a VPS; however, that seemed a little overboard just for compiling, I then realized that this would be the perfect use for a container. I could have an identical system to the one where the binaries will be deployed on, and at little cost since it can simply be blown away when I’m done. In order to set up my compile machine I used LXC.
LXC, or “Linux Containers”, are a set of tools for creating full-featured containers. Compared to other tools such as systemd-nspawn, LXC is much more complex, and it has been used to build projects such as Docker. Docker has since moved away from LXC, however LXC is still one of the huge players in the Linux container game. The Linux container project also brings LXD, a daemon that can be used to manage containers. LXD makes a larger use of system images, as opposed to templates, in order to allow quick deployment of containers. Together these projects allow easy deployment and management of containers, as well as as advanced features and customizability.
In the following post I will go through the process of setting up several LXC containers, using both the traditional template-based method, as well as experiment with LXD to download image-based containers.
On Arch Linux, which is what I’m using, the packages lxc and arch-install-scripts are required.
In order to see if the currently running kernel is properly configured to use lxc, run lxc-checkconfig, it should detect if anything is missing from the current configuration.
One of the features that it is normal to see missing is user namespaces. In order to use this feature on Arch Linux, a custom compiled kernel is necessary. The default kernel has user namespaces turned off due to security concerns.
The traditional way of setting up an LXC container is using templates. Templates are prebuilt
shell scripts that will build a system image.
View the available templates in /usr/share/lxc/templates:
To get help on container creation use lxc-create --help. lxc-create is used to create a container from the templates.
There are many options:
Settings can be specified in the lxc config. I’m using ZFS as a backing store, btrfs and lvm are also supported as well as “none” which is the default.
I added my backend, lxc path, and zfs root.
I also set some container defaults, i’m using a systemd-network bridge setup the same way I detailed in a previous post on systemd-nspawn:
I created a new ZFS dataset for my lxc containers
I created a centos container using zfs as a backing store, it requires the AUR package yum:
A new data set has been created with the name of the container.
The config file made with the container can be found at /var/lib/lxc/centos-builder/config, it can be edited if the networking needs changing.
Start the Container
Start the container, and login to the console. The password will be in the container directory /var/lib/lxc/centos-builder/tmp_root_pass.
Now attach to the console with lxc-console you should find yourself at the login of a brand-new machine.
Networking can now be setup in the container using a bridge, check the interface name and set up the network as if on any other computer.
Check the interface:
Here the interface shows up as as eth0@if5. Use the prefix eth0 as the interface.
I setup a static IP using systemd-networkd.
If a static IP is being used, the file /usr/lib/systemd/network/80-container-host0.network that would have setup DHCP needs to be masked.
Start and enable systemd-networkd.
Create the configuration file.
Restart the service and you should have an IP.
Alternatively to using templates, images that have been pre-built can also be downloaded using the lxc-download script.
Running lxc-download runs you through an interactive prompt that will let you download an image from the listed options.
You can also specify the release and architecture beforehand.
[root]# lxc-create -t download --help
LXC container image downloader
[ -h | --help ]: Print this help message and exit.
[ -l | --list ]: List all available images and exit.
[ -d | --dist <distribution> ]: The name of the distribution
[ -r | --release <release> ]: Release name/version
[ -a | --arch <architecture> ]: Architecture of the container
[ --variant <variant> ]: Variant of the image (default: "default")
[ --server <server> ]: Image server (default: "images.linuxcontainers.org")
[ --keyid <keyid> ]: GPG keyid (default: 0x...)
[ --keyserver <keyserver> ]: GPG keyserver to use
[ --no-validate ]: Disable GPG validation (not recommended)
[ --flush-cache ]: Flush the local copy (if present)
[ --force-cache ]: Force the use of the local copy even if expired
LXC internal arguments (do not pass manually!):
[ --name <name> ]: The container name
[ --path <path> ]: The path to the container
[ --rootfs <rootfs> ]: The path to the container's rootfs
[ --mapped-uid <map> ]: A uid map (user namespaces)
[ --mapped-gid <map> ]: A gid map (user namespaces)
Let’s set up Ubuntu 17.04 container non-interactively. When specifying LXC arguments, -- must be passed before the template specific arguments.
Since this container doesn’t come with a root password, we need to attach directly to the container. There we can specify the root password and from then on connect normally using lxc-console.
LXD builds on the idea of using image-based containers and functions similar to the way lxc-download works. In order to use it on Arch, install the lxd (AUR) package and start the service lxd.
lxd requires the ability to run unprivileged containers, and since Arch does not have this enabled by default, you can either build your own kernel, or run containers with the security.privileged=true command. I take the latter approach in the following examples.
To configure lxd run lxd init, an interactive prompt should follow walking you through the configuration.
Available networks can be viewed with the network command, listing them I see the bridge I made previously.
The default configuration can be edited with lxc profile edit default.
I added my network bridge.
Using LXD you can talk to remote image servers and download containers.
To list the available images run lxc image list images. A long list of available images should appear in the following form.
To create a container from one of these images, specify the distro release and architecture in the form distro/release/arch.
To view the current containers run lxc list
Individual commands can be run inside the containers by using the lxc exec command. This can be used to get a shell by running bash, or an alternate shell, as well as used to run one off commands.
This is just a small look into what LXC can do, and if you’re interested in experimenting more with LXC and LXD I would recommend reading the great series of posts by Stéphane Graber, the LXC and LXD project leader.