Working with containers is a daily task for many users and developers. Container developers often need to (re)build container images frequently. If you develop containers, have you ever thought about reducing the image size? Smaller images have several benefits. They require less bandwidth to download and they save costs when run in cloud environments. Also, using smaller container images on Fedora CoreOS, IoT and Silverblue improves overall system performance because those operating systems rely heavily on container workflows. This article will provide a few tips for reducing the size of container images.
The tools
The host operating system in the following examples is Fedora Linux 33. The examples use Podman 3.1.0 and Buildah 1.2.0. Podman and Buildah are pre-installed in most Fedora Linux variants. If you don’t have Podman or Buildah installed, run the following command to install them.
$ sudo dnf install -y podman buildah
The task
Begin with a basic example. Build a web container meeting the following requirements.
- The container must be based on Fedora Linux
- Use the Apache httpd web server
- Include a custom website
- The container should be relatively small
The following steps will also work on more complex images.
The setup
First, create a project directory. This directory will include your website and container file.
$ mkdir smallerContainer $ cd smallerContainer $ mkdir files $ touch files/index.html
Make a simple landing page. For this demonstration, you may copy the below HTML into the index.html file.
<!doctype html>
<html lang="de">
<head>
<title>Container Page</title>
</head>
<body>
<header>
<h1>Container Page</h1>
</header>
<main>
<h2>Fedora</h2>
<ul>
<li><a href="https://getfedora.org">Fedora Project</a></li>
<li><a href="https://docs.fedoraproject.org/">Fedora Documentation</a></li>
<li><a href="https://fedoramagazine.org">Fedora Magazine</a></li>
<li><a href="https://communityblog.fedoraproject.org/">Fedora Community Blog</a></li>
</ul>
<h2>Podman</h2>
<ul>
<li><a href="https://podman.io">Podman</a></li>
<li><a href="https://docs.podman.io/">Podman Documentation</a></li>
<li><a href="https://github.com/containers/podman">Podman Code</a></li>
<li><a href="https://podman.io/blogs/">Podman Blog</a></li>
</ul>
<h2>Buildah</h2>
<ul>
<li><a href="https://buildah.io">Buildah</a></li>
<li><a href="https://github.com/containers/buildah">Buildah Code</a></li>
<li><a href="https://buildah.io/blogs/">Buildah Blog</a></li>
</ul>
<h2>Skopeo</h2>
<ul>
<li><a href="https://github.com/containers/skopeo">skopeo Code</a></li>
</ul>
<h2>CRI-O</h2>
<ul>
<li><a href="https://cri-o.io/">CRI-O</a></li>
<li><a href="https://github.com/cri-o/cri-o">CRI-O Code</a></li>
<li><a href="https://medium.com/cri-o">CRI-O Blog</a></li>
</ul>
</main>
</body>
</html>
Optionally, test the above index.html file in your browser.
$ firefox files/index.html
Finally, create a container file. The file can be named either Dockerfile or Containerfile.
$ touch Containerfile
You should now have a project directory with a file system layout similar to what is shown in the below diagram.
smallerContainer/ |- files/ | |- index.html | |- Containerfile
The build
Now make the image. Each of the below stages will add a layer of improvements to help reduce the size of the image. You will end up with a series of images, but only one Containerfile.
Stage 0: a baseline container image
Your new image will be very simple and it will only include the mandatory steps. Place the following text in Containerfile.
# Use Fedora 33 as base image FROM registry.fedoraproject.org/fedora:33 # Install httpd RUN dnf install -y httpd # Copy the website COPY files/* /var/www/html/ # Expose Port 80/tcp EXPOSE 80 # Start httpd CMD ["httpd", "-DFOREGROUND"]
In the above file there are some comments to indicate what is being done. More verbosely, the steps are:
- Create a build container with the base FROM registry.fedoraproject.org/fedora:33
- RUN the command: dnf install -y httpd
- COPY files relative to the Containerfile to the container
- Set EXPOSE 80 to indicate which port is auto-publishable
- Set a CMD to indicate what should be run if one creates a container from this image
Run the below command to create a new image from the project directory.
$ podman image build -f Containerfile -t localhost/web-base
Use the following command to examine your image’s attributes. Note in particular the size of your image (467 MB).
$ podman image ls REPOSITORY TAG IMAGE ID CREATED SIZE localhost/web-base latest ac8c5ed73bb5 5 minutes ago 467 MB registry.fedoraproject.org/fedora 33 9f2a56037643 3 months ago 182 MB
The example image shown above is currently occupying 467 MB of storage. The remaining stages should reduce the size of the image significantly. But first, verify that the image works as intended.
Enter the following command to start the container.
$ podman container run -d --name web-base -P localhost/web-base
Enter the following command to list your containers.
$ podman container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d24063487f9f localhost/web-base httpd -DFOREGROUN... 2 seconds ago Up 3 seconds ago 0.0.0.0:46191->80/tcp web-base
The container shown above is running and it is listening on port 46191. Going to localhost:46191 from a web browser running on the host operating system should render your web page.
$ firefox localhost:46191
Stage 1: clear caches and remove other leftovers from the container
The first step one should always perform to optimize the size of their container image is “clean up”. This will ensure that leftovers from installations and packaging are removed. What exactly this process entails will vary depending on your container. For the above example you can just edit Containerfile to include the following lines.
[...]
# Install httpd
RUN dnf install -y httpd && \
dnf clean all -y
[...]
Build the modified Containerfile to reduce the size of the image significantly (237 MB in this example).
$ podman image build -f Containerfile -t localhost/web-clean $ podman image ls REPOSITORY TAG IMAGE ID CREATED SIZE localhost/web-clean latest f0f62aece028 6 seconds ago 237 MB
Stage 2: remove documentation and unneeded package dependencies
Many packages will pull in recommendations, weak dependencies and documentation when they are installed. These are often not needed in a container and can be excluded. The dnf command has options to indicate that it should not include weak dependencies or documentation.
Edit Containerfile again and add the options to exclude documentation and weak dependencies on the dnf install line:
[...]
# Install httpd
RUN dnf install -y httpd --nodocs --setopt install_weak_deps=False && \
dnf clean all -y
[...]
Build Containerfile with the above modifications to achieve an even smaller image (231 MB).
$ podman image build -f Containerfile -t localhost/web-docs $ podman image ls REPOSITORY TAG IMAGE ID CREATED SIZE localhost/web-docs latest 8a76820cec2f 8 seconds ago 231 MB
Stage 3: use a smaller container base image
The prior stages, in combination, have reduced the size of the example image by half. But there is still one more thing that can be done to reduce the size of the image. The base image registry.fedoraproject.org/fedora:33 is meant for general purpose use. It provides a collection of packages that many people expect to be pre-installed in their Fedora Linux containers. The collection of packages provided in the general purpose Fedora Linux base image is often more extensive than needed, however. The Fedora Project also provides a fedora-minimal base image for those who wish to start with only the essential packages and then add only what they need to achieve a smaller total image size.
Use podman image search to search for the fedora-minimal image as shown below.
$ podman image search fedora-minimal INDEX NAME DESCRIPTION STARS OFFICIAL AUTOMATED fedoraproject.org registry.fedoraproject.org/fedora-minimal 0
The fedora-minimal base image excludes DNF in favor of the smaller microDNF which does not require Python. When registry.fedoraproject.org/fedora:33 is replaced with registry.fedoraproject.org/fedora-minimal:33, dnf needs to be replaced with microdnf.
# Use Fedora minimal 33 as base image
FROM registry.fedoraproject.org/fedora-minimal:33
# Install httpd
RUN microdnf install -y httpd --nodocs --setopt install_weak_deps=0 && \
microdnf clean all -y
[...]
Rebuild the image to see how much storage space has been recovered by using fedora-minimal (169 MB).
$ podman image build -f Containerfile -t localhost/web-docs $ podman image ls REPOSITORY TAG IMAGE ID CREATED SIZE localhost/web-minimal latest e1603bbb1097 7 minutes ago 169 MB
The initial image size was 467 MB. Combining the methods detailed in each of the above stages has resulted in a final image size of 169 MB. The final total image size is smaller than the original base image size of 182 MB!
Building containers from scratch
The previous section used a container file and Podman to build a new image. There is one last thing to demonstrate — building a container from scratch using Buildah. Podman uses the same libraries to build containers as Buildah. But Buildah is considered a pure build tool. Podman is designed to work as a replacement for Docker.
When building from scratch using Buildah, the container is empty — there is nothing in it. Everything needed must be installed or copied from outside the container. Fortunately, this is quite easy with Buildah. Below, a small Bash script is provided which will build the image from scratch. Instead of running the script, you can run each of the commands from the script individually in a terminal to better understand what is being done.
#!/usr/bin/env bash set -o errexit # Create a container CONTAINER=$(buildah from scratch) # Mount the container filesystem MOUNTPOINT=$(buildah mount $CONTAINER) # Install a basic filesystem and minimal set of packages, and httpd dnf install -y --installroot $MOUNTPOINT --releasever 33 glibc-minimal-langpack httpd --nodocs --setopt install_weak_deps=False dnf clean all -y --installroot $MOUNTPOINT --releasever 33 # Cleanup buildah unmount $CONTAINER # Copy the website buildah copy $CONTAINER 'files/*' '/var/www/html/' # Expose Port 80/tcp buildah config --port 80 $CONTAINER # Start httpd buildah config --cmd "httpd -DFOREGROUND" $CONTAINER # Save the container to an image buildah commit --squash $CONTAINER web-scratch
Alternatively, the image can be built by passing the above script to Buildah. Notice that root privileges are not required.
$ buildah unshare bash web-scratch.sh $ podman image ls REPOSITORY TAG IMAGE ID CREATED SIZE localhost/web-scratch latest acca45fc9118 9 seconds ago 155 MB
The final image is only 155 MB! Also, the attack surface has been reduced. Not even DNF (or microDNF) is installed in the final image.
Conclusion
Building smaller container images has many advantages. Reducing the needed bandwidth, the disk footprint and attack surface will lead to better images overall. It is easy to reduce the footprint with just a few small changes. Many of the changes can be done without altering the functionality of the resulting image.
It is also possible to build very small images from scratch which will only hold the needed binaries and configuration files.
Alexander
Thank you for the last part. “Building from scratch.”
I’m woundering about SystemD. It must be installed as well? but not started?
Daniel Schier
systemd is installed by dependencies, but is not needed in a container. In fact, it should be even avoided if necessary. Starting a service directly in foreground should be preferred.
Rob Verduijn
Removing the layers also reduces the overhead a bit.
podman build –layers=false
Daniel Schier
will only avoid caching layers during the build. If you want to squash everything in one layer (which can be useful depending on your usecase), you can use
or
.
Rene Reichenbach
And now try with a go or rust app doing the same 😉
Phozzy
– will work well in the last scenario.
Daniel Schier
The “from scratch” example will most likely work similar, but you can also have a look at multi-stage builds. 🙂
rnx
useful article. thanks.
perhaps dnf should get a new subcommand specifically for installing with recommended defaults for container images and cache cleaning … would make things more cleaner and more readable.
dnf containerinstall xy or something
Mark
I have had issues in the past with microdnf when a fedora-minimal image must have been built prior to a package update being removed from the main repository and it found a package to downgrade, microdnf will fail instead of downgrading. Infrequent but if you care about attack surfaces you should always ‘microdnf -y update’ when building a new container so it can be an issue. So some of my docker files use microdnf to install dnf and then remove that when done.
I use docker and there is a utility ‘docker-squash’ (obtained simply with ‘pip3 install docker-squash’) which has made me a bit lazy; in that you can use clarity in Dockerfile usage by not bothering to use && to stack multiple commands to minimise layers but instead happily generate as many layers as you want and even have huge ones at the start. And docker-squash will happily compress 20-30 layers down to 2 for you which shrinks the container image size a lot. I also use that on images pulled from dockerhub where some application containers can be >1Gb in size when pulled from dockerhub and many of those can be shrunk as well. docker-squash does need docker though and won’t work with podman last time I checked.
Having said that I was interested in the comment from Rob Verduijn about the ‘podman build –layers=false’ option I will have to look at to see what it does, as it is the layers that while useful for managing changes/versions take up needless space in a container image. But it’s use would be in containers personally built at build time rather than run against containers already built externally.
I also appreciate the ‘building from scratch’ section, very easy to follow with the example you used.
On layers, it should probably also be mentioned it is best to keep large layers toward the end of a Dockerfile or config file; don’t layer1 copy in a 100Mb datafile then layer2 install a package then layer3 add a user or all three layers would have that datafile. Add the user then the package then the 100Mb database and you have saved 200Mb in your final container image; order is important. Or you could docker-squash and as I am also going to do investigate the ‘podman build –layers=false’ option Rob mentioned.
Daniel Schier
For squashing layers you can use the option
or
, which is available by default for podman build and buildah.
Diogo Lopes
Very Good
langdon
Minor complaint. I think you have a “typo” in the final script. You say “nginx” in the comments when I think you meant “httpd” (or apache). I was confused for a minute so others might be as well. Otherwise, nice article.
Gregory Bartholomew
Thanks langdon. I have made the correction.
laolux
Thanks for the article. I really like the from scratch method using buildah. Now I would like to improve on that by reusing the system’s dnf metadata cache. Any idea of how to accomplish that? Otherwise any user on my system would have to download all the metadata just for himself.
Another point: When using the btrfs backend for podman, then regular users need to execute the script with
. Then it works fine though.
Mohamed Amin
Thanks for the article. I would think of using buildah for building containers.
I pulled apache default and alpine image from docker hub and I found this:
docker.io/library/httpd alpine a4d0bee07118 8 days ago 57 MB
docker.io/library/httpd latest 0b932df43057 13 days ago 142 MB
Why the alpine image here is so small?
Daniel Schier
Hi, this is an easy/not so easy to answer question.
Alpine is very focused on minimalism. Therefore, it replaced and stripped important packages, which are common in other Linux systems. For example, you will not have gcc/glibc, but musl libc/gcc. But this also means, that you have to build software differently and may not be able to use certain software at all.
xhtml
useful article. thanks. 😀
Podman tutorial – How to install XAMPP server in Podman container?
Daniel Schier
Since XAMPP was the idea to get LAMP on Windows without knowing how to install Apache, MySQL, PHP, etc. I would suggest using each of these tools in a container is the proper way. At least I am considering XAMPP “a bunch of tools for mere testing only”. Using httpd or MariaDB in a container is a piece of cake with containers 🙂
Shaun McFee
Could multi-stage builds be used within the Containerfile itself to achieve what the bash script of Buildah commands is doing in order to keep the Containerfile as the single source of truth for how the container was built? Is there an advantage to the bash script with Buildah approach versus multi-stage builds?
Andrea R
I was going to ask this exactly, it’s what I do with docker and it’s clean and easy enough.
Daniel Schier
Multi-Stage builds can be done, too. Sometimes these are very useful. If you want to use the “from scratch” method in addition to multi-stage, you can also stick to “from scratch”.
Nevertheless, this article explains the capabilities of buildah with regards to both: https://www.redhat.com/en/blog/unleash-powerful-linux-container-building-capabilities-buildah
SergMx
I’m sorry.
Doesn’t Fedora have a similar project: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image
?
Scott McCarty
You can use UBI on Fedora. UBI is supposed to sort of be universal, can be used on Fedora or even other Linux distros.
Also, UBI micro will come out with RHEL 8.4 and that is even smaller than the minimal image with microdnf
Daniel Schier
Last time I used UBI on Fedora/CentOS (a year ago maybe), it requested a subscription. Is this still the case?
Lee Whitty
Great article! I just went through this exercise using Fedora 34 images. They must have reduced the minimal image size because the podman build using the minimal image was smaller (154 MB) than the buildah build (155 MB).
Lee Whitty
Correction: Both the minimal image size build and buildah builds were smaller under Fedora 34, with the Buildah build being the smallest. I like the trend of the container image sizes shrinking!