How to backup your git repositories with gickup

Git is by far the most used version control system out there. Originally created by Linus Torvalds, it is free and open source software, released under the GPLv2 license. Many online platforms such as Github or Gitlab allow developers to easily store and track changes in their code in public or private repositories using git as a backend.

In this article we talk about gickup, a nice tool written in Go which lets us create backups of our git repositories and mirror them locally, or from one platform to another.

In this tutorial you will learn:

How to install gickup
How to backup your repositories locally
How to mirror Github repositories to Gitlab

How to backup your git repositories with gickup

Software Requirements and Linux Command Line Conventions
Category	Requirements, Conventions or Software Version Used
System	Distribution agnostic
Software	gickup
Other	Root privileges may be needed to run the Docker container
Conventions	# – requires given linux-commands to be executed with root privileges either directly as a root user or by use of `sudo` command $ – requires given linux-commands to be executed as a regular non-privileged user

Installation

Gickup is open source software: its code is hosted itself on Github. At the moment of writing, the tool is not available in any Linux distribution repository yet, therefore, if we don’t want to compile it, there are basically two ways we can use it:

by downloading the pre-compiled binary from Github
by using the dedicated Docker Image

Let’s explore these options.

Using the pre-compiled binary

At the moment of writing 0.10.17 is the latest available version of gickup; we can download the pre-compiled binary from the releases page of the project on Github. Pre-compiled binaries exist for all the major operating systems and for the most common architectures. We want to download the Linux version, of course, together with the *checksum.txt file, in order to verify the integrity of the data. To run gickup on my machine, I need to download the “amd64” version:

$ curl --location --remote-name-all https://github.com/cooperspencer/gickup/releases/download/v0.10.17/{gickup_0.10.17_linux_amd64.tar.gz,gickup_0.10.17_checksums.txt}

In order to verify the tarball checksum, assuming the two files to be in the same directory (as it would result from running the above command), we can run:

$ sha256sum -c gickup_0.10.17_checksums.txt

If the checksum of the tarball corresponds to the one written in the file, we should see a message similar to the one below:

gickup_0.10.17_linux_amd64.tar.gz: OK

To extract the tarball we run:

$ tar -xvzf gickup_0.10.17_linux_amd64.tar.gz

In order to use the application we just need to launch the extracted binary file just as any other executable, passing the path of the configuration file (more on this in a moment) as first argument:

$ ./gickup /path/to/conf.yml

Using the dedicated Docker image

If we don’t want to use the pre-compiled binary, as an alternative, we can download the docker-compose.yml file, and run the tool in a container, by using docker or podman. We can download the file straight from the command line:

$ curl --remote-name https://raw.githubusercontent.com/cooperspencer/gickup/main/docker-compose.yml

The next step consists into placing the configuration file, which in this case must be named conf.yml, in the same directory of the file we just downloaded. In order to launch the backup, we just need to run:

$ sudo docker-compose up

Or, if using podman instead of docker:

$ podman-compose up

If you are using docker and your user is in the “docker” group, you can omit using sudo; podman, instead, can run without root privileges.

Those are the instruction we can use to run gickup. We can specify what the tool should do, and how it should do it, via the configuration file we mentioned above. Let’s see an example of its usage.

The gickup configuration file

Gickup makes use of a YAML configuration file in which we define the sources and destinations of our backups, plus other useful actions. Let’s some examples. For the sake of this tutorial I will download all my repositories from Github to my local machine:

source:
  github:
    - token: <github-token>
      username: egdoc

destination:
  local:
    - path: ~/Downloads/gitbackup
      structured: true
      zip: true
      keep: 5
      bare: true

In the example above I specified I want to backup my Github repositories using the github key in the source dictionary. Since I want to backup both public and private repos, I had to provide a token as argument of the token key. Generating a Github token is pretty easy: it’s enough to navigate to “Settings -> Developer settings -> Personal access tokens” in your Github profile, select either a fine-grained or a personal token, and grant it the minimum required set of privileges. In this case it should be enough to just check the “repo” box, as in shown the picture below:

Creating a personal token with "repo" privileges on Github — Creating a personal token with “repo” privileges on Github

Once the token is generated, keep note of it, since you will not be able to visualize it in the future.

The next key I used in the configuration file is username, to specify the user for the clone process. With the destination dictionary, instead, I specified the backups destination. In this case, since I wanted the repositories to be cloned locally, I used the path key to specify a local path, and the structured key to make gickup clone the repositories in a directory tree which reflects the source website and username. In this case the repositories hosted on Github will be cloned in the ~/Downloads/gitbackup/github.com/egdoc directory.

The zip key is used to specify we want each cloned repository to be zipped; keep, instead, is used to specify how many “zipped” backups which should be kept. Finally, by setting the value of the bare key to true, I specified I want the repositories to be cloned as “bare”.

Mirror repositories from one platform to another

In the previous example I cloned all my repositories hosted on Github on my local machine. By using gickup, however, we can also clone repositories directly from one platform to another. Suppose I want to mirror all the repositories to Gitlab. Here is what I could write:

source:
  github:
    - token: <github-token>
      username: egdoc

destination:
  gitlab:
    - token: <gitlab-token>
      username: egdoc

In all the examples you can see I specified only the platform name (e.g. “github” or “gitlab”) and not the complete platform URL. This is possible because gickup has built-in support for those platforms. Platforms supported as source are:

Github
Gitea
Gogs
Gitlab
Bitbucket
Onedev
Sourcehut

Additionally, the “Any” key can be used to specify information about platforms not included in the list above.

Platforms supported as destination, instead are:

Gitea
Gogs
Gitlab
Local

Conclusions

In this tutorial we talked about gickup, a nice open source tool written in Go we can use to mirror our git repositories locally, or from one web platform to another. We provided two little examples of how to achieve both results; to know more about the project, you can take a look at the official documentation.