111 lines
4.3 KiB
Markdown
111 lines
4.3 KiB
Markdown
---
|
|
title: Backup archives migration to Borg
|
|
date: 2019-11-21 18:30:00
|
|
---
|
|
|
|
Last weekend I found a number of encrypted hard-drives that were used to do periodic
|
|
backups from 2006 to 2014. At the time, the backups were made using rsync
|
|
with hard-links to save only one occurrence of the file if it was not changed
|
|
since the last backup.
|
|
|
|
I wanted to check that everything was still there and upgrade this to a
|
|
Borg repository so that I can profit from compression and deduplication
|
|
to reduce these backup size further down and store them in a more secure way.
|
|
|
|
## Check the backups
|
|
The backups were made using hard-links with one backup corresponding to
|
|
one folder as follow :
|
|
|
|
```
|
|
$ ls backups/feronia/
|
|
back-2014-06-19T19:05:10/ back-2014-10-10T07:30:00/
|
|
back-2014-12-24T14:34:44/ current@
|
|
```
|
|
|
|
To check that the backups were still readable, I listed the content of
|
|
the different folders and checked that some known configuration files were
|
|
present and matched what was expected. This worked until I processed some
|
|
backups done before I was using awesomewm, when I changed a lot of config
|
|
files to match my usage instead of using the default ones.
|
|
|
|
All in all, the backups were still good and readable, I could use these
|
|
as a basis for the transition to a more robust and space-efficient system.
|
|
|
|
I saw a number of freezes during the check I interpreted as signs of old
|
|
age for the spinning rust.
|
|
|
|
## Initialize the Borg backup
|
|
The first step is to initialize the borg repository. We will put it on
|
|
one of the known good backup drive that still has some room. To estimate
|
|
the space needed for the backups, I took the size of the most recent backup
|
|
and multiplied by two, as I know that I did not delete a lot of files and
|
|
that the deduplication will reduce the size of the old backups that contained
|
|
a lot of checked-out subversion repositories.
|
|
|
|
So, with a destination for my borg repository, I created a folder on the
|
|
disk and gave my user read-write rights on this folder.
|
|
|
|
```
|
|
$ sudo mkdir backups/borg-feronia
|
|
$ sudo chown 1000:1000 backups/borg-feronia -R
|
|
```
|
|
|
|
Then, the creation of the repository with borg :
|
|
|
|
```
|
|
$ borg init --encryption=repokey backups/borg-feronia
|
|
Enter new passphrase:
|
|
Enter same passphrase again:
|
|
Do you want your passphrase to be displayed for verification? [yN]: n
|
|
[...]
|
|
```
|
|
|
|
I decided to use the `repokey` encryption mode. This mode stores the key
|
|
in the repository, allowing me to only remember the passphrase and not having
|
|
to worry about backuping the key file.
|
|
|
|
## Transfer the existing backups to Borg
|
|
The borg repository has been initialized, we can now start migrating the
|
|
backups from the hard-linked folders into borg.
|
|
|
|
As borg does not care about hard-links, we can simply loop over the different
|
|
folders and create a new archive from it. It will take some time because
|
|
in each directory it will loop over the whole content, hash it, check whether
|
|
it changed, deduplicate it, compress it and then write it. Each backup
|
|
of approximately 70 GiB took one hour to migrate on my computer. It seems
|
|
that the process is limited by the single-thread performance of your CPU.
|
|
|
|
```
|
|
$ export BORG_PASSPHRASE=asdf
|
|
$ for i in back*; do \
|
|
archivename=$(echo $i | cut -c 6-15); \
|
|
pushd $i; \
|
|
borg create --stats --progress ~/backups/borg-feronia::$archivename .; \
|
|
popd; \
|
|
done;
|
|
```
|
|
|
|
The env variable will allow us to walk away at this stage and let the computer
|
|
do its magic for some hours.
|
|
|
|
## Check the migrated backups
|
|
Once the backups have been migrated, we need to check that everything is
|
|
in order before doing anything else.
|
|
|
|
I did the same as before, using this time `borg list` and `borg extract`
|
|
to check whether the files are present and their content is correct.
|
|
|
|
|
|
## Archive these backups
|
|
Once the migrated backups have been tested, we can shred the old hard drives
|
|
that were showing signs of old age.
|
|
|
|
Since storage is so cheap nowadays, I will also transfer an archive of
|
|
the Borg backup folder to an online storage service as to be able
|
|
to retrieve it in case the local storage supports are destroyed or otherwise
|
|
unreadable in the future.
|
|
|
|
I choose to simply create a tar archive of the Borg folder and upload it
|
|
to AWS S3 since these backups will not be updated. Perhaps some day I will
|
|
add the more recent backups to this setup but for now they are a read-only
|
|
window into the laptop I had during my studies and during my first jobs.
|