Vorta’s “No Matter What, Keep All…” Setting

Since switching from Pop!_OS (Gnome) to Kubuntu (KDE) for work, I have also changed my backup GUI.  There is no question that Vorta is more powerful than Pika Backup, but the price of that is the loss of simplicity.

One place I got confused was in the backup-retention rules, referred to as “pruning” by the GUI.  I have learned: when Vorta offers to “keep all backups made within…”, that is internally a separate rule with high priority. Therefore, when I set up my hourly backups to keep “one week” of hourly, two weeks of daily, and so on, but “keep everything from the last six weeks,” I ended up with seven weeks of hourly backups, followed by the two weeks of daily.

I noticed the problem when my laptop fans spun up for a while, which turned out to be Vorta verifying 300+ archives.  The work laptop is on only for work, producing backups for 8–9 hours per weekday, for around 250 extra archives over those first six weeks.

Unrelated, but one nice thing about Vorta is that, like Pika, it is a front-end to Borg.  I gave it the same repository on disk.  Now I have continuous backup history across the two GUIs, and emergency CLI access if necessary.

Revisiting Backups

Since comparing DejaDup and Pika Backup for work, I’ve also used KDE backup/kup at home, and gotten a little more experience with both systems.  How are things going?

Pika Backup

Pika has a firm internal idea of the schedule.  At the end of a vacation, where the normal weekly backup was skipped, I manually asked it to run the backup “6 days late / 1 day early”, hoping I could skip it the next day.  No such luck: it promptly asked for the drive the next morning.  No big deal; I just didn’t know the details of the system.

One small file recovery was quick and effective through the GUI.  It looked like a special folder in the normal file explorer, which allowed for drag-and-drop copying from the backup to the desired location in another file window.  As usual when coding, I had done something, changed my mind, deleted it, and then changed my mind back later.  And probably changed a couple more times.

KDE Backup

KDE Backup had been set up as a redundant system behind my handcrafted (smaller, faster, non-versioned) script, the latter of which was—er, intended—to store only my critical data.

A few months after a trial-by-fire backup test, I discovered that ~/.gnupg was not in my manual backup.  This could have been a critical fault… but KDE Backup had the data.  Clicking the “Restore” button opened a “File Digger” window, and from there it was just like using Pika Backup.

I got my key back!

I added the missing ~/.gnupg directory to manual backup, then reconfigured KDE Backup to back up into the existing repository.  That went smoothly, too.  It noticed that the directory I gave it already had data, so it verified the integrity, and then backed up into it.

Format-wise, KDE uses kup to do the backup, which is a front-end to bup, storing the data as a bare git repository.  I didn’t need a password to get data out of it, which makes sense, because I didn’t need one to set up the backup.  That’s also great news for actually recovering my data, because I have no idea what I would have chosen for a password when I started KDE Backup in the first place.

Conclusion

Both systems are working great, i.e., better than my manual one.

DejaDup and PikaBackup, early impressions (Update 2)

I tried a couple of backup programs:

• DejaDup (the one I had heard of), a front-end for Duplicity
• PikaBackup, a front-end for BorgBackup

I installed both of them as Flatpaks, although deja-dup also has a version in the Pop!_OS 22.04 repository.  I have been using DejaDup for four months, and PikaBackup for one month.  This has been long enough for DejaDup to make a second full backup, but not so long for Pika to do anything special.

Speed:

For a weekly incremental backup of my data set…

• DejaDup: about 5 minutes, lots of fan speed changes
• PikaBackup: about 1 minute, fans up the whole time

Part of Pika’s speed is probably the better exclusion rules; I can use patterns of **/node_modules and **/vendor, to exclude those folders, wherever they are in the tree.  With DejaDup, I would apparently have to add each one individually, and I did not want to bother, nor keep the list up-to-date over time.

Part of DejaDup’s slowness might be that it executes thousands of gpg calls as it works.  Watching with top, DejaDup is frequently running, and sometimes there’s a gpg process running with it.  Often, DejaDup is credited with much less than 100% of a single CPU core.

Features:

PikaBackup offers multiple backup configurations.  I keep my main backup as a weekly backup, on an external drive that’s only plugged in for the occasion. I was able to configure an additional hourly backup of my most-often-changed files in Pika.  (This goes into ~/.borg-fast, which I excluded from the weekly backups.) The hourly backups, covering about 2 GB of files, aren’t noticeable at all when using the system.

Noted under “speed,” PikaBackup offers better control of exclusions.  It tracks how long operations took, so I know that it has been exactly 53–57 seconds to make the incremental weekly backups.

On the other hand, Pika appears to always save the backup password.  DejaDup gives the user the option of whether it should be remembered.

There is a DejaDup plugin for Caja (the MATE file manager) in the OS repo, which may be interesting to MATE users.

Space Usage:

PikaBackup did the weekly backup on 2023-04-24 in 46 seconds; it reports a total backup size of 28 GB and 982 MB (0.959 GB = 3.4%) written out.

With scheduled backups, Pika offers control of the number of copies kept.  One can choose from a couple of presets, or provide custom settings.  Of note, these are count-based rather than time-based; if a laptop is only running for 8-9 hours a day, then 24 hourly backups will be able to provide up to 3 days back in time.

For unscheduled backups, it’s not clear that Pika offers any ‘cleanup’ options, because the cleanup is tied to the schedule in the UI.

I do not remember being given many options to control space usage in DejaDup.

Disaster Simulation:

To ensure backups were really encrypted, I rebooted into the OS Recovery environment and tried to access them.  Both programs’ CLI tools (duplicity and borgbackup) from the OS repository were able to verify the data sets. I don’t know what the stability guarantees are, but it’s nice that this worked in practice.

• duplicity verified the DejaDup backup in about 9m40s
• borgbackup verified the PikaBackup backup in 3m23s

This isn’t a benchmark at all; after a while, I got bored of duplicity being credited with 30% of 1 core CPU usage, and started the borgbackup task in parallel.

Both programs required the password to unlock the backup, because my login keychain isn’t available in this environment.

Curiously, borgbackup changed the permissions on a couple of files on the backup during the verification: the config and index files became owned by root.  This made it impossible to access the backups as my normal user, including to take a new one.  I needed to return to my admin user and set the ownership back to my limited account.  The error message made it clear an unexpected exception occurred, but wasn’t very useful beyond that.

Major limitations of this post:

My data set is a few GB, consisting mainly of git repos and related office documents.  The performance of other data sets is likely to vary.

I started running Pika about the same time that DejaDup wanted to make a second backup, so the full-backup date and number of incremental snapshots since should be fairly close to each other.  I expect this to make the verification times comparable.

I haven’t actually done any restores yet.

Final words:

Pika has become my primary backup method.  Together, its speed and its support for multiple configurations made hourly backups reasonable, without compromising the offline weekly backup.

Update History:

This post was updated on 2023-03-31, to add information about multiple backups to “Features,” and about BorgBackup’s file permission change during the verification test.  Links were added to the list above, and a new “Final Words” concluding section was written.

It was updated again on 2023-04-26, to add the “Space Usage” section, and to reduce “I will probably…” statements to reflect the final decisions made.