Thursday, 10 December 2015

Purging old backup files in Linux

Here's a quick script for cleaning out old backup files. We organise our backups into directories by the amount of time you want to keep the backups for. So when writing your backup script, you do not need to think about purging old backup files, this is done automatically for you. This cleanup script goes through each of the backup directories and cleans them out when the timestamp on backup files exceed the given time period.

This has since been replaced by a proper backup system, but it still remains for devices that cannot have the backup client installed, for example a config dump from a network device where it runs a small proprietary o/s

We saved this script in /usr/local/bin/purge-drbackups.sh
Here's the script in full:


#!/bin/bash

# exit if a variable is not initialised
set -u

###########################
## variables for editing
BASE_BACKUP_DIR=/srv/backup
KEEPFOR_EVER=${BASE_BACKUP_DIR}/keepfor_ever
KEEPFOR_ONEMONTH=${BASE_BACKUP_DIR}/keepfor_onemonth
KEEPFOR_THREEMONTHS=${BASE_BACKUP_DIR}/keepfor_threemonths
KEEPFOR_TWOYEARS=${BASE_BACKUP_DIR}/keepfor_twoyears
###########################

#keepfor ever
printf "== Processing keepfor ever\n"
printf "No Action needed in directory ${KEEPFOR_EVER}\n"

#keepfor onemonth
printf "== Processing onemonth dir\n"
/usr/bin/find ${KEEPFOR_ONEMONTH} -type f -mtime +31 -exec rm -vf {} \;

#keepfor threemonths
printf "== Processing threemonts dir\n"
/usr/bin/find ${KEEPFOR_THREEMONTHS} -type f -mtime +92 -exec rm -vf {} \;

#keepfor twoyears
printf "== Processing twoyears dir\n"
/usr/bin/find ${KEEPFOR_TWOYEARS} -type f -mtime +730 -exec rm -vf {} \;

Some notes on the stript

  • set -u is included at the start of the script to make the script exit if it encounters an unset variable. I mention this because we are using the rm command and could potentially delete files starting at the root directory if an unset variable is used as part of the path passed to the rm command.
  • The second point i want to make is to think about how long you want to keep backups for, you will notice that we had one directory where we only keep files for a month. This is very short I would not recommend keeping critical files for only one month. Instead, keep them for as long as possible, typically measured in years, not months. Do some research on the subject, there's books and websites that cover this topic in detail.
  • The aim of this script was to stop the server hard disk from filling up, and to keep it like that automatically. Adjust it for your own needs as required.

I would recommend having this script run once daily with cron. On debian systems you create a script in /etc/cron.daily/

Here's the contents of ours, stored in /etc/cron.daily/daily-scripts

#!/bin/bash

/usr/local/bin/purge-drbackups.sh

One final note, don't forget to make both the script and the cron executable with the chmod +x command. Thats it!