2Shirt/duplicity-backup.sh
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

v0.7 : external config file now mandatory + fix issue #1

Browse source
This commit is contained in:
Zertrin 2012-07-08 20:46:41 +02:00
parent 9489adc9d4
commit fe55d48cbb
4 changed files with 198 additions and 151 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
CHANGELOG
View file

@ -1,3 +1,8 @@
0.7 (8 July 2012)
=================
* Fix Issue #1 "Restore error if no target name"
* Rewrite to force utilisation of an external config file to facilitate updates of the script in itself
0.6.1 (5 May 2012) 0.6.1 (5 May 2012)
================= =================
Inclusion of two patches written by [shamer] + modified config file management. Inclusion of two patches written by [shamer] + modified config file management.

46
README
View file

@ -1,11 +1,11 @@
ABOUT THIS SCRIPT ABOUT THIS SCRIPT
================= =================
This bash script was designed to automate and simplify the remote backup This bash script was designed to automate and simplify the remote backup process
process of duplicity on Amazon S3 primarily. Other backup destinations are of duplicity on Amazon S3 primarily. Other backup destinations are possible.
possible. After your script is configured, you can easily backup, restore, After your script is configured, you can easily backup, restore, verify and
verify and clean (either via cron or manually) your data without having to clean (either via cron or manually) your data without having to remember lots of
remember lots of different command options and passphrases. different command options and passphrases.
Most importantly, you can easily backup the script and your gpg key in a Most importantly, you can easily backup the script and your gpg key in a
convenient passphrase-encrypted file. This comes in in handy if/when your convenient passphrase-encrypted file. This comes in in handy if/when your
@ -15,8 +15,8 @@ Optionally, you can set up an email address where the log file will be sent,
which is useful when the script is used via cron. which is useful when the script is used via cron.
This version is a rewriting of the code originally written by Damon Timm, This version is a rewriting of the code originally written by Damon Timm,
including many patches that have been brought to the original scripts by including many patches that have been brought to the original scripts by various
various forks on Github. forks on Github.
Latest version of the code available at: Latest version of the code available at:
http://github.com/zertrin/duplicity-backup http://github.com/zertrin/duplicity-backup
@ -24,7 +24,7 @@ http://github.com/zertrin/duplicity-backup
Merge requests are welcome :) Merge requests are welcome :)
More information about this script available at: More information about this script available at:
http://zertrin.org/duplicity-backup.html http://zertrin.org/projects/duplicity-backup/
The original version of the code is availabe at: The original version of the code is availabe at:
http://github.com/theterran/dt-s3-backup http://github.com/theterran/dt-s3-backup
@ -32,10 +32,11 @@ http://github.com/theterran/dt-s3-backup
BEFORE YOU START BEFORE YOU START
================ ================
This script requires user configuration. Instructions are in This script requires user configuration. Instructions are in the config file
the file itself and should be self-explanatory. Be sure to replace all the itself and should be self-explanatory. Please DO NOT edit the example config
*foobar* values with your real ones. Almost every value needs to be file, but make a copy of it and edit this one. Be sure to replace all the
configured in someway. *foobar* values with your real ones. Almost every value needs to be configured
in someway.
You can use multiple copies of the script with different settings for different You can use multiple copies of the script with different settings for different
backup scenarios. It is designed to run as a cron job and will log information backup scenarios. It is designed to run as a cron job and will log information
@ -54,11 +55,11 @@ REQUIREMENTS
CONFIGURATION CONFIGURATION
============= =============
The configuration takes place directly in the script and is documented there. The configuration takes place in a separate config file and is documented there.
You can optionnaly specifiy a custom config file in the CONFIG parameter at the The script looks for its configuration by reading the config file specified in
very beginning. Any parameter specified in this custom config file will the CONFIG parameter at the beggining of the script. Be sure to edit it to match
override thoses specified in the script. the actual location of your config file.
COMMON USAGE EXAMPLES COMMON USAGE EXAMPLES
===================== =====================
@ -103,15 +104,14 @@ TROUBLESHOOTING
This script attempts to simplify the task of running a duplicity command; if you This script attempts to simplify the task of running a duplicity command; if you
are having any problems with the script the first step is to determine if the are having any problems with the script the first step is to determine if the
script is generating an incorrect command or if duplicity itself is script is generating an incorrect command or if duplicity itself is causing your
causing your error. error.
To see exactly what is happening when you run duplicity-backup, head to the bottom To see exactly what is happening when you run duplicity-backup, head to the bottom
of the user configuration portion of the script and uncomment the `ECHO=$(which of the configuration file and uncomment the `ECHO=$(which echo)` variable. This
echo)` variable. This will stop the script from running and will, instead, will stop the script from running and will, instead, output the generated
output the generated command into your log file. You can then check to see if command into your log file. You can then check to see if what is being generated
what is being generated is causing an error or if it is duplicity causing you is causing an error or if it is duplicity causing you woe.
woe.
NEXT VERSION WISH LIST NEXT VERSION WISH LIST
====================== ======================

158
duplicity-backup.conf.example Normal file
View file

@ -0,0 +1,158 @@
#!/bin/bash
#
# Copyright (c) 2008-2010 Damon Timm.
# Copyright (c) 2010 Mario Santagiuliana.
# Copyright (c) 2012 Marc Gallet.
#
# This program is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free Software
# Foundation, either version 3 of the License, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
# details.
#
# You should have received a copy of the GNU General Public License along with
# this program. If not, see <http://www.gnu.org/licenses/>.
#
# MORE ABOUT THIS SCRIPT AVAILABLE IN THE README AND AT:
#
# http://zertrin.org/projects/duplicity-backup/ (for this version)
# http://damontimm.com/code/dt-s3-backup (for the original programi by Damon Timm)
#
# Latest code available at:
# http://github.com/zertrin/duplicity-backup
#
# ---------------------------------------------------------------------------- #
# #############################################
# # DUPLICITY-BACKUP CONFIG FILE #
# #############################################
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# ! DO NOT edit duplicity-backup.conf.example !
# ! please copy it to anywhere you want !
# ! (typically duplicity-backup.conf) !
# ! and edit this file instead !
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# AMAZON S3 INFORMATION
# Comment out this lines if you're not using Amazon S3
AWS_ACCESS_KEY_ID="foobar_aws_key_id"
AWS_SECRET_ACCESS_KEY="foobar_aws_access_key"
# ENCRYPTION INFORMATION
# Do you want your backup to be encrypted? yes/no
# If yes, please make sure you specify either PASSPHRASE or GPG_KEY
ENCRYPTION='yes'
# If you are NOT running this from a cron, comment this line out
# and duplicity should prompt you for your password.
# Comment out if you aren't using encryption
PASSPHRASE="foobar_gpg_passphrase"
# Specify which GPG key you would like to use (even if you have only one).
# Comment out if you're using only PASSPHRASE or not using encryption
GPG_KEY="foobar_gpg_key"
# BACKUP SOURCE INFORMATION
# The ROOT of your backup (where you want the backup to start);
# This can be / or somwhere else -- I use /home/ because all the
# directories start with /home/ that I want to backup.
ROOT="/home"
# BACKUP DESTINATION INFORMATION
# In my case, I use Amazon S3 use this - so I made up a unique
# bucket name (you don't have to have one created, it will do it
# for you). If you don't want to use Amazon S3, you can backup
# to a file or any of duplicity's supported outputs.
#
# NOTE: You do need to keep the "s3+http://<your location>/" format
# even though duplicity supports "s3://<your location>/".
DEST="s3+http://backup-bucket/backup-folder/"
# Other possible locations
#DEST="ftp://user[:password]@other.host[:port]/some_dir"
#DEST="rsync://user@host.com[:port]//absolute_path"
#DEST="scp://user[:password]@other.host[:port]/[/]some_dir"
#DEST="ssh://user[:password]@other.host[:port]/[/]some_dir"
#DEST="file:///home/foobar_user_name/new-backup-test/"
# INCLUDE LIST OF DIRECTORIES
# Here is a list of directories to include; if you want to include
# everything that is in root, you could leave this list empty (I think).
#
# Here is an example with multiple locations:
#INCLIST=( "/home/*/Documents" \
# "/home/*/Projects" \
# "/home/*/logs" \
# "/home/www/mysql-backups" \
# )
#
# Simpler example with one location:
INCLIST=( "/home/foobar_user_name/Documents/Prose/" )
# EXCLUDE LIST OF DIRECTORIES
# Even though I am being specific about what I want to include,
# there is still a lot of stuff I don't need.
EXCLIST=( "/home/*/Trash" \
"/home/*/Projects/Completed" \
"/**.DS_Store" "/**Icon?" "/**.AppleDouble" \
)
# STATIC BACKUP OPTIONS
# Here you can define the static backup options that you want to run with
# duplicity. I use both the `--full-if-older-than` option plus the
# `--s3-use-new-style` option (for European buckets). Be sure to separate your
# options with appropriate spacing.
STATIC_OPTIONS="--full-if-older-than 14D --s3-use-new-style"
# FULL BACKUP & REMOVE OLDER THAN SETTINGS
# Because duplicity will continue to add to each backup as you go,
# it will eventually create a very large set of files. Also, incremental
# backups leave room for problems in the chain, so doing a "full"
# backup every so often isn't not a bad idea.
#
# You can either remove older than a specific time period:
#CLEAN_UP_TYPE="remove-older-than"
#CLEAN_UP_VARIABLE="31D"
# Or, If you would rather keep a certain (n) number of full backups (rather
# than removing the files based on their age), you can use what I use:
CLEAN_UP_TYPE="remove-all-but-n-full"
CLEAN_UP_VARIABLE="2"
# LOGFILE INFORMATION DIRECTORY
# Provide directory for logfile, ownership of logfile, and verbosity level.
# I run this script as root, but save the log files under my user name --
# just makes it easier for me to read them and delete them as needed.
LOGDIR="/home/foobar_user_name/logs/test2/"
LOG_FILE="duplicity-`date +%Y-%m-%d_%H-%M`.txt"
LOG_FILE_OWNER="foobar_user_name:foobar_user_name"
VERBOSITY="-v3"
# EMAIL ALERT (*thanks: rmarescu*)
# Provide an email address to receive the logfile by email. If no email
# address is provided, no alert will be sent.
# You can set a custom from email address and a custom subject (both optionally)
# If no value is provided for the subject, the following value will be
# used by default: "duplicity-backup Alert ${LOG_FILE}"
# MTA used: mailx
#EMAIL="admin@example.com"
EMAIL_TO=
EMAIL_FROM=
EMAIL_SUBJECT=
# command to use to send mail
MAIL="mailx"
#MAIL="ssmtp"
# TROUBLESHOOTING: If you are having any problems running this script it is
# helpful to see the command output that is being generated to determine if the
# script is causing a problem or if it is an issue with duplicity (or your
# setup). Simply uncomment the ECHO line below and the commands will be
# printed to the logfile. This way, you can see if the problem is with the
# script or with duplicity.
#ECHO=$(which echo)

140
duplicity-backup.sh
View file

@ -19,7 +19,7 @@
# #
# MORE ABOUT THIS SCRIPT AVAILABLE IN THE README AND AT: # MORE ABOUT THIS SCRIPT AVAILABLE IN THE README AND AT:
# #
# http://zertrin.org/duplicity-backup.html (for this version) # http://zertrin.org/projects/duplicity-backup/ (for this version)
# http://damontimm.com/code/dt-s3-backup (for the original programi by Damon Timm) # http://damontimm.com/code/dt-s3-backup (for the original programi by Damon Timm)
# #
# Latest code available at: # Latest code available at:
@ -27,127 +27,9 @@
# #
# ---------------------------------------------------------------------------- # # ---------------------------------------------------------------------------- #
# Set config file (uncomment if you want to use a separate config file) # Set config file (don't forget to copy duplicity-backup.conf.example to
# Its content override config below ! # match that path)
#CONFIG="/some/path/to/config/file" CONFIG="duplicity-backup.conf"
# AMAZON S3 INFORMATION
# Comment out this lines if you're not using S3
AWS_ACCESS_KEY_ID="foobar_aws_key_id"
AWS_SECRET_ACCESS_KEY="foobar_aws_access_key"
# ENCRYPTION INFORMATION
# If you aren't running this from a cron, comment this line out
# and duplicity should prompt you for your password.
# Comment out if you're using only GPG_KEY or not using encryption
PASSPHRASE="foobar_gpg_passphrase"
# Specify which GPG key you would like to use (even if you have only one).
# Comment out if you're using only PASSPHRASE or not using encryption
GPG_KEY="foobar_gpg_key"
# Do you want your backup to be encrypted? yes/no
# If yes, please make sure you specify either PASSPHRASE OR GPG_KEY
ENCRYPTION='yes'
# BACKUP SOURCE INFORMATION
# The ROOT of your backup (where you want the backup to start);
# This can be / or somwhere else -- I use /home/ because all the
# directories start with /home/ that I want to backup.
ROOT="/home"
# BACKUP DESTINATION INFORMATION
# In my case, I use Amazon S3 use this - so I made up a unique
# bucket name (you don't have to have one created, it will do it
# for you). If you don't want to use Amazon S3, you can backup
# to a file or any of duplicity's supported outputs.
#
# NOTE: You do need to keep the "s3+http://<your location>/" format
# even though duplicity supports "s3://<your location>/".
DEST="s3+http://backup-bucket/backup-folder/"
# Other possible locations
#DEST="ftp://user[:password]@other.host[:port]/some_dir"
#DEST="rsync://user@host.com[:port]//absolute_path"
#DEST="ssh://user[:password]@other.host[:port]/[/]some_dir"
#DEST="file:///home/foobar_user_name/new-backup-test/"
# INCLUDE LIST OF DIRECTORIES
# Here is a list of directories to include; if you want to include
# everything that is in root, you could leave this list empty (I think).
#
# Here is an example with multiple locations:
#INCLIST=( "/home/*/Documents" \
# "/home/*/Projects" \
# "/home/*/logs" \
# "/home/www/mysql-backups" \
# )
#
# Simpler example with one location:
INCLIST=( "/home/foobar_user_name/Documents/Prose/" )
# EXCLUDE LIST OF DIRECTORIES
# Even though I am being specific about what I want to include,
# there is still a lot of stuff I don't need.
EXCLIST=( "/home/*/Trash" \
"/home/*/Projects/Completed" \
"/**.DS_Store" "/**Icon?" "/**.AppleDouble" \
)
# STATIC BACKUP OPTIONS
# Here you can define the static backup options that you want to run with
# duplicity. I use both the `--full-if-older-than` option plus the
# `--s3-use-new-style` option (for European buckets). Be sure to separate your
# options with appropriate spacing.
STATIC_OPTIONS="--full-if-older-than 14D --s3-use-new-style"
# FULL BACKUP & REMOVE OLDER THAN SETTINGS
# Because duplicity will continue to add to each backup as you go,
# it will eventually create a very large set of files. Also, incremental
# backups leave room for problems in the chain, so doing a "full"
# backup every so often isn't not a bad idea.
#
# You can either remove older than a specific time period:
#CLEAN_UP_TYPE="remove-older-than"
#CLEAN_UP_VARIABLE="31D"
# Or, If you would rather keep a certain (n) number of full backups (rather
# than removing the files based on their age), you can use what I use:
CLEAN_UP_TYPE="remove-all-but-n-full"
CLEAN_UP_VARIABLE="2"
# LOGFILE INFORMATION DIRECTORY
# Provide directory for logfile, ownership of logfile, and verbosity level.
# I run this script as root, but save the log files under my user name --
# just makes it easier for me to read them and delete them as needed.
LOGDIR="/home/foobar_user_name/logs/test2/"
LOG_FILE="duplicity-`date +%Y-%m-%d_%H-%M`.txt"
LOG_FILE_OWNER="foobar_user_name:foobar_user_name"
VERBOSITY="-v3"
# EMAIL ALERT (*thanks: rmarescu*)
# Provide an email address to receive the logfile by email. If no email
# address is provided, no alert will be sent.
# You can set a custom from email address and a custom subject (both optionally)
# If no value is provided for the subject, the following value will be
# used by default: "duplicity-backup Alert ${LOG_FILE}"
# MTA used: mailx
#EMAIL="admin@example.com"
EMAIL_TO=
EMAIL_FROM=
EMAIL_SUBJECT=
# command to use to send mail
MAIL="mailx"
#MAIL="ssmtp"
# TROUBLESHOOTING: If you are having any problems running this script it is
# helpful to see the command output that is being generated to determine if the
# script is causing a problem or if it is an issue with duplicity (or your
# setup). Simply uncomment the ECHO line below and the commands will be
# printed to the logfile. This way, you can see if the problem is with the
# script or with duplicity.
#ECHO=$(which echo)
############################################################## ##############################################################
# Script Happens Below This Line - Shouldn't Require Editing # # Script Happens Below This Line - Shouldn't Require Editing #
@ -157,9 +39,9 @@ MAIL="mailx"
if [ ! -z "$CONFIG" -a -f "$CONFIG" ]; if [ ! -z "$CONFIG" -a -f "$CONFIG" ];
then then
. $CONFIG . $CONFIG
elif [ ! -z "$CONFIG" -a ! -f "$CONFIG" ]; else
then echo "ERROR: can't find config file! (${CONFIG})" >&2
echo "ERROR: can't find config file!" >&2 exit 1
fi fi
export AWS_ACCESS_KEY_ID export AWS_ACCESS_KEY_ID
@ -507,14 +389,13 @@ case "$1" in
if [[ ! "$2" ]]; then if [[ ! "$2" ]]; then
echo "Which file do you want to restore (eg, mail/letter.txt):" echo "Which file do you want to restore (eg, mail/letter.txt):"
read -e FILE_TO_RESTORE read -e FILE_TO_RESTORE
FILE_TO_RESTORE="'"$FILE_TO_RESTORE"'"
echo echo
else else
FILE_TO_RESTORE="'"$2"'" FILE_TO_RESTORE=$2
fi fi
if [[ "$3" ]]; then if [[ "$3" ]]; then
DEST="'"$3"'" DEST=$3
else else
DEST=$(basename $FILE_TO_RESTORE) DEST=$(basename $FILE_TO_RESTORE)
fi fi
@ -530,6 +411,9 @@ case "$1" in
exit 1 exit 1
fi fi
FILE_TO_RESTORE="'"$FILE_TO_RESTORE"'"
DEST="'"$DEST"'"
echo "Restoring now ..." echo "Restoring now ..."
#use INCLUDE variable without create another one #use INCLUDE variable without create another one
INCLUDE="--file-to-restore ${FILE_TO_RESTORE}" INCLUDE="--file-to-restore ${FILE_TO_RESTORE}"