From fe55d48cbbc6832e2cfe51102bf2694b955bd7c5 Mon Sep 17 00:00:00 2001 From: Zertrin Date: Sun, 8 Jul 2012 20:46:41 +0200 Subject: [PATCH] v0.7 : external config file now mandatory + fix issue #1 --- CHANGELOG | 5 ++ README | 46 +++++----- duplicity-backup.conf.example | 158 ++++++++++++++++++++++++++++++++++ duplicity-backup.sh | 140 +++--------------------------- 4 files changed, 198 insertions(+), 151 deletions(-) create mode 100644 duplicity-backup.conf.example diff --git a/CHANGELOG b/CHANGELOG index 3249ff0..65c47b8 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,3 +1,8 @@ +0.7 (8 July 2012) +================= + * Fix Issue #1 "Restore error if no target name" + * Rewrite to force utilisation of an external config file to facilitate updates of the script in itself + 0.6.1 (5 May 2012) ================= Inclusion of two patches written by [shamer] + modified config file management. diff --git a/README b/README index 4f49a39..5199f14 100644 --- a/README +++ b/README @@ -1,11 +1,11 @@ ABOUT THIS SCRIPT ================= -This bash script was designed to automate and simplify the remote backup -process of duplicity on Amazon S3 primarily. Other backup destinations are -possible. After your script is configured, you can easily backup, restore, -verify and clean (either via cron or manually) your data without having to -remember lots of different command options and passphrases. +This bash script was designed to automate and simplify the remote backup process +of duplicity on Amazon S3 primarily. Other backup destinations are possible. +After your script is configured, you can easily backup, restore, verify and +clean (either via cron or manually) your data without having to remember lots of +different command options and passphrases. Most importantly, you can easily backup the script and your gpg key in a convenient passphrase-encrypted file. This comes in in handy if/when your @@ -15,8 +15,8 @@ Optionally, you can set up an email address where the log file will be sent, which is useful when the script is used via cron. This version is a rewriting of the code originally written by Damon Timm, -including many patches that have been brought to the original scripts by -various forks on Github. +including many patches that have been brought to the original scripts by various +forks on Github. Latest version of the code available at: http://github.com/zertrin/duplicity-backup @@ -24,7 +24,7 @@ http://github.com/zertrin/duplicity-backup Merge requests are welcome :) More information about this script available at: -http://zertrin.org/duplicity-backup.html +http://zertrin.org/projects/duplicity-backup/ The original version of the code is availabe at: http://github.com/theterran/dt-s3-backup @@ -32,10 +32,11 @@ http://github.com/theterran/dt-s3-backup BEFORE YOU START ================ -This script requires user configuration. Instructions are in -the file itself and should be self-explanatory. Be sure to replace all the -*foobar* values with your real ones. Almost every value needs to be -configured in someway. +This script requires user configuration. Instructions are in the config file +itself and should be self-explanatory. Please DO NOT edit the example config +file, but make a copy of it and edit this one. Be sure to replace all the +*foobar* values with your real ones. Almost every value needs to be configured +in someway. You can use multiple copies of the script with different settings for different backup scenarios. It is designed to run as a cron job and will log information @@ -54,11 +55,11 @@ REQUIREMENTS CONFIGURATION ============= -The configuration takes place directly in the script and is documented there. +The configuration takes place in a separate config file and is documented there. -You can optionnaly specifiy a custom config file in the CONFIG parameter at the -very beginning. Any parameter specified in this custom config file will -override thoses specified in the script. +The script looks for its configuration by reading the config file specified in +the CONFIG parameter at the beggining of the script. Be sure to edit it to match +the actual location of your config file. COMMON USAGE EXAMPLES ===================== @@ -103,15 +104,14 @@ TROUBLESHOOTING This script attempts to simplify the task of running a duplicity command; if you are having any problems with the script the first step is to determine if the -script is generating an incorrect command or if duplicity itself is -causing your error. +script is generating an incorrect command or if duplicity itself is causing your +error. To see exactly what is happening when you run duplicity-backup, head to the bottom -of the user configuration portion of the script and uncomment the `ECHO=$(which -echo)` variable. This will stop the script from running and will, instead, -output the generated command into your log file. You can then check to see if -what is being generated is causing an error or if it is duplicity causing you -woe. +of the configuration file and uncomment the `ECHO=$(which echo)` variable. This +will stop the script from running and will, instead, output the generated +command into your log file. You can then check to see if what is being generated +is causing an error or if it is duplicity causing you woe. NEXT VERSION WISH LIST ====================== diff --git a/duplicity-backup.conf.example b/duplicity-backup.conf.example new file mode 100644 index 0000000..9c6b405 --- /dev/null +++ b/duplicity-backup.conf.example @@ -0,0 +1,158 @@ +#!/bin/bash +# +# Copyright (c) 2008-2010 Damon Timm. +# Copyright (c) 2010 Mario Santagiuliana. +# Copyright (c) 2012 Marc Gallet. +# +# This program is free software: you can redistribute it and/or modify it under +# the terms of the GNU General Public License as published by the Free Software +# Foundation, either version 3 of the License, or (at your option) any later +# version. +# +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more +# details. +# +# You should have received a copy of the GNU General Public License along with +# this program. If not, see . +# +# MORE ABOUT THIS SCRIPT AVAILABLE IN THE README AND AT: +# +# http://zertrin.org/projects/duplicity-backup/ (for this version) +# http://damontimm.com/code/dt-s3-backup (for the original programi by Damon Timm) +# +# Latest code available at: +# http://github.com/zertrin/duplicity-backup +# +# ---------------------------------------------------------------------------- # + +# ############################################# +# # DUPLICITY-BACKUP CONFIG FILE # +# ############################################# + +# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! +# ! DO NOT edit duplicity-backup.conf.example ! +# ! please copy it to anywhere you want ! +# ! (typically duplicity-backup.conf) ! +# ! and edit this file instead ! +# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + +# AMAZON S3 INFORMATION +# Comment out this lines if you're not using Amazon S3 +AWS_ACCESS_KEY_ID="foobar_aws_key_id" +AWS_SECRET_ACCESS_KEY="foobar_aws_access_key" + +# ENCRYPTION INFORMATION +# Do you want your backup to be encrypted? yes/no +# If yes, please make sure you specify either PASSPHRASE or GPG_KEY +ENCRYPTION='yes' + +# If you are NOT running this from a cron, comment this line out +# and duplicity should prompt you for your password. +# Comment out if you aren't using encryption +PASSPHRASE="foobar_gpg_passphrase" + +# Specify which GPG key you would like to use (even if you have only one). +# Comment out if you're using only PASSPHRASE or not using encryption +GPG_KEY="foobar_gpg_key" + +# BACKUP SOURCE INFORMATION +# The ROOT of your backup (where you want the backup to start); +# This can be / or somwhere else -- I use /home/ because all the +# directories start with /home/ that I want to backup. +ROOT="/home" + +# BACKUP DESTINATION INFORMATION +# In my case, I use Amazon S3 use this - so I made up a unique +# bucket name (you don't have to have one created, it will do it +# for you). If you don't want to use Amazon S3, you can backup +# to a file or any of duplicity's supported outputs. +# +# NOTE: You do need to keep the "s3+http:///" format +# even though duplicity supports "s3:///". +DEST="s3+http://backup-bucket/backup-folder/" +# Other possible locations +#DEST="ftp://user[:password]@other.host[:port]/some_dir" +#DEST="rsync://user@host.com[:port]//absolute_path" +#DEST="scp://user[:password]@other.host[:port]/[/]some_dir" +#DEST="ssh://user[:password]@other.host[:port]/[/]some_dir" +#DEST="file:///home/foobar_user_name/new-backup-test/" + +# INCLUDE LIST OF DIRECTORIES +# Here is a list of directories to include; if you want to include +# everything that is in root, you could leave this list empty (I think). +# +# Here is an example with multiple locations: +#INCLIST=( "/home/*/Documents" \ +# "/home/*/Projects" \ +# "/home/*/logs" \ +# "/home/www/mysql-backups" \ +# ) +# +# Simpler example with one location: +INCLIST=( "/home/foobar_user_name/Documents/Prose/" ) + +# EXCLUDE LIST OF DIRECTORIES +# Even though I am being specific about what I want to include, +# there is still a lot of stuff I don't need. +EXCLIST=( "/home/*/Trash" \ + "/home/*/Projects/Completed" \ + "/**.DS_Store" "/**Icon?" "/**.AppleDouble" \ + ) + +# STATIC BACKUP OPTIONS +# Here you can define the static backup options that you want to run with +# duplicity. I use both the `--full-if-older-than` option plus the +# `--s3-use-new-style` option (for European buckets). Be sure to separate your +# options with appropriate spacing. +STATIC_OPTIONS="--full-if-older-than 14D --s3-use-new-style" + +# FULL BACKUP & REMOVE OLDER THAN SETTINGS +# Because duplicity will continue to add to each backup as you go, +# it will eventually create a very large set of files. Also, incremental +# backups leave room for problems in the chain, so doing a "full" +# backup every so often isn't not a bad idea. +# +# You can either remove older than a specific time period: +#CLEAN_UP_TYPE="remove-older-than" +#CLEAN_UP_VARIABLE="31D" + +# Or, If you would rather keep a certain (n) number of full backups (rather +# than removing the files based on their age), you can use what I use: +CLEAN_UP_TYPE="remove-all-but-n-full" +CLEAN_UP_VARIABLE="2" + +# LOGFILE INFORMATION DIRECTORY +# Provide directory for logfile, ownership of logfile, and verbosity level. +# I run this script as root, but save the log files under my user name -- +# just makes it easier for me to read them and delete them as needed. + +LOGDIR="/home/foobar_user_name/logs/test2/" +LOG_FILE="duplicity-`date +%Y-%m-%d_%H-%M`.txt" +LOG_FILE_OWNER="foobar_user_name:foobar_user_name" +VERBOSITY="-v3" + +# EMAIL ALERT (*thanks: rmarescu*) +# Provide an email address to receive the logfile by email. If no email +# address is provided, no alert will be sent. +# You can set a custom from email address and a custom subject (both optionally) +# If no value is provided for the subject, the following value will be +# used by default: "duplicity-backup Alert ${LOG_FILE}" +# MTA used: mailx +#EMAIL="admin@example.com" +EMAIL_TO= +EMAIL_FROM= +EMAIL_SUBJECT= + +# command to use to send mail +MAIL="mailx" +#MAIL="ssmtp" + +# TROUBLESHOOTING: If you are having any problems running this script it is +# helpful to see the command output that is being generated to determine if the +# script is causing a problem or if it is an issue with duplicity (or your +# setup). Simply uncomment the ECHO line below and the commands will be +# printed to the logfile. This way, you can see if the problem is with the +# script or with duplicity. +#ECHO=$(which echo) diff --git a/duplicity-backup.sh b/duplicity-backup.sh index ed385c0..2147180 100755 --- a/duplicity-backup.sh +++ b/duplicity-backup.sh @@ -19,7 +19,7 @@ # # MORE ABOUT THIS SCRIPT AVAILABLE IN THE README AND AT: # -# http://zertrin.org/duplicity-backup.html (for this version) +# http://zertrin.org/projects/duplicity-backup/ (for this version) # http://damontimm.com/code/dt-s3-backup (for the original programi by Damon Timm) # # Latest code available at: @@ -27,127 +27,9 @@ # # ---------------------------------------------------------------------------- # -# Set config file (uncomment if you want to use a separate config file) -# Its content override config below ! -#CONFIG="/some/path/to/config/file" - -# AMAZON S3 INFORMATION -# Comment out this lines if you're not using S3 -AWS_ACCESS_KEY_ID="foobar_aws_key_id" -AWS_SECRET_ACCESS_KEY="foobar_aws_access_key" - -# ENCRYPTION INFORMATION -# If you aren't running this from a cron, comment this line out -# and duplicity should prompt you for your password. -# Comment out if you're using only GPG_KEY or not using encryption -PASSPHRASE="foobar_gpg_passphrase" - -# Specify which GPG key you would like to use (even if you have only one). -# Comment out if you're using only PASSPHRASE or not using encryption -GPG_KEY="foobar_gpg_key" - -# Do you want your backup to be encrypted? yes/no -# If yes, please make sure you specify either PASSPHRASE OR GPG_KEY -ENCRYPTION='yes' - -# BACKUP SOURCE INFORMATION -# The ROOT of your backup (where you want the backup to start); -# This can be / or somwhere else -- I use /home/ because all the -# directories start with /home/ that I want to backup. -ROOT="/home" - -# BACKUP DESTINATION INFORMATION -# In my case, I use Amazon S3 use this - so I made up a unique -# bucket name (you don't have to have one created, it will do it -# for you). If you don't want to use Amazon S3, you can backup -# to a file or any of duplicity's supported outputs. -# -# NOTE: You do need to keep the "s3+http:///" format -# even though duplicity supports "s3:///". -DEST="s3+http://backup-bucket/backup-folder/" -# Other possible locations -#DEST="ftp://user[:password]@other.host[:port]/some_dir" -#DEST="rsync://user@host.com[:port]//absolute_path" -#DEST="ssh://user[:password]@other.host[:port]/[/]some_dir" -#DEST="file:///home/foobar_user_name/new-backup-test/" - -# INCLUDE LIST OF DIRECTORIES -# Here is a list of directories to include; if you want to include -# everything that is in root, you could leave this list empty (I think). -# -# Here is an example with multiple locations: -#INCLIST=( "/home/*/Documents" \ -# "/home/*/Projects" \ -# "/home/*/logs" \ -# "/home/www/mysql-backups" \ -# ) -# -# Simpler example with one location: -INCLIST=( "/home/foobar_user_name/Documents/Prose/" ) - -# EXCLUDE LIST OF DIRECTORIES -# Even though I am being specific about what I want to include, -# there is still a lot of stuff I don't need. -EXCLIST=( "/home/*/Trash" \ - "/home/*/Projects/Completed" \ - "/**.DS_Store" "/**Icon?" "/**.AppleDouble" \ - ) - -# STATIC BACKUP OPTIONS -# Here you can define the static backup options that you want to run with -# duplicity. I use both the `--full-if-older-than` option plus the -# `--s3-use-new-style` option (for European buckets). Be sure to separate your -# options with appropriate spacing. -STATIC_OPTIONS="--full-if-older-than 14D --s3-use-new-style" - -# FULL BACKUP & REMOVE OLDER THAN SETTINGS -# Because duplicity will continue to add to each backup as you go, -# it will eventually create a very large set of files. Also, incremental -# backups leave room for problems in the chain, so doing a "full" -# backup every so often isn't not a bad idea. -# -# You can either remove older than a specific time period: -#CLEAN_UP_TYPE="remove-older-than" -#CLEAN_UP_VARIABLE="31D" - -# Or, If you would rather keep a certain (n) number of full backups (rather -# than removing the files based on their age), you can use what I use: -CLEAN_UP_TYPE="remove-all-but-n-full" -CLEAN_UP_VARIABLE="2" - -# LOGFILE INFORMATION DIRECTORY -# Provide directory for logfile, ownership of logfile, and verbosity level. -# I run this script as root, but save the log files under my user name -- -# just makes it easier for me to read them and delete them as needed. - -LOGDIR="/home/foobar_user_name/logs/test2/" -LOG_FILE="duplicity-`date +%Y-%m-%d_%H-%M`.txt" -LOG_FILE_OWNER="foobar_user_name:foobar_user_name" -VERBOSITY="-v3" - -# EMAIL ALERT (*thanks: rmarescu*) -# Provide an email address to receive the logfile by email. If no email -# address is provided, no alert will be sent. -# You can set a custom from email address and a custom subject (both optionally) -# If no value is provided for the subject, the following value will be -# used by default: "duplicity-backup Alert ${LOG_FILE}" -# MTA used: mailx -#EMAIL="admin@example.com" -EMAIL_TO= -EMAIL_FROM= -EMAIL_SUBJECT= - -# command to use to send mail -MAIL="mailx" -#MAIL="ssmtp" - -# TROUBLESHOOTING: If you are having any problems running this script it is -# helpful to see the command output that is being generated to determine if the -# script is causing a problem or if it is an issue with duplicity (or your -# setup). Simply uncomment the ECHO line below and the commands will be -# printed to the logfile. This way, you can see if the problem is with the -# script or with duplicity. -#ECHO=$(which echo) +# Set config file (don't forget to copy duplicity-backup.conf.example to +# match that path) +CONFIG="duplicity-backup.conf" ############################################################## # Script Happens Below This Line - Shouldn't Require Editing # @@ -157,9 +39,9 @@ MAIL="mailx" if [ ! -z "$CONFIG" -a -f "$CONFIG" ]; then . $CONFIG -elif [ ! -z "$CONFIG" -a ! -f "$CONFIG" ]; -then - echo "ERROR: can't find config file!" >&2 +else + echo "ERROR: can't find config file! (${CONFIG})" >&2 + exit 1 fi export AWS_ACCESS_KEY_ID @@ -507,14 +389,13 @@ case "$1" in if [[ ! "$2" ]]; then echo "Which file do you want to restore (eg, mail/letter.txt):" read -e FILE_TO_RESTORE - FILE_TO_RESTORE="'"$FILE_TO_RESTORE"'" echo else - FILE_TO_RESTORE="'"$2"'" + FILE_TO_RESTORE=$2 fi if [[ "$3" ]]; then - DEST="'"$3"'" + DEST=$3 else DEST=$(basename $FILE_TO_RESTORE) fi @@ -530,6 +411,9 @@ case "$1" in exit 1 fi + FILE_TO_RESTORE="'"$FILE_TO_RESTORE"'" + DEST="'"$DEST"'" + echo "Restoring now ..." #use INCLUDE variable without create another one INCLUDE="--file-to-restore ${FILE_TO_RESTORE}"