Backup to Amazon S3 Bucket from CentOS 6.4

In this tutorial I’ll make mysql db backup and then upload it to AMAZON S3 Bucket using S3CMD tool and bash script. The script will also create the separate folder automatically for each day inside the bucket.

Before starting this tutorial, we need to install the EPEL for s3cmd tools:

wget http://ftp.riken.jp/Linux/fedora/epel/RPM-GPG-KEY-EPEL-6 
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm --import RPM-GPG-KEY-EPEL-6
rpm -ivh epel-release-6-8.noarch.rpm
yum repolist

Switch to the root user:

sudo su -

Install the s3cmd tools:

yum install s3cmd

After installation, run the following command in order to configure the s3cmd tools using your AMAZON ACCESS KEY and SECRET KEY.

s3cmd --configure

It is always a good practice to create a separate scripts directory:

mkdir /script
cd /script
nano mysqlbackuptos3.sh

Paste the below code into the script (Please modify the following script to suit your purposes):

#!/bin/bash
#############################################################
# Created by: Thoai Nguyen (thoai.nguyenanh{at}gmail(dot)com) #
#                                                           #                                                                                                                    #
#############################################################
## Specify the name of the database that you want to backup
DATABASE="tendodb"

## USERNAME and PASSWORD to take the backup
USER="arbab"
PASSWORD="PASSWORD"
## Defination of some necessary variables
S3_BUCKET=tendodbbackup
DATE=`date +%A-"(%d.%m.%Y)".%H`
BACKUP_LOC=/tmp/backups
S3BDIR=`date +%d.%A-"(%d.%m.%Y)"`
mysql_backup(){
/usr/bin/mysqldump -u $USER -p$PASSWORD --databases $DATABASE | gzip -9 > $BACKUP_LOC/tendodb_$DATE.sql.gz
 s3cmd ls s3://$S3_BUCKET/$S3BDIR > /tmp/log.txt
 grep -lr "$S3BDIR" /tmp/log.txt
 if [ $? -ne 0 ]
 then
 mkdir /tmp/$S3BDIR
 s3cmd put -r /tmp/$S3BDIR s3://$S3_BUCKET/
s3cmd sync -r $BACKUP_LOC/ s3://$S3_BUCKET/$S3BDIR/
else
 s3cmd sync -r $BACKUP_LOC/ s3://$S3_BUCKET/$S3BDIR/
fi
}
mysql_backup
echo "----------------------------------------------------------------------------------------------" >> /scripts/dbbackup.log
echo "Backup Successfully uploading to the S3 Bucket at `date +%A.%Y%m%d-%H.%M`" >> /script/dbbackup.log
echo "##############################################END#############################################" >> /scripts/dbbackup.log
rm -rf $BACKUP_LOC/* /tmp/$S3BDIR
exit 0

Create the backups directory inside the tmp directory (or anywhere else, as per your desire) to hold the mysql backup temporarily before uploading it to the s3 bucket:

mkdir /tmp/backups
chmod 0777 /tmp/backups

Verify the access to the bucket, where you want to upload the backup using the following command:

s3cmd ls

Give the execute right to the script and run it:

cd /script
chmod +x mysqlbackuptos3.sh
ls
./mysqlbackuptos3.sh

This bash script also creates a log file inside the /script directory:

cd /script
ls
cat dbbackup.log

Inside the S3 Bucket, it will create the mysql backup like this:



To automate this process using the cronjob, enter crontab -e at the shell prompt:

crontab -e

Add this line to run the script at midnight every day:

0 * * * * /script/mysqlbackuptos3.sh > /dev/null

Enjoy 

Hope this will help you!

Please Remember me in your prayers!