{{tag>en en:linux en:file_transfert en:backup}}
====== Synchronization with lftp ======
This page talk about folder synchronization with lftp.
===== Installation =====
Install lftp package.
===== Configuration =====
There is no specific configuration to do but bookmarks can simplify the use of lftp.
Usage: bookmark [SUBCMD]
bookmark command controls bookmarks
The following subcommands are recognized:
add [] - add current place or given location to bookmarks
and bind to given name
del - remove bookmark with the name
edit - start editor on bookmarks file
import - import foreign bookmarks
list - list bookmarks (default)
Example :
lftp :/> bookmark add site.com ftp://user:pass@ftp.site.com
lftp :/> bookmark list
site.com ftp://user:XXXX@ftp.site.com
lftp :/>
It's possible to use bookmarks with the "connect" command or directly on the command line :
lftp site.com
===== Manual synchronization =====
Before automatic synchronization, it's advised to do some tests manually.\\
:!: Be carreful, there is a risk to delete source file by mistake.
Synchronization is done with "mirror" :
Usage: mirror [OPTS] [remote [local]]
Mirror specified remote directory to local directory
-c, --continue continue a mirror job if possible
-e, --delete delete files not present at remote site
--delete-first delete old files before transferring new ones
-s, --allow-suid set suid/sgid bits according to remote site
--allow-chown try to set owner and group on files
--ignore-time ignore time when deciding whether to download
-n, --only-newer download only newer files (-c won't work)
-r, --no-recursion don't go to subdirectories
-p, --no-perms don't set file permissions
--no-umask don't apply umask to file modes
-R, --reverse reverse mirror (put files)
-L, --dereference download symbolic links as files
-N, --newer-than=SPEC download only files newer than specified time
-P, --parallel[=N] download N files in parallel
-i RX, --include RX include matching files
-x RX, --exclude RX exclude matching files
RX is extended regular expression
-v, --verbose[=N] verbose operation
--log=FILE write lftp commands being executed to FILE
--script=FILE write lftp commands to FILE, but don't execute them
--just-print, --dry-run same as --script=-
When using -R, the first directory is local and the second is remote.
If the second directory is omitted, basename of first directory is used.
If both directories are omitted, current local and remote directories are used.
The option --dry-run is usefull to make some tests without risk.
Some example of synchronisation (download) :
mirror /Data/Photos /data/transfert/photos -v --log /var/log/lftp/download_nas --parallel=10 --no-perms -e
* /Data/Photos is the source folder located on the remote server
* /data/transfert/photos is the local destination folder
* -v activate the verbose mode
* --log option set the file to use for logs, this file is overrided each time and is not used with --dry-run
* --parallel activate parralel transferts, usefull with small files
* --no-perms disable permission synchronization
* -e delete files on the destination folder that are no more present on the source
Another example (upload) :
mirror -R /space/Videos/ /data/videos/ --ignore-time --no-perms -e -v
* -R activate the reverse mode (upload)
* /space/Videos/ is the local source
* /data/videos/ is the destination folder on the remote server
* --ignore-time disable date check used to determined if the file need to be synchronized (only the file size is used)
* --no-perms disable permission synchronization
* -e delete files on the destination folder that are no more present on the source
* -v activate the verbose mode
When uploading, it is not possible to set the date/time on the files uploaded, that's why --ignore-time is needed.
When the complete command is defined, it's possible to use it inside an lftp script.
Example :
connect nas
mirror /Data/Photos /data/transfert/photos -v --log /var/log/lftp/download_nas --parallel=10 --no-perms -e
To use the script, launch lftp with -f option.
===== Automatic synchronization =====
To run automaticaly the synchronization, please find below a small batch that will launch lftp.\\
lftp is started in background like a daemon.
To use the script, you need to :
- copy it inside a new file
- change NAME variable as you like
- specify the link to your lftp script inside DAEMON_OPTS
- make the script executable
The script has the following options :
* start : start the synchronization
* stop : stop the daemon
* restart : restart the daemin
* check : indicate if the synchronization is running or not
Script :
#!/bin/sh
# lftp synchronization script by Matthieu Bouthors
# http://www.bouthors.fr
# Change NAME and DAEMON_OPTS before use
set -e
# Must be a valid filename
NAME=lftp_synchro
PIDFILE=/var/run/$NAME.pid
#This is the command to be run, give the full pathname
DAEMON=/usr/bin/lftp
DAEMON_OPTS="-f /chemin/vers/cmd.lftp"
export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"
# check if lftp is running
RUNNING=false
if [ -e $PIDFILE ]
then
currentpid=`cat $PIDFILE`
if `ps -p $currentpid > /dev/null`
then
RUNNING=true
else
rm -f $PIDFILE
fi
fi
case "$1" in
start)
date
if $RUNNING
then
started=`date -r $PIDFILE`
echo "daemon $NAME already started since $started, you need to stop it first"
else
echo -n "Starting daemon: "$NAME
start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $DAEMON -- $DAEMON_OPTS
echo "."
fi
;;
stop)
date
echo -n "Stopping daemon: "$NAME
start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
rm -f $PIDFILE
echo "."
;;
restart)
date
echo -n "Restarting daemon: "$NAME
if $RUNNING
then
start-stop-daemon --stop --quiet --oknodo --retry 30 --pidfile $PIDFILE
rm -f $PIDFILE
fi
start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $DAEMON -- $DAEMON_OPTS
echo "."
;;
check)
if $RUNNING
then
echo "$NAME is running"
else
echo "$NAME is not running"
fi
;;
*)
echo "Usage: "$1" {start|stop|restart|check}"
exit 1
esac
exit 0