Process new or changed files

From Michael's Information Zone
Revision as of 13:45, 4 October 2017 by Michael.mast (talk | contribs) (Created page with "In this example I check for all files in a source directory, create an MD5 hash, search for the hash in a database, if not found add the file to the database, copy the file to...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

In this example I check for all files in a source directory, create an MD5 hash, search for the hash in a database, if not found add the file to the database, copy the file to the destination, send an email with a list of files that were processed.

newfilesdate=$(date +%Y-%m-%d)
mkdir /tmp/workingdir
find /mnt/source/ -type f > /tmp/woringdir/filelist
while read line
do
	line2=$(md5sum "$line" | sed 's/\ /---/' | sed 's/\ //')
	date=$(ls -l --time-style="+%Y-%m-%d" "$(awk 'BEGIN {FS="---"}; {print $2}' <<<"$line2")" | awk '{print $6}')
	hash=$(awk 'BEGIN {FS="---"}; {print $1}' <<<"$line2")
	filename=$( echo "$(awk 'BEGIN {FS="---"}; {print $2}' <<<"$line2")" | sed "s|.*/||;s|\ |\\\ |g;s|"\'"|\\\'|g")
	moddate=$(awk 'BEGIN {FS="---"}; {print $3}' <<<"$line2")
	query="$( echo "use database_name; select * from table_name where MD5 = '$hash';" | mysql -u user_name)"
	if ! grep -qi $hash <<<$query; then
	        echo "use database_name; insert into table_name (name,mod_date,move_date,MD5) values ('$filename','$newfilesdate','$date','$hash');" | mysql -u user_name
	        cp "$line" /mnt/destination/
	        echo "$line" >> /tmp/workingdir/copied
	fi
done < /tmp/workingdir/filelist
if [ -a /tmp/workingdir/copied ]; then
        sed -i 's|^.*\/||g' /tmp/workingdir/copied
        echo "Subject: "Files copied" | cat - /tmp/workingdir/copied | /usr/sbin/sendmail -r your_relay_email@domain.tld youremail@domain.tld
fi
rm -rf /tmp/workingdir