Server Cleaner : A script to keep your VPS or Dedicated Server Squeky clean

adminServers

Do you run your own VPS or dedicated server, and find your server filling up with log files, yum files and unnecessary ‘stuff’.
Well heres a script you can run on your servers, in fact add a cron and have it run daily, weekly or monthly.

IMPORTANT NOTE: this script will be deleting files, and must be run as root
and if you are running this as root – IT IS YOUR RESPONSIBILITY FOR THE RESULTS.

If you would like to know what files are deleted or want to edit the script as to determine what gets deleted
see further below.

There are plenty of improvements that could be made to this script, however its a great baseline to start with.

The basic idea is that Server side disk space is expensive compared to local storage – every
extra Gb is more clients that you can host. This script will:
a) remove uneeded YUM cache files, cpanel temp files etc.
b) remove “trash” files from user accounts older than a specified period (see Variables)
c) backup old log files (from logrotate), then compress and FTP them to a backup server
d) remove the backup files
e) give you a report to so that you know how it went

The script:

First create a file named vpsclean.sh which contains the following:

[divider_advanced]

#!/bin/bash
#VPS disk cleanup by RoboRutt for the world to use as you see fit
#any comments critique and stuff welcome to robert@ruttentech.com
#latest version available from http://www.ruttentech.com (checkout downloads)
#NOTE: this does some deleting stuff, and deliberately must be run as root
#and if you are running this as root – IT IS YOUR RESPONSIBILITY FOR THE RESULTS

echo Starting…

#setup the application variables – dont change these – editable variables are in the file vpsclean.conf
source vpsclean.conf
today=`date +%G%m%d`
MaxSpam=`expr $MaxSpamM “*” 1000`
version=1.6

#remove temp folder if it already exists from a previous (bad) run
cd $RunDir
if [ -d $RunDir/backup-$today ] ;then rm -rdf $RunDir/backup-$today ;fi
if [ -f vpsclean_ftp.log ] ;then rm -f vpsclean_ftp.log ;fi
if [ -f vpsclean-$today.tar.gz ] ;then rm -f vpsclean-$today.tar.gz ;fi
if [ -f $RunDir/before_cleanup ] ;then rm -f $RunDir/before_cleanup ;fi

#get a snapshot of disk usage before cleanup
echo Getting some disk statistics
df -h >> $RunDir/before_cleanup

# FTP Backup
if test $FTPBACKUP = Y
then
echo Creating a backup of files to be removed
#create file structure in back to mirror server
mkdir $RunDir/backup-$today
mkdir $RunDir/backup-$today/var
mkdir $RunDir/backup-$today/var/log
mkdir $RunDir/backup-$today/usr
mkdir $RunDir/backup-$today/usr/local
mkdir $RunDir/backup-$today/usr/local/apache
mkdir $RunDir/backup-$today/usr/local/apache/logs
#copy files
cp /var/log/apf_log.? $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/boot.log.? $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/cron.? $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/exim_mainlog.?.gz $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/exim_paniclog.?.gz $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/exim_rejectlog.?.gz $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/maillog.? $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/messages.? $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/rpmpkgs.? $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/secure.? $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/spooler.? $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /var/log/wtmp.? $RunDir/backup-$today/var/log > /dev/null 2>&1
cp /usr/local/apache/logs/error_log $RunDir/backup-$today/usr/local/apache/logs > /dev/null 2>&1
cp /usr/local/apache/logs/suphp_log $RunDir/backup-$today/usr/local/apache/logs > /dev/null 2>&1
find /home -ipath “*/.trash/*” -type f -mtime +$OldTrash -exec tar -rf $RunDir/backup-$today/user_trash_accounts.tar {} \; > /dev/null 2>&1
#tar and ftp the complete backup folder to your ftp server
echo “Compressing backup files to send to remote FTP server ($FtpHost:$FtpDir)”
cd $RunDir
tar -czf vpsclean-$today.tar.gz backup-$today/
FtpSize=`ls -hs vpsclean-$today.tar.gz`
echo “…about to ftp, this may take sime time – your backup file is $FtpSize”
ftp -nv $FtpHost << END > vpsclean_ftp.log
user “$FtpUser” “$FtpPass”
binary
cd $FtpDir
put vpsclean-$today.tar.gz
bye
END

#remove temporary files
echo “…removing the vpsclean temporary files”
rm -rdf $RunDir/backup-$today
rm -f $RunDir/vpsclean-$today.tar.gz

#end FTP backup conditional check
else
echo “Skipping FTP backup of files”
fi

#the actual cleaning up bit
echo “Now cleaning up”
echo …clearing YUM cache
yum clean all
echo “…removing Cpanel temp files”
rm -rdf /home/.cpcpan > /dev/null 2>&1
rm -rdf /home/.cpan > /dev/null 2>&1
rm -rdf /home/MySQLInstall > /dev/null 2>&1
rm -rdf /home/cpeasyapache > /dev/null 2>&1
rm -rdf /home/cprestore > /dev/null 2>&1
rm -rdf /home/cpbackuptmp > /dev/null 2>&1
echo “…removing “trash” files from user accounts”
find /home -ipath “*/.trash/*” -type f -mtime +$OldTrash -exec rm -df {} \; > /dev/null 2>&1
echo “…removing old log files”
rm -f /var/log/apf_log.? > /dev/null 2>&1
rm -f /var/log/boot.log.? > /dev/null 2>&1
rm -f /var/log/cron.? > /dev/null 2>&1
rm -f /var/log/exim_mainlog.?.gz > /dev/null 2>&1
rm -f /var/log/exim_paniclog.?.gz > /dev/null 2>&1
rm -f /var/log/exim_rejectlog.?.gz > /dev/null 2>&1
rm -f /var/log/maillog.? > /dev/null 2>&1
rm -f /var/log/messages.? > /dev/null 2>&1
rm -f /var/log/rpmpkgs.? > /dev/null 2>&1
rm -f /var/log/secure.? > /dev/null 2>&1
rm -f /var/log/spooler.? > /dev/null 2>&1
rm -f /var/log/wtmp.? > /dev/null 2>&1
rm -f /usr/local/apache/logs/error_log > /dev/null 2>&1
touch /usr/local/apache/logs/error_log
rm -f /usr/local/apache/logs/suphp_log > /dev/bull 2>&1
touch /usr/local/apache/logs/suphp_log

#Check for and remove oversize SpamAssassin Files
if test $DELSPAM = Y
then
echo “Checking for large SpamAssassin files over “$MaxSpamM” Megabytes”
CHECK=`find /home -name “auto-whitelist” -size +”$MaxSpam”k`
if test -n “$CHECK”
then
echo “…the following files have been detected”
find /home -name “auto-whitelist” -size +”$MaxSpam”k -exec du {} -ah –max-depth=0 \;
find /home -name “bayes_seen” -size +”$MaxSpam”k -exec du {} -ah –max-depth=0 \;
echo
echo “Would you like to delete these files (y = yes) ?”
read Conf
if test $Conf = y
then
echo “…deleting files”
#find /home -name “auto-whitelist” -exec rm -f {} \;
#find /home -name “bayes-seen” -exec rm -f {} \;
else
echo “…skipping SpamAssassin file deletion”
fi
else
echo “…there are no SpamAssassin files greater than $MaxSpamM Mb”
fi
else
echo “Skipping check for large SpamAssassin files”
fi

#Clear Joomla cache and tmp files for accounts in root folders
if test $JCACHE = Y
then
echo “Clearing cache for joomla installations in account webroot folders”
find /home -path “*/public_html/cache” -type d -exec rm -f {}/* \; > /dev/null 2>&1
echo “Clearing tmp files for joomla installations in account webroot folders”
find /home -ipath “*/public_html/tmp/*” ! -iname “*index*” -maxdepth 5 -exec rm -rdf \{} \; > /dev/null 2>&1
else
echo “Skipping joomla cleanup”
fi

#display the results of your hard work so that you know you deserve a coffee
echo
echo “finished cleaning up – now the results”
echo
echo “BEFORE FILE CLEANUP (v$version)”
cat before_cleanup
echo
echo “AFTER FILE CLEANUP (v$version)”
df -h
echo
cd /root
RootSize=`du -ah –max-depth=0`
echo “JUST TO LET YOU KNOW, YOU ARE STORING $RootSize IN YOUR “/ROOT” FOLDER”

#Remove log files
rm -f $RunDir/before_cleanup
#rm -f $RunDir/vpsclean_ftp.log

#end
cd $RunDir
echo “finished”

[divider_advanced]

VARIABLES:

Now Create the configuration file named vpsclean.conf and insert this content:

[divider_advanced]

conf_version=1.2

#setup some variables that you might like to change Dont comment them out – but you can change them as noted

OldTrash=60 #the number of days that you want to keep user accounts .trash files
FtpUser=something #your username for your chosen ftp backup server
FtpPass=somehting #password for the aformentioned ftp backup server
FtpHost=127.127.127.127 #host for the now famous ftp backup server
FtpDir=something #remote ftp dir for the legendary ftp backup server – you will need to create this first
RunDir=/root #Folder that you want to run the script from (usually /root)
MaxSpamM=100 #SpamAssassin auto-whitelist or bayes_seen file to be detected IN MEGABYTES
FTPBACKUP=Y #Perform FTP backup of files, Y= yes, N = no (implies no backup of files to be removed)
JCACHE=Y #Clear cache from Joomla sites (only for Joomla sites in account web root folders)
DELSPAM=Y #Check for large SpamAssassin Files, Y= yes, N= no (you will still be asked to confirm before delete)

[divider_advanced]

The file contains all the user editable variables. Feel free to change what
you like in this file, notes on variables are contained within the file.

Here are a few variables that are included in the vpsclean.conf file:

‘OldTrash’ – the number of days that you want to keep user accounts .trash files
‘FtpUser’ – your username for your chosen ftp backup server
‘FtpPass’ – password for ftp backup server
‘FtpHost’ – host for the ftp backup server
‘FtpDir’ – remote ftp dir for the ftp backup server
‘RunDir’ – folder that you want to run the script from (usually =/root )
‘MaxSpamM’ – The largest SpamAssassin auto-whitelist or bayes_seen file to be detected IN MEGABYTES
‘FTPBACKUP’ – Perform FTP backup of files, Y= yes, N = no (implies no backup of files to be removed)
‘JCACHE’ – Choose to clear cache from Joomla sites (only for Joomla sites in account web root folders ie. /home/’acccount’/public_html/cache)
‘DELSPAM’ – Choose to check for large SpamAssassin Files, Y= yes, N= no (you will still be asked to confirm before delete)

1) save both filenames correctly
2) upload to your server /root folder
3) make the script executable (ie. ‘chmod +x vpsclean*’)
4) make the script writeable (ie. ‘chmod 775 vpsclean*’)
5) edit the script – you will need to enter your own FTP server variables “vpsclean.conf” file
6) run the script – ./vpsclean.sh

==BACKUP:==

This script works on the assumption that you would like to backup your log files to a remote FTP server. If you don’t want to, you have 2 choices. Firstly you can just delete them without a backup – the second choice is to allow the compressed tar.gz file to be stored on your server. My tests have shown it to be about half the size of the total files that it replaces. Either way, you will need to modify the script to do this.

==KNOWN ISSUES:==

* FTP connection issue. The script will drpo a logfile (vpsclean_ftp.log) in the root folder once it starts FTP’ing, and should remove it once it ends. If for some reason the FTP is interrupted you should be able to check the logfile for hint. At this stage there is no built in check to verify the file transferred.
* Currently I only test for the existance of tmp/ and cache/ folders in user accounts. On my server, this equals Joomla installations, but I can’t guarantee that others use similar systems. In other words, if you are storing valuable information in a folder called “cache/” – this script will delete it.

==THINGS TO DO:==
* Domain logs – checkout /usr/local/apache/domlogs/ – can we rotate these logs? Which ones can be deleted ?
* /var/log – see if we can tar/ftp the logs to a remote server, then delete.
* add webroot as a variable so that we can use this script on other filesystems/structures (eg. var/www)
* tar command in progress meter
* elborate on joomla cleanup to detect version.php – currently just checks for the existence of tmp and cache
* remove the need for a fixed “rundir” variable
* add option to run as su
* create, append to logfile – option for emailing logfile to user (for use as part of cronjob)
* add error traps around ftp and other functions
* do a version check on the conf file – to allow the creation of an update script

==CHANGELOG:==

v1.5 – added the removal and re-creation of the suphp logfile for those running suphp

v1.4 – fixup some bugs from 1.3
fixed a bug preventing “find” to work properly

v1.3 – added changelog to notes file 🙂
added a search/remove for tmp file in joomla installations
fixed ftp bug
changed removal process for user trash files. This will now backup file (if wanted) before removal
added notes file to explain my innermost thinking – scarey

v1.1 – added some conditional statements to allow you to “opt out” of an FTP backup **Thanks to DAWORM on NL Forums for suggestion**
– added a check and remove (plus confirm) for extra large SpamAssassin files
– added a find and remove for Joomla Cache files (some folders on our server were 100Mb +)
– improved the copy / remove functions to output to NULL so that no messages are received when file does not exist