« Leopard Site-Wide | Main | Customize Firefox’s Default Profile Without Creating One »

July 08, 2008

Prune old transcripts with tkpruner

One of the things we’ve noticed is that since we sync out our loadsets to several servers around campus, we’re wasting time syncing old loadsets that aren’t in use anymore. Even with rsync, the files need to be compared. So, to that end, I wrote a script I call “tkpruner? that walks the config file of a Radmind server, finds the loadsets that are in use, and backs everything else up to ~/transcript/obsolete and ~/file/obsolete. It also backs up unused command files, moving them to ~/command/obsolete. Everything the script backs up is timestamped: ~/transcript/foo/bar/FooBar.T would become ~/transcript/obsolete/FooBar.20080708.T if I backed it up today.

Using this script, we can automate the archival process for old transcripts. Our obsolete folder isn’t included in syncs, and it’s located on a RAID array, so we aren’t concerned about data loss, merely with ensuring that our satellite servers have the transcripts that are in use.

Here’s the script:

#!/bin/bash

##
# Walk the config file and move unused transcripts to ~/transcript/obsolete
##

# Declarations
RadmindFolder="/var/radmind"
configfile="${RadmindFolder}/config"
ExcludeFile="${RadmindFolder}/etc/tpruner.exclude"
TranscriptFolder="${RadmindFolder}/transcript"
CommandFolder="${RadmindFolder}/command"
ObsoleteTFolder="${TranscriptFolder}/obsolete"
ObsoleteKFolder="${CommandFolder}/obsolete"

# Subroutines
processconfig(){
        for x in `cat "$configfile" | grep -v "#" | awk '{ print $2 }'`; do
                isinkarray=0

                for command in ${karray[@]}; do
                        if [ "$command" == "${CommandFolder}/$x" ]; then
                                isinkarray=1
                                continue
                        fi
                done

                if [ $isinkarray == 0 ]; then
                        karray[${#karray[@]}]="${CommandFolder}/$x"
                        processk "${CommandFolder}/$x"
                fi
        done
}

processk(){
        for x in `cat $1 | awk '{ if ($1=="p" || $1=="n") print $2 }'`; do
                isintarray=0

                for transcript in ${tarray[@]}; do
                        if [ "$transcript" == "${TranscriptFolder}/$x" ]; then
                                isintarray=1
                                continue
                        fi
                done

                if [ $isintarray == 0 ]; then
                        tarray[${#tarray[@]}]="${TranscriptFolder}/$x"
                fi

        done

        for x in `cat $1 | awk '{ if ($1=="k") print $2 }'`; do
                isinkarray=0

                for command in ${karray[@]}; do
                        if [ "$command" == "${CommandFolder}/$x" ]; then
                                isinkarray=1
                                continue
                        fi
                done

                if [ $isinkarray == 0 ]; then
                        karray[${#karray[@]}]="${CommandFolder}/$x"
                        processk "${CommandFolder}/$x"
                fi

        done
}

evald(){
        cd $1

        for transcript in `ls | grep \\\\.T`; do
                if [ -f $transcript ]; then
                        evalt "`pwd`/$transcript"
                fi
        done

        for command in `ls | grep \\\\.K`; do
                if [ -f $command ]; then
                        evalk "`pwd`/$command"
                fi
        done

        for file in `ls | grep -v \\\\.T`; do
                if [ -d $file ]; then
                        directory="`pwd`/$file"
                        if [ "`grep $directory "$ExcludeFile"`" == "" ]; then
                                evald "$directory"
                        fi
                fi
        done

        cd ..
}

evalt(){
        isintarray=0

        for transcript in ${tarray[@]}; do
                if [ "$transcript" == "$1" ]; then
                        isintarray=1
                        continue
                fi
        done

        if [ $isintarray == 0 ]; then
                ttomovearray[${#ttomovearray[@]}]="$1"
        fi
}

evalk(){
        isinkarray=0

        for command in ${karray[@]}; do
                if [ "$command" == "$1" ]; then
                        isinkarray=1
                        continue
                fi
        done

        if [ $isinkarray == 0 ]; then
                ktomovearray[${#ktomovearray[@]}]="$1"
        fi
}

movet(){
        Transcript=$1
        TranscriptNoPath="`echo $Transcript | awk 'BEGIN { FS = "/" } { print $NF }'`"
        TranscriptNoEnding="`echo $TranscriptNoPath | sed -e \"s/\.T//\"`"
        Today="`date +%Y%m%d`"
        Target="${ObsoleteTFolder}/${TranscriptNoEnding}.${Today}.T"

        mv $1 $Target

        sourcefiledir="`echo $1 | sed -e \"s/\/var\/radmind\/transcript/\/var\/radmind\/file/\"`"
        targetfiledir="`echo $Target | sed -e \"s/\/var\/radmind\/transcript/\/var\/radmind\/file/\"`/"

        mv $sourcefiledir $targetfiledir
}

movek(){
        Command=$1
        CommandNoPath="`echo $Command | awk 'BEGIN { FS = "/" } { print $NF }'`"
        CommandNoEnding="`echo $CommandNoPath | sed -e \"s/\.K//\"`"
        Today="`date +%Y%m%d`"
        Target="${ObsoleteKFolder}/${CommandNoEnding}.${Today}.K"

        mv $1 $Target
}

# Initiate the array.
processconfig

# Walk the filesystem.
evald "$TranscriptFolder"
evald "$CommandFolder"

echo The following will be backed up:
echo "${#ktomovearray[@]} command file(s)"
echo "${#ttomovearray[@]} transcript(s)"
echo
echo "(R)eview changes, (A)pply changes or (C)ancel?"

read response

if [ "$response" == "R" -o "$response" == "r" ]; then
        echo "Command file(s) to back up:"
        for x in `jot ${#ktomovearray[@]} 0`; do
                echo "  ${ktomovearray[$x]}"
        done

        echo

        echo "Transcript(s) to back up:"
        for x in `jot ${#ttomovearray[@]} 0`; do
                echo "  ${ttomovearray[$x]}"
        done

        echo
        echo "(A)pply changes or (C)ancel?"

        read response
fi


if [ "$response" == "A" -o "$response" == "a" ]; then
        for x in `jot ${#ktomovearray[@]} 0`; do
                movek "${ktomovearray[$x]}"
        done

        for x in `jot ${#ttomovearray[@]} 0`; do
                movet "${ttomovearray[$x]}"
        done
fi

exit 0

Posted by slauncha at July 8, 2008 10:06 AM

Trackback Pings

TrackBack URL for this entry:

Comments

Login to leave a comment. Create a new account.