LOGFILE="logs/$(date +%F)_backup.log"
# Append to the file
restic backup /etc/ >> $LOGFILE
find /root/scripts/ -type f -name "restic*.log" -mtime +14 -delete -print >> $LOGFILE

That's great for small scripts. But sometimes you find that its not logging errors, which are also helpful, and you need to add those:

LOGFILE="logs/$(date +%F)_backup.log"
# Append to the file
restic backup /etc/ 2>&1 >> $LOGFILE
find /root/scripts/ -type f -name "restic*.log" -mtime +14 -delete -print 2>&1 >> $LOGFILE

And different commands behave differently, so it all starts to become complicated. But there's an easier way. Two actually. The first is to use curly brackets to contain all the commands you want to log, and then pipe it through tee at the end. Commands after that block won't be logged.

LOGFILE="logs/$(date +%F)_backup.log"
{
echo "Starting Restic backup at $(date)"
restic backup /etc/
find /root/scripts/ -type f -name "restic*.log" -mtime +14 -delete -print
echo "Restic backup and maintenance completed"
} 2>&1 | tee -a "${LOGFILE}"

And then there's the slightly less readable command substitution method.

exec > >(tee -a "${LOGFILE}") 2>&1
echo "Starting Restic backup at $(date)"
$RESTIC backup /etc/
find /root/scripts/ -type f -name "restic*.log" -mtime +14 -delete -print
echo "Restic backup and maintenance completed"

Comparison

Both methods will log all output of all commands, including errors. I find the curly brackets way slightly easier to read. It also allows you to select which commands are logged, but putting them inside the brackets or not. On the other hand, the exec method will log everything until the end of the script, which may or may not be desirable. Curly braces also seems to be more widely compatible.

There may be other considerations, different performance considerations and buffering for heavy duty scripts, but that's beyond my regular use cases.

Will just add a poor-man's log rotation trick for compleness, and so I can copy and paste it.

# Start logging to rolling logfile
LOGFILE=/path/to.log
KEEPLINES=5000
{
            ### commands
} 2>&1 | tee -a "${LOGFILE}"

# Keep last XXXX lines of rolling logfile, defined above in KEEPLINES
tail -n $KEEPLINES "$LOGFILE" > "$LOGFILE".tmp
mv -f "$LOGFILE".tmp "$LOGFILE"

pm.status_path = /status

in your fpm pool file, then add a block to your nginx config so that it can only be accessed by certain IP addresses.

        location ~ ^/status {
         access_log off;
         allow 127.0.0.1;    # localhost
         allow 12.23.34.45;   # my remote IP
         deny all;  
         fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
         fastcgi_index index.php;
         include fastcgi_params;
         fastcgi_pass unix:/run/php/php-fpm-mysite.sock;
        }

You access the URL mysite.com/status and all is good.

Things get more complicated when you have a server where each domain has its own fpm pool file. The /status line in this case is just referring to that domain. If you access the mydomain.com/status URL from an external browser, the domain name resolves and nginx picks the correct php-fpm pool file, and you see the stats.

Now consider using curl from localhost.

 # On a server where there's only one pool file, this works. But when there are multiple pool files it doesn't know which one to use
curl http://127.0.0.1/status
# Can we resolve it with a Host header? No
 curl -H "Host: mysite.com" http://127.0.0.1/status
# Resolve looks hopeful, but actually it doesn't work with SSL
 curl --resolve 'mysite.com:443:127.0.0.1' https://127.0.0.1/status
# Maybe --insecure will help? No
 curl --resolve 'mysite.com:443:127.0.0.1' https://127.0.0.1/status --insecure
# Wait, maybe we can connect directly to the socket, and target the website that way! Nope.
curl --unix-socket /var/run/php/php-fpm-mysite.sock http://status/

After exhausting the Internet's wisdom on this, there didn't seem to be a way to get curl to fetch the information. It was needed locally so that it could be incorporated into a monitoring script, so allowing a remote IP to gather the info was not a desirable solution, although rapidly looking like the only option.

Salvation came in the form of a program called cgi-fcgi which talks pure cgi-ese to the fpm socket. Or something like that. (apt install libfcgi-bin or yum install fcgi). Once installed, we can ask it for the information like this:

SCRIPT_NAME=/status SCRIPT_FILENAME=/status REQUEST_METHOD=GET cgi-fcgi -bind -connect /var/run/php/php-fpm-mysite.sock

So, incorporating that into a script we can monitor our website's fpm usage. I guess this could also be used in telegraf, zabbix and other monitoring solutions. And you can have a different status URL for each site/pool on the server.

So you just take that and append s3: to it in your restic config, don't you? Of course you do. Except you don't. That will just give you a weird client.BucketExists error.

So what it actually wants is this:

s3:eur3.digitaloceanspaces.com/myspace/

Or if you have created a subfolder, then use

s3:eur3.digitaloceanspaces.com/myspace/subfolder

If you're scripting this and using environment variables, your whole thing will look like this:

# Digital Ocean API Key
export AWS_ACCESS_KEY_ID=THESHORTCAPSANDNUMBERSTRING
# Digital Ocean API SECRET
export AWS_SECRET_ACCESS_KEY=Th3LongMix3dC4seString123213jkadfadsfasdadf
export RESTIC_REPOSITORY="s3:eur3.digitaloceanspaces.com/myspace/subfolder"
export RESTIC_PASSWORD="SecretEncryptionPW"

I hope this saved you some time, because I just spent a couple of hours trying to figure it out.

I play guitar, and I'd previously written a small script to act as a guitar tuner. I generated some .ogg files of the correct pitch and then wrote a script to loop through them in sequence. "Pretty good", I thought, "Clever Me."

Then a few days ago I came across this little gem of a script.

for n in E2 A2 D3 G3 B3 E4;do play -n synth 4 pluck $n repeat 2;done

That's it. Sheer brilliance. Of course you'll need to install the sox package first.

Anyway, on getting it back up, I poked around in the logs and found that it was running out of memory, which it really shouldn't do given that it was a fairly low load on the webserver, and only apache, mysql and php were running on it. I looked around the Interwebs and found an excellent resource called Troubleshooting Memory Usage, which gave me some pointers about how to rein in apache and stop it from eating up memory so quickly. Thanks!

Anyway on the page was a script for summarising memory usage, which I thought was a good idea, so i grabbed that one and adapted it to my own purposes. I thought I'd make the results available here, so feel free to grab it and adapt it to your own needs.

#!/bin/bash
# Mem-info.sh. Version 1.5 2008-12-20
# Adapted from various sources around the Internet.
# Requires mutt to be installed to use the email function. 

#################### Functions ####################
function print_help {
	echo "  Usage:"
	echo "    $0 [-a full|summary] -f /tmp/memoryreport.txt"
	echo "    Parameters:"
	echo "    -a              Specify full or summary report"
	echo "    -f              Specify alternate filename "
	echo "    -e <adddr>      Send report to specified email address"             
	echo "    -h | --help     This usage information"
	echo "    Running the script with no parameters writes a summary report to ./output.txt"
}

function summary_report {
	echo "=========================== SUMMARY ============================" > $mem_outputfile
	date >> $mem_outputfile
	echo  >> $mem_outputfile
	echo "=========================== uptime ==============================" >> $mem_outputfile
	uptime >> $mem_outputfile
	echo  >> $mem_outputfile
	echo "========================== free -m ==============================" >> $mem_outputfile
	free -m >> $mem_outputfile
	echo  >> $mem_outputfile
	echo "========================= vmstat 1.5 ============================" >> $mem_outputfile
	vmstat 1 5 >> $mem_outputfile
	echo  >> $mem_outputfile
	echo "================== ps top 20 Processes by CPU ===================" >> $mem_outputfile
	ps -eo user,%mem,%cpu,pid,cmd --sort -%cpu | head -n 20 >> $mem_outputfile
	echo  >> $mem_outputfile

}

function full_report {
	summary_report
	echo "========================= FULL DETAIL ===========================" >> $mem_outputfile
	echo "======================== top raw output =========================" >> $mem_outputfile
	top -b -n 1 >> $mem_outputfile
	echo " " >> $mem_outputfile
	echo "======================= ps auxf raw output ======================" >> $mem_outputfile
	ps auxf --width=200 >> $mem_outputfile
	echo  >> $mem_outputfile
	echo "=========================== end ================================ " >> $mem_outputfile
	echo  >> $mem_outputfile
	echo  >> $mem_outputfile
}

#################### Process Command line ####################

while [ "$1" != "" ]; do
    case $1 in
        -f )           	shift
                                mem_outputfile=$1
                                ;;
        -a )    	shift
				mem_action=$1
                                ;;
        -e )            shift
				email_to=$1
				;;
        -h | --help )           print_help
                                exit
                                ;;
        * )                     print_help
                                exit
				;;
    esac
    shift
done

######################### Set Defaults #########################
if [ -z "${mem_action}" ]; then 
    echo "    No action Specified. Defaulting to summary report"
    mem_action="summary"
fi
if [ -z "${mem_outputfile}" ]; then 
    echo "    No output file specified. Writing to `pwd`/output.txt"
    mem_outputfile="output.txt"
fi

######################### Do Stuff #########################

if [ "$mem_action" = "full" ]; then
		full_report
	else
		summary_report
fi

if [ "$email_to" = "" ]; then
		echo "    No email address specified. Not emailing report."
	else
		mutt -s "Memory Usage Report" $email_to < $mem_outputfile
fi

Or you can download the file mem-info1 if the formatting above is all messed up. Make sure you go to the page I linked to above for excellent advice on interpreting the report.

Ubuntu Hardy 8.04

First of all, you need to edit your /etc/xorg.conf file, for which you'll need root privileges – use sudo.You just need to add one line in the input device section, which is the one in bold below. (Intrepid 8.10, see below)

Section "InputDevice"

        Identifier      "Synaptics Touchpad"

        Driver          "synaptics"

        Option          "SendCoreEvents"        "true"

        Option          "Device"                "/dev/psaux"

        Option          "Protocol"              "auto-dev"

        Option          "HorizEdgeScroll"       "0"

        Option          "SHMConfig"

EndSection

OK, now after you restart X, by logging out and in again, you can turn off the touchpad with

    synclient TouchpadOff=0

And on with

    synclient TouchpadOff=0

I put this together in a script which will look to see if it is on or off, and toggle it to the opposite.

if synclient -l | grep TouchpadOff | grep 1

then

    synclient TouchpadOff=0

else

    synclient TouchpadOff=1

fi

I attached the script to an icon in my toolbar using the Add to Panel > Launcher route. Now I just have to click to toggle it off and on.

Ubuntu 8.10, Intrepid Whatsit

While the script above still works, editing xorg.conf doesn't work any more, as Ubuntu is moving functionality out of that file. So, instead of that we create a file here.

sudo nano /etc/hal/fdi/policy/shmconfig.fdi

and cut and paste the following into it, saving afterwards.

<?xml version="1.0" encoding="ISO-8859-1"?>
<deviceinfo version="0.2">
<device>
<match key="input.x11_driver" string="synaptics">
<merge key="input.x11_options.SHMConfig" type="string">True</merge>
<merge key="input.x11_options.HorizEdgeScroll" type="string">1</merge>
</match>
</device>
</deviceinfo>

This file only works with Synaptics Touchpad driver. If you have another touchpad, the filename may be different. There are hundreds of other options which you can activate in this file, but the SHMConfig line is the one you need to allow the script to work.

OK, now you've done that, the commands and script above should work.

Ubuntu 9.04 Jaunty Jackass

…. it should work unless you upgrade to 9.04 that is. Now neither of those methods work for disabling the touchpad, which is causing me no end of irritation. So now there's a third solution which I discovered after finding the new touchpad control panel under System > Preferences > Mouse > Touchpad. In there there is a switch to turn off the touchpad, so I figured there must be a way to change this setting with gconftool. Here's what  came up with:

gconftool-2 –toggle /desktop/gnome/peripherals/mouse/touchpad_enabled

This works from the commandline, in a script and even typed directly into a launcher button on your menubar for lightning fast toggling. Which is the only way to toggle.

Ubuntu 9.10 Karmic Koala

Why do they have to change this every frigging version. OK, so back to a script into which the following is inserted.

if xinput list-props "SynPS/2 Synaptics TouchPad" | grep "Device Enabled" | grep 1

then

    xinput set-int-prop "SynPS/2 Synaptics TouchPad" "Device Enabled" 8 0

else

     xinput set-int-prop "SynPS/2 Synaptics TouchPad" "Device Enabled" 8 1

fi

Then assign this to a key (in my case F6) using System > Preferences > Keyboard Shortcuts. All seems to work again … until the next upgrade.