I have a small VPS server which hosts a couple of low traffic websites. I've been battling it for a couple of years, trying to reduce the memory consumption. The VPS has 0.5Gb RAM, and the same sized swap file. Sure, I could upgrade the VPS, but sometimes this geeky battling stuff is half the fun!
I've been using the two obvious Mysql tuning scripts and altering the configuration gradually over the last year or so. If you've ever looked at any of this yourselves you'll know them: mysqltuner.pl and tuning-primer.sh
So the problem was this. I'd start up Mysql and it would eat up all the memory, start swapping, and eventually invoke oom_killer. The first change I made was to disable innodb. I didn't need it on this server, and immediately memory usage of MySQL dropped to about half. Now it would start up fine, and run for a week or two, but eventually it would crash out. It was using about 300Mb of the available 512Mb RAM, but would eventually use up all the swap as well. I set up monit to watch it and restart it when that happened, but that wasn't a great solution, so I started chipping away at the my.cnf file, to reduce memory footprint.