On Ubuntu there is an option to create an encrypted directory in your home directory called ~/.Private, which is mounted at ~/Private. To set this up you need to issue two commands:
sudo apt-get install ecryptfs-utils
ecryptfs-setup-private
It asks you for a mount password. Log out and log back in again and everything you drop in the Private directory is encrypted and stored in the .Private directory, so that no-one can access your files if, for example, they log in to the machine in Single user mode, or take the hard disk out. So far so good.
But what happens when you move your encrypted files to a different machine? The instructions on this weren't so clear, so I'm just writing down a step-by-step approach to help others who are unsure.