I’ve been using Dreamhost to host about 10 sites for the past five years, including this one. I have no complaints and many kudos. If you want dead-simple administration of your sites plus ssh and FTP access, they’re really good. They have one-click installs for WordPress, Joomla, MediaWiki, ZenCart, Pligg, Gallery, phpBB, and a bunch of other popular open source tools. Unlimited e-mail accounts for all of your domains, unlimited MySQL databases, the ability to create your own DNS entries, and more. If you click this Dreamhost link they’ll give you $50 off your fees.
If you want root access on a Linux box, your computing needs are greater than simple hosting (multiple servers, dedicated IP addresses, private LAN IPs, etc.), and you know how to get everything going from a bare bones Linux install via ssh, try Linode. Their setup is excellent and you get exactly what you need. If you want a small one-person mail server or a remote Nagios monitoring system it’s yours for $20/month. They support current versions of OpenSUSE, Ubuntu, Debian, Fedora, Arch, CentOS and Gentoo. I’ve used them for a couple of projects over the past several years and I’ve had no problems at all.
If you go this route, I highly recommend learning/using Ansible for configuration management since you can rebuild your servers at will or add more servers with identical configurations in minutes.