The Export function is under Tools in the admin menu. That'll just export your content. If you'd rather take a copy of the whole site, you can use a plugin called
All-in-One WP Migration which I've found very handy in the past. Bear in mind that it'll do absolutely everything - users, plugins, themes etc. It works best if the host your moving to already has the same domain name set up, but you can still get it working without.
I don't know if the out of the box export tool caters for this but if your new site has a different domain name altogether, you'll need to sweep through your content and update any URLs from the old to the new one. Fortunately there's a good tool for that as well -
Better Search ReplaceAs for hosting, I can only recommend the
one I used based on quality as they've been the fastest ones I've used, but they're not the cheapest. My crappy
blog sits on their £2.95 a month (plus VAT) package and
ConsoleMAD is on their £8.95 a month (plus VAT) package. Fast, but not cheap.
Could you just have the blog hosted on wordpress.com itself?