With the VPS expansion you can now get a (very) small virtual private server (VPS) for a very affordable price. However, when you get a server with something like 256MB or 512MB RAM and a portion of CPU power, using default MySQL/PHP/Apache settings is a pretty bad idea.
Performance and scalability tuning of a server is more of an art than science, in the sense that there're no ready-to-use formulas. Optimal server settings depend on many unique factors: web-app code, traffic to the site, site's information architecture among other things. It's virtually impossible to really optimize server settings without thorough understanding of the web application and a lot of testing.
That said, you are not going to run newsweek.com or huffingtonpost.com on a 256MB slice. Also, the default settings are typically so off that it is possible to give you a much better starting point. I'd like to share with you some settings that have worked well for me. I am assuming you are running a small site or a blog, with the traffic of several thousand page-views/day, using Wordpress, Drupal or something of that kind. I highly recommend getting at least a 512MB VPS, but these settings are better than the default for a 256MB server, as well.
Following are some variables for various settings files that have been modified from their default values.