Sorry to point out the obvious, but a logical place to start, when evaluating your website’s usability on mobile devices, is to pull out your phone and look at it. Sometimes you’ll get lucky and by chance a non-responsive, non-mobile site will actually be somewhat usable on a phone. Oddly enough these tend to be old-school sites with all the navigation on the left. Check out http://yakyudori.hinotez.com. You can’t see the nice pictures of the food without manipulating the screen, but you can see the navigation, address, business hours and even the quaint non-interactive map. It even looks kind of cool in a 1998 kind of way.
Another example is http://www.spicyhousesandiego.com. In this case, on a phone, you can’t see the whole navigation menu but you can see the address and read thru their enticing food menu without much effort. On the other hand http://www.koonthaikitchen.com looks terrible and is extremely frustrating to use. Which is a shame because it’s a great restaurant.
If you want to conveniently see how your website looks on a variety of screen sizes, check out http://quirktools.com/screenfly
Beyond basic usability the next most important variable to examine is loading speed. A great place to test that is http://www.webpagetest.org. There are other such resources, but this one lets you select a region to test from. My own website is much faster when accessed from Southern California than from Brazil.
The most common reason a website loads slowly on a phone is that the image files are too big, and there are too many of them. I can generally reduce image file sizes a lot with just a tiny loss of image quality. I can also write media queries that remove images completely at small screen sizes. On the homepage of this website, for example, the big banner goes away completely on an iPad or anything smaller. I can also replace a bigger image with a smaller one for the small screens. There is a right way and a wrong way to do this. (hint for developers: use the CSS property display not visibility).