Here's some sample code, with XXXXs marking what wasn't working for me (in any browser).
Now it turns out the reason these functions didn't work is not that browsers suck and this whole HTML5 is hugely overrated (my initial reaction), but was in fact that the server serving the MP3 files did not support streaming. Of course if would have been nice if somehow the audio object would warn me about this rather than sitting there doing nothing, but you can't have everything.
Anyway, now the question was how do I make my server support streaming? Do I need to sign up for some kind of premium service? Or spend weeks implementing a streaming sever?
Thankfully not. It turns out you 'simply' need to make your server support HTTP Range requests. If your server uses PHP, you're in luck. I've got a complete working example which I can share with you right now.
Note I did not write this code, I found it here and added it to gist for posterity.
Simply add this code to your PHP server, and when you're about to serve an MP3 (or OGG or whatever) call this function.
For reference, I call the function like this (where $fullPath is the absolute path to my MP3 Resource).
smartReadFile($fullPath, basename($fullPath), 'audio/mpeg, audio/x-mpeg, audio/x-mpeg-3, audio/mpeg3');