This always was a pain. The traditional way to do it was to write it in your ~/.Xdefaults as

Emacs*font: [some XLFD font name or alias]

So it turns out there’s a much easier way. In your .emacs, just say

(set-default-font "somefont")

Where somefont can even be an alias. Its much easier, and you can also change it at runtime. Now I just wait until it supports gtk2 and some kind of Xft or cairo.