It stores the last 50 (well, between 25 and 200, depending on what you want) messages. You want more? It'll get it on the fly.
Are you sure you're not confusing POP and IMAP? IMAP is the one that stores messages on the server whereas POP3 is the one that you have to download every message, one at a time. Yes, if someone sends you 30 mails of lolcats in a row, mail 31 won't be viewable until you've downloaded the cats on POP3. On IMAP, it just downloads the headers first.
I have 8GB on my iPhone, 2GB of mails on one account on the server, 3GB of IMAPed Gmail and a few MB on another account. I store about 50MB of mail on the phone at the most. It's probably closer to 5MB. On IMAP, attachments are only downloaded when you click on them.