There is a setting in Outlook that will allow you to remove emails from the server after download, or leave them on the server. This is useful when you want to get the same emails to 2 different computers, such as work and home. If they're left on the server, your client may recognize them as new mail and download again.
Do you get the emails multiple times on a simple "send/receive," or do you get a second copy the next time you get new mail? If that's the case, you can change the settings in your account configuration through Outlook.