Outlook 2019 While connecting an IMAP account in "classic" Outlook 2024 I caused a massive duplication of emails on the server (death loop)

[tomka]

Operating system::    Windows 10 (64bit)
Outlook version:     2019, classic 2024
Email type or host:    IMAP

---

INCIDENT: During migration of an IMAP email account to Outlook 2024, a large-scale message duplication occurred on the server, nearly filling the mailbox quota (9.1/9.9 GB).

INFRASTRUCTURE: I have several devices that have access to one mailbox via IMAP:

• My old PC (Outlook 2019)
• My new PC ("classic" Outlook 2024), which I wanted to connect
• My Android Phone (which uses Android's standard Mail app)
• I can also access this mail account directly in the browser (very limited web UI).

TIMELINE:

1. Initial State (March 12, 2025):
   • Inbox: 22,712 messages
   • PST backup available from right before the migration (There seems to be some issues with the file, but after opening it in PST Walker and exporting it again, I can see all the messages inside of it in Outlook without issue.)
   • Normal email operation, all devices (minus the new PC) work together without issue
2. Migration Process (March 13, 2025):
   • Installed Outlook 2024 on a new PC
   • Set up IMAP account
   • Imported PST file from Outlook 2019
   • Switched to Online Mode, started sync
   • Observed "synchronization problems" folder growing rapidly
     • Error codes mentioned frequently in the messages of this folder are [8004DF0B-0-0-322] (for individual messages) and [8004DF0B-0-0-560] (for folders)
   • Sync took forever, cancelled
   • Deleted account, set up fresh account without PST import, sync took forever again, cancelled again
3. Post-Incident State
   • Suddenly discovered: Inbox (server side) now has 33,800 messages (+11,088 duplicates)
   • Duplicates have minor differences (missing new lines, "'" around receiver names, one message marked with "replied to", the other one not)
   • 4 messages show even bigger signs of corruption (mangled headers, wrong dates)
   • After syncing Outlook 2019 the duplication occurred there as well. Had to abort because it was taking forever.

WHAT I TRIED:

• Contacting server provider: No server side back ups available, I'm on my own.
• Accessing the mailbox via Thunderbird over IMAP (in order to have more control than just via WebUI). This took forever to sync but is working to this point. No duplication issues there.
• Installing a deduplication extension in Thunderbird. Not possible because all of them are only compatible with older versions of Thunderbird. → Currently trying this again because I found the last compatible version of Thunderbird as a Portable Apps release.
• Used an Outlook deduplication tool (MAPILab Duplicate Email Remover) to remove duplicates from the Outlook 2019 mailbox (slightly suboptimal solution as some messages were not just duplicated, but also corrupted), tried syncing to the server again, which caused even more duplicates to appear (now 9.2/9.9 GB taken). I'm scared.
• On a (much) smaller mailbox I tried the following clean slate approach semi-successfully:
  • Deleted all older messages from the server including duplicates (possible via Web UI or Thunderbird)
  • Deleted all older messages from my local Outlook 2019 installation
  • Synced both systems → no more sync errors
  • Now I copied all older messages from the PST (which doesn't have duplicates) to the correct Outlook 2019 folder.
  • Synced Outlook 2019 to the server and Thunderbird. No sync issues still.
  • This worked for the Notebooks but my Android phone (even after deleting and readding the IMAP account) still doesn't understand what exactly happened and doesn't show me most of my messages from 2024 and 2025; when I refresh, the latest messages disappear.
  • Another small disadvantage of this approach: Copying messages from the PST to Outlook somehow changes their encoding (?) causing emojis in subject lines to be shown as "??" in Thunderbird and the WebUI.
  • I'm having scruples using this approach on my main account as it would mean I have to delete tons of messages.

If anyone has any idea what I did wrong or what a better way to fix this would be, let me know.

 

[Diane Poremsky]

If anyone has any idea what I did wrong or what a better way to fix this would be, let me know.

Importing mail you exported from the account back into the same account is where you went wrong.

When you are using IMAP you don't need to import mail from the other computer. If you have calendar and contacts in folders named “(This computer only)" they need exported/imported on the new computer, but the mail in the IMAP account is on the server. No need to export it to a PST - other than if you want to have a backup.

If you have post files, you can open them in Outlook but should not import into IMAP accounts.

If you need to move mail from a PST to an IMAP mailbox, move or copy a few hundred at a time and wait for it to sync.

• Suddenly discovered: Inbox (server side) now has 33,800 messages (+11,088 duplicates)
• Duplicates have minor differences (missing new lines, "'" around receiver names, one message marked with "replied to", the other one not)

Since you have a PST you exported - just delete older messages. Or use the clean up old mail feature to archive old mail to a new pst. Either will remove older mail from the online mailbox. Then you only need to clean up deuplicates in newer mail.

Used an Outlook deduplication tool (MAPILab Duplicate Email Remover) to remove duplicates from the Outlook 2019 mailbox (slightly suboptimal solution as some messages were not just duplicated, but also corrupted), tried syncing to the server again, which caused even more duplicates to appear (now 9.2/9.9 GB taken). I'm scared.

I don't know why it would cause more dupes - did you have the dupes it found going into a folder in the mailbox? In this case, you'd want them in a pst. It's been a while since I used that app - maybe it can't move the dupes to a pst.

But... because the imported messages have differences from the original, dup removers may miss a lot.

Accessing the mailbox via Thunderbird over IMAP (in order to have more control than just via WebUI). This took forever to sync but is working to this point. No duplication issues there.

If the messages are online and in outlook, they should be in Thunderbird too, unless it hides them (kinda the way Gmail hides dupes does).

This worked for the Notebooks but my Android phone (even after deleting and readding the IMAP account) still doesn't understand what exactly happened and doesn't show me most of my messages from 2024 and 2025; when I refresh, the latest messages disappear.

Did you remove the account and add it back to the android?

 

[Diane Poremsky]

BTW - eM client has a dedupe function (its on the hamburg > tools menu). It can move the dupes to a local folder or delete them.

 

[tomka]

Hi Diane,

First let me say thank you not just for your quick response here but also for your work in general. I have profited quite a few times from your published work when I was dealing with some annoying Outlook issues. So thank you!

Importing mail you exported from the account back into the same account is where you went wrong.

I see. That makes sense. I was just surprised that Outlook didn't realize by itself that the messages were duplicates. (I imported them to the fresh account when it contained no messages at all, before syncing.)

When you are using IMAP you don't need to import mail from the other computer. If you have calendar and contacts in folders named “(This computer only)" they need exported/imported on the new computer, but the mail in the IMAP account is on the server. No need to export it to a PST - other than if you want to have a backup.

I guess I both under- (regarding the ability to just sync 5 GB of emails) and overestimated (regarding dupe detection) IMAP and Outlook at the same time.

Since you have a PST you exported - just delete older messages. Or use the clean up old mail feature to archive old mail to a new pst. Either will remove older mail from the online mailbox. Then you only need to clean up deuplicates in newer mail.

You're right. I was just keeping everything in the same folder for convenience while searching. So far this was not an issue, storage was only half full. I guess after removing the dupes and old messages I could go back to that even.

I don't know why it would cause more dupes - did you have the dupes it found going into a folder in the mailbox? In this case, you'd want them in a pst. It's been a while since I used that app - maybe it can't move the dupes to a pst.

I mean the duplicates were supposed to be deleted, but they were moved to "Deleted elements", so maybe that was it?

The DER-software could also move them to a PST, I just checked.

When I was discussing this whole issue with Claude 3.5 Sonnet it also mentioned that "IMAP UID validity" could have become corrupted. Maybe that would explain why also my Android mail client has issues and tends to show messages from years ago at the top? It started doing this first when I started syncing Outlook.

If the messages are online and in outlook, they should be in Thunderbird too, unless it hides them (kinda the way Gmail hides dupes does).

The dupes are present and visible in Thunderbird, it just doesn't cause any more dupes to appear when I sync it.

Did you remove the account and add it back to the android?

I did that, but the issue persists. Maybe I should delete the user data of the mail app and clear its cache before readding to be sure.

My current plan (for Outlook):

• Remove both copies of dupes from the mailbox completely (via Thunderbird)
• Sync
• Just for those emails copy a good record from the PST file back to the account (via Outlook)
• Sync and pray

Advantages (over just removing dupes via Thunderbird):

Since there is no easy way to select which of the duplicate emails is the good one (see below), automatically removing one of the version would risk leaving the bad one only in my mailbox.

Examples of corruption:

• Some of the duplicates are marked with "replied to", others not.
• Some of them contain more new lines than the other version.
• In some of them the text is even enforced to be a certain width (!), which wasn't the case originally (according to the PST), so for example this:

Lorem ipsum dolor sit amet consectetur adipisici elit, sed eiusmod tempor incidunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquid ex ea commodi consequat.

... became this in the duplicate version:

Lorem ipsum dolor sit amet consectetur adipisici elit, sed
eiusmod tempor incidunt ut labore et dolore magna aliqua.
Ut enim ad minim veniam, quis nostrud exercitation ullamco
laboris nisi ut aliquid ex ea commodi consequat.

Btw, if you have any idea what could have caused this last phenomen, I would be really interested to hear it.

Disadvantages (compared to the original state of the mailbox, RIP):

Emails retrieved from the PST file will have their encoding (?) changed in a way that won't be understood by non-Outlook clients. For example this subject line in Outlook:

Will look like this in Thunderbird (and also in Webmail), the emojis are replaced by "?? ??":

NB: When I send an email with the same subject line directly to the mailbox, everything looks correct, even in Thunderbird. This only happens with emails I copy from the PST to Outlook and then sync. Weird!

 

[tomka]

Update for the smaller mailbox I mentioned above:

I found the reason why my Android phone (GMail-App) showed older messages first: It's because it sorts emails not by Date but by UID. Those UIDs, however, became out of order when I imported from the PST, so some older messages now have a higher UID than some younger messages. Compare the Date and the Message UIDs in the following table (screenshot from an old freeware called imapSize):

To fix this mess I did the following:

1. Moved all messages in Inbox to another folder.
2. Synced with the server.
3. Sorted them in ascending order (oldest message at the top).
4. Selected all, copied them back to Inbox
5. Synced again

Now everything is in order with the newest messages having the highest Message UID (as they should):

 

[Diane Poremsky]

I mean the duplicates were supposed to be deleted, but they were moved to "Deleted elements", so maybe that was it?

Yes... that would do it. If you are worried they might not all be dupes, move or archive them to a PST file, otherwise, delete the folder then empty deleted items.

When I was discussing this whole issue with Claude 3.5 Sonnet it also mentioned that "IMAP UID validity" could have become corrupted. Maybe that would explain why also my Android mail client has issues and tends to show messages from years ago at the top? It started doing this first when I started syncing Outlook.

It's possible it is an issue with the UID. Especially if it started after you tried to import the pst.

The dupes are present and visible in Thunderbird, it just doesn't cause any more dupes to appear when I sync it.

That is because you didn't import a PST into it.

Will look like this in Thunderbird (and also in Webmail), the emojis are replaced by "?? ??":

Definitely something changed the encoding. Best guess: the export / import process.

I found the reason why my Android phone (GMail-App) showed older messages first: It's because it sorts emails not by Date but by UID. Those UIDs, however, became out of order when I imported from the PST, so some older messages now have a higher UID than some younger messages. Compare the Date and the Message UIDs in the following table (screenshot from an old freeware called imapSize):

It doesn't have a sort by date option? But yeah, the import will change the UID - every thing imported will have newer UID. Too bad you can't view that in Outlook and delete the mail with consecutive numbers.