If you've ever merged two contact lists or two spreadsheet exports, you know the mess that follows: the same email address three times, the same name copied twice. This tool scans a list, line by line, and removes any repeats, keeping the first one it sees. It's built for exactly that kind of cleanup.
How to use it
- Paste your list in, one item per line.
- Turn on ignore case if 'Email@site.com' and 'email@site.com' should count as the same thing.
- Look at the cleaned list in the output box.
- Check the removed count to see how many duplicates were found.
- Copy the result into your spreadsheet or script.
When it helps
- Merging two mailing lists without ending up with the same email address twice.
- Cleaning a list of usernames before importing them into a database.
- Tidying up a list of URLs before running them through a checker.
- Removing repeated keywords after a few rounds of brainstorming.
- Deduplicating log entries so you only see the unique ones.
Good to know
- Turn on ignore case for anything like emails or usernames, where capitalisation is inconsistent but shouldn't matter.
- Trim trailing spaces before pasting your list, since 'item' and 'item ' with a space at the end count as different lines.
- Sort the result afterward in a spreadsheet if order matters to you, since this tool keeps whatever order the list was originally in.
- If your list has extra formatting mixed in, like bullet points, strip that first, since '1. Apple' and 'Apple' will otherwise be treated as different lines.
- If two entries look identical but aren't matching, check for an invisible character like a stray space, which is a common and easy to miss cause.
Questions
Does it keep the first copy or the last one?
The first one. Everything after that gets removed, and the original order of the list is kept.
Does it remove blank lines too?
Blank lines get treated like any other line, so several blank lines in a row will collapse down to one. Use the whitespace remover first if you want them gone entirely.
Can it handle a really long list?
It runs in your browser, so a list with tens of thousands of lines might take a moment, but there's no hard limit built into the tool.
Does the order of my original list matter?
Yes, the first occurrence of each unique line is the one kept, so sort your list into the order you want first if that matters.
Can this help clean a mailing list before a campaign?
Yes, deduplicating a merged list before sending helps avoid sending the same email twice to one person, which can hurt sender reputation.
Will it catch near duplicates, like a typo in one entry?
No, it only matches exact lines, or lines that match after case is ignored if you turn that on. A typo makes two lines different even if they were meant to be the same.