How to remove duplicate lines
- Paste your list into the box, one item per line.
- Set the options. Turn off Case sensitive to treat Apple and apple as the same.
- Choose to keep the First occurrence or the Last occurrence.
- Read the tiles to see how many lines went in, how many came out, and how many duplicates were removed. Then copy or download the result.
How it works
The tool reads your text line by line. For each line it builds a comparison key, which is the line itself, optionally trimmed and optionally lower-cased. The first time it sees a key, it keeps the line. Any later line with the same key is dropped. When you pick Last occurrence, it works from the bottom up, so the final copy stays and the order of your list is still preserved.
Empty lines are treated like any other line, so only one empty line survives. Turn on Remove empty lines to drop them all. Windows and Mac line endings are both understood, and your original line ending style is kept.
Tips
- Cleaning an email list? Turn off Case sensitive and turn on Trim spaces, because addresses are not case sensitive and pasted lists often carry stray spaces.
- To get a tidy alphabetical list after removing duplicates, use sort lines.
- If your data is on one line separated by commas, replace the commas with line breaks first using find and replace.
- Spacing noise inside lines is handled by remove extra spaces.
Good to know
Two lines are duplicates only if they match completely under your options. A line with an extra word is a different line. If your list has near matches, such as the same name spelled two ways, fix the spelling first with find and replace, then run the clean-up. The tool never changes the text of a line, apart from trimming when you ask for it.
Privacy
Your list is processed on your device. Nothing is uploaded, stored on a server, or sent in the page address.