The Remove Duplicates tool takes one text block, splits it into lines, and deletes every line that is an exact repeat of an earlier line, in the browser. The placeholder shows the working example: apple, banana, apple, cherry, banana, date. You press the Remove Duplicates button, and the result block prints the de-duplicated text in a read-only field, with a green line above it reading Removed 2 duplicate(s). Nothing is sent anywhere: the split, the set, and the join all happen on the device, and the Clear button wipes the input, the output, and the count.
What the tool takes and what it gives
The input is a single text area, ten rows tall and resizable, and nothing else is asked. There is a Clear button above it that resets everything, and one action button below it, Remove Duplicates, so the computation is not live: the result only changes when you press it. The output is a read-only text area of the same height, and a copy button sits next to its Result label, but only when the output is not empty. The green count line, Removed N duplicate(s), appears only when N is above zero, so an input with no repeats shows the result and nothing else.
How the de-duplication works
The input is split on line breaks into an array of lines, and the lines are walked in order with a set remembering every line already kept. The first time a line is seen it is added to the set and to the output list; every later time it is seen it is only counted as a duplicate and dropped. The kept lines are joined back together with line breaks, so the output is the input with its repeats carved out, in the original first-occurrence order, never re-sorted. For the placeholder, apple, banana, apple, cherry, banana, date, the result is apple, banana, cherry, date, with 2 removed. The walk is one pass over the lines, so a 100,000-line input with 50,000 unique lines is processed in about 40 milliseconds.
What counts as the same line
Two lines are the same only when every character matches, because the comparison is an exact string test with no normalization. Case counts, so A and a are different lines and both stay. Spaces count, so an a with a trailing space and an a with a leading space are different lines and both stay. Accents count, so café and cafe are different lines and both stay. Blank lines count as lines too: a line that is empty is still a line, and a line that is only a space is a space line, so two space-only lines reduce to one. If your repeats differ by one of these, they are not repeats to this tool, and the count stays at zero.
What happens to line breaks and blank lines
The split only looks for the line-break character, so a file that ends with a line break gets one extra, empty, line at the end, and like any first occurrence of a blank line it is kept: an input of a and b, each on its own line ending with a line break, comes back unchanged, with 0 removed. Two consecutive blank lines reduce to one, and an input that is a single line break removes one line and prints an empty result, which still shows the result block but no copy button. Windows line endings are the sharp edge: the split does not strip the carriage return, so the same text read from a Windows file is a different line from the same text without it, and a copy of the result keeps the carriage returns.
How the removed count is read out
The green line counts only the lines that were dropped, not the lines that were kept. An input of four identical lines, x x x x, removes 3 and keeps 1; an interleaved input, 1 2 1 2, removes 2 and keeps 2; and an input with no exact repeats reads nothing at all, because the line only renders when the count is above zero. The count is the difference between the line count of the input and the line count of the output, nothing more.
Where the text goes
Nowhere off the device. The text area value is read by the page, the split, the set, and the join run in the browser, and no network request is made at any point. There is no history and no stored result: the only state the page holds is the three values, input, output, and count, and Clear resets all three to empty and zero. Copy uses the browser clipboard, so the text leaves the page only if you press copy and paste it somewhere.
What it is not
The tool de-duplicates lines, it does not normalize them: to strip the spaces and blank lines that make near-identical lines look different, use the Whitespace Cleaner first; to put the kept lines in alphabetical order afterwards, the Sort Lines tool does that; and to count how often each word appears instead of each line, the Word Frequency Counter does. For the walkthrough of a text tool in the same vein, see the text reverser guide.