How to Remove Duplicate Characters from Text Online
Paste your text into a character-dedupe tool, keep the first occurrence of each character, and let it drop every later repeat. The order of the characters that survive stays exactly as it was, so “bookkeeper” becomes “bokepr” — the first b, the first o, the first k, then e, p and r, with the second o, second k, second e and second r removed.
That one sentence covers most cases. The rest of this guide exists because the details decide whether the result is useful or ruined: whether spaces count, whether a capital A and a small a are the same character, whether the tool works on the whole string at once or restarts at every word, and whether you wanted the first occurrence kept or the last one.
Key takeaways
- Character dedupe works on single characters, not words or lines. It is a different job from removing duplicate words.
- Most tools keep the first occurrence and preserve order. If you need the last occurrence instead, say so before you run it.
- Decide on case, spaces and punctuation first. Those three settings change the output more than anything else.
- Word, Excel and most editors have no one-click character-dedupe button. An online text tool or a short script is the honest route.
- Never run it on a password, key or code you have not copied elsewhere first. Deleted characters cannot be guessed back.
What “duplicate characters” actually means
A character is one letter, digit, symbol, space or punctuation mark. Removing duplicate characters means scanning the string from left to right, remembering each character you have already seen, and deleting any character you have seen before. Nothing is sorted, nothing is reordered, and no new characters are added.
Take “Mississippi”. The first M, i, s and p survive. The second i, the extra s characters and the extra p characters go, leaving “Misp”. That looks odd, and it should — the point of this operation is not to produce readable prose. It produces a set of unique characters in their original order, which is useful for cleaning codes, checking character sets, building a quick list of letters used in a string, or stripping accidental repeats from pasted data.
It helps to separate this from its neighbours. Removing duplicate words treats “the the” as one repeated word. Removing duplicate lines treats a whole repeated line as one item. Character dedupe goes smaller: in “hello hello”, word dedupe leaves one “hello”, while character dedupe across the whole string leaves “helo ” — h, e, l, o and the space — because the second “hello” contributes no new character. If that is not what you expected, you picked the wrong unit, not the wrong tool.
Choose the scope before you delete anything
Scope is the setting most people skip, and it is the one that causes the most surprises. Four scopes cover nearly every real use.
| Scope | What it does | Example: “banana split” | Use it when |
|---|---|---|---|
| Whole text | One pass over the entire string | “ban split” | You want the unique character set of the whole input |
| Per word | Restarts at each word | “ban split” | You want each word cleaned on its own (here both scopes coincide; try “book book” to see them split) |
| Per line | Restarts at each line break | Each line deduped separately | You are cleaning a list, one entry per line |
| Consecutive only | Removes only repeats that sit next to each other | “banana split” is unchanged except doubled letters | You only want to fix “helllo” style typing slips |
The per-word example deserves a second look, because it catches people out. With whole-text scope, “book book” becomes “bok ” — b, o, k and a space. With per-word scope, each “book” becomes “bok”, giving “bok bok”. Same input, same button name, different result. If a tool does not say which scope it uses, test it on “book book” before trusting it with real data.
Consecutive-only is the gentlest mode and the one to pick when you are tidying prose rather than analysing a string. It turns “comming” into “coming” and “bookkeeper” into “bokeper” only where letters repeat side by side — which, in “bookkeeper”, is the double o, double k and double e. It leaves later, non-adjacent repeats alone, so ordinary words survive in recognisable form.
How to remove duplicate characters from text online, step by step
You do not need to install anything or create an account for this. A browser-based text tool does the work on the page, and the steps are the same on a phone or a desktop.
- Copy your text and paste it into the tool’s input box. Work on a copy, not the only version of something important.
- Pick the scope: whole text, per word, per line, or consecutive only. Whole text is the usual default.
- Set case handling. Case-sensitive is the safer default: “A” and “a” stay separate. Case-insensitive merges them and keeps whichever form appeared first.
- Decide whether spaces, punctuation and digits count as characters. Most tools count them by default, which is why a result can end with a single space where several words used to be.
- Run the tool, read the output character by character, and only then copy it out. A quick scan catches the common surprise — a missing letter you actually needed.
One practical habit saves more rework than any setting: run the tool on a short sample first. “Bookkeeper Mississippi 112233” is a good test string because it contains doubled letters, a repeated word pattern, digits and a space, and you can see every decision the tool makes in a few seconds.
Case, spaces and punctuation: the three decisions that change everything
Case sensitivity is the first fork. With case-sensitive matching, “AaBb” is already unique — four different characters — and nothing is removed. With case-insensitive matching, the second a and second b are treated as repeats, leaving “AB” (or “ab”, depending on which form the tool keeps). For codes and passwords, case-sensitive is the only sensible choice. For a rough letter inventory, case-insensitive gives a tidier list.
Spaces are characters too, and this is where whole-text dedupe looks strangest. In “the cat sat”, the spaces after “the” and “cat” are repeats of the first space, so whole-text dedupe keeps one space and joins the words in an odd way. If you want readable output, either exclude spaces from dedupe, use per-word scope, or clean the spaces separately afterwards — for example by removing spaces in a second, deliberate pass, rather than letting dedupe do it by accident.
Punctuation behaves the same way. Commas, full stops and hyphens each count once under whole-text scope, so a paragraph’s punctuation collapses to a single comma, a single full stop and so on. If punctuation matters to you, strip it first with a dedicated pass — see removing special characters — or choose a tool that lets you exclude punctuation from the dedupe entirely. Digits follow the same rule: “112233” becomes “123”, which is fine for a character inventory and disastrous if those digits were a quantity, a date or part of an identifier.
Accented letters, emoji and other characters that are not plain ASCII
Most modern tools handle accented letters as distinct characters: “é” is not “e”, and “ñ” is not “n”. That is usually what you want, but it means a case-insensitive pass may still leave both “É” and “é” in place if the tool compares them byte by byte rather than by letter. If your text mixes accented and unaccented forms, check the output rather than assuming they merged.
Emoji need a similar warning. Some emoji are a single character, others are several code points joined together — a base emoji plus a skin-tone modifier, or a family group joined by invisible connectors. A character-level tool may split such a sequence apart and leave a broken symbol behind. If your text contains emoji you care about, dedupe a copy first and inspect the result, or strip the emoji in a separate step before you start.
Doing it in Word, Google Docs, Excel and code editors
There is no single “remove duplicate characters” button in Word or Google Docs, and pretending otherwise would waste your time. Word’s Find and Replace works on patterns, not on a running memory of characters already seen, so it cannot express “delete this letter if it appeared earlier in the string” in one search. The realistic route in a document is to copy the string out to a text tool, dedupe it there, and paste the result back.
Excel is a useful contrast because it does have a distinct-values feature — but it works at the wrong level for this job. Microsoft’s UNIQUE function returns a list of unique values from a range, meaning unique rows or cells, not unique characters inside one cell. Microsoft documents it as returning “a list of unique values in a list or range”, and its syntax is =UNIQUE(array, [by_col], [exactly_once]). That is the right tool when you want one copy of each row in a list; it will not dedupe the letters inside a single cell. For that, you still need a text tool or a formula built around the same keep-first logic, applied character by character, which is more work than the job usually justifies.
In VS Code or Notepad++, the same gap applies. Regular expressions can find doubled characters next to each other — the consecutive-only case — but a general “keep first occurrence anywhere in the string” rule does not fit a single find-and-replace pattern. If you are already comfortable with a scripting language, the shortest honest version is a one-liner. In Python, for example, ”.join(dict.fromkeys(text)) keeps the first occurrence of each character and preserves order, because a dictionary remembers insertion order and drops later repeats. Run it on a copy, print the result, and check it before writing anything back.
What can go wrong
The most common failure is not a bug. It is running whole-text dedupe on prose and being surprised that the prose is gone. Character dedupe destroys readability by design: vowels repeat, consonants repeat, spaces repeat. It is an analysis and cleaning operation, not an editing operation. If you need the text to stay readable, you almost certainly want consecutive-only mode, or a different tool altogether.
The second failure is data loss in identifiers. Product codes, serial numbers, phone numbers, dates and passwords all rely on repeated characters. “112233” becoming “123” is not a cleanup; it is a different number. The same applies to codes where a repeated letter carries meaning, such as a grade or a rating scale. Keep the original somewhere safe until you have checked the output against it.
The third failure is subtler: assuming the tool’s defaults match yours. Defaults vary. One tool keeps the first occurrence, another keeps the last, a third sorts the survivors alphabetically — which destroys the original order entirely. “Bookkeeper” sorted by unique character gives “bekopr”, not “bokepr”. If order matters to you, confirm that the tool preserves it; the word “preserve order” or “keep order” in its description is what you are looking for.
When to keep the duplicates instead
Sometimes the right answer is not to run the tool at all. Keep duplicates when the text must stay readable, when repeats carry meaning (counts, codes, ratings, musical notation, repeated letters in a name), when you are working on the only copy of something, or when a simpler operation would do — removing a doubled letter here and there, for instance, rather than deduping the entire string. A tool that deletes characters should be used the way you would use a pair of scissors: on a copy, with a clear reason, and with the original still in the drawer.
Questions people ask after trying it
What exactly happens to my text?
Each character is checked in order. The first time a character appears, it stays. Every later appearance of that same character is deleted. Nothing is added and nothing is reordered, unless you chose a sorting option.
Does it keep the order of the surviving characters?
Good tools do, and you should check. If a tool sorts the result, the output is a character inventory, not a cleaned version of your string, and it will not read back in any sensible order.
Are capital and small letters treated as the same character?
Only if you choose case-insensitive matching. By default, most tools treat them as different characters, so “Aa” stays “Aa”. Change the setting only when you mean to merge them.
Do spaces and punctuation count?
Yes, unless the tool lets you exclude them. That is why whole-text dedupe on a sentence leaves a single space and a single instance of each punctuation mark. Exclude them, or use per-word scope, if you want the words to stay separated.
Can Excel do this?
Not directly for characters inside a cell. Excel’s UNIQUE function, documented by Microsoft Support, returns unique rows or values from a range. For characters inside one cell, use a text tool or a short script instead.
Will it damage a password or a code?
It can, permanently. Repeated characters in a password or code are part of the value. Never dedupe the only copy of one, and never paste a live password into an online tool.
A short way to decide
Start from the unit: character, word or line. If it is genuinely characters, pick the scope, set case and space handling, and test on a short sample. If the sample looks wrong, change one setting at a time until it looks right, and only then run the full text. That discipline takes about a minute, and it is the difference between a cleaned string and a string you have to reconstruct from memory.
