Internationalized email domains: punycode and normalization

An internationalized email domain like bücher.de travels through DNS and SMTP as punycode: xn--bcher-kva.de. To handle these addresses correctly, split the address, convert only the domain to its ASCII form with a UTS 46-compliant library (which also lowercases and Unicode-normalizes it), store and compare that ASCII form, and keep the Unicode form purely for display. Don't touch the local part; it can't be punycoded.

That's the whole recipe. The rest of this post is about why the details matter, because a naive implementation turns one domain into several different strings, or one string into the wrong domain.

Two halves, two sets of rules

An email address is a local part and a domain, and internationalization works completely differently for each.

The domain is handled by IDNA. Each non-ASCII label is converted to an "A-label": the prefix xn-- followed by the Punycode encoding of the label. Punycode, per RFC 3492, "uniquely and reversibly transforms a Unicode string into an ASCII string" and works one label at a time, so пример.рф becomes xn--e1afmkfd.xn--p1ai and bücher.de keeps its plain de. DNS only ever sees A-labels. Every MX lookup, every DoH query, every blocklist match should use them.

The local part has no ASCII encoding at all. A Unicode local part like josé@ is legal only under the SMTPUTF8 extension in RFC 6531, and only if every server on the path supports it. If a receiving server doesn't advertise SMTPUTF8, the RFC says the sender must not send the internationalized address; it can downgrade the message if that's possible, otherwise reject it. For a SaaS signup, the practical question is whether your transactional email provider supports sending to such addresses. If you haven't confirmed that, reject non-ASCII local parts with a clear message rather than accepting addresses you can't email.

So the common, safe policy is: Unicode domains yes (they're plain ASCII once converted), Unicode local parts only when you know your mail stack handles them.

The normalization pipeline

For each address:

1. Trim surrounding whitespace.
2. Split on the last @.
3. Strip a single trailing dot from the domain (bücher.de. is the same domain, and UTS 46 libraries preserve the dot).
4. Run the domain through UTS 46 processing to get the A-label form. This step lowercases, applies Unicode normalization and compatibility mappings, and punycodes each non-ASCII label.
5. Reject the address if conversion fails.
6. Optionally reduce to the registrable domain (mail.bücher.de to xn--bcher-kva.de) for company-level grouping. See eTLD+1 explained.
7. Store the A-label form as the key. Derive the Unicode form from it for display.

UTS 46 is the Unicode standard for this processing, and it's what browsers use for hostnames. In practice you call a library function rather than implement it.

JavaScript

Node's node:url exports domainToASCII, and in any runtime with the URL API, new URL() does the same conversion for hostnames:

import { domainToASCII, domainToUnicode } from 'node:url';

export function normalizeEmailDomain(email) {
  const at = email.trim().lastIndexOf('@');
  if (at < 1) return null;
  const local = email.trim().slice(0, at);
  const host = email.trim().slice(at + 1).replace(/\.$/, '');
  const ascii = domainToASCII(host); // '' when the name is invalid
  if (!ascii) return null;
  return { local, domain: ascii, domainDisplay: domainToUnicode(ascii) };
}

normalizeEmailDomain('Jane@BÜCHER.de');
// { local: 'Jane', domain: 'xn--bcher-kva.de', domainDisplay: 'bücher.de' }

In Workers, Deno or the browser, where node:url may not exist, use new URL('http://' + host).hostname. It throws on invalid hosts, so wrap it in try.

Python

Use the third-party idna package with uts46=True:

import idna

def normalize_email_domain(email: str):
    local, sep, host = email.strip().rpartition("@")
    if not sep or not local:
        return None
    host = host.rstrip(".")
    try:
        ascii_domain = idna.encode(host, uts46=True).decode("ascii")
    except idna.IDNAError:
        return None
    return {"local": local, "domain": ascii_domain, "domain_display": idna.decode(ascii_domain)}

normalize_email_domain("Jane@BÜCHER.de")
# {'local': 'Jane', 'domain': 'xn--bcher-kva.de', 'domain_display': 'bücher.de'}

uts46=True is not optional decoration. Without it, idna.encode("BÜCHER.de") raises InvalidCodepoint on the capital B, because strict IDNA2008 doesn't do case mapping.

Five ways to get it wrong

Each of these was checked against Node 23 and Python's idna 3.x while writing this post.

1. Punycoding without lowercasing

The low-level punycode encoder does exactly what RFC 3492 says and nothing more. Node's deprecated punycode module turns BÜCHER.de into xn--BCHER-2pa.de. The correct A-label is xn--bcher-kva.de. Those are different strings for the same domain, so your dedupe, cache and blocklist all miss. Lowercase first, or better, use a UTS 46 function that does it for you. (Node prints a DEP0040 deprecation warning for punycode anyway.)

2. Skipping Unicode normalization

ü can be one code point (U+00FC) or two (u followed by combining diaeresis U+0308). They render identically. Raw punycode turns the decomposed form of bücher.de into xn--bucher-xyd.de, which is not the same domain. UTS 46 normalizes to NFC first, so domainToASCII gives xn--bcher-kva.de for both. Compatibility characters get the same treatment: the fi ligature in file.com maps to file.com.

3. Using IDNA2003 by accident

Python's built-in "faß.de".encode("idna") returns fass.de. The idna package and Node both return xn--fa-hia.de. The built-in codec implements IDNA2003, which mapped ß to ss; IDNA2008 treats ß as a real letter. Those are two different registrable domains that may belong to different people. The Python docs themselves point you to the idna package if you need IDNA2008. Do that.

4. Trusting anything that starts with xn--

xn--zz.com looks like an A-label but isn't valid Punycode. domainToASCII returns an empty string for it and idna.decode raises. If you accept A-labels as input without round-tripping them, you can store names that no resolver will ever answer for.

5. Displaying the Unicode form without thinking about lookalikes

аpple.com with a Cyrillic а converts to xn--pple-43d.com. Stored as an A-label, it's obviously different from apple.com, which is one more reason to key on A-labels. Displayed as Unicode in an admin panel, it isn't. If staff make trust decisions from a list of signup domains, show the A-label next to the Unicode form whenever a domain contains non-ASCII characters, or at least whenever it mixes scripts.

Where the A-label must be used

• DNS queries. Cloudflare's and Google's DoH JSON endpoints both return HTTP 400 for bücher.de and answer normally for xn--bcher-kva.de. Details in DNS-over-HTTPS for domain checks.
• Blocklist and allowlist matching. Lists are distributed in ASCII. A Unicode lookup never matches.
• Database keys and unique constraints. One canonical spelling per domain.
• Logs. Grep-friendly and unambiguous.

The U-label belongs in exactly one place: text shown to humans, such as "We couldn't find bücher.de" in a form error.

Duplicate accounts and the local part

Since you shouldn't punycode or Unicode-normalize the local part for sending, keep two values: the address as entered (for sending) and a comparison key with a lowercased local part plus the A-label domain (for uniqueness). Provider-specific rules like Gmail's ignored dots go into that comparison key too; Gmail dots, plus addressing and duplicate accounts covers which rules are safe to apply. Shape validation for the local part is in why an email regex is never enough.

What pingutil returns

pingutil accepts Unicode input, lowercases it, converts the hostname to punycode, reduces it to the registrable domain, and returns both forms:

curl -X POST https://pingutil.com/v1/lookup \
  -H "Authorization: Bearer pk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{ "input": "jane@mail.bücher.de", "skip_enrichment": true }'
# "domain": "xn--bcher-kva.de", "domain_unicode": "bücher.de"

Use domain as your key and domain_unicode for display. We'd still recommend running your own UTS 46 normalization before calling any API, ours included, so that the string you log, cache and send is already canonical. Field details are in the API reference.

Checklist: split on the last @, strip the trailing dot, UTS 46 to ASCII, reject on failure, key on the A-label, display the U-label, and leave the local part as typed.