URL Encoder

What percent-encoding is, and why a URL needs it

A URL is allowed a small alphabet: letters, digits, a handful of marks, and a set of characters reserved to mean something structural. ? starts the query, & separates parameters, = splits a name from its value, # begins the fragment, / divides path segments. Put any of those inside a value and the parser reads them as punctuation, because it has no way to tell they were meant literally.

Percent-encoding is the escape hatch. Each byte that cannot appear literally is written as % followed by its value in hex, so & becomes %26 and the parser sees one ordinary character where a separator would have been. Everything outside ASCII is encoded as its UTF-8 bytes first, which is why an accented letter takes two escapes and an emoji takes four.

The three modes, and when each one is right

• One value for something that goes inside a query string or a path segment: a search term, a campaign name, a redirect target. Escapes & = ? / # and spaces. This is what encodeURIComponent does, and the mode you want nine times out of ten.
• Whole URL for an address that is already assembled and just needs its illegal characters cleaned up. Leaves : / ? # & = intact so the URL keeps working. This is encodeURI, and using it on a single value is the classic mistake: the & inside your value survives and splits your parameter in two.
• Form field for an application/x-www-form-urlencoded body, where a space is written + by convention older than the URL spec. Identical to the first mode otherwise.
• Reading the parameter table when the input contains a query string, the parameters are listed underneath, decoded. It is usually faster than reading the encoded line.

The cases that break a link

A redirect target inside a parameter is where this bites hardest. To send someone to ?next=https://example.com/a?b=1, the whole target has to be encoded as a value, otherwise its own ? and & are read as part of the outer URL and the destination arrives truncated. Encoded properly it becomes https%3A%2F%2Fexample.com%2Fa%3Fb%3D1, which looks unreadable and is exactly right.

Double encoding is the other direction of the same mistake. Encode an already-encoded string and %20 becomes %2520, because the % itself gets escaped. The visible symptom is a value that arrives with literal percent sequences in it, and the cause is almost always a value encoded once by the code and once by the framework.

And the plus sign is a trap with two sides. In a form body, + means space, so the string "C++" submitted raw arrives as "C ". Outside a form body, + means a literal plus, so decoding it as a space corrupts any base64 value that happens to contain one. This is why the mode is a choice here and never a guess.

When it goes wrong

• A parameter arrives cut in half the value contained an unencoded & and the server read it as the start of the next parameter. Encode the value, not the URL.
• Spaces arrive as %20 in the page text something encoded a value that was already encoded, or encoded a value that was then displayed instead of being used.
• %2520 anywhere double encoding, every time. Decode once and check whether the encoding belongs in your code or in the framework, then remove one of them.
• Accents come out as è the bytes were encoded as Latin-1 rather than UTF-8 somewhere in the chain. Modern percent-encoding is UTF-8; anything else needs saying explicitly.
• A fragment never reaches the server by design. Everything after # stays in the browser and is never sent, so encoding it changes nothing about what the server receives.

The neighbouring vocabulary

• Reserved characters : / ? # [ ] @ ! $ & ’ ( ) * +, ; =: the ones with a structural job, which have to be escaped to be used literally.
• Unreserved characters A–Z a–z 0–9 - . _ ~: never need encoding, and encoding them anyway is legal but pointless.
• encodeURI against encodeURIComponent the whole-URL and single-value modes above. The names are JavaScript’s; the distinction is RFC 3986’s.
• Punycode a different mechanism for a different part: non-ASCII domain names become xn-- prefixed ASCII, because the host part cannot carry percent escapes.
• Base64 and base64url encoding for bytes rather than for URLs. Standard Base64 contains + and /, which is why the URL-safe variant swaps them for - and _.