Text

A String in Mar is a sequence of Unicode scalar values, and a Char is one of them. That sentence looks like a footnote and is really a design decision, because “how long is this string?” has three defensible answers, and a language that runs on three runtimes has to pick one and hold all three to it.

How long is “🇧🇷”?

Ask three platforms and get three numbers. Go counts code points and says 2. A browser counts UTF-16 units and says 4. Swift counts grapheme clusters — what a person would call one character — and says 1. None is wrong; they are answers to three different questions, and each platform picked one long before your program existed.

For ASCII the three agree, which is exactly what makes the disagreement dangerous. Every accented letter in a Brazilian address book agrees too, and so does every CJK ideograph. The gap opens only at emoji and combining marks, which means a program can be wrong for a year and look right the whole time.

Mar’s answer is the middle one, and the reason is not aesthetic:

A Char was already a scalar. String.toList yields scalars on every runtime; Char.toCode '😀' is 128512 everywhere. Counting anything else would make this false:

String.length s == List.length (String.toList s)

A reader assumes that law without being told. Under UTF-16 or grapheme counting it quietly fails.

Scalars need no table. Grapheme segmentation is an algorithm over Unicode data, and each platform ships its own copy on its own release cycle. Choosing graphemes would make the answer depend on the version of the operating system the program is running on — the same app, two phones, two answers. Scalars are arithmetic over a fixed encoding. There is nothing to version.

And the cost is visible. Counting graphemes in a browser costs about 2,000 nanoseconds on a ten-character string and about 50,000 on a thousand-character one, against roughly ten for the scalar count. String.length is the kind of function that ends up inside a render loop.

The function that answered three ways

Here is what the decision buys, and it is worth seeing concretely. A validation function in Shared — the module the browser and the server both compile:

nameIsShort : String -> Bool
nameIsShort name =
    String.length name <= 3

Call it with a flag emoji. Under platform-native counting the server accepts, the browser blocks, and the phone accepts: one function, three verdicts, and nothing anywhere reports the disagreement. The user sees a form that refuses to submit something the server would have been happy with.

Sharing code between client and server is one of Mar’s central promises. A String whose length depends on which side is asking would quietly cancel it.

Two spellings are two strings

The accent in café can be written two ways: one character that already carries it, or a plain e followed by a combining acute. They render identically. They are different sequences of scalars, and in Mar they are different strings.

This is not a fine point, because Dict and Set are built on == and <:

Dict.fromList [ ( "café", 1 ), ( "cafe" ++ combining, 2 ) ]

Under scalar equality that dictionary has two entries, everywhere. Under a comparison that normalises first it would have one, and Dict.get would hand you the value stored under the other spelling — a dictionary whose contents depend on the platform. Equality is load-bearing enough that it should not be clever.

The cost is real and worth stating plainly: if your app needs those two spellings to match, you have to normalise on purpose. Mar’s position is that an explicit transformation you can see is better than a rule one runtime applies on your behalf and another does not.

Case is not a per-letter operation

Uppercasing looks like it maps each letter to another letter. Unicode disagrees. The upper case of the German ß is two letters. The upper case of the fi ligature is two letters. And lowercasing a Greek capital sigma depends on whether it ends a word — the same letter, two answers, decided by its neighbours.

So String.toUpper applies Unicode’s full mapping and may return a longer string:

String.toUpper "straße"     -- "STRASSE"

A Char holds exactly one scalar and has nowhere to put a two-letter answer. Rather than truncate — which would silently turn ß into S, a different letter — Char.toUpper uses Unicode’s simple mapping and hands the character back unchanged when the full answer would not fit:

Char.toUpper 'ß'            -- 'ß'
String.toUpper "ß"          -- "SS"

Two functions, two jobs, and the type of each one tells you which you are getting. The same split runs through the predicates: Char.isDigit is true for a decimal digit in any script, not only 0 through 9, because Unicode is what defines a digit.

The honest costs

Scalars are not what a person means by “character”. A flag is two, a family emoji is seven, and an e with a combining accent is two. Code that truncates a string for display can cut one of those in half, and Mar does not stop you. A future String.graphemes would answer the human question for the cases that need it, and it would be a separate function with a visible price, rather than a hidden cost on String.length.

Normalisation is missing today too. Text arriving from different sources can carry the same word spelled two ways, and matching it takes a transformation the language does not yet provide.

The design instinct to take away

Mar keeps making the same trade: one answer everywhere, even when a platform offers a more convenient one locally. Numbers gave up floating point to get arithmetic that agrees on three runtimes. Trigonometry gave up each platform’s sin for one shared table. Text gives up the platform’s own idea of a character for the same reason.

The pattern is worth naming, because it is what makes sharing code between a browser, a phone and a server more than a slogan. A shared function is only shared if it answers the same.