# Arabic sorting that stays correct in production

A customer list sorted with `array.sort()` may look harmless in staging. In production, the same shortcut can scatter Arabic names by code point, put "محمود 10" before "محمود 2", disagree with the database, and make the next page repeat or skip records.

The fix is not an Arabic alphabet array. It is a small set of explicit policies.

[Unicode's collation algorithm](https://www.unicode.org/reports/tr10/#Scope) compares text through weighted levels and allows locale-specific tailoring. [CLDR supplies the locale data and settings](https://www.unicode.org/reports/tr35/tr35-collation.html#Setting_Options) used by implementations such as ICU. That machinery is designed for ordering human language. It should not quietly become the definition of account identity, uniqueness, search relevance and API pagination too.

## Start by naming the operation

"Compare these two strings" is incomplete. The product has to say why it is comparing them.

| Operation | Product question | Sensible owner |
|---|---|---|
| Display sorting | In what order should a user see these labels? | Locale-aware collator |
| Equality | Should these values count as the same for this feature? | Feature-specific rule |
| Search | Is this candidate a useful match for the query? | Search pipeline or search collator |
| Database constraint | May both values exist? | Schema and business identity policy |
| Cursor pagination | Which record comes next, without ambiguity? | Database order plus a unique tie-breaker |

This separation matters because collation can judge distinct strings equal at the selected strength. It can also change as language data and implementations change. Unicode is explicit that [collation order is not fixed](https://www.unicode.org/reports/tr10/#Common_Misperceptions), and that stable sorting is a property of the sorting algorithm, not the comparison mechanism.

![Five production text operations mapped to separate policies: display sort, equality, search, constraints and cursor pagination.](https://cdn.hashnode.com/uploads/covers/60ecf4a0fc37a15ec15655e8/7c1936cc-8e3b-4a30-9c65-45671c358a0d.png)

*One text field can participate in five operations. Give each operation its own contract instead of inheriting one global collation. Sources: [Unicode UTS #10 v17.0.0](https://www.unicode.org/reports/tr10/), [CLDR v48.2](https://www.unicode.org/reports/tr35/tr35-collation.html), [ECMA-402](https://tc39.es/ecma402/#collator-objects), [PostgreSQL 18](https://www.postgresql.org/docs/18/collation.html), [MySQL 8.4](https://dev.mysql.com/doc/refman/8.4/en/charset-unicode-sets.html) and the [ICU User Guide](https://unicode-org.github.io/icu/userguide/collation/). Graphic: SultanByte editorial artwork.*

## Sort Arabic labels in the interface

JavaScript already exposes the right primitive. `Intl.Collator` performs language-sensitive comparison, and its `compare` function can be passed to `sort`. [MDN documents the API and its locale-dependent results](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/Collator).

```js
const rows = [
  { id: 5, name: "محمود 10" },
  { id: 2, name: "أحمد" },
  { id: 4, name: "محمود 2" },
  { id: 1, name: "إبراهيم" },
  { id: 3, name: "محمد" },
];

const collator = new Intl.Collator("ar", {
  usage: "sort",
  sensitivity: "variant",
  numeric: true,
});

const sorted = [...rows].sort(
  (a, b) => collator.compare(a.name, b.name) || a.id - b.id,
);
```

On Node.js 22.23.1 with its installed internationalization data, that produced:

```text
إبراهيم
أحمد
محمد
محمود 2
محمود 10
```

The important parts are the explicit locale, `usage: "sort"`, and the final comparison by `id`. Numeric collation treats a run of decimal digits by numeric value, which is why 2 precedes 10; [CLDR defines that behavior at the primary comparison level](https://www.unicode.org/reports/tr35/tr35-collation.html#table-collation-settings).

Do not snapshot that exact list and call it a universal Arabic order. ECMA-402 says the underlying locale data is implementation-dependent and may vary over time. Inspect `collator.resolvedOptions()` in diagnostics, and run a fixture of product-relevant names against every supported runtime during upgrades.

If the server produces the list, let the server own the order. Re-sorting a page in the browser can make page one look polished while the database still decides page boundaries with a different comparator.

## Search comparison is not sort order

A forgiving match can be useful in a typeahead:

```js
const matcher = new Intl.Collator("ar", {
  usage: "search",
  sensitivity: "base",
});

const sameForThisMatch = matcher.compare("أحمد", "احمد") === 0;
```

The tested runtime returned `true`. That does not make the two strings identical, and it does not justify merging two customer records.

The distinction is built into [ECMA-402](https://tc39.es/ecma402/#sec-initializecollator). It selects different locale data for `usage: "sort"` and `usage: "search"`, and warns that a search collation is only for finding matches because it is not guaranteed to have a particular order. ICU likewise exposes [string search as a higher-level operation](https://unicode-org.github.io/icu/userguide/collation/#overview), separate from comparison and sort-key generation.

Use the search comparator for small in-memory candidate sets. A production search service still needs tokenization, ranking, field weighting and an explicit query contract. Collation alone does not supply relevance.

## Put the database order in the schema

PostgreSQL can apply a collation per column or per operation. With an ICU-enabled build, create a named collation so queries and indexes refer to one reviewed object:

```sql
CREATE COLLATION ar_display (
  provider = icu,
  locale = 'ar-u-kn',
  deterministic = true
);

CREATE TABLE people (
  id bigint PRIMARY KEY,
  display_name text NOT NULL,
  login_handle text COLLATE "C" NOT NULL UNIQUE
);

CREATE INDEX people_name_ar_idx
  ON people ((display_name COLLATE ar_display), id);

SELECT id, display_name
FROM people
ORDER BY display_name COLLATE ar_display, id;
```

This DDL and query were executed on PostgreSQL 18.6 with ICU 78.3. The `kn` Unicode locale key enables numeric ordering. The index uses the same collation expression and the same tie-breaker as the query, which gives the planner an index that matches the requested order once the table is large enough to justify using it.

[PostgreSQL's collation documentation](https://www.postgresql.org/docs/current/collation.html) distinguishes ICU from operating-system `libc` collations and notes that provider version and definition affect stability. Avoid relying on whatever locale happened to be installed when a host was built. Name the provider and locale in migrations, then verify them in each environment.

MySQL encodes some of this policy in collation names. For example, [MySQL documents `utf8mb4_0900_ai_ci` as using UCA 9.0.0 weights](https://dev.mysql.com/doc/refman/8.4/en/charset-unicode-sets.html#charset-unicode-sets-uca). It also documents a behavior change that catches migrations: UCA 9.0 collations use `NO PAD`, while older UCA collations commonly use `PAD SPACE`, so trailing-space comparisons can change. Record the exact MySQL collation, not just `utf8mb4`, in schema reviews and migration tests.

## Keep equality and uniqueness deliberate

PostgreSQL deterministic collations only treat byte-identical strings as equal after the locale comparison. Nondeterministic ICU collations can treat different byte sequences as equal. The documentation shows how lower comparison strengths can ignore selected differences, but also notes [a performance cost and restrictions on some pattern-matching operations](https://www.postgresql.org/docs/current/collation.html#COLLATION-NONDETERMINISTIC).

That behavior can be correct for a particular equality rule. It is dangerous as an accidental default for a unique constraint.

A loose collation may cause a unique index to reject two spellings that the business considers separate. A strict or binary constraint may allow values that the login flow later treats as equivalent. Decide identity first, then encode it. Keep display labels out of authentication identifiers whenever possible, and test the exact pairs that product and support teams care about.

The same warning applies to `GROUP BY`, joins and deduplication. If collation equality is used there, it can change counts and merge buckets. Display order is rarely a safe deduplication policy.

## Make cursor pagination a total order

`ORDER BY display_name COLLATE ar_display` is not enough for a cursor. Several rows can share a name, and some collations can consider different strings equal at the active comparison level. Add an immutable unique key:

```sql
SELECT id, display_name
FROM people
WHERE display_name COLLATE ar_display > $1 COLLATE ar_display
   OR (
     display_name COLLATE ar_display = $1 COLLATE ar_display
     AND id > $2
   )
ORDER BY display_name COLLATE ar_display, id
LIMIT 50;
```

The cursor must carry both the last `display_name` and `id`, plus a version for the cursor format. Sign or authenticate it if clients must not edit it.

A unique tie-breaker prevents ambiguity inside one collation version. It cannot freeze order across an ICU, CLDR, operating-system or database upgrade. [UTS #10 says binary sort-key values are not stable between versions](https://www.unicode.org/reports/tr10/#Non-Goals). Do not put opaque library sort keys into long-lived cursors. Expire cursors across collation migrations, rebuild affected database indexes as required by the database upgrade procedure, and run before-and-after ordering fixtures.

Keyset pagination also does not create a snapshot by itself. If records are inserted or renamed between requests, their position can move. Use a database snapshot or a product-level cutoff when the workflow requires a frozen export rather than a live directory.

## A deployment contract worth keeping

Write the following into the feature specification and migration review:

1. The locale and collation provider used for display order.
2. The options that matter, including numeric ordering, sensitivity and punctuation handling.
3. Separate equality rules for login, deduplication, grouping and search.
4. The exact collation on every relevant database column, expression index and query.
5. A unique final sort key for every paginated order.
6. A fixture containing Arabic names, mixed scripts, digits, punctuation, empty values and duplicate labels.
7. An upgrade plan that records runtime, ICU or UCA versions and invalidates old cursors when ordering can change.

The practical boundary is simple: collators answer language-order questions. Constraints answer identity questions. Search answers relevance questions. Pagination needs a total database order. Once those contracts are separate, Arabic sorting stops being a UI patch and becomes an ordinary, testable part of the data model.

*Cover credit: SultanByte editorial artwork based on the cited Unicode, ECMA-402, ICU, PostgreSQL and MySQL technical sources.*