Algorithms for Unicode script detection

Simon Cozens via Unicode unicode at unicode.org
Wed Jul 5 18:43:29 CDT 2017


I want to segment a Unicode text into runs according to their script.
I've had a look through UAX#24 in the hope of finding a standard
algorithm for doing this, but there isn't one specified. The
implementation section gives some good pointers for what to be careful
with (paired punctuation, etc.) but I can't find a step-by-step
algorithm similar to the bidi algorithm or collation algorithm.

Equally, I don't see anything in ICU that segments into script-based
runs. You can get script properties, but that doesn't help you resolve
common characters in the context of a run.

Does anyone know of an open-source algorithm for doing this?


More information about the Unicode mailing list