Concise term for non-ASCII Unicode characters

[Sean Leonard]

On 9/29/2015 9:40 AM, Daniel Bünzli wrote:
> I would say there's already enough terminology in the Unicode world to add more to it. This thread already hinted at enough ways of expressing what you'd like, the simplest one being "scalar values greater than U+001F". This is the clearest you can come up with and anybody who has basic knowledge of the Unicode standard
Uh...I think you mean U+007F? :)

Perhaps it's because I'm writing to the Unicode crowd, but honestly 
there are a lot of very intelligent software engineers/standards folks 
who do not have the "basic knowledge of the Unicode standard" that is 
being presumed. They want to focus on other parts of their systems or 
protocols, and when it comes to the "text part", they just hand-wave and 
say "Unicode!" and call it a day. In particular there is a flow-down 
effect where terms from one standards body don't match with another 
standards body, perhaps because they got redefined over time for various 
reasons. The distinction between "characters", "abstract characters", 
"code points", and "scalar values" is not intuitively obvious to people 
without specialized knowledge of text processing issues. The fact that 
(modern implementations of) UTF-8 encoders and decoders are not supposed 
to process the surrogate code points (arbitrarily), for example, is a 
rather advanced topic that presumes knowledge of the interaction between 
UTF-16, UTF-8, what surrogate code points actually are, and the security 
implications of so-doing (UTR-36). Furthermore one has to parse the 
distinction between "well-formed" and "ill-formed".

In the twenty minutes since my last post, I got two different 
responses...and as you pointed out, there are a lot of ways to express 
what one would like. I would prefer one, uniform way (hence, 
"standardized way"). Just surveying the various standards that have 
tried to tackle this distinction with their own organic terminology will 
probably be revealing. Evidence-based should be the yardstick.

Best regards,

Sean