Re: TR35 § 3.1 and keyboards

Wed Nov 21 08:42:39 CST 2018

On 21/11/2018 01:47, Hugh Paterson via CLDR-Users wrote:
> Greetings,
> 
> I am reading over the TR35 documentation [1] for categorizing letters
> as Auxiliary vs. Main for the purposes of designing keyboard 
> layouts.
> 
> I read the following guidance for classifying letters between 
> Auxiliary and Main:
> 
> For a given language, there are a few factors that help for 
> determining whether a character belongs in the auxiliary set, instead
> of the main set:
> 
> * The character is not available on all normal keyboards.
> * It is acceptable to always use spellings that avoid that character.
> 
> 
> So my questions are as follows:
> 
> 1. is "all normal keyboards" supposed to be interpreted with language
> scope such that: "all normal German keyboards" when considering
> German "letters" and the task of determining if they are Auxiliary
> vs. Main?
> 
> 2. I'm not actually working on German. I am working with Eastern Dan,
> a language of the Ivory Coast. If the answer to question #1 is "yes,
> interpret with language scope", then I have a follow on question: Are
> "keyboards" then in the TR35 context understood to be keyboard
> layouts (such as may be switched with software) or are they 
> considered to be physical keyboards? In the Eastern Dan context 
> keyboards are French or English as those are the two types of 
> physical computers which make it into the language using context. 
> However, it seems a bit silly to consider some of the characters in 
> the Eastern Dan orthography as "Auxiliary" just because they don't 
> appear on the set of "all physical keyboards".
> 
> 3. When designing an new keyboard layout, or working with a language
>  which does not have a keyboard (or keyboard layout) how is one 
> advised to approach the distinction between Auxiliary vs. Main?
> 
> 
> 
> [1]: https://www.unicode.org/reports/tr35/tr35-general.html#Character_Elements

I think that the advice to use the so-called normal keyboards as
a shibboleth is likely to be an unfortunate remainder in the spec.
The Information Hub for Linguists [2] does not mention it when
quoting this snippet:

| “The test to see whether or not a letter belongs in the main set
|  is based on whether it is acceptable in your language to always use
|  spellings that avoid that character. For example, English characters
|  do not contain the accented letters that are sometimes seen in words
|  like résumé or naïve, because it is acceptable in common practice to
|  spell those words without the accents.”

We see that the keyboards can safely remain out of consideration when
setting up standard vs auxiliary. Eg the German keyboard you cited has
support for much more than German standard letters, given it has dead
keys for accented letters, let alone that the new standard keyboard
for Germany [3] has support for all European official languages.

The reverse is also true: Designing keyboard layouts based on which
letters are standard and which are auxiliary is at risk of resulting
in suboptimal layouts, at least so far as I can tell based on my
experience. Eg in French we have "œ" and "æ" as mandatory standard
letters, still they may be accessed in a Group 2 defined not as a
label for levels 3 and 4, but as a real group accessed via a dead key
group selector. That is already implemented on the new German standard
multilingual keyboard referred to above, but it may also fit a French
keyboard because it makes for a less disruptive, more streamlined,
less confusing and more respectful keyboard layout that is both
powerful and easy to use. That is achieved by not sticking with
the rule stipulating that all standard national letters must be
in Group 1, and all auxiliary ie foreign letters are relegated
into Group 2. Æ and œ now happen to be conveniently grouped together
with ĳ, ð, ŋ, ɲ, ɛ, ɔ and many more letters; so far I’m happy with
it and am currently working on the documentation.

1. When considering level 1 of the German keyboard shipping with
Windows, we find indeed all German standard lowercase letters, but
the same won’t hold true for French given all circumflex-accented
vowels are accessed through a dead key, while e and i with diaeresis
are typed using another dead key position (or place, so as not to
interfere with part 7 of UTS #35), still they are standard letters,
while doing the same on a German keyboard yields auxiliary letters.
Hence normal keyboards are not a means to safely dispatch letters
between categories.

That also helps answering question 2.

3. The second rule applies:
| “It is acceptable to always use spellings that avoid that character.”
The stress is on *always*. Eg in French it is (still) acceptable to use
spellings avoiding "œ" and "æ", but not always, ie not in all books nor
in all handwriting. It may be inacceptable in other written material
either.

A point that I consider very important is implementability, because it
conditions usability. Marc Durdin advised long ago that we’d be well
advised not to implement on Windows the third level as a Ctrl+Alt key
combo (0x06), whether or not mapped to a single key on the right:

https://blog.keyman.com/2008/06/robust-key-mess/

But given in Windows that is the only CapsLock-sensitive level pair
beside the classic Base and Shift levels, mapping letters on levels 3
and 4 on Windows we risk to run into issues. Using a dead key group
like Karl Pentzlin did for Germany is the way to go.

I’m not qualified to advise on designing keyboards for languages
of Africa, but I remember that a similar experience took place
in Togo as it was discussed on the Unicode Public Mailing List:

https://unicode.org/mail-arch/unicode-ml/y2016-m02/0071.html

Continued:
https://unicode.org/mail-arch/unicode-ml/y2016-m11/0005.html

Hope that helps. Good luck!

Best regards,
Marcel

[2] https://sites.google.com/site/cldr/translation/characters#TOC-Exemplar-Characters
[3] DIN 2137-2