Teletext control codes
kent.b.karlsson at bahnhof.se
Sun Dec 19 05:34:16 CST 2021
> 17 dec. 2021 kl. 01:06 skrev Harriet Riddle via Unicode <unicode at corp.unicode.org>:
> William_J_G Overington via Unicode wrote:
>> In relation to teletext control codes, my opinion is that they need to be encoded separately from the C0 control range. This would ensure that in interchange that none of the teletext control codes is ever misinterpreted as having the basic C0 character meaning.
> What is the "basic C0 character meaning"?
> In fact, the method of declaring when alternative C0 and C1 sets are in use is already explicitly covered by ISO 10646 in section 13.4, which I excerpt as follows (I've added some explanations in hard brackets):
> For other C0 or C1 sets, the final octet F shall be obtained from the International Register of Coded Character Sets. The identifier sequences for these sets shall be
> ESC 02/01 F [i.e. 0x1B 0x21 then a byte from 0x40–7E (or 0x30–3F for a private use C0 set)] identifies a C0 set
> ESC 02/02 F [i.e. 0x1B 0x22 then a byte from 0x40–7E (or 0x30–3F for a private use C1 set)] identifies a C1 set
> If such an escape sequence appears within a code unit sequence conforming to ISO/IEC 2022 [this strictly speaking includes e.g. ISO-8859-2, EUC-JP and ISO-2022-JP but not e.g. Windows-1252, Shift_JIS, EBCDIC or any Unicode encoding—although this specific provision is applicable to any ASCII‑ish encoding], it shall consist only of the sequences of bit combinations as shown above [i.e. each byte value in the escape sequence shall be emitted as a single byte of the specified binary value without transcoding].
> If such an escape sequence appears within a code unit sequence conforming to this document [i.e. in a Unicode string], it shall be padded in accordance with Clause 12 [i.e. in UTF-16 or UTF-32, a whole code unit shall be emitted for each byte in the escape sequence, not just the single byte; this has no actual effect on UTF-8].
> (end of excerpt)
That is a total, utter, complete, one thousand percent non-starter. Not just for Teletext, but for everything. The quoted section really needs to be deleted. It is an absolutely horrible suggestion, and flies in the face of the design of Unicode and current 10646.
>> One possibility is to encode the teletext control characters as a block of 32 code points in plane 14, without closing up the unused points. These characters in plane 14 would be displayable characters and thus not control characters in non-teletext-emulating systems, each displayed as a glyph specified in The Unicode Standard as two small capital letters arranged one above the other, but not overlapping. For example A above G for Alphanumerics Green.
That is also a non-starter. But William is right on one point: The Teletext (Text-TV) control codes do (already!) have a graphical representation. And that is … a SPACE.
> Control "character" is a bit of a wooly term. The term "control code" refers specifically to the category Cc characters (a closed category), which don't have formal names (although they do have formal aliases) and mostly have behaviour defined by higher level protocols rather than by Unicode itself, some of which actually carry instructions for things other than the text renderer (BEL, for example). Category Cf, Zl and Zp characters are format effectors, which are a type of nonprinting character which semantically constitute part of the text itself and affect only how it's displayed by triggering (effecting) a particular format behaviour (line break, RTL override, permitted or forbidden line break, superscript etc). Some Cc characters from particular C0 or C1 sets are format effectors (LF for example), but the Cf/Zl/Zp ones are full-fledged Unicode characters with names and semantics defined by Unicode itself, not a higher level protocol such as ECMA-48 or the aforementioned IR-056.
> The Teletext controls are control codes, in existing systems that incorporate them
I’m not sure what that is supposed to explain. I just got confused, it did not appear to make any sense.
However, there are appropriate ways of dealing with (most) Teletext controls in a Unicode context. We discussed that over a year ago. Hint: take your pick: HTML (can handle most of Teletext, with special fonts; and that is done for many Teletext pages on a daily basis currently, indeed, minute-by-minute basis, to show them in web pages and smart phone apps) or ECMA-48 (needs some extensions to handle all Teletext controls).)
Note sure why I have been listed as one of the authors of https://www.unicode.org/L2/L2021/21235-terminals-supplement.pdf <https://www.unicode.org/L2/L2021/21235-terminals-supplement.pdf>. Yes, I have had comments on an earlier version, and I have made suggestions related to it. But I have had no part in writing that document, I have not even been consulted about it. And as you can see from this email, I strongly oppose some of the things suggested there.
>> Could we discuss this please?
>> William Overington
>> Thursday 16 December 2021
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Unicode