From wjgo_10009 at btinternet.com  Mon Jan  3 10:28:41 2022
From: wjgo_10009 at btinternet.com (William_J_G Overington)
Date: Mon, 3 Jan 2022 16:28:41 +0000 (GMT)
Subject: A teletext control codes encoding suggestion
Message-ID: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>

http://www.users.globalnet.co.uk/~ngo/a_teletext_control_codes_encoding_suggestion.pdf

William Overington

Monday 3 January 2022



From kent.b.karlsson at bahnhof.se  Tue Jan  4 07:11:26 2022
From: kent.b.karlsson at bahnhof.se (Kent Karlsson)
Date: Tue, 4 Jan 2022 14:11:26 +0100
Subject: A teletext control codes encoding suggestion
In-Reply-To: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
References: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
Message-ID: <B6279E55-4D82-40A8-BEE4-BF33C9221FDC@bahnhof.se>

1) These ?controls? are not ligatures in any way whatsoever.
2) These ?controls? are very much geared towards a particular codepage architecture that is specific to Teletext. Thus making them completely unsuitable for everything else.
3) You are right that they are spacing graphic characters, but usually rendered as SPACE (but in some circumstances as a ?mosaic? character).

While it is not the worst suggestion in relation to how to handle these (far from), it is still not an appropriate approach in a Unicode context (why? see the three points above).

And? It is strange that you refer (only) to a specification from 1976. The most recent specifications are:

Enhanced Teletext specification, ETSI EN 300 706 V1.2.1, https://www.etsi.org/ <https://www.etsi.org/>deliver/etsi_en/300700_300799/300706/01.02.01_60/en_300706v010201p.pdf, 2003.
Digital Video Broadcasting (DVB); Specification for conveying ITU-R System B Teletext in DVB bitstreams, ETSI EN 300 472 V1.4.1, https://www.etsi.org/deliver/etsi_en/300400_300499/300472/01.04.01_60/en_300472v010401p.pdf <https://www.etsi.org/deliver/etsi_en/300400_300499/300472/01.04.01_60/en_300472v010401p.pdf>, 2017.

These are the standards for Teletext implemented in all currently produced, and all produced for several years (decades) back, TV sets and TV ?boxes?. The specifications go through hoops to be backwards compatible with the original Teletext, only providing ?enhancements? (and having several ?presentation levels?). The enhancements include: more colours and support for bold, italic and proportional font (via non-spacing out-of-line attributes).

It is true that using Teletext for news services is rapidly declining (though not quite dead yet). But it is still used for optional subtitling for accessibility reasons (subtitling is using the Box start and Box end controls for the subtitle lines, plus different colours for participants in a dialogue), despite that DVB provides another way of doing optional subtitles (image based, so that any fonts used need only be on the broadcaster side, not in the TV set).

/Kent K


> 3 jan. 2022 kl. 17:28 skrev William_J_G Overington via Unicode <unicode at corp.unicode.org>:
> 
> http://www.users.globalnet.co.uk/~ngo/a_teletext_control_codes_encoding_suggestion.pdf
> 
> William Overington
> 
> Monday 3 January 2022
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220104/941e49a5/attachment.htm>

From beckiergb at gmail.com  Tue Jan  4 13:07:00 2022
From: beckiergb at gmail.com (Rebecca Bettencourt)
Date: Tue, 4 Jan 2022 11:07:00 -0800
Subject: A teletext control codes encoding suggestion
In-Reply-To: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
References: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
Message-ID: <CAH=y87bn6dAF361SWvSE4ZUh9XfyCPvWrV9hX7gatDzTKsUEgg@mail.gmail.com>

Can we please add teletext control characters to the list of topics William
Overington is not allowed to discuss on this list?

He has shown he knows nothing about Unicode or teletext, and has caused us
a great amount of undeserved grief over it.

-- Rebecca Bettencourt


On Mon, Jan 3, 2022 at 9:56 AM William_J_G Overington via Unicode <
unicode at corp.unicode.org> wrote:

>
> http://www.users.globalnet.co.uk/~ngo/a_teletext_control_codes_encoding_suggestion.pdf
>
> William Overington
>
> Monday 3 January 2022
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220104/5a658d8e/attachment.htm>

From mark at kli.org  Tue Jan  4 13:14:27 2022
From: mark at kli.org (Mark E. Shoulson)
Date: Tue, 4 Jan 2022 14:14:27 -0500
Subject: A teletext control codes encoding suggestion
In-Reply-To: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
References: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
Message-ID: <26800eec-809b-8318-b594-198b69cd1ca8@shoulson.com>

On 1/3/22 11:28, William_J_G Overington via Unicode wrote:
> http://www.users.globalnet.co.uk/~ngo/a_teletext_control_codes_encoding_suggestion.pdf 
>
>
I guess it's a protocol for transmitting and representing control codes, 
but it certainly isn't plain text (in plain text, "~AG" represents the 
characters "~AG".)? Nothing wrong with that, there are lots of 
non-plain-text protocols and formats.? But we don't regulate them with 
Unicode.? Go and get people who interchange this information interested 
in using your protocol, good luck. But not here.

~mark



From sosipiuk at gmail.com  Tue Jan  4 16:12:32 2022
From: sosipiuk at gmail.com (=?utf-8?Q?S=C5=82awomir_Osipiuk?=)
Date: Tue, 4 Jan 2022 17:12:32 -0500
Subject: A teletext control codes encoding suggestion
In-Reply-To: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
References: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
Message-ID: <000101d801b8$2bf44a70$83dcdf50$@gmail.com>

But what if I want to display a literal tilde followed by capital A and G in my teletext page without changing colour? This proposal has no method to escape the "controls".

If the idea is just "use existing Unicode characters in special combinations to encode functionality" (and a lot of Mr. Overington's ideas seem to revolve around this) then why not use one of the innumerable EXISTING standards which use special combinations of characters? I seem to recall one that uses greater-than and less-than signs to surround key-value pairs, something like <font color="green">.

Or if that's too space-inefficient, Harriet Riddle's idea of using the C1 control space along with the proper ISO-2022 designation is about as compact as it gets, and stays closest to the spirit of control characters.

There's no compelling reason for yet another markup. "Reuse" is not a dirty concept.

S?awomir Osipiuk




From kent.b.karlsson at bahnhof.se  Tue Jan  4 17:59:53 2022
From: kent.b.karlsson at bahnhof.se (Kent Karlsson)
Date: Wed, 5 Jan 2022 00:59:53 +0100
Subject: A teletext control codes encoding suggestion
In-Reply-To: <000101d801b8$2bf44a70$83dcdf50$@gmail.com>
References: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
 <000101d801b8$2bf44a70$83dcdf50$@gmail.com>
Message-ID: <5E28E222-8A5F-4256-8656-A889F6C1D982@bahnhof.se>

(A bit long reply, sorry for that.)

> 4 jan. 2022 kl. 23:12 skrev S?awomir Osipiuk via Unicode <unicode at corp.unicode.org>:
> 
> But what if I want to display a literal tilde followed by capital A and G in my teletext page without changing colour? This proposal has no method to escape the "controls".
> 
> If the idea is just "use existing Unicode characters in special combinations to encode functionality" (and a lot of Mr. Overington's ideas seem to revolve around this) then why not use one of the innumerable EXISTING standards which use special combinations of characters? I seem to recall one that uses greater-than and less-than signs to surround key-value pairs, something like <font color="green?>.

HTML/CSS is possible. But a bit heavy-handed. Not everything has to be HTML (or XML), however popular it is.

HTML has been and is used for presenting Teletext pages on the web. Some used to use styled text, but now generating an image seems to have taken over, and that image is embedded in HTML? Quite heavy-handed (except for browsers), and not searchable or indexable unless the text in the image is also given as text (which does not seem to be the case). And certainly not suitable for archiving Teletext pages.

> Or if that's too space-inefficient, Harriet Riddle's idea of using the C1 control space along with the proper ISO-2022 designation is about as compact as it gets, and stays closest to the spirit of control characters.

That is out of the question, I?d say, and a very bad idea, much worse than what William has proposed (there has been several proposals).

However, CSI 32m is the control sequence for foreground ?green? in ECMA-48 (ISO/IEC 6429) encoding. By popular extension, CSI 92m is the control sequence for foreground ?bright green?. This is implemented in just about all terminal emulators, but ECMA-48 is not at all limited to terminal emulators. See https://en.wikipedia.org/wiki/ANSI_escape_code.

So there is already a good candidate ?styling? (and other things) standard for handling this, and its SGR (Set Graphics Rendition) control sequences already cover a lot of the Teletext styling (functionality-wise), though some additions needed to cover all.

> There's no compelling reason for yet another markup. "Reuse" is not a dirty concept.

I agree. But one does need a few extensions to ECMA-48 styling to cover the functionality of Teletext styling. Most importantly for Box start and Box end (which are used for subtitling). That can be done by extending SGR (in ECMA-48) to handle also Box start/end, plus a few other extensions to cover all of Teletext styling. (Yes, I have looked into this in detail.)

One also needs to separate the functionalities of the Teletext ?control? codes. For most of them, they each serve three functions:

1) A graphic character, usually SPACE (but sometimes a ?mosaic? character); rendered before or after the colour (or other) change, as specified in the Teletext standard. That must be handled by the converter (to Unicode plus ECMA-48 (ISO/IEC 6429)).

2) Code page switching specific to Teletext, and need be converted away.

3) Most of them change foreground colour (and one changes background colour); the ECMA-48 control sequences for colour change are non-spacing (cmp. point 1).

In addition, as I have mentioned multiple times, the Teletext protocol allows for more colours and styling to underline, bold, italic or proportional font ?out-of-band? of the text (there is a start index when giving such attributes). No surprise, ECMA-48 can support all of that (in-band), and several of them (except proportional font) commonly implemented in terminal emulators (but ECMA-48, ISO/IEC 6429, itself is not limited to terminal emulators).

Teletext has never embraced any part of ECMA-48, which is a little bit of a pity, and instead goes through hoops to be fully backwards compatible with the original Teletext design, which is highly old-fashioned, in particular the codepage setup, clashing royally with such things as Unicode/10646, or indeed with anything that is not 7-bit with special code pages. On the other hand, ECMA-48 works well also with Unicode. And? may also be suitable for archiving of Teletext pages, if one wants to get out of the 7-bit codepage switching approach (which indeed makes searches and indexing hard).

/Kent K

> S?awomir Osipiuk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220105/77e3c3b3/attachment.htm>

From sosipiuk at gmail.com  Tue Jan  4 18:20:01 2022
From: sosipiuk at gmail.com (=?UTF-8?Q?S=C5=82awomir_Osipiuk?=)
Date: Tue, 4 Jan 2022 19:20:01 -0500
Subject: A teletext control codes encoding suggestion
In-Reply-To: <5E28E222-8A5F-4256-8656-A889F6C1D982@bahnhof.se>
References: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
 <000101d801b8$2bf44a70$83dcdf50$@gmail.com>
 <5E28E222-8A5F-4256-8656-A889F6C1D982@bahnhof.se>
Message-ID: <CAM+ijLgosLMmym1UoOnTRv5stBzGy7L=MZ5fehA=Fdksz-iWwQ@mail.gmail.com>

On Tue, Jan 4, 2022 at 6:59 PM Kent Karlsson <kent.b.karlsson at bahnhof.se> wrote:
>
> That is out of the question, I?d say, and a very bad idea, much worse than what William has proposed (there has been several proposals).

You've repeated this, but still without explanation. I don't
understand what's so horrible about the C1 control approach.

The code points for C0 and C1 controls are (with a handful of
exceptions) specifically reserved in Unicode to allow systems to
assign meaning to them. ISO 2022 provides a standard way of
designating that meaning, ISO 10646 gives a brief nod to it, and a set
of controls for teletext is already registered and available for use.
As Harriet Riddle said, the solution already exists. What are the
negatives?


From beckiergb at gmail.com  Tue Jan  4 18:42:29 2022
From: beckiergb at gmail.com (Rebecca Bettencourt)
Date: Tue, 4 Jan 2022 16:42:29 -0800
Subject: A teletext control codes encoding suggestion
In-Reply-To: <CAM+ijLgosLMmym1UoOnTRv5stBzGy7L=MZ5fehA=Fdksz-iWwQ@mail.gmail.com>
References: <73a6e73d.3ea43.17e20c581c2.Webtop.96@btinternet.com>
 <000101d801b8$2bf44a70$83dcdf50$@gmail.com>
 <5E28E222-8A5F-4256-8656-A889F6C1D982@bahnhof.se>
 <CAM+ijLgosLMmym1UoOnTRv5stBzGy7L=MZ5fehA=Fdksz-iWwQ@mail.gmail.com>
Message-ID: <CAH=y87Z5EeNA8N+wQKptcV+0hxASTq6y6pjrzoJ7mSk4NpX4Xg@mail.gmail.com>

Shall I quote the very standard for which this list was created?

23.1 Control Codes

There are 65 code points set aside in the Unicode Standard for
compatibility with the C0 and C1 control codes defined in the ISO/IEC 2022
framework. The ranges of these code points are U+0000..U+001F, U+007F, and
U+0080..U+009F, which correspond to the 8- bit controls 0016 to 1F16 (C0
controls), 7F16 (delete), and 8016 to 9F16 (C1 controls), respectively. For
example, the 8-bit legacy control code character tabulation (or tab) is the
byte value 0916; the Unicode Standard encodes the corresponding control
code at U+0009. The Unicode Standard provides for the intact interchange of
these code points, *neither adding to nor subtracting from their semantics*.
The semantics of the control codes are generally *determined by the
application with which they are used*. However, in the absence of specific
application uses, they *may be* interpreted according to the control
function semantics specified in ISO/IEC 6429:1992. In general, *the use of
control codes constitutes a higher-level protocol and is beyond the scope
of the Unicode Standard*. For example, the use of ISO/IEC 6429 control
sequences for controlling bidirectional formatting would be a legitimate
higher-level protocol layered on top of the plain text of the Unicode
Standard. *Higher-level protocols are not specified by the Unicode Standard*;
their existence cannot be assumed without a separate agreement between the
parties interchanging such data. [emphasis mine]

In other words, you can use C0 and C1 controls in any way you
want, especially if it's according to another standard such as ISO 6429 or
ISO 2022, or even, *gasp* ETSI 300, also known as teletext! Sorry Kent,
just like your understanding of our proposal is wrong, so is your
understanding of control characters in Unicode.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220104/0242b7f3/attachment.htm>

From wjgo_10009 at btinternet.com  Wed Jan 12 10:53:19 2022
From: wjgo_10009 at btinternet.com (William_J_G Overington)
Date: Wed, 12 Jan 2022 16:53:19 +0000 (GMT)
Subject: The control codes of the 1976 teletext specification are a
 brilliant solution, given the boundary condition
Message-ID: <286a76c9.54ce0.17e4f354c06.Webtop.96@btinternet.com>

In the document

https://www.unicode.org/L2/L2022/22013-c0-c1-stability.pdf

Kent Karlsson writes:

> But there are some character encodings that are ?a bit crazy? when it 
> comes to control codes. They override all of C0 (or C1) with something 
> that cannot even be regarded as pure control characters. In particular 
> Teletext ...

In my opinion, the control codes of the 1976 teletext specification are 
a brilliant solution, given the boundary condition that existed at the 
time.

In order to work, a teletext-equipped television set needed enough solid 
state memory, to which data could be wriiten and from which data could 
be read, to store a whole teletext page.

At the time such solid state memory was expensive, so using two 
kilobytes of memory rather than one kilobyte of memory would have added 
significant cost to each teletext-equipped television set.

So the decision was made to design the specification such that one 
kilobyte of solid state memory would be sufficient to store a complete 
teletext page.

I was told that originally the BBC (British Broadcasting Corporation) 
and the IBA (Independent Broadcasting Authority) had each developed a 
prototype system of a text-based information system of its own and that 
the best features of each were included in the agreed common teletext 
technical specification.

In an era where personal computing was only starting and computers were 
mostly in businesses, universities and polytechnics, and mostly in 
monochrome and just text-based, colourful teletext with its graphics was 
very futuristic and often on view displaying a multipage (a teletext 
page on a fixed page number yet such that there were a number of 
different page displays broadcast in sequence, changing, say, every 
thirty seconds) in the window display of a shop.

William Overington

Wednesday 12 January 2022







From kent.b.karlsson at bahnhof.se  Thu Jan 13 17:25:13 2022
From: kent.b.karlsson at bahnhof.se (Kent Karlsson)
Date: Fri, 14 Jan 2022 00:25:13 +0100
Subject: The control codes of the 1976 teletext specification are a
 brilliant solution, given the boundary condition
In-Reply-To: <286a76c9.54ce0.17e4f354c06.Webtop.96@btinternet.com>
References: <286a76c9.54ce0.17e4f354c06.Webtop.96@btinternet.com>
Message-ID: <8A6620BB-EDB7-4B35-AE92-61D4CD609A99@bahnhof.se>



> 12 jan. 2022 kl. 17:53 skrev William_J_G Overington via Unicode <unicode at corp.unicode.org>:
> 
> In the document
> 
> https://www.unicode.org/L2/L2022/22013-c0-c1-stability.pdf
> 
> Kent Karlsson writes:
> 
>> But there are some character encodings that are ?a bit crazy? when it comes to control codes. They override all of C0 (or C1) with something that cannot even be regarded as pure control characters. In particular Teletext ...
> 
> In my opinion, the control codes of the 1976 teletext specification are a brilliant solution, given the boundary condition that existed at the time.

Key: 1976. Then ?all there was? was 7-bit codepages. (7-bit, not 8-bit; the 8th bit was then commonly used for parity, as it still is in the Teletext protocol.)

> In order to work, a teletext-equipped television set needed enough solid state memory, to which data could be wriiten and from which data could be read, to store a whole teletext page.

Again, 1976. And has been carried on with backwards compatibility until now. However, that is far from making that solution appropriate today, especially not in a Unicode context.

> At the time such solid state memory was expensive, so using two kilobytes of memory rather than one kilobyte of memory would have added significant cost to each teletext-equipped television set.

Sure. 1976.

> So the decision was made to design the specification such that one kilobyte of solid state memory would be sufficient to store a complete teletext page.
> 
> I was told that originally the BBC (British Broadcasting Corporation) and the IBA (Independent Broadcasting Authority) had each developed a prototype system of a text-based information system of its own and that the best features of each were included in the agreed common teletext technical specification.

I don?t know the historical details, but one precursor to Teletext was apparently ?Videotex? (no ?t? at the end?).

> In an era where personal computing was only starting and computers were mostly in businesses, universities and polytechnics, and mostly in monochrome and just text-based, colourful teletext with its graphics was very futuristic and often on view displaying a multipage (a teletext page on a fixed page number yet such that there were a number of different page displays broadcast in sequence, changing, say, every thirty seconds) in the window display of a shop.

That?s fine (1976).

However, that is far from making that solution appropriate today, especially not in a Unicode context. I would agree that not everything need be turned into HTML, however popular it is (HTML/CSS is great, but needlessly heavy-handed for, say, archiving Teletext pages). Though ECMA-48 is also quite old, it still has solutions for colouring and other styling that 1) are compatible with Unicode, 2) is quite popular (to an extent) in all terminal emulators (but ECMA-48 is not limited to terminal emulators), 3) can handle ?later? extension to Teletext w.r.t. colours and styling (though certain extensions to ECMA-48 would be needed for handling conversion of certain Teletext features, esp. subtitling), 4) are viable also outside of the Teletext protocol (the Teletext triple-function ?controls? have no business outside of the Teletext protocol).

Getting back to what I wrote in https://www.unicode.org/L2/L2022/22013-c0-c1-stability.pdf: It is super-highly inappropriate to treat C0/C1 in Unicode as a private-use area, which some have proposed.

/Kent Karlsson

> William Overington
> 
> Wednesday 12 January 2022

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220114/7b907564/attachment.htm>

From jonathan.coxhead at gmail.com  Fri Jan 21 20:29:12 2022
From: jonathan.coxhead at gmail.com (Jonathan Coxhead)
Date: Fri, 21 Jan 2022 18:29:12 -0800
Subject: Repertoire
Message-ID: <CABWUGktf0UAo011dsxzhWDb7t7cmn3KJHK5kA6jv-gGHaABOhg@mail.gmail.com>

Hi Unicodets

   I just updated a web page I once created (now at <
http://twojays.me/unichar-14.0.0.html>), which lists the whole Unicode
repertoire, including all the decompositions, aliases, cross references and
comments. I find this summary page to be the only character reference I
need in my day-to-day life. It's as short as I could make it, though if it
were printed out, it would be about 1000 pages long(!).

   But in updating it, I came upon a problem:

   Some combining characters are clearly "printing characters", for example,

  ? COMBINING ACUTE ACCENT

which can be shown graphically, as above, by displaying it on a space. Some
are control characters, and have no possible visual display, such as

 \u034F COMBINING GRAPHEME JOINER

which can only be shown as a code: it has no printable nature at all.

   Now, in the case of non-combining characters, this distinction is made
very clearly, as it has been all the way back to the days of the C
isprint() and iscntrl() macros. But for combining characters, the
distinction between printable and control seems not to be made. The only
way I could see to do was to special-case the character names

    VARIATION SELECTOR-[0-9]+
    MONGOLIAN FREE VARIATION SELECTOR (ONE|TWO|THREE|FOUR)
    COMBINING GRAPHEME JOINER
    TIFINAGH CONSONANT JOINER
    BRAHMI NUMBER JOINER

which isn't very satisfactory.

   Am I missing something? And if not, should there be something in
UnicodeData.txt that gives me this information?

   I was also wondering idly if anyone has any practical uses for the
legacy computing characters, specifically the ones with "BLOCK DIAGONAL" in
the name. They look tantalisingly as though they must be good for
something, but I don't know what it could be?

   Cheers

?Jonathan Coxhead

?*ballads not bombs, songs not surveillance*? ?Thom Hartmann
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220121/21b89beb/attachment-0001.htm>

From markus.icu at gmail.com  Fri Jan 21 20:40:54 2022
From: markus.icu at gmail.com (Markus Scherer)
Date: Fri, 21 Jan 2022 18:40:54 -0800
Subject: Repertoire
In-Reply-To: <CABWUGktf0UAo011dsxzhWDb7t7cmn3KJHK5kA6jv-gGHaABOhg@mail.gmail.com>
References: <CABWUGktf0UAo011dsxzhWDb7t7cmn3KJHK5kA6jv-gGHaABOhg@mail.gmail.com>
Message-ID: <CAN49p6otStkb8d+ZgYEbyO4G8yMiK2s0kaphfkQt_C54ph3tfQ@mail.gmail.com>

I would show all format controls (gc=Cf
<https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=%5B%3Agc%3DCf%3A%5D&g=&i=>)
with a code. A few might have a non-empty default glyph, but it's probably
not worth getting more precise.

Best regards,
markus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220121/bf7019f2/attachment.htm>

From asmusf at ix.netcom.com  Sat Jan 22 03:27:34 2022
From: asmusf at ix.netcom.com (Asmus Freytag)
Date: Sat, 22 Jan 2022 01:27:34 -0800
Subject: Repertoire
In-Reply-To: <CAN49p6otStkb8d+ZgYEbyO4G8yMiK2s0kaphfkQt_C54ph3tfQ@mail.gmail.com>
References: <CABWUGktf0UAo011dsxzhWDb7t7cmn3KJHK5kA6jv-gGHaABOhg@mail.gmail.com>
 <CAN49p6otStkb8d+ZgYEbyO4G8yMiK2s0kaphfkQt_C54ph3tfQ@mail.gmail.com>
Message-ID: <f506b8e6-efa5-79b3-1f35-12b001791a0d@ix.netcom.com>

An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220122/d2672f5d/attachment.htm>

From gwidion at gmail.com  Sat Jan 22 21:09:27 2022
From: gwidion at gmail.com (Joao S. O. Bueno)
Date: Sun, 23 Jan 2022 00:09:27 -0300
Subject: Repertoire
In-Reply-To: <f506b8e6-efa5-79b3-1f35-12b001791a0d@ix.netcom.com>
References: <CABWUGktf0UAo011dsxzhWDb7t7cmn3KJHK5kA6jv-gGHaABOhg@mail.gmail.com>
 <CAN49p6otStkb8d+ZgYEbyO4G8yMiK2s0kaphfkQt_C54ph3tfQ@mail.gmail.com>
 <f506b8e6-efa5-79b3-1f35-12b001791a0d@ix.netcom.com>
Message-ID: <CAH0mxTSAmb4LhTbBLFQC_tE6UmeEt4HjWBKXn3NxBrH+pXmhEw@mail.gmail.com>

>
>
>    I was also wondering idly if anyone has any practical uses for the
> legacy computing characters, specifically the ones with "BLOCK DIAGONAL" in
> the name. They look tantalisingly as though they must be good for
> something, but I don't know what it could be?


I have those on my roadmap.  :-)
I am author to the "terminedia" framework for Unicode Art and terminal
based applications -
the code enables one to use a drawing API to draw with characters, anyone,
but including
full block (\U2588) as pixels, and then uses 1/4 block characters for "hi
resolution",
with each character block being able to convey 4 pixels. There is a hackish
mode
with braille characters, with 8 pixels/block, and, few months after the
legacy
computer characters were added and generally available, I added support
for the 1/6 block sextant characters as a drawing mode.

The other legacy characters currenntly have no better support in the
project than most emojis - but the "BLOCK DIAGONAL" in specific are
in my roadmap: when I get to implement the "1/2 pixel per block" drawing
mode,
taking two character blocks per "pixel", I will add a transform to fill in
corners
with the appropriate "BLOCK DIAGONAL" characters to make curves
"smooth'.

I use a similar approach for using the line/table drawing characters now
("HEAVY QUADRUPLE DASH *", for example): one draws boxes and lines
with fullblocks, apply a transform, and have the characters replaced by
the apropriate frame drawing chars.

Unfortunately, I am not getting a lot of time to put in the project
over the last few months, and I am currently in the task of
improving the usability of the text-based widgets, so that
the project can be used for real-world apps.

The project is at
https://github.com/jsbueno/terminedia
 the main gap is documentation. It installs along with
some example scripts, and there is a separate project that uses
it, and can be used to draw interactively on the terminal using the above
mentioned "pixel" modes -
https://github.com/jsbueno/terminedia-paint

(installable with `pip3 install --user terminedia-paint`, currently no
Windows support,
and on MacOS it needs a terminal program that supports full color ANSI
codes. )

And finally - yes, I'd also greatly appreciate proper responses for your
main query -
as being able to introspect the full unicode charset and make developers
life
easier when dealing with Unicode is one of the project's missions.
I

On Sat, 22 Jan 2022 at 06:31, Asmus Freytag via Unicode <
unicode at corp.unicode.org> wrote:

> On 1/21/2022 6:40 PM, Markus Scherer via Unicode wrote:
>
> I would show all format controls (gc=Cf
> <https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=%5B%3Agc%3DCf%3A%5D&g=&i=>)
> with a code. A few might have a non-empty default glyph, but it's probably
> not worth getting more precise.
>
> Best regards,
> markus
>
> The code charts use special glyphs for these, for those users who don't
> memorize hex codes.
>
> A./
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220123/b5ae083c/attachment.htm>

From jonathan.coxhead at gmail.com  Sun Jan 23 23:50:46 2022
From: jonathan.coxhead at gmail.com (Jonathan Coxhead)
Date: Sun, 23 Jan 2022 21:50:46 -0800
Subject: Repertoire
In-Reply-To: <CAN49p6otStkb8d+ZgYEbyO4G8yMiK2s0kaphfkQt_C54ph3tfQ@mail.gmail.com>
References: <CABWUGktf0UAo011dsxzhWDb7t7cmn3KJHK5kA6jv-gGHaABOhg@mail.gmail.com>
 <CAN49p6otStkb8d+ZgYEbyO4G8yMiK2s0kaphfkQt_C54ph3tfQ@mail.gmail.com>
Message-ID: <CABWUGksWa-MwO0emmzvLd=Gw5gerz0=6vho=JRsmisBtrzHNdQ@mail.gmail.com>

   I show all control characters (category Cx) and all space characters
(category Zx) with a code. This is necessary for a coherent layout, if
nothing else. The issue is that some non-spacing marks (category Mx) also
need to be shown with a code, namely

    VARIATION SELECTOR-[0-9]+
    MONGOLIAN FREE VARIATION SELECTOR (ONE|TWO|THREE|FOUR)
    COMBINING GRAPHEME JOINER
    TIFINAGH CONSONANT JOINER
    BRAHMI NUMBER JOINER

but there is nothing in the standard to tell me that. But maybe I'm the
only one with this problem.

   Cheers

?Jonathan

On Fri, 21 Jan 2022 at 18:41, Markus Scherer <markus.icu at gmail.com> wrote:

> I would show all format controls (gc=Cf
> <https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=%5B%3Agc%3DCf%3A%5D&g=&i=>)
> with a code. A few might have a non-empty default glyph, but it's probably
> not worth getting more precise.
>
> Best regards,
> markus
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220123/ba1c2638/attachment.htm>

From jonathan.coxhead at gmail.com  Mon Jan 24 00:04:52 2022
From: jonathan.coxhead at gmail.com (Jonathan Coxhead)
Date: Sun, 23 Jan 2022 22:04:52 -0800
Subject: Repertoire
In-Reply-To: <CAH0mxTSAmb4LhTbBLFQC_tE6UmeEt4HjWBKXn3NxBrH+pXmhEw@mail.gmail.com>
References: <CABWUGktf0UAo011dsxzhWDb7t7cmn3KJHK5kA6jv-gGHaABOhg@mail.gmail.com>
 <CAN49p6otStkb8d+ZgYEbyO4G8yMiK2s0kaphfkQt_C54ph3tfQ@mail.gmail.com>
 <f506b8e6-efa5-79b3-1f35-12b001791a0d@ix.netcom.com>
 <CAH0mxTSAmb4LhTbBLFQC_tE6UmeEt4HjWBKXn3NxBrH+pXmhEw@mail.gmail.com>
Message-ID: <CABWUGktzos+qdxti5HtS0yhQGerHCa0PNgnAitNwNbX7tgwG2g@mail.gmail.com>

   This is very fine! I'm very pleased to see work like this. My own
interest led me to write a script to use the light box, heavy box and
double box characters to make pretty tables out of CSV files, like this:

?????  ?????  ?????  ?????
?1?2?  ?1?2?  ?1?2?  ?1?2?
?????  ?????  ?????  ?????
?3?4?  ?3?4?  ?3?4?  ?3?4?
?????  ?????  ?????  ?????

   One thing I noticed is that your code is built around using the 1/8
block characters, but the BLOCK DIAGONALS (sorry, don't mean to shout) are
supposed to work with the SEXTANT (1/6) characters. Not an insurmountable
problem, I'm sure.

   Thanks for posting!

On Sat, 22 Jan 2022 at 19:12, Joao S. O. Bueno via Unicode <
unicode at corp.unicode.org> wrote:

>
>>    I was also wondering idly if anyone has any practical uses for the
>> legacy computing characters, specifically the ones with "BLOCK DIAGONAL" in
>> the name. They look tantalisingly as though they must be good for
>> something, but I don't know what it could be?
>
>
> I have those on my roadmap.  :-)
> I am author to the "terminedia" framework for Unicode Art and terminal
> based applications -
> the code enables one to use a drawing API to draw with characters, anyone,
> but including
> full block (\U2588) as pixels, and then uses 1/4 block characters for "hi
> resolution",
> with each character block being able to convey 4 pixels. There is a
> hackish mode
> with braille characters, with 8 pixels/block, and, few months after the
> legacy
> computer characters were added and generally available, I added support
> for the 1/6 block sextant characters as a drawing mode.
>
> The other legacy characters currenntly have no better support in the
> project than most emojis - but the "BLOCK DIAGONAL" in specific are
> in my roadmap: when I get to implement the "1/2 pixel per block" drawing
> mode,
> taking two character blocks per "pixel", I will add a transform to fill in
> corners
> with the appropriate "BLOCK DIAGONAL" characters to make curves
> "smooth'.
>
> I use a similar approach for using the line/table drawing characters now
> ("HEAVY QUADRUPLE DASH *", for example): one draws boxes and lines
> with fullblocks, apply a transform, and have the characters replaced by
> the apropriate frame drawing chars.
>
> Unfortunately, I am not getting a lot of time to put in the project
> over the last few months, and I am currently in the task of
> improving the usability of the text-based widgets, so that
> the project can be used for real-world apps.
>
> The project is at
> https://github.com/jsbueno/terminedia
>  the main gap is documentation. It installs along with
> some example scripts, and there is a separate project that uses
> it, and can be used to draw interactively on the terminal using the above
> mentioned "pixel" modes -
> https://github.com/jsbueno/terminedia-paint
>
> (installable with `pip3 install --user terminedia-paint`, currently no
> Windows support,
> and on MacOS it needs a terminal program that supports full color ANSI
> codes. )
>
> And finally - yes, I'd also greatly appreciate proper responses for your
> main query -
> as being able to introspect the full unicode charset and make developers
> life
> easier when dealing with Unicode is one of the project's missions.
> I
>
> On Sat, 22 Jan 2022 at 06:31, Asmus Freytag via Unicode <
> unicode at corp.unicode.org> wrote:
>
>> On 1/21/2022 6:40 PM, Markus Scherer via Unicode wrote:
>>
>> I would show all format controls (gc=Cf
>> <https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=%5B%3Agc%3DCf%3A%5D&g=&i=>)
>> with a code. A few might have a non-empty default glyph, but it's probably
>> not worth getting more precise.
>>
>> Best regards,
>> markus
>>
>> The code charts use special glyphs for these, for those users who don't
>> memorize hex codes.
>>
>> A./
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20220123/45c70af5/attachment.htm>