Proposal for BiDi in terminal emulators

[Egmont Koblinger via Unicode]

Hi Kent,

On Sat, Feb 2, 2019 at 12:41 AM Kent Karlsson via Unicode
<unicode at unicode.org> wrote:

> [...] neither of which
> should directly consult the font [...]
> But terminals
> (read terminal emulators) can deal with mixed single width and double
> width characters (which is, IIUC, the motivation for the datafile
> EastAsianWidth.txt).

Yup, exactly; and for this reason, no terminal I'm aware of takes the
single vs. double width property from the font. The logical behavior,
i.e. knowing which logical cell contains what character (or which half
of what character, in case of double wide ones) isn't influenced by
the font. It's taken from EastAsianWidth (or other means, which we're
working on: https://gitlab.freedesktop.org/terminal-wg/specifications/issues/9
, to address e.g. incompatibilities arising from different Unicode
version used by the app vs. the terminal, as you pointed out).

Also think of cases like when the user modifies the font of the
terminal run-time, or a headless terminal emulator, or a screen/tmux
attached to multiple terminal emulators of different fonts at once...
Adjusting the logical behavior according to the font would definitely
be a wrong path to take.

> Likewise non-spacing combining characters should
> be possible to deal reasonably with.

Most terminal emulators handle non-spacing combining marks, it's a
piece of cake. (Spacing marks are more problematic.)

> All sorts of problems arise; feeding
> the emulator a character (or "short" strings) at a time not allowed
> to buffer for display (causing reshaping or movement of already
> displayed characters, edit position movement even within a single
> line, etc.).

Emulators need to update their screen to reflect whatever is in the
logical buffer, and the contents of the logical buffer mustn't depend
on the timing of the incoming data. As a consequence, when the input
stream contains a base character + a combining accent, there is a slim
chance that the base character without the combining accent makes it
to the display for a short time. It's the emulator's job to "fix" it
(that is, redraw the glyph with the combining accent) once the accent
is received. If an emulator doesn't do it correctly, it's simply a bug
in that emulator.

On a side note, we're also working on an extension for atomic updates
at https://gitlab.freedesktop.org/terminal-wg/specifications/issues/9
which should significantly further decrease the chance of such
intermittent screen updates.

cheers,
egmont