Bidi paragraph direction in terminal emulators (was: Proposal for BiDi in terminal emulators)

[Eli Zaretskii via Unicode]

> From: Egmont Koblinger <egmont at gmail.com>
> Date: Fri, 8 Feb 2019 13:30:42 +0100
> Cc: Richard Wordingham <richard.wordingham at ntlworld.com>, 
> 	unicode Unicode Discussion <unicode at unicode.org>
> 
> Hi Eli,
> 
> > Not sure why.  There are terminal emulators out there which support
> > proportional fonts.
> 
> Well, of course, a terminal emulator can load any font, even
> proportional, but as it places them in the grid, it will look ugly as
> hell

Maybe so, but the original text was this:

  Emacs and 'M-x term' are the route to take if one only has
  proportional fonts.

Which I don't understand, since the terminal emulator in Emacs doesn't
do anything special about proportional fonts, AFAIK.

> In Emacs-25.2's terminal emulator I executed "cat TUTORIAL.he". For
> the entire contents, LTR paragraph direction was used and was aligned
> to the left. Maybe something has changed for 26.x, I don't know.

I told you what changed: Emacs 25 forces LTR paragraph direction,
whereas Emacs 26 and later does not.  You can get dynamic paragraph
direction in your Emacs 25 as well if you set bidi-paragraph-direction
to nil in the *term* buffer.

> And now you suddenly tell that Emacs's terminal supports BiDi more or
> less in full???

Emacs implements the latest UBA from Unicode 11; and the Emacs
terminal emulator inserts all the text into a "normal" Emacs buffer,
and displays that buffer as any other buffer.  So yes, you have there
full UBA support.  I thought this was clear, sorry if it wasn't.  One
caveat with this is that the Emacs emulator works only on Posix
platforms, it doesn't work on MS-Windows.

> Sorry, I just don't buy it. If you retain this claim, I'd pretty
> please like to see a specification of its behavior

The specification is the latest version of the UBA, augmented with
three deviations, two of them allowed by the UBA, the third isn't:

  . Emacs uses HLA1 for determining base paragraph direction: it
    decides on base direction only once for every chunk of text
    delimited by empty lines;

  . Emacs doesn't by default remove bidi formatting controls from
    display;

  . Emacs wraps long lines _after_ reordering, not before.

I think that's it.  If I forget something, please forgive me: I
implemented this 10 years ago, so maybe something evades me at the
moment.

> one which addresses at least all the major the issues I address in
> my work, one which I could replace my work with, one which I'd be
> happy to implement in gnome-terminal in the solid belief that it's
> about as good as my proposal, and would wholeheartedly recommend for
> other terminal emulators to adopt.
> 
> Or maybe, by any chance, when you said Emacs's terminal supported BiDi
> more or less in full, did you perhaps went with your own idea what a
> BiDi-aware terminal emulator needs to support; ignoring all those
> things I detail in my work, such as the inevitable need for explicit
> mode, the need for deciding the scope of implicit vs. explicit mode,
> and much more?

Sorry, I cannot afford testing everything you wrote in your
specification.  I think most, if not all, of that is covered, but I
certainly didn't test that, so maybe I'm wrong.  Please feel free to
test the relevant aspects and ask questions if you need more "inside
information".  I do hope that my impression about "most everything
being supported" is correct, because that would give you a working
implementation/prototype of most of the features you want to see in
terminal emulators, so you could actually try the behavior to see if
it's convenient, causes problems, etc.

One other feature you may find interesting (something that I don't
think you covered in your document, at least not explicitly) is that
Emacs supports visual-order cursor motion, in addition to the "usual"
logical-order.  The latter is, of course, the default, but you can
switch to the former if you set the visual-order-cursor-movement
option to a non-nil value.