Bidi paragraph direction in terminal emulators BiDi in terminal emulators

Richard Wordingham via Unicode unicode at unicode.org
Wed Feb 6 17:32:43 CST 2019


On Wed, 6 Feb 2019 22:01:59 +0100
Egmont Koblinger via Unicode <unicode at unicode.org> wrote:

> Hi Eli,
> 
> (I'm getting lost where to reply, and how the subject gets mangled and
> the thread split into different ones.)
> 
> 
> I've thought about it a lot, experimented with Emacs's behavior, and
> I've arrived at the conclusion that we are actually much closer to
> each other than I had thought. Probably there's a lot of
> misunderstanding due to different terminology we used.
> 
> I've set my terminal to RTL paragraph direction (via the relevant
> escape sequence), then did a "cat TUTORIAL.he" (the file taken from
> 26.1), and compared to what I see in Emacs 25.2.2 – both the graphical
> one, and the one running in a terminal of no BiDi.
> 
> Apart from a few minor irrelevant differences, they look the same!
> Hooray!!!
> 
> (The differences are:
> 
> - I had to slightly modify TUTORIAL.he to make sure none of the lines
> start with a BiDi control (I added a preceding character) because
> currently VTE doesn't support them, there's no character cell to store
> this data. This definitely needs to be fixed in the second version of
> my proposal.
> 
> - Emacs running in a terminal shows an underscore wherever there's a
> BiDi control in the source file – while the graphical one doesn't.
> This looks like a simple bug to me, right?
> 
> - Line 1007, the copyright line of this file uses visual indentation,
> and Emacs detects LTR paragraph for that line. I think it should
> rather use BiDi controls to have an overall RTL paragraph direction
> detected, and within that BiDi controls to force LTR for the text. The
> terminal shows it with RTL direction, as I manually set it.
> 
> Again, all these three details are irrelevant to my point, namely that
> in WIP gnome-terminal it looks the same as in Emacs.)
> 
> 
> You define paragraphs as emptyline-separated blocks on which you
> perform autodetection of the paragraph direction. This is great! As
> I've mentioned, I'd love to have such a mode in terminals, but it's
> subject to underlying improvements, like knowing when a prompt starts
> and ends, because prompts also have to be paragraph delimiters.

Not necessarily.  One could allow the first strong character in the
prompt to determine the paragraph directions.  That's what the Emacs
terminal (invoked by M-x term; top level definition in term.el) does.

> On a nitpicking side note:
> 
> It's damn ugly not to terminate a text file with a newline. Newline is
> much better thought of a "terminator" than a "delimiter". For example,
> if you do a "cat file1 file2", you expect file2 to start on its own
> line.

Not necessarily.  One might use cat to glue together files that had
split into 1400k chunks, in which case it is not even reasonable to
expect the end of file to be at a character boundary.  (Yes, floppy
disks still have their uses.)

> Shouldn't this apply to paragraphs, too, especially when BiDi is in
> the game? I'd argue that an empty line (double newline) shouldn't be a
> delimiter, it should be a terminator for a paragraph.

But the white space between paragraphs is a separator, not a
terminator.  One doesn't require it at the end when formatting
paragraphs within the cell of a table. 

Richard.



More information about the Unicode mailing list