<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">15 dec. 2022 kl. 08:23 skrev Asmus Freytag via Unicode <<a href="mailto:unicode@corp.unicode.org" class="">unicode@corp.unicode.org</a>>:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
<div class="">
<div class="moz-cite-prefix">On 12/13/2022 3:36 AM, Kent Karlsson
via Unicode wrote:<br class="">
</div>
<blockquote type="cite" cite="mid:8B168E84-2EA2-403A-AE80-232C7ED58C08@bahnhof.se" class="">
<pre class="moz-quote-pre" wrap="">(Hoping that this goes through ok; I did have some problems with the sum sign when copying this text…)
I've deviced a (or rather, several) new format(s) for representing math expressions.
Why, you may wonder... Isn't MathML the answer to everyting math? Well, not quite.
</pre>
</blockquote><p class="">Looks like it came through.</p><p class="">The real audience for this would be people creating / editing
mathematical / technical / scientific papers, not the character
encoders (although there's some small overlap).</p><p class="">I think it is valuable to explore the solution space, but have
you heard from people actively using mathematical notation (in
whatever form it is currently supported) to find out what they are
looking for?</p><p class="">What would you have to offer to make your new mousetrap one that
people would be willing to leave investment in established
technologies behind?</p><p class="">I'm sure you've thought of all of that already.</p><p class="">A./<br class="">
</p>
</div>
</div></blockquote></div><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class=""> </span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">MathML has been “around” for more than 20 years. It hasn’t been a great hit. And I think I know why: it is way too verbose.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">So what about other ways of getting “math on the web”. There is one that is popular: integrating LaTeX math expressions with HTML web pages. But the integration was, <b class="">and still is</b>, very much a workaround. The result is (smallish) images with math expressions in the web page. Not really what one would like to see. I guess MathML is supposed to remedy that, but MathML is still too verbose. And “authoring tools”, such as that in MS Word, are quite tedious to use (and the result does not always look good). I know, MS Word has three different ways of representing math expressions: UnicodeMath, TeX math expressions, and OMML which is XML based but is not MathML.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">Imagine writing a math textbook, with tens of math expressions per page, and hundreds of pages. Using MathML or OMML, even with authoring tools, would be tedious indeed. Ok, maybe that is not the target for MathML or OMML. But still, why should there be vastly different representations for shorter texts (like a web page) vs. longer texts (like a math textbook). They should at least be perfectly equivalent.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">There should be a format that works both for the web (HTML) and other contexts (some other markup, even plain text), one that is light-footed, and equivalent in all “surface forms” (HTML, some other markup, even plain text).<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">Further, in a Unicode context handling bidi properly is almost a requirement. Out of the mentioned formats, it is, IIUC, only MathML that attempts to cover bidi. And it is not done very well, at least it is not reliable, especially not w.r.t. arrows. It should be noted that for bidi there are different conventions when to “mirror” a math subexpression. E.g., for division it is not always reversed (RTL). Indeed, how does one make a horizontal division textually reversed? And for subtraction it would be very confusing to “swap the arguments” since the operator is left/right symmetric. So it must be up to the author to decide which arguments to “swap”; no blind textual reversal. That goes also for which direction “operators” (including arrows) are to be “mirrored”. One cannot blindly mirror math operators just because the context is RTL. Indeed, arrows (and arrow-like) characters are treated especially handwavingly in the bidi algorithm specification.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">And how to avoid the heavy-footedness of MathML and OMML? Much of that is due to the convention of “fully bracket” everything based on XML. But if one ignores that convention, then we can have a notation that is compatible with XML, and still be more “light-footed”, almost like TeX or UnicodeMath. But then an extra level of parsing is needed, a level akin to the parsing done for eqn, TeX or UnicodeMath. That is unusual, but I don’t think that is contrary to what is allowed in XML.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">The solution I’ve presented “goes back to basics”. Math expressions basically have not changed for decades. I hesitate to say centuries, since it has changed over several centuries. Still, there is a basic structure, regardless of semantics, of placements of math expression parts. Just the placement, no reference to semantics, or even operator precedence. Using a system like XML, where there is a strong tradition of “full bracketing”, is fine when the (textual, in number of characters) size of the content is much greater than the bracketing (in case of XML, they are called tags). But for math expressions, full bracketing for the structure notation, with clumsy brackets (tags) to boot, is not a great idea. I don’t think there is anything in XML that *forces* the use of full bracketing. Hence, the proposal I make (XML variant) uses (invisible) structure *operators* (with priorities expressed in the grammar), in the XML variant these operators are “empty tags”, to allow avoiding bracketing when it can be derived by parsing according to the grammar.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">But not everything “is XML”. There needs to be a <i class="">fully equivalent</i> representation suitable for context that are not XML. And I presented one such, based on C1 control codes. Much as the UTC loves to hate (C0 and) C1 control codes, the UTC has also declared that there “will be no more control codes”. Fortunately, ECMA-48 control codes allows for several extension mechanisms (that would be “available/reserved for future standardisation” as well as explicitly “private use” ones), voiding the need to allocate new control codes in Unicode; the extension mechanisms are still available. For the math expressions, using SCI, Single Character Introducer, seems to be the handiest. That, together with using a few more hitherto unused C1 control codes enabled a representation of math expressions that can reasonably be called plain text (or a plain text protocol). XML, on the other hand, uses pure printable character substrings as controls, and is a higher-level protocol.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">But what about typeability (direct from the keyboard)? Well, the XML version is technically typable, especially the verbosity is kept down quite a bit in my proposal (but not when it is verbose, typing by hand is simply too much). But it is still XML, even if less verbose, so not so handy. And the C1 version? Well, it is very compact, and truly plain text. But most text editors are not friendly when it comes to C1 control characters, and those characters are also essentially impossible to type directly from the keyboard. So a short, non-verbose, markup (in “markdown style”)? We already have UnicodeMath (which is NOT a plain text format, claiming that it is is misleading). Two reasons why that is not quite right: 1) It is not exactly equivalent to any other format. 2) As a markup language (which it is), it is not particularly well designed. It’s a bit haphazard, and frankly a hack. Furthermore, 3) it does not seem to cater for bidi.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">My sketchy proposal in appendix C is based on the C1 version of the proposed math expression formats. Admittedly a bit quirky, since C1 controls, in particular SCI, is a bit quirky. I don’t know the original intent for SCI, but it was left unspecified in use, except that it takes <i class="">one</i>, and only one,<i class=""> <u class="">character</u></i> after the SCI to create a whole lot of control sequences. But asking for new control characters is a non-starter, so this one seemed fine to use. Some of the quirkiness is avoided in the “mark-down” version hinted at in appendix C, and maybe more can be avoided.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">And, since I’m sure someone will ask (or even protest): So-called “MATHEMATICAL” letters and digits are forbidden in the proposed formats. Why? The fundamental reason is that they are ill-conceived. Variables are often multi-letter, especially in computer science. And they need not be in English (even though that is commonly so), so other (Latin) letters than a-zA-Z may be used. Therefore, there is a more general math letters (and numerals, and some symbols) styling mechanism proposed. This mechanism can handle multiple-letter variables easily, as well as handle other letters than those that have been allocated as “MATHEMATICAL” ones.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">Note that the distinction that TeX makes between “default math variable style” (almost the same as \mathnormal) and \mathit. Very often one must apply \mathit to get a proper typesetting of multiletter variables. I come across such things as “coefficient” written (by others than me) in MS Word math expressions, and it looks horrible. Just like when forgetting to apply \mathit in TeX (and I have written a few texts where I needed to apply \mathit quite generously; not counting TeX’s ability to define macros/commands). Neither MS Word nor MathML seems to cater well for what in TeX is \mathit (it may be applicable, but not easily so). Further, combining characters apply to a base character in Unicode, never to a math expression, despite what other math formats say (unfortunately, MathML is among the formats that mistreat combining characters).<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">So while I’m not keen on referring to math expression representation formats as “mouse traps”, I do think I have a solution that avoids several of the problems of other formats, their flaws (verbosity, single context only), shortcomings (not handling RTL arrow-like characters right, inability or difficulty to handle multiletter variables, haphazard design of mark-down style mark-up), and outright errors (misinterpretation of combining characters, forcing very strict RTL on RTL math expression). And, of course, the non-use of the ill-conceived “MATHEMATICAL” letters.<o:p class=""></o:p></span></p><p class="MsoNormal" style="margin: 0cm 0cm 8pt; line-height: 15.546667098999023px; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-GB" class="">Of course, there is no claim of perfection, so comments are welcome.</span></p><div class=""><br class=""></div><div class="">Kind regards</div><div class="">/Kent K</div><div class=""><br class=""></div></body></html>