Private Use Area characters and the eudcedit program

[Peter Constable]

Adding to Jim's excellent answer: the EUDC mechanism that has been in Windows since the 90s is integrated into platform font fallback mechanisms (more specifically, font linking). A privately-defined character is "linked" (associated with) particular fonts all fonts installed in the system, and will display anywhere that makes use of Win32 (GDI, User...), GDI+, Uniscribe or DWrite _unless_ lower-level APIs for drawing text are used that bypass the platform font fallback mechanisms. Affinity products use DWrite, but probably use only those lower-level APIs.


Peter

From: Unicode <unicode-bounces at corp.unicode.org> On Behalf Of Jim DeLaHunt via Unicode
Sent: Friday, February 2, 2024 5:05 PM
To: unicode at corp.unicode.org
Subject: Re: Private Use Area characters and the eudcedit program


On 2024-02-02 14:33, William_J_G Overington via Unicode wrote:
Regarding the issue raised in the thread

https://forum.affinity.serif.com/index.php?/topic/197938-private-characters-created-with-microsoft-eudceditexe
The issue appears to be (copying text from that thread to this):

I have created two "private" characters using the Windows built-in eudcedit utility. The first one I have saved to a specific font and the second one I have saved to all fonts.

I can locate and copy both characters in Character Map and paste them successfully into Notepad and into my CAD programs, but not Affinity Publisher (v.1)  Is there a special procedure in Publisher that will overcome this, or is the programe not yet equipped to deal with private characters?


On 2024-02-02 14:33, William_J_G Overington via Unicode wrote:
can anyone explain what is happening please?

I can perhaps shed some light, if not explain definitively.

Anyone using EUDCedit would be well advised to learn what Windows has to say about what EUDC is and how it works in Windows. A web search finds:

End-User-Defined and Private Use Area Characters (2021) <https://learn.microsoft.com/en-us/windows/win32/intl/end-user-defined-characters><https://learn.microsoft.com/en-us/windows/win32/intl/end-user-defined-characters>

End-user-defined characters (EUDC) in double-byte character sets<https://learn.microsoft.com/en-us/windows/win32/intl/double-byte-character-sets> (DBCSs) and private use area (PUA) characters in Unicode<https://learn.microsoft.com/en-us/windows/win32/intl/unicode> are custom characters. They can be defined and implemented either by an end user or by another party.... Their use enables users to form names and other words using characters that are not available in standard screen and printer fonts.

The EUDC and PUA characters can be assigned differently, or not assigned at all, on different computers. Some code pages have extensions that reuse the EUDC range, ... a manufacturer might provide a custom set of characters in one of these ranges, ... user groups can attempt to provide additional characters in the PUA. Different combinations of these cases can cause conflict. When creating applications that rely on EUDC or PUA characters, you should keep in mind the conflicting interpretations of an individual code point....


Character Sets and Fonts (2021) <https://learn.microsoft.com/en-us/windows/win32/intl/character-sets-and-fonts><https://learn.microsoft.com/en-us/windows/win32/intl/character-sets-and-fonts>
To create an EUDC or PUA character, the user chooses a character value that is within the specified range and adds the glyph<https://learn.microsoft.com/en-us/windows/win32/intl/uniscribe-glossary> to the font in the entry that corresponds to that character value. The user creates the glyph using an EUDC editor or using a font package purchased from a font vendor. Any DBCS font can contain EUDCs, and any Unicode font can contain PUA characters. The font is called a "separate" EUDC/PUA font if it contains only EUDCs. The font is an "integrated" EUDC/PUA font if it contains standard characters as well as EUDCs....
TrueType fonts can be installed either as .ttf files or as .tte files. Since the operating system hides .tte files, applications cannot enumerate or otherwise examine the installed fonts using GDI API functions. On many operating systems, the system default EUDC/PUA font and separate EUDC/PUA fonts are installed as .tte files. Applications such as EUDC editors and the Control Panel must use registry entries to add, modify, and delete such fonts....



The backstory is that end-user defined character handling is a text requirement originating from ideographic scripts, especially Japan, and an era when the glyph complement of Japanese fonts was small (c 5,000 glyphs) compared to the range of ideographic characters listed in dictionaries and fair game to use in text (c 70,000 characters). Authors wanting to use such "outside characters" (known as "gaiji" in Japanese) in their publications had to resort to special measures like EUDCs. OS and application vendors who wanted to sell to serious publishers in the Japanese market need to provide EUDC tools.

The need for special measures like EUDC has receded greatly with the arrival of ideographic script fonts with very large glyph repertoires. Use of EUDC tools in ideographic script documents is likely now a niche. Use of EUDC outside of ideographic script context is even more of a niche.

The original question was, "I can [not] locate and copy [my EUDC] characters in... Affinity Publisher (v.1)  Is ... the programe not yet equipped to deal with private characters?"

It seems pretty likely to me that the program is not equipped to deal with EUDC characters. The feature list for Affinity Publisher <https://affinity.serif.com/en-us/publisher/full-feature-list/><https://affinity.serif.com/en-us/publisher/full-feature-list/> does not mention Japanese or Chinese typography support. If they do not have ideographic typography as a major feature, they are even more unlikely to have Windows-specific EUDC support.

--

.   --Jim DeLaHunt, jdlh at jdlh.com<mailto:jdlh at jdlh.com>     http://blog.jdlh.com/ (http://jdlh.com/)

      multilingual websites consultant, Vancouver, B.C., Canada
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://corp.unicode.org/pipermail/unicode/attachments/20240203/9ce6f5c0/attachment.htm>