Keyboard Suddenly Outputting in NFD

Richard Wordingham richard.wordingham at ntlworld.com
Mon Mar 22 17:16:24 CDT 2021


I'm asking here because my searches turned up nothing.

I've just noticed that when I use my handrolled keyboard designed to
output NFC, what appears on the terminal (Gnome-terminal) or browser
(Firefox into a Wikimedia form), my text is being stored as NFD UTF-8.
I use an M17n definition with fcitx on Ubuntu 16.04.3 as the input
method. It used to generate NFC; I'm not sure when it suddenly changed
to generating NFD text. The keyboard used to generate NFC output.

This change causes me grief because I am using grep to search data files
stored in NFC; grep does not respect canonical equivalence, so a typed
in sequence in NFD does not match the NFC data in the file. 

Does anyone know where this change has occurred?  Are there any quick
fixes?

I do have a grep-like search utility that respects canonical
equivalence, but it's a bit slow with a million-line input file.

Richard.



More information about the Unicode mailing list