Chinese Handwriting Recognition

Some random thoughts on Chinese handwriting recognition:

• This problem appears to me more interesting than speech recognition. Chinese (Mandarin or Cantonese) recognition in general has no difference with English recognition. Handwriting, on the other hand, makes a huge difference. Think about the amount of Chinese character and roman character. Another interesting observation is that there are quite a lot dialects or spoken languages in China while the writing is almost the same: simplified Chinese for mainland; traditional for Taiwan, Hong Kong, and some other areas.
• There are some commercialized software on market. There is even a Korea Internet company developed online dictionary accepting handwriting as input. The open sourced solution for this problem is very rare. I can list a few from Japan originally for recognizing Kanji, e.g. tomoe, Tegaki (based on Zinnia). Considering Japanese also having katakana and hiragana, this problem is actually more appealing to Chinese.
• The problem is not hard to solve. I can easily think of several methods, e.g. based on stroke classification and ordering, or feature based pattern matching. The difficult part is data sparsity. It is relatively hard to get training data for all Chinese characters no matter what methods is used. The number of characters I guess is more than 100,000.
• I can think of some practical ways to work around the lack of training data. For example, wubi (five strokes) input method has the mapping between character to the five stroke order. Another good data maybe the bitmap font files. Chinese bitmap font essentially are font images to characters mapping. The images may be used for pattern matching.
• Another direction is to incorporate language model like in speech recognition system. Actually most pinyin based input methods are using some kinds language model. The good pinyin input method often results from a good vocabulary frequency table which is essentially a language model. So it is no wonder two internet search engine, Google and Sogou, are supplying the two most successful Chinese input methods.
• I am also thinking another recognition problem: handwriting math formula recognition. Since handwriting formula is much quick er than typing (thinking LaTex or Office formula editor). More thoughts on this to be filled next time...

- Written on Sat Feb 14 17:27:01 2009.

There is a handwriting math formula recognition demo: http://www.cs.rit.edu/~rlaz/#research
- Reply from Y. Shao on Tue Feb 17 04:55:41 2009