High performance chinese tokenizer with both GBK and UTF-8 charset support developed by ANSI C - View it on GitHub