an example tokenizer for a simplified C style language - View it on GitHub