Changing tokenizer's attribute at preprocessing stage will result in saving a wrong tokenizer. for example, baichuan2
lvziandGitHub