kyy/llm_trainer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,709 Commits 1 Branch 0 Tags
0a2ec5fe2049a6d6bc3955c7e4794bd6115ccbcd
Commit Graph
1 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
d 6979f3f848 经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降 2024-06-11 16:23:40 +08:00