kyy/llm_trainer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,838 Commits 1 Branch 0 Tags
d74244d56858d837044e5c9cea57a1b3c2ca0214
Commit Graph
39 Commits
Author SHA1 Message Date
hiyouga d74244d568 fix #4398 #4592 2024-06-30 21:28:51 +08:00
hiyouga 2f4b89ace1 loose gemma2 attention 2024-06-29 01:42:14 +08:00
hiyouga 4d35e218b1 bf16 by default, gemma2 attns 
Gemma2 finetuning cannot work until merging https://github.com/huggingface/transformers/pull/31674
 2024-06-28 06:00:26 +08:00
hiyouga 96a5044394 add quant checks 2024-06-27 01:12:25 +08:00
hiyouga ad144c2265 support HQQ/EETQ #4113 2024-06-27 00:29:42 +08:00
hiyouga addca926de improve autogptq integration 2024-06-26 22:11:44 +08:00
hiyouga 1e9d0aa1e4 fix #4432 2024-06-25 02:34:04 +08:00
hiyouga fca893d73c fix #4410 2024-06-24 22:34:31 +08:00
stceum 3ed063f281 Bug Fix: off is parsed as False in yaml file, changed to disabled to avoid this. 2024-06-24 20:39:31 +08:00
hiyouga 8d4f5093cf tiny fix 2024-06-20 22:56:05 +08:00
hiyouga 3b040e8e0f update patcher 2024-06-19 21:27:00 +08:00
hiyouga 4bd77d8563 fix #4357 2024-06-18 22:42:45 +08:00
hiyouga e2665e71c7 fix #4326 2024-06-17 18:17:48 +08:00
hiyouga 8c1046d78a support pissa 2024-06-16 01:08:12 +08:00
hiyouga 38b6b0f52e tiny fix 2024-06-16 01:06:41 +08:00
hiyouga d87108daa6 add license 2024-06-15 17:54:33 +08:00
hiyouga b27269bd2b add test cases 2024-06-15 04:05:54 +08:00
hiyouga 2ed8270112 clean code 2024-06-13 01:58:16 +08:00
hiyouga cca6f35108 fix deepspeed version 2024-06-11 16:52:36 +08:00
hiyouga 89f2bd8c8c fix #4198 2024-06-11 15:38:38 +08:00
hiyouga 3f24337a8a tiny fix 2024-06-11 01:04:16 +08:00
hiyouga a793e8456b fix #4160 
The split heads should be concatenated in dim=2
 2024-06-11 00:37:17 +08:00
hiyouga c907d81667 fix #2666 2024-06-10 21:24:15 +08:00
hiyouga 54cd743ebf reorganize adapter code 2024-06-08 00:47:23 +08:00
hoshi-hiyouga cfd62283a9 fix #4139 2024-06-08 00:45:02 +08:00
hiyouga 06e5d136a4 add resume args in webui 2024-06-08 00:22:16 +08:00
hiyouga 74f96efef9 rename files 2024-06-07 00:09:06 +08:00
hiyouga 451b6693c0 fix torch gc 2024-06-06 20:30:25 +08:00
hiyouga a12a506c3d support train from scratch #4033 #4075 2024-06-06 02:43:19 +08:00
hiyouga 08564838bd fix full/freeze tuning for mllm 2024-05-27 20:37:57 +08:00
BUAADreamer 57eb13b75d add regex of only tune lm and mm_proj 2024-05-27 18:59:00 +08:00
BUAADreamer 60170a1da4 Merge branch 'hiyouga:main' into main 2024-05-25 14:18:49 +08:00
hiyouga 063f91cc80 fix #3853 2024-05-24 23:29:45 +08:00
BUAADreamer 29a6d5bdb8 support pretraining of llava 2024-05-21 08:57:14 +08:00
hiyouga 31a0564d4f fix zero2 high ram usage 2024-05-19 21:53:54 +08:00
hiyouga 1ebc890a5f fix #3807 2024-05-19 17:07:57 +08:00
hiyouga d43822fcc2 fix jetmoe z3 block 2024-05-18 22:28:45 +08:00
hiyouga d9f190ff1e better dtype handle in loading 2024-05-17 02:14:56 +08:00
hiyouga 308edbc426 rename package 2024-05-16 18:39:08 +08:00