基于llama-factory的昇腾实战
【摘要】 环境安装(在安装好cann的容器执行)git clone https://github.com/hiyouga/LLaMA-Factory.gitcd LLaMA-Factorypip install -e ".[torch-npu,metrics]"# 查看安装的版本llamafactory-cli env 全流程实践 原始模型推理cd LLaMA-Factory# 需要提前下载好模型权...
环境安装(在安装好cann的容器执行)
git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch-npu,metrics]"
# 查看安装的版本
llamafactory-cli env
全流程实践
原始模型推理
cd LLaMA-Factory
# 需要提前下载好模型权重:如/weights/Qwen1.5-0.5B-Chat/
ASCEND_RT_VISIBLE_DEVICES=0 llamafactory-cli chat --model_name_or_path /weights/Qwen1.5-0.5B-Chat/ --template qwen
# 模型部署好可以进行对话
基于LORA的sft指令微调
- 创建yaml文件,如qwen.yaml
### model
model_name_or_path: /weights/Qwen1.5-0.5B-Chat
### method
stage: sft
do_train: true
finetuning_type: lora
lora_target: q_proj,v_proj
### ddp(开启deepspeed需要跑多卡)
#ddp_timeout: 180000000
#deepspeed: examples/deepspeed/ds_z0_config.json
### dataset
dataset: identity,alpaca_en_demo
template: qwen
cutoff_len: 1024
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16
### output
output_dir: saves/Qwen1.5-0.5B-Chat/lora/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 2
learning_rate: 0.0001
num_train_epochs: 1.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
fp16: true
### eval
val_size: 0.1
per_device_eval_batch_size: 1
#evaluation_strategy: steps
eval_steps: 500
- 执行微调
cd LLaMA-Factory
ASCEND_RT_VISIBLE_DEVICES=0 llamafactory-cli train qwen1_5_lora_sft_ds.yaml
动态合并LoRA的推理
ASCEND_RT_VISIBLE_DEVICES=0 llamafactory-cli chat --model_name_or_path /weights/Qwen1.5-0.5B-Chat/ --template qwen --adapter_name_or_path saves/Qwen1.5-0.5B-Chat/lora/sft --finetuning_type lora
benchmark评测
llamafactory-cli eval --model_name_or_path /weights/Qwen1.5-0.5B-Chat/ --template fewshot --task mmlu_test --lang en --n_shot 5 --batch_size 1 --trust_remote_code true
【版权声明】本文为华为云社区用户原创内容,未经允许不得转载,如需转载请自行联系原作者进行授权。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱:
cloudbbs@huaweicloud.com
- 点赞
- 收藏
- 关注作者
评论(0)