AI开发平台MODELARTS-Llama 3.2-Vision基于DevServer适配Pytorch NPU训练微调指导（6.3.911）:步骤九：开始训练

时间：2024-12-25 09:49:50

AI开发平台MODELARTS

步骤九：开始训练

单机训练

cd ${container_work_dir}
# 指定model_path为步骤八：下载模型权重下载的Llama-3.2-11B-Vision-Instruct权重路径
# 指定dataset参数为步骤七：数据集下载与制作中所述custom_dataset_info_demo.json中文件设置的数据集名称：coco2014_train_40k_demo
# 修改custom_dataset_info参数路径为${container_work_dir}/ms-swift/swift/llm/data/custom_dataset_info_demo.json 
# finetune sft全参微调场景
model_path=path/to/Llama-3.2-11B-Vision-Instruct \
dataset=coco2014_train_40k_demo \
custom_dataset_info=path/to/ms-swift/swift/llm/data/custom_dataset_info_demo.json \
work_dir=${container_work_dir} \
globol_batch_size=32 \
per_device_batch_size=2 \
bash llama32_vision_11b_finetune_sft.sh  
# finetune lora微调场景
model_path=path/to/Llama-3.2-11B-Vision-Instruct \
dataset=coco2014_train_40k_demo \
custom_dataset_info=path/to/ms-swift/swift/llm/data/custom_dataset_info_demo.json \
work_dir=${container_work_dir} \
globol_batch_size=32 \
per_device_batch_size=4 \
bash llama32_vision_11b_finetune_lora.sh

多机训练

cd ${container_work_dir}
# 指定model_path为步骤八：下载模型权重下载的Llama-3.2-11B-Vision-Instruct权重路径
# 指定dataset参数为步骤七：数据集下载与制作中所述custom_dataset_info_demo.json中文件设置的数据集名称：coco2014_train_40k_demo
# 指定custom_dataset_info参数路径为${container_work_dir}/ms-swift/swift/llm/data/custom_dataset_info_demo.json 
# finetune sft场景
model_path=path/to/Llama-3.2-11B-Vision-Instruct \
dataset=coco2014_train_40k_demo \
custom_dataset_info=path/to/ms-swift/swift/llm/data/custom_dataset_info_demo.json \
work_dir=${container_work_dir} \
node_num=${NODE_NUM} \
node_rank=${NODE_RANK} \
master_addr=${MASTER_ADDR} \
globol_batch_size=32*${NODE_NUM} \
per_device_batch_size=2 \
bash llama32_vision_11b_finetune_sft.sh  
# finetune lora场景
model_path=path/to/Llama-3.2-11B-Vision-Instruct \
dataset=coco2014_train_40k_demo \
custom_dataset_info=path/to/ms-swift/swift/llm/data/custom_dataset_info_demo.json \
work_dir=${container_work_dir} \
node_num=${NODE_NUM} \
node_rank=${NODE_RANK} \
master_addr=${MASTER_ADDR} \
globol_batch_size=32*${NODE_NUM} \
per_device_batch_size=4 \
bash llama32_vision_11b_finetune_lora.sh

参数说明：