AI开发平台ModelArts-Yaml配置文件参数配置说明:dpo_yaml样例模板

时间：2025-02-12 15:14:13

AI开发平台ModelArts

dpo_yaml样例模板

### modelmodel_name_or_path: /home/ma-user/ws/tokenizers/Qwen2-72B### methodstage: dpodo_train: true# lorafinetuning_type: loralora_target: allpref_beta: 0.1pref_loss: sigmoiddeepspeed: examples/deepspeed/ds_z3_config.json### datasetdataset: dpo_en_demodataset_dir: /home/ma-user/ws/llm_train/LLaMAFactory/LLaMA-Factory/data template: qwencutoff_len: 4096packing: truemax_samples: 50000overwrite_cache: truepreprocessing_num_workers: 16dataloader_num_workers: 8### outputoutput_dir: /home/ma-user/ws/saves/dpo/llama3-8b/loralogging_steps: 2save_steps: 5000plot_loss: trueoverwrite_output_dir: true### trainper_device_train_batch_size: 1gradient_accumulation_steps: 8learning_rate: 5.0e-6num_train_epochs: 3.0lr_scheduler_type: cosinewarmup_ratio: 0.1bf16: trueflash_attn: sdpaddp_timeout: 180000000include_tokens_per_second: trueinclude_num_input_tokens_seen: true

上一篇：AI开发平台ModelArts-Yaml配置文件参数配置说明:ppo_yaml样例模板

下一篇：AI开发平台ModelArts-Yaml配置文件参数配置说明:ppo_yaml样例模板