AI开发平台MODELARTS-附录:训练常见问题:问题3:训练过程报错:ImportError: XXX not found in your environment: flash_attn

时间:2024-11-12 16:42:44

问题3:训练过程报错:ImportError: XXX not found in your environment: flash_attn

根因:昇腾环境暂时不支持flash_attn接口

规避措施:修改dynamic_module_utils.py文件,将180-184行代码注释掉

vim /home/ma-user/anaconda3/envs/PyTorch-2.1.0/lib/python3.9/site-packages/transformers/dynamic_module_utils.py
support.huaweicloud.com/bestpractice-modelarts/modelarts_llm_train_90936.html