AI开发平台MODELARTS-附录:大模型推理常见问题:问题2:在推理预测过程中遇到ValueError:User-specified max_model_len is greater than the drived max_model_len

时间:2024-12-17 18:06:49

问题2:在推理预测过程中遇到ValueError:User-specified max_model_len is greater than the drived max_model_len

解决方法:

修改config.json文件中的"seq_length"的值,"seq_length"需要大于等于 --max-model-len的值。config.json存在模型对应的路径下,例如:/data/nfs/benchmark/tokenizer/chatglm3-6b/config.json

support.huaweicloud.com/bestpractice-modelarts/modelarts_llm_infer_91117.html