AI开发平台MODELARTS-部署推理服务:Step2 配置pod

时间：2024-12-09 20:36:20

AI开发平台MODELARTS

Step2 配置pod

在节点自定义目录${node_path}下创建config.yaml文件

apiVersion: apps/v1
kind: Deployment
metadata:
  name: yourapp
  labels:
      app: infers
spec:
  replicas: 1
  selector:
    matchLabels:
      app: infers
  template:
    metadata: 
      labels:
         app: infers
    spec:
      schedulerName: volcano
      nodeSelector:
        accelerator/huawei-npu: ascend-1980
      containers:
      - image: ${image_name}                  # 推理镜像名称
        imagePullPolicy: IfNotPresent
        name: ${container_name}
        securityContext:
          runAsUser: 0
        ports:
        - containerPort: 8080
        command: ["/bin/bash", "-c"]
        args: ["${node-path}/run_vllm.sh"]          # 节点自定义目录，该目录下包含pod配置文件config.yaml和推理服务启动脚本run_vllm.sh
        resources:
          requests:
            huawei.com/ascend-1980: "8"             # 需求卡数，key保持不变。
          limits:
            huawei.com/ascend-1980: "8"             # 限制卡数，key保持不变。
        volumeMounts:                             # 容器内部映射路径
        - name: ascend-driver               #驱动挂载，保持不动
          mountPath: /usr/local/Ascend/driver
        - name: ascend-add-ons           #驱动挂载，保持不动
          mountPath: /usr/local/Ascend/add-ons
        - name: hccn                             #驱动hccn配置，保持不动
          mountPath: /etc/hccn.conf
        - name: localtime                       
          mountPath: /etc/localtime
        - name: npu-smi                             # npu-smi
          mountPath: /usr/local/sbin/npu-smi
        - name: model-path                       # 模型权重路径
          mountPath: ${model-path}
        - name: node-path                       
          mountPath: ${node-path}
      volumes:                                   # 物理机外部路径
      - name: ascend-driver
        hostPath:
          path: /usr/local/Ascend/driver
      - name: ascend-add-ons
        hostPath:
          path: /usr/local/Ascend/add-ons
      - name: hccn
        hostPath:
          path: /etc/hccn.conf
      - name: localtime
        hostPath:
          path: /etc/localtime
      - name: npu-smi
        hostPath:
          path: /usr/local/sbin/npu-smi
      - name: model-path
        hostPath:
          path: ${model-path}
      - name: node-path
        hostPath:
          path: ${node-path}

参数说明：

${container_name}：容器名称，此处可以自己定义一个容器名称，例如ascend-vllm。
${image_name}：Step3 制作推理镜像构建的推理镜像名称。
${node-path}：节点自定义目录，该目录下包含pod配置文件config.yaml和推理服务启动脚本run_vllm.sh，run_vllm.sh内容见Step3 创建服务启动脚本。
${model-path}：Step1 上传权重文件中上传的模型权重路径。

上一篇：AI开发平台MODELARTS-部署推理服务:Step4 创建pod

下一篇：AI开发平台MODELARTS-部署推理服务:Step4 创建pod