Atlas 310P 单卡部署实测
原创
大圣
大圣
TQSec网络安全
2026年11月5日 17:48
上海
去阅读
环境信息
使用官网https://hidevlab.huawei.com单卡环境部署
- vLLM-Ascend 0.26.0rc1
- 1NPU(Altas 310P)
- 测试模型 Qwen3.5-4B
- 精度 float16
- 上下文 4096
- 生成速度 10.87 tok/s
测试过程
export ASCEND_WORK_PATH=/tmp/ascend_cache && mkdir -p $ASCEND_WORK_PATH# 目的：vLLM-Ascend 在编译算子时，默认尝试在只读的模型目录下创建 kernel_meta 文件夹，导致 Read-only file system 失败。# 现在让 Ascend 将编译缓存文件生成到可写路径。执行下面这条命令，指定一个临时目录（/tmp/ascend_cache）作为工作路径。 vllm serve /workspace/shared_assets/models/Qwen/Qwen3.5-4B \\ --served-model-name qwen3.5-4b \\ --port 8000 \\ --dtype float16 \\ --max-model-len 4096 \\ --enforce-eager \\ --mamba-ssm-cache-dtype float16# 目的：加载模型并启动服务
运行完成后的输出应当如下(仅截取部分输出)
(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/chat/completions/batch, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/responses, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/completions, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/messages, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/messages/count_tokens, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /generative_scoring, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/chat/completions/render, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/completions/render, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/chat/completions/derender, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /v1/completions/derender, Methods: POST(APIServer pid=65735) INFO 11-05 08:55:26 [launcher.py:46] Route: /inference/v1/generate, Methods: POST(APIServer pid=65735) INFO: Started server process [65735](APIServer pid=65735) INFO: Waiting for application startup.(APIServer pid=65735) INFO: Application startup complete.
测试模型
curl -s http://127.0.0.1:8000/v1/chat/completions \\ -H ”Content-Type: application/json” \\ -d '{ ”model”: ”qwen3.5-4b”, ”messages”: [{”role”: ”user”, ”content”: ”你好，用一句话介绍你自己”}], ”max_tokens”: 128, ”temperature”: 0.7, ”chat_template_kwargs”: {”enable_thinking”: false} }'
注意: 该模型上下文仅4096，所以必须关闭思考，不然没有输出
资源占用信息
npu-smi info && echo ”-----” && free -h
root@876806df0804:/workspace # npu-smi info && echo ”-----” && free -h+--------------------------------------------------------------------------------------------------------+| npu-smi 26.0.rc1 Version: 26.0.rc1 |+-------------------------------+-----------------+------------------------------------------------------+| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) || Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) |+===============================+=================+======================================================+| 32768 310P3 | OK | NA 49 18904 / 18904 || 0 0 | 0000:81:00.0 | 0 39331/ 44278 |+-------------------------------+-----------------+------------------------------------------------------+| 32768 310P3 | OK | NA 45 0 / 0 || 1 1 | 0000:81:00.0 | 0 1479 / 43693 |+===============================+=================+======================================================++-------------------------------+-----------------+------------------------------------------------------+| NPU Chip | Process id | Process name | Process memory(MB) |+===============================+=================+======================================================+| 32768 0 | 66557 | VLLMEngineCor | 37902 |+===============================+=================+======================================================+----- total used free shared buff/cache availableMem: 470Gi 22Gi 98Gi 111Mi 350Gi 445GiSwap: 4.0Gi 364Mi 3.6Gi
NPU 0 号芯片：显存占用 39331 / 44278 MB（约 89%），其中 VLLMEngineCor 进程占 37902 MB。这是 vLLM 默认 --gpu-memory-utilization 0.9 预分配 KV cache 的结果，属于正常现象，剩余约 4.9 GB 留给激活值和临时张量。
Hugepages：18904 / 18904 已满，310P 上正常。
系统内存：470 GB 总量，可用 434 GB。
NPU1号芯片：空闲（1479MB为系统占用），符合我们只用了ASCEND\_RT\_VISIBLE\_DEVICES=0。
Token速度测试
vllm bench serve \\ --model /workspace/shared_assets/models/Qwen/Qwen3.5-4B \\ --served-model-name qwen3.5-4b \\ --base-url http://127.0.0.1:8000 \\ --dataset-name random \\ --random-input-len 128 \\ --random-output-len 256 \\ --num-prompts 10 \\ --max-concurrency 1 \\ --ignore-eos
============ Serving Benchmark Result ============Successful requests: 10Failed requests: 0Maximum request concurrency: 1Benchmark duration (s): 235.50Total input tokens: 1280Total generated tokens: 2560Request throughput (req/s): 0.04Output token throughput (tok/s): 10.87Peak output token throughput (tok/s): 12.00Peak concurrent requests: 2.00Total token throughput (tok/s): 16.31---------------Time to First Token----------------Mean TTFT (ms): 567.26Median TTFT (ms): 523.29P99 TTFT (ms): 917.19-----Time per Output Token (excl. 1st token)------Mean TPOT (ms): 90.12Median TPOT (ms): 90.00P99 TPOT (ms): 91.40---------------Inter-token Latency----------------Mean ITL (ms): 90.12Median ITL (ms): 90.23P99 ITL (ms): 97.23==================================================
单请求性能（batch=1）Output token throughput: 10.87 tok/s，即约 90 ms/token（TPOT 90.12 ms）TTFT 均值 567 ms（128 输入 token 的 prefill）
建议:在请求4B模型时建议\"temperature\": 0.0，不然4B模型输出的内容过于不准确。
版权声明
本站仅做备份收录，仅供研究与教学参考之用。
读者将信息用于其他用途的，全部法律及连带责任由读者自行承担，本站不承担任何责任。
评论