Record successful baseline restoration after MTP test

audit/runtime/acceptance-after-mtp.jsonl

@@ -0,0 +1,4 @@
+{"models": {"object": "list", "data": [{"id": "qwen3.8-flash-next", "object": "model", "created": 1789700529, "owned_by": "vllm", "root": "/model", "parent": null, "max_model_len": 131072, "permission": [{"id": "modelperm-933247eb769488b6", "object": "model_permission", "created": 1789700529, "allow_create_engine": false, "allow_sampling": true, "allow_logprobs": true, "allow_search_indices": false, "allow_view": true, "allow_fine_tuning": false, "organization": "*", "group": null, "is_blocking": false}]}]}}
+{"test": "math", "passed": true, "content": "\n\n323", "reasoning_chars": 111, "ttft_s": 0.20431093200022588, "elapsed_s": 0.9834625690000394, "finish_reason": "stop", "usage": {"prompt_tokens": 55, "total_tokens": 115, "completion_tokens": 60, "completion_tokens_details": {"reasoning_tokens": 54}}}
+{"test": "chinese", "passed": true, "content": "\n\n模型已就绪", "reasoning_chars": 53, "ttft_s": 0.07846190500004013, "elapsed_s": 0.5019190799994249, "finish_reason": "stop", "usage": {"prompt_tokens": 47, "total_tokens": 80, "completion_tokens": 33, "completion_tokens_details": {"reasoning_tokens": 27}}}
+{"test": "tool", "passed": true, "response": {"id": "chatcmpl-b6b3720854c14b05", "object": "chat.completion", "created": 1789700530, "model": "qwen3.8-flash-next", "choices": [{"index": 0, "message": {"role": "assistant", "content": null, "refusal": null, "annotations": null, "audio": null, "function_call": null, "tool_calls": [{"id": "chatcmpl-tool-a6205fc98e7ef570", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Shanghai\"}"}}], "reasoning": "The user wants me to check the weather in Shanghai using the get_weather tool. This is a straightforward single tool call.\n"}, "logprobs": null, "finish_reason": "tool_calls", "stop_reason": null, "token_ids": null, "routed_experts": null}], "service_tier": null, "system_fingerprint": "vllm-0.3.1.dev3+g0bfc7a15d-9c4a3436", "usage": {"prompt_tokens": 301, "total_tokens": 355, "completion_tokens": 54, "prompt_tokens_details": null, "completion_tokens_details": {"reasoning_tokens": 25}}, "prompt_logprobs": null, "prompt_token_ids": null, "prompt_text": null, "kv_transfer_params": null, "ec_transfer_params": null, "metrics": null}}

audit/runtime/mtp2-0985-server.log

@@ -63,18 +63,30 @@
 (EngineCore pid=112) WARNING 09-18 02:56:06 [compilation.py:1350] Op 'quant_fp8' not present in model, enabling with '+quant_fp8' has no effect
 (EngineCore pid=112) INFO 09-18 02:56:06 [weight_utils.py:900] Filesystem type for checkpoints: EXT4. Checkpoint size: 123.57 GiB. Available RAM: 142.29 GiB.
 (EngineCore pid=112) INFO 09-18 02:56:06 [weight_utils.py:923] Auto-prefetch is disabled because the filesystem (EXT4) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:   0% Completed | 0/11 [00:00<?, ?it/s]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:   9% Completed | 1/11 [00:01<00:16,  1.63s/it]
+Loading safetensors checkpoint shards:   0% Completed | 0/11 [00:00<?, ?it/s]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  18% Completed | 2/11 [00:06<00:33,  3.78s/it]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  27% Completed | 3/11 [00:12<00:36,  4.57s/it]
+Loading safetensors checkpoint shards:   9% Completed | 1/11 [00:01<00:16,  1.63s/it]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  36% Completed | 4/11 [00:17<00:34,  4.95s/it]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  45% Completed | 5/11 [00:23<00:31,  5.20s/it]
+Loading safetensors checkpoint shards:  18% Completed | 2/11 [00:06<00:33,  3.78s/it]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  55% Completed | 6/11 [00:29<00:26,  5.36s/it]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  64% Completed | 7/11 [00:34<00:21,  5.45s/it]
+Loading safetensors checkpoint shards:  27% Completed | 3/11 [00:12<00:36,  4.57s/it]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  73% Completed | 8/11 [00:40<00:16,  5.53s/it]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  82% Completed | 9/11 [00:42<00:08,  4.25s/it]
+Loading safetensors checkpoint shards:  36% Completed | 4/11 [00:17<00:34,  4.95s/it]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:42<00:00,  2.43s/it]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:42<00:00,  3.89s/it]
+Loading safetensors checkpoint shards:  45% Completed | 5/11 [00:23<00:31,  5.20s/it]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards:  55% Completed | 6/11 [00:29<00:26,  5.36s/it]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards:  64% Completed | 7/11 [00:34<00:21,  5.45s/it]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards:  73% Completed | 8/11 [00:40<00:16,  5.53s/it]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards:  82% Completed | 9/11 [00:42<00:08,  4.25s/it]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:42<00:00,  2.43s/it]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:42<00:00,  3.89s/it]
 (EngineCore pid=112)
 (EngineCore pid=112) INFO 09-18 02:56:49 [default_loader.py:430] Loading weights took 42.97 seconds
 (EngineCore pid=112) INFO 09-18 02:56:49 [nvfp4.py:611] Using MoEPrepareAndFinalizeNoDPEPModular
@@ -84,17 +96,28 @@
 (EngineCore pid=112) INFO 09-18 02:56:49 [compilation.py:331] Enabled custom fusions: norm_quant, act_quant
 (EngineCore pid=112) INFO 09-18 02:56:50 [fp8.py:433] Using DEEPGEMM Fp8 MoE backend out of potential backends: ['AITER', 'FLASHINFER_TRTLLM', 'FLASHINFER_CUTLASS', 'DEEPGEMM', 'TRITON', 'MARLIN', 'HUMMING', 'BATCHED_DEEPGEMM', 'BATCHED_TRITON', 'XPU', 'CPU', 'HPC'].
 (EngineCore pid=112) INFO 09-18 02:56:50 [weight_utils.py:900] Filesystem type for checkpoints: EXT4. Checkpoint size: 123.57 GiB. Available RAM: 142.10 GiB.
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:   0% Completed | 0/11 [00:00<?, ?it/s]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  18% Completed | 2/11 [00:00<00:01,  5.24it/s]
+Loading safetensors checkpoint shards:   0% Completed | 0/11 [00:00<?, ?it/s]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  27% Completed | 3/11 [00:00<00:01,  4.08it/s]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  36% Completed | 4/11 [00:01<00:01,  3.67it/s]
+Loading safetensors checkpoint shards:  18% Completed | 2/11 [00:00<00:01,  5.24it/s]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  45% Completed | 5/11 [00:01<00:01,  3.45it/s]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  55% Completed | 6/11 [00:01<00:01,  3.32it/s]
+Loading safetensors checkpoint shards:  27% Completed | 3/11 [00:00<00:01,  4.08it/s]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  64% Completed | 7/11 [00:01<00:01,  3.26it/s]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  73% Completed | 8/11 [00:02<00:00,  3.21it/s]
+Loading safetensors checkpoint shards:  36% Completed | 4/11 [00:01<00:01,  3.67it/s]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards:  82% Completed | 9/11 [00:02<00:00,  3.71it/s]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:02<00:00,  3.89it/s]
+Loading safetensors checkpoint shards:  45% Completed | 5/11 [00:01<00:01,  3.45it/s]
-(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:02<00:00,  3.70it/s]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards:  55% Completed | 6/11 [00:01<00:01,  3.32it/s]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards:  64% Completed | 7/11 [00:01<00:01,  3.26it/s]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards:  73% Completed | 8/11 [00:02<00:00,  3.21it/s]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards:  82% Completed | 9/11 [00:02<00:00,  3.71it/s]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:02<00:00,  3.89it/s]
+(EngineCore pid=112)
+Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:02<00:00,  3.70it/s]
 (EngineCore pid=112)
 (EngineCore pid=112) INFO 09-18 02:56:53 [default_loader.py:430] Loading weights took 2.98 seconds
 (EngineCore pid=112) INFO 09-18 02:56:53 [deep_gemm.py:196] deep_gemm not found in site-packages, trying vendored vllm.third_party.deep_gemm
@@ -116,10 +139,19 @@
 (EngineCore pid=112) WARNING 09-18 02:57:23 [kv_cache_utils.py:2219] Speculative decoding (method=mtp) is enabled but no KV cache group could be identified as the draft model's.
 (EngineCore pid=112) WARNING 09-18 02:57:23 [compilation.py:1415] CUDAGraphMode.FULL is not supported with GDNAttentionBackend backend (support: AttentionCGSupport.UNIFORM_BATCH); setting cudagraph_mode=FULL_DECODE_ONLY
 (EngineCore pid=112) INFO 09-18 02:57:23 [speculator.py:119] Fused multi-step draft decode is not supported by attention backend(s) QWEN4_EXP_EXP_QSA_STATE; falling back to rebuilding attention metadata between draft steps.
-(EngineCore pid=112) ␍Capturing CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]␍Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:01<00:00,  1.83s/it]␍Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:01<00:00,  1.83s/it]
+(EngineCore pid=112)
+Capturing CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]
+Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:01<00:00,  1.83s/it]
+Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:01<00:00,  1.83s/it]
 (EngineCore pid=112) INFO 09-18 02:57:25 [speculator.py:150] Capturing model for speculator...
-(EngineCore pid=112) ␍Capturing prefill CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]␍Capturing prefill CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00,  2.15it/s]␍Capturing prefill CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00,  2.15it/s]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Capturing decode CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]␍Capturing decode CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00,  5.63it/s]␍Capturing decode CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00,  5.62it/s]
+Capturing prefill CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]
+Capturing prefill CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00,  2.15it/s]
+Capturing prefill CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00,  2.15it/s]
+(EngineCore pid=112)
+Capturing decode CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]
+Capturing decode CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00,  5.63it/s]
+Capturing decode CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00,  5.62it/s]
 (EngineCore pid=112) INFO 09-18 02:57:27 [model_runner.py:1057] Graph capturing finished in 3 secs, took 0.14 GiB
 (EngineCore pid=112) INFO 09-18 02:57:27 [gpu_worker.py:641] Available KV cache memory: 2.78 GiB
 (EngineCore pid=112) INFO 09-18 02:57:27 [gpu_worker.py:656] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9850 is equivalent to --gpu-memory-utilization=0.9833 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9867. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
@@ -133,10 +165,16 @@
 (EngineCore pid=112) INFO 09-18 02:57:28 [qwen4_exp_qsa_warmup.py:71] Warmed up Qwen4Exp QSA decode kernels: ((1, 1), (2, 1), (3, 1)).
 (EngineCore pid=112) INFO 09-18 02:57:34 [qwen4_exp_qsa_warmup.py:85] Warmed up Qwen4Exp QSA sparse attention kernels: ((32, 2, 1), (32, 4, 4), (64, 1, 2), (64, 4, 4), (64, 8, 4), (64, 33, 8), (128, 4, 4), (128, 8, 4)).
 (EngineCore pid=112) INFO 09-18 02:57:34 [kernel_warmup.py:254] Skipping FlashInfer autotune because it is disabled.
-(EngineCore pid=112) ␍Capturing CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]␍Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 10.02it/s]
+(EngineCore pid=112)
+Capturing CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]
+Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 10.02it/s]
 (EngineCore pid=112) INFO 09-18 02:57:42 [speculator.py:150] Capturing model for speculator...
-(EngineCore pid=112) ␍Capturing prefill CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]␍Capturing prefill CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 122.54it/s]
+(EngineCore pid=112)
-(EngineCore pid=112) ␍Capturing decode CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]␍Capturing decode CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 178.09it/s]
+Capturing prefill CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]
+Capturing prefill CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 122.54it/s]
+(EngineCore pid=112)
+Capturing decode CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]
+Capturing decode CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 178.09it/s]
 (EngineCore pid=112) INFO 09-18 02:57:43 [model_runner.py:1057] Graph capturing finished in 1 secs, took 0.13 GiB
 (EngineCore pid=112) INFO 09-18 02:57:43 [gpu_worker.py:824] CUDA graph pool memory: 0.13 GiB (actual), 0.14 GiB (estimated), difference: 0.01 GiB (5.8%).
 (EngineCore pid=112) INFO 09-18 02:57:43 [gpu_worker.py:887] Free memory on device (82.59/83.05 GiB) on startup. Desired GPU memory utilization is (0.985, 81.8 GiB). Actual usage is 77.85 GiB for consumed memory (weights + non-torch), 1.17 GiB for peak activation, and 0.13 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=2685284312` (2.5 GiB) to fit into requested memory, or `--kv-cache-memory=3531951104` (3.29 GiB) to fully utilize gpu memory. Current kv cache memory in use is 2.78 GiB.

docs/mtp-test.md

@@ -42,3 +42,5 @@
 **默认仍保留无 MTP 的 128K 多模态基线。** MTP 2 有明显短输出吞吐收益，但需要 98.5% 显存预算，且多模态测试只覆盖小图片和短视频，没有长视频、较长输出、多任务真实质量或长期压力测试。现有证据不足以把它称为更稳定的默认配置。
 
 实验配置保存在 `audit/runtime/compose-mtp2-0985.yaml`，关闭了自动重启。根目录 Compose 不修改；结束测试后恢复原基线。没有测试 MTP 1、3 或更多预测步，当前结果不代表参数最优。
+
+恢复验收完成：原配置重新启动后中文、数学和工具调用通过；容器 healthy、OOMKilled=false、restart=unless-stopped。回执为 `audit/runtime/acceptance-after-mtp.jsonl`。日志导出时已遮蔽 API key，并规范化行尾空白。