Document MTP 2 speed and memory tradeoff; retain stable default
This commit is contained in:
@@ -8,6 +8,7 @@
|
||||
|
||||
- `compose.yaml`、`Dockerfile`：已验收部署配置与固定镜像。
|
||||
- `patches/`：最小补丁、精确源码保护及许可证。
|
||||
- [MTP 实测](docs/mtp-test.md)：约 1.93 倍短输出吞吐，需要更高显存预算，暂不设为默认。
|
||||
- [部署心得](docs/lessons.md)：取舍、踩坑、优化方向及升级方法。
|
||||
- `scripts/`：功能、上下文、速度和资源验证工具。
|
||||
- `audit/runtime/`：2026-09-18 历史原始回执，不代表实时状态。
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
{"models": {"object": "list", "data": [{"id": "qwen3.8-flash-next", "object": "model", "created": 1789700272, "owned_by": "vllm", "root": "/model", "parent": null, "max_model_len": 131072, "permission": [{"id": "modelperm-939fb8389720742a", "object": "model_permission", "created": 1789700272, "allow_create_engine": false, "allow_sampling": true, "allow_logprobs": true, "allow_search_indices": false, "allow_view": true, "allow_fine_tuning": false, "organization": "*", "group": null, "is_blocking": false}]}]}}
|
||||
{"test": "math", "passed": true, "content": "\n\n323", "reasoning_chars": 111, "ttft_s": 0.667495921000409, "elapsed_s": 1.0160472059997119, "finish_reason": "stop", "usage": {"prompt_tokens": 55, "total_tokens": 115, "completion_tokens": 60, "completion_tokens_details": {"reasoning_tokens": 54}}}
|
||||
{"test": "chinese", "passed": true, "content": "\n\n模型已就绪", "reasoning_chars": 49, "ttft_s": 0.13881150700035505, "elapsed_s": 0.427693612999974, "finish_reason": "stop", "usage": {"prompt_tokens": 47, "total_tokens": 79, "completion_tokens": 32, "completion_tokens_details": {"reasoning_tokens": 26}}}
|
||||
{"test": "tool", "passed": true, "response": {"id": "chatcmpl-8d87f3f47ce4459d", "object": "chat.completion", "created": 1789700274, "model": "qwen3.8-flash-next", "choices": [{"index": 0, "message": {"role": "assistant", "content": null, "refusal": null, "annotations": null, "audio": null, "function_call": null, "tool_calls": [{"id": "chatcmpl-tool-8b192b2a4b8a087e", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Shanghai\"}"}}], "reasoning": "The user wants me to check the weather in Shanghai using the get_weather tool. This is a straightforward single tool call.\n"}, "logprobs": null, "finish_reason": "tool_calls", "stop_reason": null, "token_ids": null, "routed_experts": null}], "service_tier": null, "system_fingerprint": "vllm-0.3.1.dev3+g0bfc7a15d-9e1c6e32", "usage": {"prompt_tokens": 301, "total_tokens": 355, "completion_tokens": 54, "prompt_tokens_details": null, "completion_tokens_details": {"reasoning_tokens": 25}}, "prompt_logprobs": null, "prompt_token_ids": null, "prompt_text": null, "kv_transfer_params": null, "ec_transfer_params": null, "metrics": null}}
|
||||
@@ -0,0 +1,4 @@
|
||||
{"run": 0, "warmup": true, "ttft_s": 0.07895353000003524, "elapsed_s": 1.795279004000804, "decode_tps": 148.5732186946995, "usage": {"prompt_tokens": 58, "total_tokens": 314, "completion_tokens": 256, "completion_tokens_details": {"reasoning_tokens": 247}}, "content": "\n\n# Merging Overlapping Intervals", "reasoning": "The user wants a Python function that merges overlapping intervals, along with an explanation of the algorithm, edge cases, and complexity analysis.\n\nLet me think about this:\n\n1. **Algorithm**: The classic approach is:\n - Sort intervals by start time\n - Iterate through sorted intervals, merging when there's overlap\n - Two intervals overlap if the start of the next is <= the end of the current\n\n2. **Edge cases to consider**:\n - Empty input\n - Single interval\n - All intervals overlap into one\n - No intervals overlap\n - Intervals that touch (e.g., [1,3] and [3,5]) \u2014 should these merge? Typically yes, since they share a point.\n - Intervals with same start\n - Intervals with same end\n - Unsorted input\n\n3. **Complexity**:\n - Time: O(n log n) due to sorting\n - Space: O(n) for the output (or O(1) extra if we modify in place, but typically we return a new list)\n\nLet me write a clean, well-documented solution with type hints and a thorough explanation.\n"}
|
||||
{"run": 1, "warmup": false, "ttft_s": 0.08640880700022535, "elapsed_s": 1.8029339949998757, "decode_tps": 148.55593252154011, "usage": {"prompt_tokens": 58, "total_tokens": 314, "completion_tokens": 256, "completion_tokens_details": {"reasoning_tokens": 247}}, "content": "\n\n# Merging Overlapping Intervals", "reasoning": "The user wants a Python function that merges overlapping intervals, along with an explanation of the algorithm, edge cases, and complexity analysis.\n\nLet me think about this:\n\n1. **Algorithm**: The classic approach is:\n - Sort intervals by start time\n - Iterate through sorted intervals, merging when there's overlap\n - Two intervals overlap if the start of the next is <= the end of the current\n\n2. **Edge cases to consider**:\n - Empty input\n - Single interval\n - All intervals overlap into one\n - No intervals overlap\n - Intervals that touch (e.g., [1,3] and [3,5]) \u2014 should these merge? Typically yes, since they share a point.\n - Intervals with same start\n - Intervals with same end\n - Unsorted input\n\n3. **Complexity**:\n - Time: O(n log n) due to sorting\n - Space: O(n) for the output (or O(1) extra if we modify in place, but typically we return a new list)\n\nLet me write a clean, well-documented solution with type hints and a thorough explanation.\n"}
|
||||
{"run": 2, "warmup": false, "ttft_s": 0.08631457799947384, "elapsed_s": 1.8029650199996468, "decode_tps": 148.54509325898877, "usage": {"prompt_tokens": 58, "total_tokens": 314, "completion_tokens": 256, "completion_tokens_details": {"reasoning_tokens": 247}}, "content": "\n\n# Merging Overlapping Intervals", "reasoning": "The user wants a Python function that merges overlapping intervals, along with an explanation of the algorithm, edge cases, and complexity analysis.\n\nLet me think about this:\n\n1. **Algorithm**: The classic approach is:\n - Sort intervals by start time\n - Iterate through sorted intervals, merging when there's overlap\n - Two intervals overlap if the start of the next is <= the end of the current\n\n2. **Edge cases to consider**:\n - Empty input\n - Single interval\n - All intervals overlap into one\n - No intervals overlap\n - Intervals that touch (e.g., [1,3] and [3,5]) \u2014 should these merge? Typically yes, since they share a point.\n - Intervals with same start\n - Intervals with same end\n - Unsorted input\n\n3. **Complexity**:\n - Time: O(n log n) due to sorting\n - Space: O(n) for the output (or O(1) extra if we modify in place, but typically we return a new list)\n\nLet me write a clean, well-documented solution with type hints and a thorough explanation.\n"}
|
||||
{"run": 3, "warmup": false, "ttft_s": 0.0862306319995696, "elapsed_s": 1.8031676069995228, "decode_tps": 148.52030314042653, "usage": {"prompt_tokens": 58, "total_tokens": 314, "completion_tokens": 256, "completion_tokens_details": {"reasoning_tokens": 247}}, "content": "\n\n# Merging Overlapping Intervals", "reasoning": "The user wants a Python function that merges overlapping intervals, along with an explanation of the algorithm, edge cases, and complexity analysis.\n\nLet me think about this:\n\n1. **Algorithm**: The classic approach is:\n - Sort intervals by start time\n - Iterate through sorted intervals, merging when there's overlap\n - Two intervals overlap if the start of the next is <= the end of the current\n\n2. **Edge cases to consider**:\n - Empty input\n - Single interval\n - All intervals overlap into one\n - No intervals overlap\n - Intervals that touch (e.g., [1,3] and [3,5]) \u2014 should these merge? Typically yes, since they share a point.\n - Intervals with same start\n - Intervals with same end\n - Unsorted input\n\n3. **Complexity**:\n - Time: O(n log n) due to sorting\n - Space: O(n) for the output (or O(1) extra if we modify in place, but typically we return a new list)\n\nLet me write a clean, well-documented solution with type hints and a thorough explanation.\n"}
|
||||
@@ -0,0 +1,46 @@
|
||||
name: qwen38-flash-rtx6000d
|
||||
services:
|
||||
vllm:
|
||||
image: local/qwen38-flash-6000d:0bfc7a15
|
||||
build: .
|
||||
container_name: qwen38-flash-6000d
|
||||
gpus: all
|
||||
shm_size: 8g
|
||||
mem_limit: 210g
|
||||
memswap_limit: 210g
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "${BIND_ADDRESS:-127.0.0.1}:${API_PORT:-8000}:8000"
|
||||
environment:
|
||||
HF_HUB_OFFLINE: "1"
|
||||
VLLM_USE_V2_MODEL_RUNNER: "1"
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
OMP_NUM_THREADS: "8"
|
||||
TORCHINDUCTOR_COMPILE_THREADS: "2"
|
||||
CUTE_DSL_ARCH: sm_120a
|
||||
volumes:
|
||||
- /data/flash-next/models/nvidia-Qwen3.8-Flash-Next-NVFP4:/model:ro
|
||||
- /data/flash-next/cache:/root/.cache
|
||||
- ./secrets/api-key:/run/secrets/api-key:ro
|
||||
entrypoint: ["/bin/bash", "-lc"]
|
||||
command:
|
||||
- >-
|
||||
exec vllm serve /model
|
||||
--served-model-name qwen3.8-flash-next
|
||||
--host 0.0.0.0 --port 8000
|
||||
--tensor-parallel-size 1 --dtype bfloat16
|
||||
--engram-config '{"cpu_offload":true}'
|
||||
--kv-cache-dtype fp8 --gpu-memory-utilization 0.96
|
||||
--max-model-len 131072 --max-num-seqs 1 --max-num-batched-tokens 2048
|
||||
--enable-chunked-prefill --enable-prefix-caching
|
||||
--speculative-config '{"method":"mtp","num_speculative_tokens":2}'
|
||||
--compilation-config '{"mode":0,"cudagraph_mode":"FULL","cudagraph_capture_sizes":[1,3]}' --no-enable-flashinfer-autotune
|
||||
--load-format safetensors
|
||||
--reasoning-parser qwen3 --tool-call-parser qwen3_xml --enable-auto-tool-choice
|
||||
--api-key "$$(cat /run/secrets/api-key)"
|
||||
healthcheck:
|
||||
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 30m
|
||||
@@ -0,0 +1,46 @@
|
||||
name: qwen38-flash-rtx6000d
|
||||
services:
|
||||
vllm:
|
||||
image: local/qwen38-flash-6000d:0bfc7a15
|
||||
build: .
|
||||
container_name: qwen38-flash-6000d
|
||||
gpus: all
|
||||
shm_size: 8g
|
||||
mem_limit: 210g
|
||||
memswap_limit: 210g
|
||||
restart: "no"
|
||||
ports:
|
||||
- "${BIND_ADDRESS:-127.0.0.1}:${API_PORT:-8000}:8000"
|
||||
environment:
|
||||
HF_HUB_OFFLINE: "1"
|
||||
VLLM_USE_V2_MODEL_RUNNER: "1"
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
OMP_NUM_THREADS: "8"
|
||||
TORCHINDUCTOR_COMPILE_THREADS: "2"
|
||||
CUTE_DSL_ARCH: sm_120a
|
||||
volumes:
|
||||
- /data/flash-next/models/nvidia-Qwen3.8-Flash-Next-NVFP4:/model:ro
|
||||
- /data/flash-next/cache:/root/.cache
|
||||
- ./secrets/api-key:/run/secrets/api-key:ro
|
||||
entrypoint: ["/bin/bash", "-lc"]
|
||||
command:
|
||||
- >-
|
||||
exec vllm serve /model
|
||||
--served-model-name qwen3.8-flash-next
|
||||
--host 0.0.0.0 --port 8000
|
||||
--tensor-parallel-size 1 --dtype bfloat16
|
||||
--engram-config '{"cpu_offload":true}'
|
||||
--kv-cache-dtype fp8 --gpu-memory-utilization 0.985
|
||||
--max-model-len 131072 --max-num-seqs 1 --max-num-batched-tokens 2048
|
||||
--enable-chunked-prefill --enable-prefix-caching
|
||||
--speculative-config '{"method":"mtp","num_speculative_tokens":2}'
|
||||
--compilation-config '{"mode":0,"cudagraph_mode":"FULL","cudagraph_capture_sizes":[1,3]}' --no-enable-flashinfer-autotune
|
||||
--load-format safetensors
|
||||
--reasoning-parser qwen3 --tool-call-parser qwen3_xml --enable-auto-tool-choice
|
||||
--api-key "$$(cat /run/secrets/api-key)"
|
||||
healthcheck:
|
||||
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 30m
|
||||
@@ -0,0 +1,45 @@
|
||||
name: qwen38-flash-rtx6000d
|
||||
services:
|
||||
vllm:
|
||||
image: local/qwen38-flash-6000d:0bfc7a15
|
||||
build: .
|
||||
container_name: qwen38-flash-6000d
|
||||
gpus: all
|
||||
shm_size: 8g
|
||||
mem_limit: 210g
|
||||
memswap_limit: 210g
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "${BIND_ADDRESS:-127.0.0.1}:${API_PORT:-8000}:8000"
|
||||
environment:
|
||||
HF_HUB_OFFLINE: "1"
|
||||
VLLM_USE_V2_MODEL_RUNNER: "1"
|
||||
VLLM_WORKER_MULTIPROC_METHOD: spawn
|
||||
OMP_NUM_THREADS: "8"
|
||||
TORCHINDUCTOR_COMPILE_THREADS: "2"
|
||||
CUTE_DSL_ARCH: sm_120a
|
||||
volumes:
|
||||
- /data/flash-next/models/nvidia-Qwen3.8-Flash-Next-NVFP4:/model:ro
|
||||
- /data/flash-next/cache:/root/.cache
|
||||
- ./secrets/api-key:/run/secrets/api-key:ro
|
||||
entrypoint: ["/bin/bash", "-lc"]
|
||||
command:
|
||||
- >-
|
||||
exec vllm serve /model
|
||||
--served-model-name qwen3.8-flash-next
|
||||
--host 0.0.0.0 --port 8000
|
||||
--tensor-parallel-size 1 --dtype bfloat16
|
||||
--engram-config '{"cpu_offload":true}'
|
||||
--kv-cache-dtype fp8 --gpu-memory-utilization 0.96
|
||||
--max-model-len 131072 --max-num-seqs 1 --max-num-batched-tokens 2048
|
||||
--enable-chunked-prefill --enable-prefix-caching
|
||||
--compilation-config '{"mode":0,"cudagraph_mode":"FULL","cudagraph_capture_sizes":[1]}' --no-enable-flashinfer-autotune
|
||||
--load-format safetensors
|
||||
--reasoning-parser qwen3 --tool-call-parser qwen3_xml --enable-auto-tool-choice
|
||||
--api-key "$$(cat /run/secrets/api-key)"
|
||||
healthcheck:
|
||||
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 30m
|
||||
@@ -0,0 +1,2 @@
|
||||
{"repetition": 0, "content": "\n\nVIOLET-74219", "usage": {"prompt_tokens": 127988, "total_tokens": 128042, "completion_tokens": 54, "completion_tokens_details": {"reasoning_tokens": 42}}, "finish_reason": "stop", "ttft_s": 14.75369103399953, "elapsed_s": 15.083149690000027, "passed": true}
|
||||
{"repetition": 1, "content": "\n\nVIOLET-74219", "usage": {"prompt_tokens": 127988, "total_tokens": 128047, "completion_tokens": 59, "completion_tokens_details": {"reasoning_tokens": 47}}, "finish_reason": "stop", "ttft_s": 0.627140304999557, "elapsed_s": 1.0119056880002972, "passed": true}
|
||||
@@ -0,0 +1,26 @@
|
||||
# HELP vllm:spec_decode_num_drafts_total Number of spec decoding drafts.
|
||||
# TYPE vllm:spec_decode_num_drafts_total counter
|
||||
vllm:spec_decode_num_drafts_total{engine="0",model_name="qwen3.8-flash-next"} 537.0
|
||||
# HELP vllm:spec_decode_num_drafts_created Number of spec decoding drafts.
|
||||
# TYPE vllm:spec_decode_num_drafts_created gauge
|
||||
vllm:spec_decode_num_drafts_created{engine="0",model_name="qwen3.8-flash-next"} 1.78970026442134e+09
|
||||
# HELP vllm:spec_decode_num_draft_tokens_total Number of draft tokens.
|
||||
# TYPE vllm:spec_decode_num_draft_tokens_total counter
|
||||
vllm:spec_decode_num_draft_tokens_total{engine="0",model_name="qwen3.8-flash-next"} 1074.0
|
||||
# HELP vllm:spec_decode_num_draft_tokens_created Number of draft tokens.
|
||||
# TYPE vllm:spec_decode_num_draft_tokens_created gauge
|
||||
vllm:spec_decode_num_draft_tokens_created{engine="0",model_name="qwen3.8-flash-next"} 1.7897002644213574e+09
|
||||
# HELP vllm:spec_decode_num_accepted_tokens_total Number of accepted tokens.
|
||||
# TYPE vllm:spec_decode_num_accepted_tokens_total counter
|
||||
vllm:spec_decode_num_accepted_tokens_total{engine="0",model_name="qwen3.8-flash-next"} 906.0
|
||||
# HELP vllm:spec_decode_num_accepted_tokens_created Number of accepted tokens.
|
||||
# TYPE vllm:spec_decode_num_accepted_tokens_created gauge
|
||||
vllm:spec_decode_num_accepted_tokens_created{engine="0",model_name="qwen3.8-flash-next"} 1.7897002644213684e+09
|
||||
# HELP vllm:spec_decode_num_accepted_tokens_per_pos_total Accepted tokens per draft position.
|
||||
# TYPE vllm:spec_decode_num_accepted_tokens_per_pos_total counter
|
||||
vllm:spec_decode_num_accepted_tokens_per_pos_total{engine="0",model_name="qwen3.8-flash-next",position="0"} 495.0
|
||||
vllm:spec_decode_num_accepted_tokens_per_pos_total{engine="0",model_name="qwen3.8-flash-next",position="1"} 411.0
|
||||
# HELP vllm:spec_decode_num_accepted_tokens_per_pos_created Accepted tokens per draft position.
|
||||
# TYPE vllm:spec_decode_num_accepted_tokens_per_pos_created gauge
|
||||
vllm:spec_decode_num_accepted_tokens_per_pos_created{engine="0",model_name="qwen3.8-flash-next",position="0"} 1.7897002644213843e+09
|
||||
vllm:spec_decode_num_accepted_tokens_per_pos_created{engine="0",model_name="qwen3.8-flash-next",position="1"} 1.7897002644213886e+09
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"graph": {
|
||||
"decode_tps_median": 77.11544284916513,
|
||||
"ttft_median": 0.06954390600003535,
|
||||
"reasoning_tokens": [
|
||||
212,
|
||||
212,
|
||||
212
|
||||
]
|
||||
},
|
||||
"mtp2": {
|
||||
"decode_tps_median": 148.54509325898877,
|
||||
"ttft_median": 0.08631457799947384,
|
||||
"reasoning_tokens": [
|
||||
247,
|
||||
247,
|
||||
247
|
||||
]
|
||||
},
|
||||
"speedup": 1.9262690814022467,
|
||||
"identical_output": false,
|
||||
"min_host_available_gib": 171.57903671264648,
|
||||
"max_oom_kill": 0,
|
||||
"safeguard_triggered": false
|
||||
}
|
||||
@@ -0,0 +1,230 @@
|
||||
(APIServer pid=1) INFO 09-18 02:55:12 [api_utils.py:347]
|
||||
(APIServer pid=1) INFO 09-18 02:55:12 [api_utils.py:347] █ █ █▄ ▄█
|
||||
(APIServer pid=1) INFO 09-18 02:55:12 [api_utils.py:347] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.3.1.dev3+g0bfc7a15d
|
||||
(APIServer pid=1) INFO 09-18 02:55:12 [api_utils.py:347] █▄█▀ █ █ █ █ model /model
|
||||
(APIServer pid=1) INFO 09-18 02:55:12 [api_utils.py:347] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
|
||||
(APIServer pid=1) INFO 09-18 02:55:12 [api_utils.py:347]
|
||||
(APIServer pid=1) INFO 09-18 02:55:12 [api_utils.py:286] non-default args: {'model_tag': '/model', 'enable_auto_tool_choice': True, 'tool_call_parser': 'qwen3_xml', 'host': '0.0.0.0', 'api_key': '***', 'model': '/model', 'dtype': 'bfloat16', 'max_model_len': 131072, 'served_model_name': ['qwen3.8-flash-next'], 'load_format': 'safetensors', 'reasoning_parser': 'qwen3', 'gpu_memory_utilization': 0.985, 'kv_cache_dtype': 'fp8', 'enable_prefix_caching': True, 'max_num_batched_tokens': 2048, 'max_num_seqs': 1, 'enable_chunked_prefill': True, 'enable_flashinfer_autotune': False, 'speculative_config': {'method': 'mtp', 'num_speculative_tokens': 2}, 'compilation_config': {'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': [], 'ir_enable_torch_wrap': None, 'splitting_ops': None, 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': None, 'compile_ranges_endpoints': None, 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL: 2>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [1, 3], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': None, 'pass_config': {}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': None, 'static_all_moe_layers': []}, 'engram_config': EngramConfig(cpu_offload=True, embedding_across_dp=False, dp_shared_memory=False)}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) INFO 09-18 02:55:12 [model.py:691] Resolved architecture: Qwen4ExpForConditionalGeneration
|
||||
(APIServer pid=1) INFO 09-18 02:55:12 [model.py:2024] Using max model len 131072
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) INFO 09-18 02:55:15 [cache.py:345] Using fp8 data type to store kv cache. It reduces the GPU memory footprint and boosts the performance. Meanwhile, it may cause accuracy drop without a proper scaling factor
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) INFO 09-18 02:55:15 [model.py:691] Resolved architecture: Qwen4ExpMTP
|
||||
(APIServer pid=1) INFO 09-18 02:55:15 [model.py:2024] Using max model len 262144
|
||||
(APIServer pid=1) WARNING 09-18 02:55:15 [speculative.py:1360] Enabling num_speculative_tokens > 1 will run multiple times of forward on same MTP layer,which may result in lower acceptance rate
|
||||
(APIServer pid=1) INFO 09-18 02:55:15 [speculative.py:1653] Overriding draft model max model len from 262144 to 131072
|
||||
(APIServer pid=1) INFO 09-18 02:55:15 [config.py:625] Mamba cache mode is set to 'align' for Qwen4ExpForConditionalGeneration by default when prefix caching is enabled
|
||||
(APIServer pid=1) INFO 09-18 02:55:15 [vllm.py:1271] Resolved Engram configuration: EngramConfig(cpu_offload=True, embedding_across_dp=False, dp_shared_memory=False)
|
||||
(APIServer pid=1) INFO 09-18 02:55:15 [vllm.py:781] Auto-enabling VLLM_USE_BREAKABLE_CUDAGRAPH=1. Set VLLM_USE_BREAKABLE_CUDAGRAPH=0 to opt out.
|
||||
(APIServer pid=1) INFO 09-18 02:55:15 [kernel.py:408] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'], gelu_and_mul_sparse=['triton', 'native'])
|
||||
(APIServer pid=1) WARNING 09-18 02:55:15 [vllm.py:2152] max_num_scheduled_tokens is set to 2048 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens.
|
||||
(APIServer pid=1) INFO 09-18 02:55:15 [compilation.py:331] Enabled custom fusions: norm_quant, act_quant
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) INFO 09-18 02:55:28 [core.py:123] Initializing a V1 LLM engine (v0.3.1.dev3+g0bfc7a15d) with config: model='/model', speculative_config=SpeculativeConfig(method='mtp', model='/model', num_spec_tokens=2), tokenizer='/model', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=131072, download_dir=None, load_format=safetensors, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=modelopt_mixed, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=fp8, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='qwen3', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=qwen3.8-flash-next, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['+quant_fp8', 'all', '+quant_fp8'], 'ir_enable_torch_wrap': False, 'splitting_ops': [], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL: 2>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 3], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 3, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'], gelu_and_mul_sparse=['triton', 'native']), enable_flashinfer_autotune=False, enable_cutedsl_warmup=True, enable_jit_warmup=True, moe_backend='auto', sparse_indexer_topk_backend='auto', linear_backend='auto', linear_backend_per_quant=None)
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) INFO 09-18 02:55:29 [parallel_state.py:1825] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_12361652d17045ad82ebf3d8d70939b6 backend=nccl
|
||||
(EngineCore pid=112) INFO 09-18 02:55:29 [parallel_state.py:2269] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, ETP rank 0, EP rank 0, EPLB rank N/A
|
||||
(EngineCore pid=112) INFO 09-18 02:55:29 [gpu_worker.py:441] Using V2 Model Runner
|
||||
(EngineCore pid=112) INFO 09-18 02:55:30 [model_runner.py:387] Loading model from scratch...
|
||||
(EngineCore pid=112) INFO 09-18 02:55:30 [cuda.py:595] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
|
||||
(EngineCore pid=112) INFO 09-18 02:55:30 [mm_encoder_attention.py:372] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
|
||||
(EngineCore pid=112) INFO 09-18 02:55:30 [qwen_gdn_linear_attn.py:176] Using FlashInfer GDN prefill kernel (requested=auto, head_k_dim=128).
|
||||
(EngineCore pid=112) INFO 09-18 02:55:30 [qwen_gdn_linear_attn.py:528] GDN decode kernel: cuda
|
||||
(EngineCore pid=112) INFO 09-18 02:55:32 [nvfp4.py:302] Using 'FLASHINFER_CUTLASS' NvFp4 MoE backend out of potential backends: ['FLASHINFER_TRTLLM', 'FLASHINFER_CUTEDSL', 'FLASHINFER_CUTEDSL_BATCHED', 'FLASHINFER_CUTLASS', 'VLLM_CUTLASS', 'MARLIN', 'HUMMING', 'EMULATION'].
|
||||
(APIServer pid=1) [transformers] Qwen3VL video processing does not apply the per-frame pixel cap the reference implementation (qwen-vl-utils) applies, so some videos cost far more tokens than they would there. In v5.22 the capped behavior will become the default and `cap_pixels_per_frame` will be removed. Pass `cap_pixels_per_frame=True` to adopt the reference behavior now, or `False` to keep the current behavior and silence this warning.
|
||||
(APIServer pid=1) INFO 09-18 02:55:34 [base.py:261] Multi-modal warmup completed in 12.188s
|
||||
(APIServer pid=1) INFO 09-18 02:55:35 [base.py:261] Readonly multi-modal warmup completed in 1.309s
|
||||
(EngineCore pid=112) INFO 09-18 02:56:05 [ngram_embedding.py:720] Initialized PLE embedding language_model.model.layers.1.ple.ple_embedding.ngram_embedding: quantization_method=Qwen4ExpPLEFp8EmbeddingMethod, weight_dtype=torch.float8_e4m3fn, weight_device=cpu, pinned=True
|
||||
(EngineCore pid=112) INFO 09-18 02:56:05 [flash_attn.py:1115] Using FlashAttention version 2
|
||||
(EngineCore pid=112) WARNING 09-18 02:56:06 [compilation.py:1350] Op 'quant_fp8' not present in model, enabling with '+quant_fp8' has no effect
|
||||
(EngineCore pid=112) INFO 09-18 02:56:06 [weight_utils.py:900] Filesystem type for checkpoints: EXT4. Checkpoint size: 123.57 GiB. Available RAM: 142.29 GiB.
|
||||
(EngineCore pid=112) INFO 09-18 02:56:06 [weight_utils.py:923] Auto-prefetch is disabled because the filesystem (EXT4) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 0% Completed | 0/11 [00:00<?, ?it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 9% Completed | 1/11 [00:01<00:16, 1.63s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 18% Completed | 2/11 [00:06<00:33, 3.78s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 27% Completed | 3/11 [00:12<00:36, 4.57s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 36% Completed | 4/11 [00:17<00:34, 4.95s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 45% Completed | 5/11 [00:23<00:31, 5.20s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 55% Completed | 6/11 [00:29<00:26, 5.36s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 64% Completed | 7/11 [00:34<00:21, 5.45s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 73% Completed | 8/11 [00:40<00:16, 5.53s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 82% Completed | 9/11 [00:42<00:08, 4.25s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:42<00:00, 2.43s/it]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:42<00:00, 3.89s/it]
|
||||
(EngineCore pid=112)
|
||||
(EngineCore pid=112) INFO 09-18 02:56:49 [default_loader.py:430] Loading weights took 42.97 seconds
|
||||
(EngineCore pid=112) INFO 09-18 02:56:49 [nvfp4.py:611] Using MoEPrepareAndFinalizeNoDPEPModular
|
||||
(EngineCore pid=112) INFO 09-18 02:56:49 [vllm.py:1271] Resolved Engram configuration: EngramConfig(cpu_offload=True, embedding_across_dp=False, dp_shared_memory=False)
|
||||
(EngineCore pid=112) INFO 09-18 02:56:49 [kernel.py:408] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'], gelu_and_mul_sparse=['triton', 'native'])
|
||||
(EngineCore pid=112) WARNING 09-18 02:56:49 [vllm.py:2152] max_num_scheduled_tokens is set to 2048 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens.
|
||||
(EngineCore pid=112) INFO 09-18 02:56:49 [compilation.py:331] Enabled custom fusions: norm_quant, act_quant
|
||||
(EngineCore pid=112) INFO 09-18 02:56:50 [fp8.py:433] Using DEEPGEMM Fp8 MoE backend out of potential backends: ['AITER', 'FLASHINFER_TRTLLM', 'FLASHINFER_CUTLASS', 'DEEPGEMM', 'TRITON', 'MARLIN', 'HUMMING', 'BATCHED_DEEPGEMM', 'BATCHED_TRITON', 'XPU', 'CPU', 'HPC'].
|
||||
(EngineCore pid=112) INFO 09-18 02:56:50 [weight_utils.py:900] Filesystem type for checkpoints: EXT4. Checkpoint size: 123.57 GiB. Available RAM: 142.10 GiB.
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 0% Completed | 0/11 [00:00<?, ?it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 18% Completed | 2/11 [00:00<00:01, 5.24it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 27% Completed | 3/11 [00:00<00:01, 4.08it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 36% Completed | 4/11 [00:01<00:01, 3.67it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 45% Completed | 5/11 [00:01<00:01, 3.45it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 55% Completed | 6/11 [00:01<00:01, 3.32it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 64% Completed | 7/11 [00:01<00:01, 3.26it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 73% Completed | 8/11 [00:02<00:00, 3.21it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 82% Completed | 9/11 [00:02<00:00, 3.71it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:02<00:00, 3.89it/s]
|
||||
(EngineCore pid=112) ␍Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:02<00:00, 3.70it/s]
|
||||
(EngineCore pid=112)
|
||||
(EngineCore pid=112) INFO 09-18 02:56:53 [default_loader.py:430] Loading weights took 2.98 seconds
|
||||
(EngineCore pid=112) INFO 09-18 02:56:53 [deep_gemm.py:196] deep_gemm not found in site-packages, trying vendored vllm.third_party.deep_gemm
|
||||
(EngineCore pid=112) INFO 09-18 02:56:53 [deep_gemm.py:223] DeepGEMM PDL enabled on vllm.third_party.deep_gemm.
|
||||
(EngineCore pid=112) INFO 09-18 02:56:53 [deep_gemm.py:136] DeepGEMM E8M0 enabled on current platform.
|
||||
(EngineCore pid=112) INFO 09-18 02:56:56 [fp8.py:733] Using MoEPrepareAndFinalizeNoDPEPModular
|
||||
(EngineCore pid=112) WARNING 09-18 02:56:56 [speculator.py:235] Draft model Qwen4ExpMTP does not support external multimodal embeddings. Embeddings from the target model will not be passed to the drafter; using text-only draft inputs instead.
|
||||
(EngineCore pid=112) INFO 09-18 02:56:57 [model_runner.py:419] Model loading took 76.36 GiB memory and 86.921433 seconds
|
||||
(EngineCore pid=112) INFO 09-18 02:56:57 [topk_topp_sampler.py:78] Using FlashInfer for top-p & top-k sampling.
|
||||
(EngineCore pid=112) INFO 09-18 02:56:57 [interface.py:918] Setting attention block size to 3184 tokens to ensure that attention page size is >= mamba page size.
|
||||
(EngineCore pid=112) INFO 09-18 02:56:57 [interface.py:942] Padding mamba page size by 0.38% to ensure that mamba page size and attention page size are exactly equal.
|
||||
(EngineCore pid=112) INFO 09-18 02:56:57 [utils.py:320] Using BLNHC KV cache layout.
|
||||
(EngineCore pid=112) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) [transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
|
||||
(EngineCore pid=112) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(EngineCore pid=112) INFO 09-18 02:57:01 [encoder_runner.py:131] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
|
||||
(EngineCore pid=112) WARNING 09-18 02:57:23 [kv_cache_utils.py:2219] Speculative decoding (method=mtp) is enabled but no KV cache group could be identified as the draft model's.
|
||||
(EngineCore pid=112) WARNING 09-18 02:57:23 [compilation.py:1415] CUDAGraphMode.FULL is not supported with GDNAttentionBackend backend (support: AttentionCGSupport.UNIFORM_BATCH); setting cudagraph_mode=FULL_DECODE_ONLY
|
||||
(EngineCore pid=112) INFO 09-18 02:57:23 [speculator.py:119] Fused multi-step draft decode is not supported by attention backend(s) QWEN4_EXP_EXP_QSA_STATE; falling back to rebuilding attention metadata between draft steps.
|
||||
(EngineCore pid=112) ␍Capturing CUDA graphs (FULL): 0%| | 0/1 [00:00<?, ?it/s]␍Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:01<00:00, 1.83s/it]␍Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:01<00:00, 1.83s/it]
|
||||
(EngineCore pid=112) INFO 09-18 02:57:25 [speculator.py:150] Capturing model for speculator...
|
||||
(EngineCore pid=112) ␍Capturing prefill CUDA graphs (FULL): 0%| | 0/1 [00:00<?, ?it/s]␍Capturing prefill CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 2.15it/s]␍Capturing prefill CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 2.15it/s]
|
||||
(EngineCore pid=112) ␍Capturing decode CUDA graphs (FULL): 0%| | 0/1 [00:00<?, ?it/s]␍Capturing decode CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 5.63it/s]␍Capturing decode CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 5.62it/s]
|
||||
(EngineCore pid=112) INFO 09-18 02:57:27 [model_runner.py:1057] Graph capturing finished in 3 secs, took 0.14 GiB
|
||||
(EngineCore pid=112) INFO 09-18 02:57:27 [gpu_worker.py:641] Available KV cache memory: 2.78 GiB
|
||||
(EngineCore pid=112) INFO 09-18 02:57:27 [gpu_worker.py:656] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9850 is equivalent to --gpu-memory-utilization=0.9833 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9867. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
|
||||
(EngineCore pid=112) WARNING 09-18 02:57:27 [kv_cache_utils.py:2219] Speculative decoding (method=mtp) is enabled but no KV cache group could be identified as the draft model's.
|
||||
(EngineCore pid=112) INFO 09-18 02:57:27 [kv_cache_utils.py:2404] GPU KV cache size: 146,622 tokens, Maximum concurrency for 131,072 tokens per request: 1.12x
|
||||
(EngineCore pid=112) INFO 09-18 02:57:27 [kernel_warmup.py:171] JIT kernel warmup starting.
|
||||
(EngineCore pid=112) INFO 09-18 02:57:27 [kernel_warmup.py:184] JIT kernel warmup finished in 0.00s.
|
||||
(EngineCore pid=112) INFO 09-18 02:57:27 [qwen_vl_triton_warmup.py:57] Warmed position embedding and vision rotary kernels on grids=[(1, 16, 16), (1, 16, 2), (1, 2, 16), (1, 2, 2)].
|
||||
(EngineCore pid=112) INFO 09-18 02:57:28 [qwen_vl_triton_warmup.py:98] Warmed M-RoPE Triton kernels.
|
||||
(EngineCore pid=112) INFO 09-18 02:57:28 [mamba_triton_warmup.py:42] Warmed Mamba batch_memcpy_kernel.
|
||||
(EngineCore pid=112) INFO 09-18 02:57:28 [qwen4_exp_qsa_warmup.py:71] Warmed up Qwen4Exp QSA decode kernels: ((1, 1), (2, 1), (3, 1)).
|
||||
(EngineCore pid=112) INFO 09-18 02:57:34 [qwen4_exp_qsa_warmup.py:85] Warmed up Qwen4Exp QSA sparse attention kernels: ((32, 2, 1), (32, 4, 4), (64, 1, 2), (64, 4, 4), (64, 8, 4), (64, 33, 8), (128, 4, 4), (128, 8, 4)).
|
||||
(EngineCore pid=112) INFO 09-18 02:57:34 [kernel_warmup.py:254] Skipping FlashInfer autotune because it is disabled.
|
||||
(EngineCore pid=112) ␍Capturing CUDA graphs (FULL): 0%| | 0/1 [00:00<?, ?it/s]␍Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 10.02it/s]
|
||||
(EngineCore pid=112) INFO 09-18 02:57:42 [speculator.py:150] Capturing model for speculator...
|
||||
(EngineCore pid=112) ␍Capturing prefill CUDA graphs (FULL): 0%| | 0/1 [00:00<?, ?it/s]␍Capturing prefill CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 122.54it/s]
|
||||
(EngineCore pid=112) ␍Capturing decode CUDA graphs (FULL): 0%| | 0/1 [00:00<?, ?it/s]␍Capturing decode CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 178.09it/s]
|
||||
(EngineCore pid=112) INFO 09-18 02:57:43 [model_runner.py:1057] Graph capturing finished in 1 secs, took 0.13 GiB
|
||||
(EngineCore pid=112) INFO 09-18 02:57:43 [gpu_worker.py:824] CUDA graph pool memory: 0.13 GiB (actual), 0.14 GiB (estimated), difference: 0.01 GiB (5.8%).
|
||||
(EngineCore pid=112) INFO 09-18 02:57:43 [gpu_worker.py:887] Free memory on device (82.59/83.05 GiB) on startup. Desired GPU memory utilization is (0.985, 81.8 GiB). Actual usage is 77.85 GiB for consumed memory (weights + non-torch), 1.17 GiB for peak activation, and 0.13 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=2685284312` (2.5 GiB) to fit into requested memory, or `--kv-cache-memory=3531951104` (3.29 GiB) to fully utilize gpu memory. Current kv cache memory in use is 2.78 GiB.
|
||||
(EngineCore pid=112) INFO 09-18 02:57:43 [jit_monitor.py:84] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
|
||||
(EngineCore pid=112) WARNING 09-18 02:57:44 [torch_utils.py:274] OMP_NUM_THREADS=8 is set; leaving Torch threads at 8 for serving. Multi-threaded torch CPU ops during serving can degrade performance through spin-wait contention and cgroup CPU-quota throttling.
|
||||
(EngineCore pid=112) INFO 09-18 02:57:44 [core.py:380] init engine (profile, create kv cache, warmup model) took 47.35 s
|
||||
(EngineCore pid=112) INFO 09-18 02:57:44 [kv_cache_utils.py:762] kv cache group sizes [3184, 3184, 3184, 3184, 8, 3184]
|
||||
(EngineCore pid=112) INFO 09-18 02:57:44 [kv_cache_utils.py:763] kv lcm block sizes 3184
|
||||
(EngineCore pid=112) INFO 09-18 02:57:44 [kernel.py:408] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'], gelu_and_mul_sparse=['triton', 'native'])
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [entry.py:132] Supported tasks: ['generate']
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [factories.py:76] Scale-out endpoints are disabled. Set --enable-scale-out to enable them.
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [parser_manager.py:34] "auto" tool choice has been enabled.
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) [transformers] Unrecognized keys in `rope_parameters` for 'rope_type'='default': {'mrope_interleaved', 'mrope_section'}
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [hf.py:642] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this.
|
||||
(APIServer pid=1) WARNING 09-18 02:57:44 [model.py:1769] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 1.0, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [entry.py:136] Starting vLLM server on http://0.0.0.0:8000
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:60] Available routes are:
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /openapi.json, Methods: GET, HEAD
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /docs, Methods: GET, HEAD
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /docs/oauth2-redirect, Methods: GET, HEAD
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /redoc, Methods: GET, HEAD
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /load, Methods: GET
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /version, Methods: GET
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /health, Methods: GET
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /metrics, Methods: GET
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /tokenize, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /detokenize, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /v1/models, Methods: GET
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /ping, Methods: GET
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /ping, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /invocations, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /v1/chat/completions, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /v1/chat/completions/batch, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /v1/responses, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /v1/responses/{response_id}, Methods: GET
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /v1/responses/{response_id}/cancel, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /v1/completions, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /v1/messages, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /v1/messages/count_tokens, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /generative_scoring, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /scale_elastic_ep, Methods: POST
|
||||
(APIServer pid=1) INFO 09-18 02:57:44 [launcher.py:69] Route: /is_scaling_elastic_ep, Methods: POST
|
||||
(APIServer pid=1) INFO: Started server process [1]
|
||||
(APIServer pid=1) INFO: Waiting for application startup.
|
||||
(APIServer pid=1) INFO: Application startup complete.
|
||||
(APIServer pid=1) INFO: 127.0.0.1:48906 - "GET /health HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:45276 - "GET /health HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:45292 - "GET /v1/models HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:45294 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(EngineCore pid=112) WARNING 09-18 02:57:52 [jit_monitor.py:140] Triton kernel JIT compilation during inference: layer_norm_fwd_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
|
||||
(EngineCore pid=112) WARNING 09-18 02:57:53 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _count_expert_num_tokens. This causes a latency spike; consider extending warmup to cover this shape/config.
|
||||
(EngineCore pid=112) WARNING 09-18 02:57:53 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _compute_local_logits_stats_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
|
||||
(EngineCore pid=112) WARNING 09-18 02:57:53 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _rejection_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
|
||||
(EngineCore pid=112) WARNING 09-18 02:57:53 [jit_monitor.py:140] Triton kernel JIT compilation during inference: _resample_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
|
||||
(APIServer pid=1) INFO: 172.21.0.1:45300 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47570 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47586 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO 09-18 02:57:55 [loggers.py:323] Engine 000: Avg prompt throughput: 78.5 tokens/s, Avg generation throughput: 19.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.5%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
|
||||
(APIServer pid=1) INFO 09-18 02:57:55 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.60, Accepted throughput: 11.82 tokens/s, Drafted throughput: 14.73 tokens/s, Accepted: 130 tokens, Drafted: 162 tokens, Per-position acceptance rate: 0.864, 0.741, Avg Draft acceptance rate: 80.2%
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47598 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) [transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (263582 > 262144). Running this sequence through the model will result in indexing errors
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47602 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47608 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47610 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47624 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47634 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47640 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47650 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47662 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47678 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47688 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47698 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47712 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47716 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47726 - "POST /tokenize HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:47732 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO 09-18 02:58:05 [loggers.py:323] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 9.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 60.0%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
|
||||
(APIServer pid=1) INFO 09-18 02:58:05 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.77, Accepted throughput: 6.20 tokens/s, Drafted throughput: 7.00 tokens/s, Accepted: 62 tokens, Drafted: 70 tokens, Per-position acceptance rate: 0.971, 0.800, Avg Draft acceptance rate: 88.6%
|
||||
(APIServer pid=1) INFO: 172.21.0.1:36086 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:36096 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO 09-18 02:58:15 [loggers.py:323] Engine 000: Avg prompt throughput: 13184.9 tokens/s, Avg generation throughput: 34.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 21.5%, Prefix cache hit rate: 48.3%, MM cache hit rate: 0.0%
|
||||
(APIServer pid=1) INFO 09-18 02:58:15 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.73, Accepted throughput: 21.60 tokens/s, Drafted throughput: 25.00 tokens/s, Accepted: 216 tokens, Drafted: 250 tokens, Per-position acceptance rate: 0.936, 0.792, Avg Draft acceptance rate: 86.4%
|
||||
(APIServer pid=1) INFO: 172.21.0.1:33248 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 127.0.0.1:45376 - "GET /health HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:33262 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO: 172.21.0.1:33272 - "POST /v1/chat/completions HTTP/1.1" 200 OK
|
||||
(APIServer pid=1) INFO 09-18 02:58:25 [loggers.py:323] Engine 000: Avg prompt throughput: 17.4 tokens/s, Avg generation throughput: 79.7 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 48.3%, MM cache hit rate: 0.0%
|
||||
(APIServer pid=1) INFO 09-18 02:58:25 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.68, Accepted throughput: 49.80 tokens/s, Drafted throughput: 59.20 tokens/s, Accepted: 498 tokens, Drafted: 592 tokens, Per-position acceptance rate: 0.926, 0.757, Avg Draft acceptance rate: 84.1%
|
||||
(APIServer pid=1) INFO 09-18 02:58:35 [loggers.py:323] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 48.3%, MM cache hit rate: 0.0%
|
||||
(APIServer pid=1) INFO: 127.0.0.1:60766 - "GET /health HTTP/1.1" 200 OK
|
||||
@@ -0,0 +1 @@
|
||||
passed
|
||||
@@ -0,0 +1,2 @@
|
||||
{"test": "image", "passed": true, "elapsed_s": 0.516706766000425, "response": {"id": "chatcmpl-b4bf7bbadb6dd5f9", "object": "chat.completion", "created": 1789700274, "model": "qwen3.8-flash-next", "choices": [{"index": 0, "message": {"role": "assistant", "content": "\n\nRed", "refusal": null, "annotations": null, "audio": null, "function_call": null, "reasoning": "The user wants the main color of the image in one English word. The image is a solid red field. I need to provide a single word answer. The dominant and only color is red. I'll respond concisely with just that word.\n"}, "logprobs": null, "finish_reason": "stop", "stop_reason": null, "token_ids": null, "routed_experts": null}], "service_tier": null, "system_fingerprint": "vllm-0.3.1.dev3+g0bfc7a15d-9e1c6e32", "usage": {"prompt_tokens": 137, "total_tokens": 192, "completion_tokens": 55, "prompt_tokens_details": null, "completion_tokens_details": {"reasoning_tokens": 51}}, "prompt_logprobs": null, "prompt_token_ids": null, "prompt_text": null, "kv_transfer_params": null, "ec_transfer_params": null, "metrics": null}}
|
||||
{"test": "video", "passed": true, "elapsed_s": 0.8257456709998223, "response": {"id": "chatcmpl-b06902fcb6aa9899", "object": "chat.completion", "created": 1789700275, "model": "qwen3.8-flash-next", "choices": [{"index": 0, "message": {"role": "assistant", "content": "\n\nRed, Green, Blue", "refusal": null, "annotations": null, "audio": null, "function_call": null, "reasoning": "The user wants the background colors in chronological order from the provided frames. Looking at the timestamps: 0.0s and 1.0s show red, 2.0s shows green, 3.0s shows blue. So the sequence of distinct background colors over time is red, then green, then blue. I need to reply with only three English color names. The answer is \"Red, Green, Blue\". I'll provide just that, no extra text.\n"}, "logprobs": null, "finish_reason": "stop", "stop_reason": null, "token_ids": null, "routed_experts": null}], "service_tier": null, "system_fingerprint": "vllm-0.3.1.dev3+g0bfc7a15d-9e1c6e32", "usage": {"prompt_tokens": 323, "total_tokens": 430, "completion_tokens": 107, "prompt_tokens_details": null, "completion_tokens_details": {"reasoning_tokens": 99}}, "prompt_logprobs": null, "prompt_token_ids": null, "prompt_text": null, "kv_transfer_params": null, "ec_transfer_params": null, "metrics": null}}
|
||||
@@ -0,0 +1,282 @@
|
||||
{"time": 1789699784.8557913, "available_gib": 173.268310546875, "oom_kill": 0, "gpu": "80181, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699786.8766131, "available_gib": 174.90249252319336, "oom_kill": 0, "gpu": "80181, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699788.8985825, "available_gib": 175.68295288085938, "oom_kill": 0, "gpu": "80181, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699790.91918, "available_gib": 175.66968536376953, "oom_kill": 0, "gpu": "80181, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699792.9395878, "available_gib": 175.68845748901367, "oom_kill": 0, "gpu": "80181, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699800.8114684, "available_gib": 179.87840270996094, "oom_kill": 0, "gpu": "3, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699802.834334, "available_gib": 242.46122360229492, "oom_kill": 0, "gpu": "0, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699804.8571448, "available_gib": 242.3736572265625, "oom_kill": 0, "gpu": "0, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699806.8817687, "available_gib": 242.27896881103516, "oom_kill": 0, "gpu": "0, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699808.9074936, "available_gib": 242.22193145751953, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699810.932665, "available_gib": 242.20484161376953, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699812.957609, "available_gib": 242.17076110839844, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699814.9819207, "available_gib": 242.11393356323242, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699817.005921, "available_gib": 242.04299545288086, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699819.0289435, "available_gib": 241.47203063964844, "oom_kill": 0, "gpu": "3, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699821.062485, "available_gib": 241.26908493041992, "oom_kill": 0, "gpu": "3, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699823.0869603, "available_gib": 241.22373580932617, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789699825.1192734, "available_gib": 240.7252311706543, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789699827.1438193, "available_gib": 241.52180099487305, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789699829.1859672, "available_gib": 241.16375732421875, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789699831.2292492, "available_gib": 240.9925537109375, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789699833.2707527, "available_gib": 240.5682601928711, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789699835.3125954, "available_gib": 240.0120506286621, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789699837.3552454, "available_gib": 239.8315200805664, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789699839.3975637, "available_gib": 239.30066299438477, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789699841.4175756, "available_gib": 239.12970352172852, "oom_kill": 0, "gpu": "703, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699843.437184, "available_gib": 238.37808227539062, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699845.4552722, "available_gib": 233.91911697387695, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699847.4740174, "available_gib": 229.94686126708984, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699849.4928708, "available_gib": 226.03619384765625, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699851.5111096, "available_gib": 222.09746932983398, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699853.5298238, "available_gib": 218.20297622680664, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699855.5481977, "available_gib": 214.18994140625, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699857.5675807, "available_gib": 210.1709442138672, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699859.5862136, "available_gib": 206.13785552978516, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699861.6048002, "available_gib": 202.08191680908203, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699863.6235912, "available_gib": 198.0450782775879, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699865.6427789, "available_gib": 193.9893913269043, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699867.6612194, "available_gib": 189.95485305786133, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699869.679868, "available_gib": 185.93933868408203, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699871.6985507, "available_gib": 181.9157257080078, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699873.7171915, "available_gib": 177.8902702331543, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699875.736203, "available_gib": 174.32772064208984, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699877.7578192, "available_gib": 173.71036529541016, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789699879.7790232, "available_gib": 173.81313705444336, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789699881.8009658, "available_gib": 173.8381690979004, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789699883.822374, "available_gib": 173.9555320739746, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789699885.8436148, "available_gib": 173.97101211547852, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699887.8657157, "available_gib": 173.9713020324707, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699889.8866572, "available_gib": 173.96550369262695, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699891.906882, "available_gib": 173.96137237548828, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699893.927697, "available_gib": 173.96295547485352, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699895.9530587, "available_gib": 173.97043991088867, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699897.973925, "available_gib": 173.98381805419922, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699899.9951556, "available_gib": 173.99149703979492, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699902.016985, "available_gib": 174.02708435058594, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699904.037938, "available_gib": 174.0359764099121, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699906.0589933, "available_gib": 174.04009628295898, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699908.0798283, "available_gib": 174.04407119750977, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699910.100448, "available_gib": 174.0757827758789, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699912.121421, "available_gib": 174.0846405029297, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699914.1430233, "available_gib": 174.09698104858398, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699916.1640205, "available_gib": 174.0961456298828, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699918.1853118, "available_gib": 174.11655044555664, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789699920.2060194, "available_gib": 174.06558990478516, "oom_kill": 0, "gpu": "78809, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789699922.226115, "available_gib": 173.94388961791992, "oom_kill": 0, "gpu": "82531, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699924.2462852, "available_gib": 173.94985961914062, "oom_kill": 0, "gpu": "82559, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699926.2655146, "available_gib": 173.94927978515625, "oom_kill": 0, "gpu": "82559, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699928.2854214, "available_gib": 173.38904571533203, "oom_kill": 0, "gpu": "82613, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699930.3055093, "available_gib": 173.25407791137695, "oom_kill": 0, "gpu": "79131, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699932.3254104, "available_gib": 172.94384002685547, "oom_kill": 0, "gpu": "79131, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699934.3455124, "available_gib": 172.37591171264648, "oom_kill": 0, "gpu": "81095, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699936.3656113, "available_gib": 172.36243057250977, "oom_kill": 0, "gpu": "81095, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699938.3857331, "available_gib": 172.38327407836914, "oom_kill": 0, "gpu": "81095, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699940.40693, "available_gib": 172.39754104614258, "oom_kill": 0, "gpu": "82317, 85651, 100, 40", "stop": false}
|
||||
{"time": 1789699942.4280798, "available_gib": 172.43234252929688, "oom_kill": 0, "gpu": "82317, 85651, 100, 42", "stop": false}
|
||||
{"time": 1789699944.4486258, "available_gib": 172.45121002197266, "oom_kill": 0, "gpu": "82317, 85651, 100, 43", "stop": false}
|
||||
{"time": 1789699946.468317, "available_gib": 172.4884033203125, "oom_kill": 0, "gpu": "82319, 85651, 0, 34", "stop": false}
|
||||
{"time": 1789699948.4880044, "available_gib": 172.42709350585938, "oom_kill": 0, "gpu": "82319, 85651, 0, 34", "stop": false}
|
||||
{"time": 1789699950.507294, "available_gib": 172.47681045532227, "oom_kill": 0, "gpu": "82319, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789699952.5260417, "available_gib": 172.41959762573242, "oom_kill": 0, "gpu": "82319, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789699954.5465488, "available_gib": 172.17332077026367, "oom_kill": 0, "gpu": "80519, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789699956.567162, "available_gib": 171.95952224731445, "oom_kill": 0, "gpu": "80519, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789699958.5875938, "available_gib": 171.57903671264648, "oom_kill": 0, "gpu": "80553, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789699960.6085618, "available_gib": 171.84549713134766, "oom_kill": 0, "gpu": "80189, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789699962.6290948, "available_gib": 172.0185546875, "oom_kill": 0, "gpu": "80239, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789699964.6503246, "available_gib": 172.32759857177734, "oom_kill": 0, "gpu": "80197, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789699966.676846, "available_gib": 172.48663330078125, "oom_kill": 0, "gpu": "80197, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789699968.697056, "available_gib": 174.07500839233398, "oom_kill": 0, "gpu": "80197, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789699970.7173376, "available_gib": 174.9847412109375, "oom_kill": 0, "gpu": "80197, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789699972.737336, "available_gib": 175.0500602722168, "oom_kill": 0, "gpu": "80197, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789699974.7615445, "available_gib": 175.09698104858398, "oom_kill": 0, "gpu": "80197, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789699982.4932203, "available_gib": 180.97797012329102, "oom_kill": 0, "gpu": "3, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789699984.5346766, "available_gib": 242.31150436401367, "oom_kill": 0, "gpu": "3, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699986.5568547, "available_gib": 242.04119110107422, "oom_kill": 0, "gpu": "0, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789699988.5815885, "available_gib": 242.06488800048828, "oom_kill": 0, "gpu": "0, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699990.6061108, "available_gib": 242.0366439819336, "oom_kill": 0, "gpu": "0, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699992.630566, "available_gib": 241.96244049072266, "oom_kill": 0, "gpu": "0, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699994.6744077, "available_gib": 241.48827743530273, "oom_kill": 0, "gpu": "3, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789699996.7181695, "available_gib": 241.28020095825195, "oom_kill": 0, "gpu": "3, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789699998.7595315, "available_gib": 241.0977783203125, "oom_kill": 0, "gpu": "3, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700000.801106, "available_gib": 240.78686904907227, "oom_kill": 0, "gpu": "3, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700002.8429427, "available_gib": 240.23579788208008, "oom_kill": 0, "gpu": "3, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700004.8777978, "available_gib": 240.06826400756836, "oom_kill": 0, "gpu": "3, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700006.927845, "available_gib": 239.48531341552734, "oom_kill": 0, "gpu": "3, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700008.9487274, "available_gib": 238.55954360961914, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700010.9686577, "available_gib": 234.62310791015625, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700012.9886026, "available_gib": 230.06606674194336, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700015.0087287, "available_gib": 226.08770751953125, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700017.028833, "available_gib": 222.08605575561523, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700019.0487628, "available_gib": 218.09417724609375, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700021.068506, "available_gib": 214.07280731201172, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700023.0881393, "available_gib": 210.02310180664062, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700025.1080492, "available_gib": 206.0057144165039, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700027.1281407, "available_gib": 201.98720932006836, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700029.1478603, "available_gib": 197.94964599609375, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700031.1675708, "available_gib": 193.9510269165039, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700033.1873636, "available_gib": 189.92237854003906, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700035.207234, "available_gib": 185.88830947875977, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700037.2265067, "available_gib": 181.86507415771484, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700039.2458131, "available_gib": 177.82470703125, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700041.2655752, "available_gib": 174.4015007019043, "oom_kill": 0, "gpu": "4285, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700043.287092, "available_gib": 174.46878051757812, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700045.3077006, "available_gib": 174.47719192504883, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700047.328204, "available_gib": 174.48582077026367, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700049.3485699, "available_gib": 174.45703887939453, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700051.3691437, "available_gib": 174.4439811706543, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700053.3893752, "available_gib": 174.42322158813477, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700055.4098642, "available_gib": 174.4094581604004, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700057.4303467, "available_gib": 174.40129852294922, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700059.4506557, "available_gib": 176.9820098876953, "oom_kill": 0, "gpu": "76995, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700061.4705217, "available_gib": 177.71730422973633, "oom_kill": 0, "gpu": "76995, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700063.490828, "available_gib": 177.7105712890625, "oom_kill": 0, "gpu": "76995, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700065.5109022, "available_gib": 177.6934700012207, "oom_kill": 0, "gpu": "76995, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700067.53109, "available_gib": 177.72324752807617, "oom_kill": 0, "gpu": "76995, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700074.1739979, "available_gib": 194.71649169921875, "oom_kill": 0, "gpu": "0, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700076.1959038, "available_gib": 242.32111358642578, "oom_kill": 0, "gpu": "0, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700078.2183936, "available_gib": 242.2687110900879, "oom_kill": 0, "gpu": "0, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700080.2433918, "available_gib": 242.21891403198242, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700082.2683237, "available_gib": 242.2096939086914, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700084.2929466, "available_gib": 242.16974639892578, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700086.3170736, "available_gib": 242.17188262939453, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700088.3417797, "available_gib": 242.07259368896484, "oom_kill": 0, "gpu": "0, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700090.3671076, "available_gib": 241.59612274169922, "oom_kill": 0, "gpu": "0, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700092.4088504, "available_gib": 241.66643524169922, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700094.4781263, "available_gib": 241.26079177856445, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700096.5219333, "available_gib": 241.02852249145508, "oom_kill": 0, "gpu": "3, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700098.5478, "available_gib": 242.32376098632812, "oom_kill": 0, "gpu": "0, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700100.5917299, "available_gib": 242.18223571777344, "oom_kill": 0, "gpu": "0, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700102.6330614, "available_gib": 242.18046188354492, "oom_kill": 0, "gpu": "0, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700104.6753898, "available_gib": 242.11222076416016, "oom_kill": 0, "gpu": "0, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700106.7176976, "available_gib": 242.0906639099121, "oom_kill": 0, "gpu": "0, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700108.7608912, "available_gib": 242.08220672607422, "oom_kill": 0, "gpu": "0, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700110.804005, "available_gib": 242.03167343139648, "oom_kill": 0, "gpu": "0, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700112.8491402, "available_gib": 241.9895896911621, "oom_kill": 0, "gpu": "0, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700114.8809855, "available_gib": 241.31512069702148, "oom_kill": 0, "gpu": "3, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700116.9228864, "available_gib": 241.4677734375, "oom_kill": 0, "gpu": "3, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700118.9670393, "available_gib": 241.2608985900879, "oom_kill": 0, "gpu": "3, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700121.008349, "available_gib": 240.94713592529297, "oom_kill": 0, "gpu": "3, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700123.0505216, "available_gib": 240.44035720825195, "oom_kill": 0, "gpu": "3, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700125.0926425, "available_gib": 240.10882949829102, "oom_kill": 0, "gpu": "3, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700127.135921, "available_gib": 239.90147399902344, "oom_kill": 0, "gpu": "3, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700129.1780832, "available_gib": 239.06969833374023, "oom_kill": 0, "gpu": "3, 85651, 0, 27", "stop": false}
|
||||
{"time": 1789700131.1975424, "available_gib": 239.08580017089844, "oom_kill": 0, "gpu": "2915, 85651, 0, 28", "stop": false}
|
||||
{"time": 1789700133.216903, "available_gib": 236.34324645996094, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700135.2349987, "available_gib": 232.00969314575195, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700137.2533488, "available_gib": 228.20048904418945, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700139.2720008, "available_gib": 224.26771926879883, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700141.2904346, "available_gib": 220.33390426635742, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700143.3088236, "available_gib": 216.4178237915039, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700145.3273628, "available_gib": 212.44153594970703, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700147.3463542, "available_gib": 208.3773651123047, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700149.3647766, "available_gib": 204.3053741455078, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700151.3832946, "available_gib": 200.27174758911133, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700153.4018042, "available_gib": 196.2129783630371, "oom_kill": 0, "gpu": "4285, 85651, 0, 29", "stop": false}
|
||||
{"time": 1789700155.4205408, "available_gib": 192.17596054077148, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700157.4392505, "available_gib": 188.12572860717773, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700159.4579098, "available_gib": 184.0816307067871, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700161.4766073, "available_gib": 180.06254959106445, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700163.4951246, "available_gib": 176.02914428710938, "oom_kill": 0, "gpu": "4285, 85651, 0, 30", "stop": false}
|
||||
{"time": 1789700165.5196788, "available_gib": 173.59284591674805, "oom_kill": 0, "gpu": "4413, 85651, 32, 30", "stop": false}
|
||||
{"time": 1789700167.5409636, "available_gib": 174.47128295898438, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700169.5620625, "available_gib": 174.4740219116211, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700171.5827205, "available_gib": 174.47975540161133, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700173.6033459, "available_gib": 174.46636581420898, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700175.625155, "available_gib": 174.47333908081055, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700177.6462028, "available_gib": 174.48141479492188, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700179.6672688, "available_gib": 174.48285675048828, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700181.6880727, "available_gib": 174.4776496887207, "oom_kill": 0, "gpu": "76995, 85651, 6, 30", "stop": false}
|
||||
{"time": 1789700183.7094233, "available_gib": 174.46028518676758, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700185.730227, "available_gib": 174.43238830566406, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700187.7518756, "available_gib": 174.4477767944336, "oom_kill": 0, "gpu": "76995, 85651, 4, 30", "stop": false}
|
||||
{"time": 1789700189.772606, "available_gib": 174.4392433166504, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700191.793387, "available_gib": 174.43968200683594, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700193.8145614, "available_gib": 174.4402618408203, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700195.8353708, "available_gib": 174.45610427856445, "oom_kill": 0, "gpu": "76995, 85651, 6, 31", "stop": false}
|
||||
{"time": 1789700197.856705, "available_gib": 174.46376419067383, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700199.8782845, "available_gib": 174.4733009338379, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700201.8992653, "available_gib": 174.47993850708008, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700203.9204733, "available_gib": 174.4811134338379, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700205.9425795, "available_gib": 174.46759414672852, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700207.9636607, "available_gib": 174.4632797241211, "oom_kill": 0, "gpu": "76995, 85651, 4, 31", "stop": false}
|
||||
{"time": 1789700209.9835656, "available_gib": 174.40138244628906, "oom_kill": 0, "gpu": "78809, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700212.0033963, "available_gib": 174.2780303955078, "oom_kill": 0, "gpu": "82531, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700214.0232139, "available_gib": 174.26416015625, "oom_kill": 0, "gpu": "82559, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700216.0458455, "available_gib": 174.2457504272461, "oom_kill": 0, "gpu": "82587, 85651, 96, 36", "stop": false}
|
||||
{"time": 1789700218.0656052, "available_gib": 174.06429290771484, "oom_kill": 0, "gpu": "79131, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700220.087584, "available_gib": 173.7898826599121, "oom_kill": 0, "gpu": "79131, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700222.10688, "available_gib": 173.32328414916992, "oom_kill": 0, "gpu": "81095, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700224.126727, "available_gib": 173.26789474487305, "oom_kill": 0, "gpu": "81095, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700226.1464965, "available_gib": 173.2430763244629, "oom_kill": 0, "gpu": "81095, 85651, 0, 31", "stop": false}
|
||||
{"time": 1789700228.1671848, "available_gib": 173.30466079711914, "oom_kill": 0, "gpu": "82317, 85651, 100, 40", "stop": false}
|
||||
{"time": 1789700230.188617, "available_gib": 173.31119918823242, "oom_kill": 0, "gpu": "82317, 85651, 100, 41", "stop": false}
|
||||
{"time": 1789700232.208453, "available_gib": 173.28583526611328, "oom_kill": 0, "gpu": "82317, 85651, 100, 42", "stop": false}
|
||||
{"time": 1789700234.2289078, "available_gib": 173.2372932434082, "oom_kill": 0, "gpu": "82319, 85651, 0, 34", "stop": false}
|
||||
{"time": 1789700236.2490773, "available_gib": 173.2351951599121, "oom_kill": 0, "gpu": "82319, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700238.269133, "available_gib": 173.20354461669922, "oom_kill": 0, "gpu": "82319, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700240.2894697, "available_gib": 173.2050018310547, "oom_kill": 0, "gpu": "82319, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700242.3098028, "available_gib": 172.9349822998047, "oom_kill": 0, "gpu": "80553, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700244.3320792, "available_gib": 173.1237335205078, "oom_kill": 0, "gpu": "80225, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700246.352452, "available_gib": 173.32941055297852, "oom_kill": 0, "gpu": "80303, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700248.3720224, "available_gib": 173.30435943603516, "oom_kill": 0, "gpu": "83033, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700250.3918135, "available_gib": 173.2729606628418, "oom_kill": 0, "gpu": "83033, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700252.4116886, "available_gib": 173.2642707824707, "oom_kill": 0, "gpu": "83033, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700254.4326909, "available_gib": 173.27268981933594, "oom_kill": 0, "gpu": "83033, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700256.4531732, "available_gib": 173.28126907348633, "oom_kill": 0, "gpu": "83035, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700258.4731524, "available_gib": 173.13495254516602, "oom_kill": 0, "gpu": "83041, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700260.493189, "available_gib": 173.013671875, "oom_kill": 0, "gpu": "83041, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700262.5140247, "available_gib": 173.01695251464844, "oom_kill": 0, "gpu": "83087, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700264.5341256, "available_gib": 173.04697036743164, "oom_kill": 0, "gpu": "83175, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700266.553813, "available_gib": 173.0420799255371, "oom_kill": 0, "gpu": "83175, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700268.573798, "available_gib": 173.0691909790039, "oom_kill": 0, "gpu": "83175, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700270.59406, "available_gib": 173.10009765625, "oom_kill": 0, "gpu": "83175, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700272.6137307, "available_gib": 173.07026290893555, "oom_kill": 0, "gpu": "83175, 85651, 0, 32", "stop": false}
|
||||
{"time": 1789700274.639571, "available_gib": 173.02909469604492, "oom_kill": 0, "gpu": "83183, 85651, 100, 38", "stop": false}
|
||||
{"time": 1789700276.65941, "available_gib": 172.89321899414062, "oom_kill": 0, "gpu": "83185, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700278.6844745, "available_gib": 172.90272903442383, "oom_kill": 0, "gpu": "83547, 85651, 100, 44", "stop": false}
|
||||
{"time": 1789700280.710171, "available_gib": 172.9180564880371, "oom_kill": 0, "gpu": "83547, 85651, 100, 44", "stop": false}
|
||||
{"time": 1789700282.7349055, "available_gib": 172.88683700561523, "oom_kill": 0, "gpu": "83547, 85651, 100, 45", "stop": false}
|
||||
{"time": 1789700284.7583632, "available_gib": 172.87765884399414, "oom_kill": 0, "gpu": "83547, 85651, 100, 46", "stop": false}
|
||||
{"time": 1789700286.7827125, "available_gib": 172.8911247253418, "oom_kill": 0, "gpu": "83547, 85651, 100, 47", "stop": false}
|
||||
{"time": 1789700288.812623, "available_gib": 172.87197875976562, "oom_kill": 0, "gpu": "83719, 85651, 100, 47", "stop": false}
|
||||
{"time": 1789700290.837241, "available_gib": 172.86443328857422, "oom_kill": 0, "gpu": "84169, 85651, 100, 49", "stop": false}
|
||||
{"time": 1789700292.857805, "available_gib": 172.86686325073242, "oom_kill": 0, "gpu": "84895, 85651, 47, 42", "stop": false}
|
||||
{"time": 1789700294.8838098, "available_gib": 172.8665657043457, "oom_kill": 0, "gpu": "84895, 85651, 100, 45", "stop": false}
|
||||
{"time": 1789700296.9096725, "available_gib": 172.86813735961914, "oom_kill": 0, "gpu": "84895, 85651, 100, 45", "stop": false}
|
||||
{"time": 1789700298.9358406, "available_gib": 172.8501434326172, "oom_kill": 0, "gpu": "84895, 85651, 100, 45", "stop": false}
|
||||
{"time": 1789700300.9623625, "available_gib": 172.901123046875, "oom_kill": 0, "gpu": "84895, 85651, 100, 45", "stop": false}
|
||||
{"time": 1789700302.982952, "available_gib": 172.91755294799805, "oom_kill": 0, "gpu": "84895, 85651, 0, 40", "stop": false}
|
||||
{"time": 1789700305.0032096, "available_gib": 172.92973709106445, "oom_kill": 0, "gpu": "84895, 85651, 0, 40", "stop": false}
|
||||
{"time": 1789700307.023144, "available_gib": 172.9438362121582, "oom_kill": 0, "gpu": "84895, 85651, 0, 39", "stop": false}
|
||||
{"time": 1789700309.043444, "available_gib": 172.9451789855957, "oom_kill": 0, "gpu": "84895, 85651, 0, 38", "stop": false}
|
||||
{"time": 1789700311.0636864, "available_gib": 172.94405364990234, "oom_kill": 0, "gpu": "84895, 85651, 0, 38", "stop": false}
|
||||
{"time": 1789700313.0836325, "available_gib": 172.91239166259766, "oom_kill": 0, "gpu": "84895, 85651, 0, 38", "stop": false}
|
||||
{"time": 1789700315.103711, "available_gib": 172.9315185546875, "oom_kill": 0, "gpu": "84895, 85651, 0, 37", "stop": false}
|
||||
{"time": 1789700317.12361, "available_gib": 172.93312454223633, "oom_kill": 0, "gpu": "84895, 85651, 0, 37", "stop": false}
|
||||
{"time": 1789700319.14374, "available_gib": 172.9223403930664, "oom_kill": 0, "gpu": "84895, 85651, 0, 37", "stop": false}
|
||||
{"time": 1789700321.1637733, "available_gib": 172.91282272338867, "oom_kill": 0, "gpu": "84895, 85651, 0, 36", "stop": false}
|
||||
{"time": 1789700323.1840858, "available_gib": 172.92692947387695, "oom_kill": 0, "gpu": "84895, 85651, 0, 36", "stop": false}
|
||||
{"time": 1789700325.2040703, "available_gib": 172.92637252807617, "oom_kill": 0, "gpu": "84895, 85651, 0, 36", "stop": false}
|
||||
{"time": 1789700327.2239447, "available_gib": 172.9365463256836, "oom_kill": 0, "gpu": "84895, 85651, 0, 36", "stop": false}
|
||||
{"time": 1789700329.243812, "available_gib": 172.96002960205078, "oom_kill": 0, "gpu": "84895, 85651, 0, 35", "stop": false}
|
||||
{"time": 1789700331.264149, "available_gib": 172.95891189575195, "oom_kill": 0, "gpu": "84895, 85651, 0, 35", "stop": false}
|
||||
{"time": 1789700333.2835627, "available_gib": 172.94894790649414, "oom_kill": 0, "gpu": "84895, 85651, 0, 35", "stop": false}
|
||||
{"time": 1789700335.303892, "available_gib": 172.96074295043945, "oom_kill": 0, "gpu": "84895, 85651, 0, 35", "stop": false}
|
||||
{"time": 1789700337.3236537, "available_gib": 172.9765968322754, "oom_kill": 0, "gpu": "84895, 85651, 0, 35", "stop": false}
|
||||
{"time": 1789700339.3438094, "available_gib": 172.9586296081543, "oom_kill": 0, "gpu": "84895, 85651, 0, 34", "stop": false}
|
||||
{"time": 1789700341.3641162, "available_gib": 172.97516250610352, "oom_kill": 0, "gpu": "84895, 85651, 0, 34", "stop": false}
|
||||
{"time": 1789700343.3842447, "available_gib": 172.96298599243164, "oom_kill": 0, "gpu": "84895, 85651, 0, 34", "stop": false}
|
||||
{"time": 1789700345.4077187, "available_gib": 172.9586181640625, "oom_kill": 0, "gpu": "84895, 85651, 0, 34", "stop": false}
|
||||
{"time": 1789700347.4277773, "available_gib": 172.9544677734375, "oom_kill": 0, "gpu": "84895, 85651, 0, 34", "stop": false}
|
||||
{"time": 1789700349.4475327, "available_gib": 172.94096755981445, "oom_kill": 0, "gpu": "84895, 85651, 0, 34", "stop": false}
|
||||
{"time": 1789700351.467543, "available_gib": 172.94974517822266, "oom_kill": 0, "gpu": "84895, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700353.487416, "available_gib": 172.9431915283203, "oom_kill": 0, "gpu": "84895, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700355.507813, "available_gib": 172.96563339233398, "oom_kill": 0, "gpu": "84895, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700357.5280263, "available_gib": 172.95469665527344, "oom_kill": 0, "gpu": "84895, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700359.548705, "available_gib": 174.7944564819336, "oom_kill": 0, "gpu": "84895, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700361.5688958, "available_gib": 175.5979118347168, "oom_kill": 0, "gpu": "84895, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700363.5890248, "available_gib": 175.59072494506836, "oom_kill": 0, "gpu": "84895, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700365.6091769, "available_gib": 175.58855438232422, "oom_kill": 0, "gpu": "84895, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700367.6296282, "available_gib": 175.56648635864258, "oom_kill": 0, "gpu": "84895, 85651, 0, 33", "stop": false}
|
||||
{"time": 1789700374.2080383, "available_gib": 193.37168884277344, "oom_kill": 0, "gpu": "3, 85651, 0, 32", "stop": false}
|
||||
@@ -0,0 +1,44 @@
|
||||
# MTP 实机测试
|
||||
|
||||
日期：2026-09-18。初始对照：main `7e53821`，128K + FP8 KV + FULL CUDA Graph，无 MTP。
|
||||
|
||||
## 方案
|
||||
|
||||
1. 保持模型 revision、镜像 digest、128K、视觉编码器、单活动请求及 GPU 预算不变。
|
||||
2. 首先添加 `--speculative-config '{"method":"mtp","num_speculative_tokens":2}'`。Graph 捕获尺寸增加验证所需的 3 token 形状。
|
||||
3. 等待健康检查，依次验证中文、数学、工具调用、图片、视频、127988 token 实际输入检索及重复前缀。
|
||||
4. 使用原 256 token 计时任务：一次预热、三次测量，记录 TTFT、总时间、解码速度与实际生成文本。采集投机接受率及显存/KV预算。
|
||||
5. 如加载失败、OOM、功能回归或无实际收益，恢复原基线。通过后才选择是否部署，并保留测试配置和回执。
|
||||
|
||||
状态：MTP 2 两轮测试完成；结论及限制见下文。
|
||||
|
||||
## 第一轮：MTP 2 / GPU 预算 0.96
|
||||
|
||||
启动失败，明确原因是 KV 缓存不足。模型加载占用 76.36 GiB（无 MTP 约 73.77 GiB）；可供分配的 KV 缓存约 0.66 GiB，而 131072 token 检查需要 2.47 GiB。没有进入请求测试，不能报告 MTP 速度。
|
||||
|
||||
补充试验：仅把 GPU 预算提高到 0.985，保留 128K 和视觉能力，测试期间关闭自动重启以避免失败后循环加载。这是余量较小的探索设置，不自动替换稳定基线。
|
||||
|
||||
## 第二轮：MTP 2 / GPU 预算 0.985
|
||||
|
||||
通过中文、数学、工具调用、图片、视频和 127988 token 输入检索。没有主机 OOM 或触发监测保护。模型显存 76.36 GiB，KV 预算 2.78 GiB，日志容量 146622 token；无 MTP 基线容量 218453 token。最终 Graph 捕获约 0.13 GiB。
|
||||
|
||||
| 项目 | 无 MTP 基线 | MTP 2 |
|
||||
|---|---:|---:|
|
||||
| 固定 256 token 解码中位数 | 77.115 token/s | 148.545 token/s |
|
||||
| 计时任务首 token 中位数 | 0.070 秒 | 0.086 秒 |
|
||||
| 思考 token 数 | 212 | 247 |
|
||||
| 128K 首次首 token | 14.141 秒 | 14.754 秒 |
|
||||
| 128K 前缀复用首 token | 0.454 秒 | 0.627 秒 |
|
||||
| GPU memory utilization | 0.96 | 0.985 |
|
||||
|
||||
同提示、temperature 0、seed 6000、固定输出长度、一次预热加三次测量。MTP 解码计时为 1.926 倍，但生成的正文/思考与无 MTP **不同**，不能视为逐字相同的性能对照，也不能据此判断质量提升或退化。固定长度吞吐包括思考 token，不等于完成同一业务任务的提速比例。
|
||||
|
||||
运行日志中两个统计窗口的草稿接受率为 86.4% 和 84.1%，平均接受长度约 2.73 和 2.68（含 bonus token）；不是整个请求集的统一平均值。原始累计指标见 `audit/runtime/metrics-mtp2.txt`。
|
||||
|
||||
运行器提示 draft 模型不接收外部多模态 embedding，使用 text-only draft 输入；目标模型仍保留多模态，合成图片和短视频测试通过。后端自动把 FULL 调整为 FULL_DECODE_ONLY，并对多步 draft 重建注意力元数据。
|
||||
|
||||
## 决定
|
||||
|
||||
**默认仍保留无 MTP 的 128K 多模态基线。** MTP 2 有明显短输出吞吐收益，但需要 98.5% 显存预算，且多模态测试只覆盖小图片和短视频，没有长视频、较长输出、多任务真实质量或长期压力测试。现有证据不足以把它称为更稳定的默认配置。
|
||||
|
||||
实验配置保存在 `audit/runtime/compose-mtp2-0985.yaml`，关闭了自动重启。根目录 Compose 不修改；结束测试后恢复原基线。没有测试 MTP 1、3 或更多预测步，当前结果不代表参数最优。
|
||||
Reference in New Issue
Block a user