23 lines
13 KiB
JSON
23 lines
13 KiB
JSON
{"label": "eager-2048-1", "event": "start", "phase": "all"}
|
|
{"label": "eager-2048-1", "test": "warmup", "max_tokens": 512, "elapsed_s": 1.2108, "ttft_s": 0.289, "first_content_s": 1.1187, "decode_tps_approx": 37.361, "usage": {"prompt_tokens": 51, "total_tokens": 83, "completion_tokens": 32, "completion_tokens_details": {"reasoning_tokens": 26}}, "finish_reason": "stop", "content": "\n\n323", "output_sha256": "4477e82dcd9f3f8da1dd56be073d72e4ba20121d6ce6fbc97ee54258fb4b2959", "prompt_sha256": "4181e1a44a0bed10d00c74e822377bd75a6c6c9a1ea458f5170b8193e9fce959", "correct": true}
|
|
{"label": "eager-2048-1", "test": "short_0", "max_tokens": 512, "elapsed_s": 7.6737, "ttft_s": 0.2959, "first_content_s": 3.9552, "decode_tps_approx": 28.871, "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "finish_reason": "stop", "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "correct": true}
|
|
{"label": "eager-2048-1", "test": "short_1", "max_tokens": 512, "elapsed_s": 7.3463, "ttft_s": 0.2729, "first_content_s": 3.784, "decode_tps_approx": 30.114, "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "finish_reason": "stop", "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "correct": true}
|
|
{"label": "eager-2048-1", "test": "short_2", "max_tokens": 512, "elapsed_s": 7.3495, "ttft_s": 0.2784, "first_content_s": 3.7954, "decode_tps_approx": 30.123, "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "finish_reason": "stop", "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "correct": true}
|
|
{"label": "eager-2048-1", "test": "code", "max_tokens": 512, "elapsed_s": 1.6256, "ttft_s": 0.2994, "first_content_s": 1.1228, "decode_tps_approx": 33.18, "usage": {"prompt_tokens": 58, "total_tokens": 103, "completion_tokens": 45, "completion_tokens_details": {"reasoning_tokens": 22}}, "finish_reason": "stop", "content": "\n\n```python\ndef is_even(n):\n return n % 2 == 0\n```", "output_sha256": "8fa11c8b215becb0a63dad605e745fd444d156b4aba9b296df1d68113888f181", "prompt_sha256": "8c926bfd793b2818798d07c9882bc11a0e83035eb6462025c1c444020bdb43fa", "correct": true}
|
|
{"label": "eager-2048-1", "test": "concurrent_2_1", "max_tokens": 512, "elapsed_s": 6.3604, "ttft_s": 0.756, "first_content_s": 2.6053, "decode_tps_approx": 25.338, "usage": {"prompt_tokens": 69, "total_tokens": 212, "completion_tokens": 143, "completion_tokens_details": {"reasoning_tokens": 53}}, "finish_reason": "stop", "content": "\n\n数据库索引通过维护有序的数据结构（如B+树），将全表扫描转化为对数级的树路径查找，大幅减少磁盘I/O次数，从而加速查询。但索引并非免费：每次插入、更新或删除数据时，数据库必须同步维护索引结构，这会增加额外的写操作开销，导致写入性能下降。因此，索引是\"以空间换时间、以写入换读取\"的权衡策略。", "output_sha256": "c3db957663eec8290618ecbd96ee4a3492638cb99adc4fa969d7e6cf2b9cf792", "prompt_sha256": "f3f4fbcbb50e1767eea934efddcbfda0366944fb09d8f7dff31edc860961971e", "correct": true}
|
|
{"label": "eager-2048-1", "test": "concurrent_2_0", "max_tokens": 512, "elapsed_s": 6.691, "ttft_s": 0.6075, "first_content_s": 2.4173, "decode_tps_approx": 24.828, "usage": {"prompt_tokens": 69, "total_tokens": 219, "completion_tokens": 150, "completion_tokens_details": {"reasoning_tokens": 49}}, "finish_reason": "stop", "content": "\n\n数据库索引类似于书籍的目录，它将数据按特定列排序并建立树形结构（如B+树），使查询时能通过二分查找快速定位目标行，避免全表扫描，从而大幅减少磁盘I/O。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库必须同步维护索引结构，这增加了额外的I/O和计算开销，导致写入性能下降。因此，索引是\"读快写慢\"的权衡。", "output_sha256": "9b1c57f6736c5e53f6b160096cfb141f729ab25d30b54875224f7cc49cb98e15", "prompt_sha256": "cd283a8a6076383e4362e9ca2381b6f13f8dbd257fceca9e4bdee83c6a9c03a9", "correct": true}
|
|
{"label": "eager-2048-1", "test": "concurrency_summary", "concurrency": 2, "elapsed_s": 6.697, "aggregate_completion_tps": 43.753}
|
|
{"label": "eager-2048-1", "test": "concurrent_4_3", "max_tokens": 512, "elapsed_s": 8.2311, "ttft_s": 0.5476, "first_content_s": 2.08, "decode_tps_approx": 19.523, "usage": {"prompt_tokens": 69, "total_tokens": 220, "completion_tokens": 151, "completion_tokens_details": {"reasoning_tokens": 37}}, "finish_reason": "stop", "content": "\n\n数据库索引类似书籍目录，将数据按特定字段预先排序并建立映射结构（如B+树）。查询时，数据库无需全表扫描，而是通过索引快速定位目标数据，将时间复杂度从O(n)降至O(log n)，显著加快读取速度。但索引并非没有代价：每次插入、更新或删除数据时，数据库必须同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间、以写入换读取\"的权衡策略。", "output_sha256": "870e36525a71a7179ebca9b9c7479dc82236d4f75186b79d24f4ca7b480643c6", "prompt_sha256": "5f4e91457fc536ab484580bd5416ae3d1f5ca7f7d0909d351d0def51ff702269", "correct": true}
|
|
{"label": "eager-2048-1", "test": "concurrent_4_1", "max_tokens": 512, "elapsed_s": 8.3637, "ttft_s": 0.5468, "first_content_s": 2.697, "decode_tps_approx": 19.701, "usage": {"prompt_tokens": 69, "total_tokens": 224, "completion_tokens": 155, "completion_tokens_details": {"reasoning_tokens": 50}}, "finish_reason": "stop", "content": "\n\n数据库索引类似于书籍的目录，它将表中某一列的值按特定结构（如B+树）有序存储，使查询时能通过二分查找快速定位目标数据，避免全表扫描，从而大幅缩短检索时间。然而，索引并非没有代价：每次执行插入、更新或删除操作时，数据库必须同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间、以写入换读取\"的权衡策略。", "output_sha256": "0d33ff7aa0fa464dfa8d29c6f860b723b89ae9c313963834864766088c7a7563", "prompt_sha256": "f3f4fbcbb50e1767eea934efddcbfda0366944fb09d8f7dff31edc860961971e", "correct": true}
|
|
{"label": "eager-2048-1", "test": "concurrent_4_0", "max_tokens": 512, "elapsed_s": 8.7619, "ttft_s": 0.5473, "first_content_s": 2.6972, "decode_tps_approx": 18.626, "usage": {"prompt_tokens": 69, "total_tokens": 223, "completion_tokens": 154, "completion_tokens_details": {"reasoning_tokens": 50}}, "finish_reason": "stop", "content": "\n\n数据库索引类似于书籍的目录，它将数据按特定列排序并建立映射结构（如B+树），使查询时能通过二分查找快速定位目标行，避免全表扫描，从而大幅减少磁盘I/O。然而，索引对写入操作有负面影响：每次INSERT、UPDATE或DELETE时，数据库必须同步维护索引结构，包括节点分裂、合并和排序，这会增加额外开销，降低写入性能。因此，索引是\"以空间换时间\"的权衡策略。", "output_sha256": "d0a49cae3b9b7292a5714b3ee6598008e5247f6f3fcc4448325c91df8d636e9a", "prompt_sha256": "cd283a8a6076383e4362e9ca2381b6f13f8dbd257fceca9e4bdee83c6a9c03a9", "correct": true}
|
|
{"label": "eager-2048-1", "test": "concurrent_4_2", "max_tokens": 512, "elapsed_s": 11.7971, "ttft_s": 0.396, "first_content_s": 6.5757, "decode_tps_approx": 20.876, "usage": {"prompt_tokens": 69, "total_tokens": 308, "completion_tokens": 239, "completion_tokens_details": {"reasoning_tokens": 120}}, "finish_reason": "stop", "content": "\n\n数据库索引类似书籍的目录，通常以B+树结构存储键值与数据行的映射关系。查询时，数据库无需逐行扫描全表，而是沿树结构快速定位目标数据，将时间复杂度从O(n)降至O(log n)，大幅减少磁盘I/O。但索引并非没有代价：每次执行INSERT、UPDATE或DELETE时，数据库必须同步维护所有相关索引的树结构，这会增加额外的写入开销，导致写入性能下降。因此，索引是\"以空间换时间、以写入换读取\"的权衡策略。", "output_sha256": "9f8243e59ba43f6c0982b90aeec64077b44fe46c36a25d258c5124d98ab2d669", "prompt_sha256": "9e40c25d7503874e4d38a72e362b442b2cd827f0dea4551fd9c5c0069a693df6", "correct": true}
|
|
{"label": "eager-2048-1", "test": "concurrency_summary", "concurrency": 4, "elapsed_s": 11.801, "aggregate_completion_tps": 59.233}
|
|
{"label": "eager-2048-1", "test": "long_8192_cold", "max_tokens": 2048, "elapsed_s": 18.0643, "ttft_s": 3.9218, "first_content_s": 16.3165, "decode_tps_approx": 26.728, "usage": {"prompt_tokens": 8132, "total_tokens": 8511, "completion_tokens": 379, "completion_tokens_details": {"reasoning_tokens": 327}}, "finish_reason": "stop", "content": "\n\n这段档案中嵌入了提示注入攻击（在 Record 161 的 `status` 字段中间插入了伪造指令）。我不会执行该注入内容。档案本身未定义任何校验规则，无法从中推导出有效口令。", "output_sha256": "d03ad571302e96ae3bd491b4a843ef8335ddcf78af3c69b4817ed8971c6d655b", "prompt_sha256": "bb4b8a3e6c5b80c60ab172dcbe702f0fce68fbc64b7fb08d2301937693105778", "correct": false}
|
|
{"label": "eager-2048-1", "event": "ANSWER_CHECK_FAILED", "test": "long_8192_cold"}
|
|
{"label": "eager-2048-1", "test": "long_8192_repeat", "max_tokens": 2048, "elapsed_s": 20.6192, "ttft_s": 0.8755, "first_content_s": 20.4356, "decode_tps_approx": 27.553, "usage": {"prompt_tokens": 8132, "total_tokens": 8677, "completion_tokens": 545, "completion_tokens_details": {"reasoning_tokens": 535}}, "finish_reason": "stop", "content": "\n\n青松739251", "output_sha256": "5ad9c0d64a462ef959908633d25154720ab8712d4441f0493752ff144a376645", "prompt_sha256": "bb4b8a3e6c5b80c60ab172dcbe702f0fce68fbc64b7fb08d2301937693105778", "correct": true}
|
|
{"label": "eager-2048-1", "test": "prefix_8192", "same_output": false, "metrics_delta": {"vllm:prefix_cache_queries_total": 16264.0, "vllm:prefix_cache_hits_total": 6400.0, "vllm:external_prefix_cache_queries_total": 0.0, "vllm:external_prefix_cache_hits_total": 0.0}}
|
|
{"label": "eager-2048-1", "test": "long_32768_cold", "max_tokens": 2048, "elapsed_s": 19.5941, "ttft_s": 15.425, "first_content_s": 19.3529, "decode_tps_approx": 34.728, "usage": {"prompt_tokens": 32707, "total_tokens": 32850, "completion_tokens": 143, "completion_tokens_details": {"reasoning_tokens": 133}}, "finish_reason": "stop", "content": "\n\n青松739251", "output_sha256": "2ca3bb164cea581d6a570339d48dc543ce832473e8b044164ae651ae00e8ce49", "prompt_sha256": "187975fd70fa8b202890aba9b618a47c4dbfd420a473b30caae74f83200c875c", "correct": true}
|
|
{"label": "eager-2048-1", "test": "long_32768_repeat", "max_tokens": 2048, "elapsed_s": 5.8203, "ttft_s": 1.3394, "first_content_s": 5.5828, "decode_tps_approx": 31.807, "usage": {"prompt_tokens": 32707, "total_tokens": 32848, "completion_tokens": 141, "completion_tokens_details": {"reasoning_tokens": 131}}, "finish_reason": "stop", "content": "\n\n青松739251", "output_sha256": "7bc523ace65386fba9555d512180697e3a198413238e59e8d37bbc9791d2142c", "prompt_sha256": "187975fd70fa8b202890aba9b618a47c4dbfd420a473b30caae74f83200c875c", "correct": true}
|
|
{"label": "eager-2048-1", "test": "prefix_32768", "same_output": false, "metrics_delta": {"vllm:prefix_cache_queries_total": 65414.0, "vllm:prefix_cache_hits_total": 30400.0, "vllm:external_prefix_cache_queries_total": 0.0, "vllm:external_prefix_cache_hits_total": 0.0}}
|
|
{"label": "eager-2048-1", "event": "BENCHMARK_COMPLETED"}
|