docs(perf): profile CUDA graph coverage and PLE costs on Spark
This commit is contained in:
@@ -0,0 +1,127 @@
|
||||
# CUDA Graph 性能剖析：图已生效，主要收益受 GPU 工作占比限制
|
||||
|
||||
2026-09-18，Asia/Shanghai。方案与脚本见 [profile128k](../experiments/profile128k/plan.md)。
|
||||
|
||||
CUDA Graph 确实被重放，并非参数失效。本轮短回答生成速度中位数由 **29.954 → 31.316 tokens/s（约 +4.5%）**，
|
||||
与此前约 +4.4% 的复测一致。32K 首次输入首 token 则为 **15.416 → 15.376 秒**，基本不变。
|
||||
|
||||
原因有两层：当前小尺寸图没有覆盖实际的长输入预填充分块；同时，GPU 时间线上本来就没有很多空隙可消除。
|
||||
不能据此认定 SSD、PLE mmap 或 CPU 调度是主要瓶颈，也不能把 CUDA Graph 的小收益解释为 CUDA/GPU 没有发挥作用。
|
||||
**eager 和 graph 两组都在使用 GPU 执行 CUDA 内核；本次比较的是是否通过 CUDA Graph 提交这些工作。**
|
||||
|
||||
## 正常计时与正确性
|
||||
|
||||
- 固定 nightly `0bfc7a15`、相同模型 revision、BF16 KV、MTP=2、前缀缓存、两处 Mamba 修正、GPU memory=0.80。
|
||||
- 实验上下文上限 131072；max_num_seqs=4，但测试串行；max_num_batched_tokens=2048。
|
||||
- 小尺寸图为 breakable PIECEWISE，capture sizes `[1,2,4,8,12]`。
|
||||
- 短回答预算 512，长检索预算 2048，思考与正文共用；temperature=0、seed=42、reasoning_effort=low。
|
||||
- 两组各 19 次请求：1 次算术预热、三轮各 5 次计时请求、3 次 trace 请求。38 次均通过各自判据。
|
||||
长检索要求精确答案和正常 stop，短说明仅检查非空及正常 stop，不代表进行了语言质量评估。
|
||||
- 19 个对应请求的输入 SHA256 全部一致；服务返回 prompt_tokens 与预检逐条一致。
|
||||
- 每轮首次输入使用不同前缀，避免跨轮缓存命中；重复输入紧随首次输入。
|
||||
- 下表为三轮中位数，不含预热及 trace 请求。短回答每次均输出 214 tokens；长检索输出约 62–74 tokens，有少量差异。
|
||||
|
||||
| 指标 | eager | 小尺寸 graph |
|
||||
|---|---:|---:|
|
||||
| 短回答解码速度（近似 tokens/s） | 29.954 | 31.316 |
|
||||
| 短回答首 token（秒，含思考） | 0.2627 | 0.2624 |
|
||||
| 短回答完整耗时（秒） | 7.374 | 7.064 |
|
||||
| 8K 首次首 token（秒） | 3.788 | 3.770 |
|
||||
| 8K 复用首 token（秒） | 0.880 | 0.874 |
|
||||
| 32K 首次首 token（秒） | 15.416 | 15.376 |
|
||||
| 32K 复用首 token（秒） | 1.368 | 1.327 |
|
||||
|
||||
原始请求记录和自动汇总见 [results/profile128k](results/profile128k/benchmark-summary.json)。
|
||||
计时阶段没有开启 capture，但 Nsight launcher 和 NVTX 包装仍存在；不是完全卸载 profiler 的独立测速。
|
||||
执行顺序 eager → graph，未随机化、未清 OS 页缓存，小样本不代表统计显著性或长期稳定性。
|
||||
|
||||
## 时间线证据
|
||||
|
||||
Nsight Systems 2025.3.2，CUDA profiler API 控制三个短采样窗口，node 级 graph trace。
|
||||
NVTX 临时标记 PLE 同步、CPU 去重、CPU gather、lookup，以及图调度描述符。
|
||||
计时阶段和 trace 阶段分开；下表不用于跨模式的端到端性能排名。
|
||||
|
||||
“GPU 活动占比”是 kernel、memcpy、memset 时间区间并集，除以首次至末次 GPU 活动的跨度。
|
||||
它不是 SM 利用率、内存带宽利用率，也不包含完整 HTTP 请求前后的所有工作。
|
||||
|
||||
| 采样请求 | eager 活动占比 | graph 活动占比 | graph 中来自图节点的 kernel 数量占比 |
|
||||
|---|---:|---:|---:|
|
||||
| 短回答 | 93.01% | 95.22% | 82.43% |
|
||||
| 32K 首次输入及回答 | 97.38% | 97.50% | 44.25% |
|
||||
| 32K 复用及回答 | 95.22% | 95.29% | 77.15% |
|
||||
|
||||
### 图生效，但覆盖范围有限且分成多段
|
||||
|
||||
- eager 的三个窗口都没有 graph-node kernel。
|
||||
- graph 短回答窗口记录 **4,664 次 `cudaGraphLaunch`，160,160 / 194,303 个 kernel 来自图节点**。
|
||||
- 调度记录显示解码 `actual=3` 被 padding 到 4，模式为 PIECEWISE；MTP 路径也存在图重放。
|
||||
同时保留部分 NONE 路径，不能说整次请求被捕获成一张图。
|
||||
- 该窗口有 88 次 `actual=3` 的图调度，累计 graph launch 约为其 53 倍。
|
||||
这是目标模型与辅助路径合计的实测比值，说明提交仍分成多段；不是每生成一个 token 就固定调用 53 次。
|
||||
- 32K 首次请求中的 **1600 和 742-token 预填充调度均为 NONE**；后续解码才命中图。
|
||||
- 日志确认 attention/KV block size 为 1600。虽然批处理上限是 2048，实际主要分块是 1600，
|
||||
因此不能仅凭配置里的 2048 推断捕获或预填充形状。
|
||||
|
||||
### PLE 的“等待时间”不是磁盘读表时间
|
||||
|
||||
32K 首次输入 trace：
|
||||
|
||||
| 时间项 | eager | graph |
|
||||
|---|---:|---:|
|
||||
| 首次至末次 GPU 活动跨度 | 17.458 s | 17.614 s |
|
||||
| GPU 活动区间并集 | 17.000 s | 17.173 s |
|
||||
| PLE CPU gather | 0.275 s | 0.292 s |
|
||||
| PLE CPU 去重（标记中的 np.unique） | 0.019 s | 0.021 s |
|
||||
| PLE 等待 GPU | 6.389 s | 7.292 s |
|
||||
| 其中与 GPU 活动重叠 | 6.341 s | 7.233 s |
|
||||
|
||||
CPU wait 与 GPU 工作几乎完全重叠，不是额外叠加在 GPU 计算之后的同等开销。
|
||||
不能把所有 API/NVTX/GPU 时长相加，也不能依据同步函数耗时最长就优先优化同步本身。
|
||||
图版本 wait 更长不证明 PLE 退化：提交时序和输出长度都不同。
|
||||
|
||||
本轮 gather 包括页缓存或实际存储读取，未测物理 NVMe I/O；没有证据说明每次都从 SSD 读取，
|
||||
也没有证据支持将整张 47.7 GiB 表强制驻留能带来显著收益。
|
||||
短回答中 CPU gather 约 0.20 秒，相比约 7 秒 GPU 活动跨度亦非主导项。
|
||||
|
||||
## 决策与后续方向
|
||||
|
||||
本轮**不扩大图捕获尺寸**。长输入 GPU 活动已接近连续，消除提交间隙的空间有限；
|
||||
扩大图还可能增加 padding、捕获内存和启动成本。当前证据不足以让它成为稳定基线的优先改动。
|
||||
这不等于证明大图一定没有收益；该候选未实测，不报告虚构加速或稳定性结果。
|
||||
按方案，只有新候选通过才追加 128K 验证，因此本轮未重跑 128K，也未再次触碰 260K。
|
||||
|
||||
稳定基线建议仍是：**eager + 前缀缓存 + MTP=2，实际使用限制在 128K 以内**。
|
||||
愿意维护实验补丁时，小尺寸图可作为约 4–5% 短回答收益的可选项；目前证据不足以要求默认切换。
|
||||
下一个有依据的性能工作应聚焦耗时 GPU 内核及其 GB10 后端：32K trace 的主要内核包含
|
||||
CUTLASS 分组量化 GEMM、稀疏注意力和归一化组合；短回答还有显著 BF16 GEMM 时间。
|
||||
要判定算力受限还是带宽受限，需要另做针对性内核分析，不能用本轮活动占比替代结论。
|
||||
|
||||
## 资源与实验修正
|
||||
|
||||
实验采样最低 MemAvailable **15.748 GiB**，主机 `oom_kill` 从 25 到 25，没有新增 OOM。
|
||||
监控每秒采样，低于 1 GiB 立即停止，持续低于 2 GiB 或主机 OOM 增加则停止。
|
||||
监控覆盖两组实验，恢复阶段另做最终系统检查；没有为了验证保护阈值主动制造内存压力。
|
||||
|
||||
首次诊断启动因临时包装函数缺少 PyTorch schema 所需类型注解而失败，尚未加载完成或执行请求。
|
||||
自动恢复触发后，修复注解并验证 `torch.library.infer_schema`，随后才运行完整两组。
|
||||
该失败属于诊断脚本问题，不是模型/vLLM 的新 bug；失败日志保留在 Spark 的 attempt1 目录。
|
||||
|
||||
原始 trace 和 SQLite 只保留在 Spark `optimization-results/profile128k/{eager,graph}`，父目录权限 0700。
|
||||
它们可能包含进程参数，不提交 Git。仓库的 [证据清单](results/profile128k/evidence-manifest.json)
|
||||
记录文件大小、SHA256、Nsight 版本、配置要点；仅提交 CUDA/NVTX 摘要、请求结果及资源记录。
|
||||
|
||||
## 恢复状态
|
||||
|
||||
2026-09-18 01:21（Asia/Shanghai）已恢复原 eager 镜像，容器 healthy、重启策略 `unless-stopped`。
|
||||
算术、中文、工具调用 smoke test 全部通过；原 Compose 与测试前逐字一致，临时 profiler 参数与挂载均已移除。
|
||||
可用内存约 18.85 GiB，主机 OOM 计数仍为 25，系统及用户 failed units 均为 0。
|
||||
恢复证据见 [recovery.json](results/profile128k/recovery.json) 和 [smoke log](results/profile128k/recovery-smoke.log)。
|
||||
|
||||
默认配置仍保留原来的 262144 上限，本轮未修改服务容量；该配置值不代表已验证的稳定容量。
|
||||
实际使用仍建议限制在 128K 以内，接近 260K 的既有 OOM 结论不变。
|
||||
|
||||
## 参考
|
||||
|
||||
- [vLLM profiling 文档](https://docs.vllm.ai/en/latest/contributing/profiling/)
|
||||
- [NVIDIA Nsight Systems User Guide](https://docs.nvidia.com/nsight-systems/UserGuide/)
|
||||
- [NVIDIA CUDA Graph 性能问题说明](https://docs.nvidia.com/dl-cuda-graph/latest/troubleshooting/performance-issues/)
|
||||
@@ -0,0 +1,394 @@
|
||||
{
|
||||
"eager": {
|
||||
"total_requests": 19,
|
||||
"correct": 19,
|
||||
"groups": {
|
||||
"short": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 0.2627224419993581,
|
||||
"values": [
|
||||
0.33354272600263357,
|
||||
0.2627224419993581,
|
||||
0.26176839799882146
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 7.373869219998596,
|
||||
"values": [
|
||||
7.68168595500174,
|
||||
7.373869219998596,
|
||||
7.35648784199293
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 29.953694756255302,
|
||||
"values": [
|
||||
28.987645227728436,
|
||||
29.953694756255302,
|
||||
30.02293954286349
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
214,
|
||||
214,
|
||||
214
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
111,
|
||||
111,
|
||||
111
|
||||
]
|
||||
},
|
||||
"prefill-8192": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 3.7875278329956927,
|
||||
"values": [
|
||||
4.006232126004761,
|
||||
3.7754420729979756,
|
||||
3.7875278329956927
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 5.739436170995759,
|
||||
"values": [
|
||||
5.835497587999271,
|
||||
5.626387860997056,
|
||||
5.739436170995759
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 35.041074313706375,
|
||||
"values": [
|
||||
35.584837627199924,
|
||||
35.041074313706375,
|
||||
32.64277985613848
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
63,
|
||||
63,
|
||||
62
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
55,
|
||||
55,
|
||||
54
|
||||
]
|
||||
},
|
||||
"reuse-8192": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 0.8803304789980757,
|
||||
"values": [
|
||||
0.8856951920024585,
|
||||
0.8737719170021592,
|
||||
0.8803304789980757
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 2.811309409997193,
|
||||
"values": [
|
||||
2.7496982970042154,
|
||||
2.835690722997242,
|
||||
2.811309409997193
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 36.047200027797416,
|
||||
"values": [
|
||||
34.74998178667343,
|
||||
36.047200027797416,
|
||||
36.74879746387076
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
63,
|
||||
69,
|
||||
69
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
55,
|
||||
61,
|
||||
61
|
||||
]
|
||||
},
|
||||
"prefill-32768": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 15.41581593700539,
|
||||
"values": [
|
||||
15.595506220997777,
|
||||
15.403379977004079,
|
||||
15.41581593700539
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 17.35202403000585,
|
||||
"values": [
|
||||
17.837196191001567,
|
||||
17.295454844999767,
|
||||
17.35202403000585
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 36.438073002765556,
|
||||
"values": [
|
||||
33.795316059955795,
|
||||
36.438073002765556,
|
||||
36.713832650404036
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
74,
|
||||
67,
|
||||
69
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
66,
|
||||
59,
|
||||
61
|
||||
]
|
||||
},
|
||||
"reuse-32768": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 1.367758443004277,
|
||||
"values": [
|
||||
1.368702627994935,
|
||||
1.367758443004277,
|
||||
1.357998088002205
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 3.5468969909998123,
|
||||
"values": [
|
||||
3.579059300995141,
|
||||
3.5468969909998123,
|
||||
3.3747847780032316
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 32.877578996597116,
|
||||
"values": [
|
||||
32.419700549347965,
|
||||
32.877578996597116,
|
||||
33.71865583479496
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
70,
|
||||
70,
|
||||
69
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
62,
|
||||
62,
|
||||
61
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"graph": {
|
||||
"total_requests": 19,
|
||||
"correct": 19,
|
||||
"groups": {
|
||||
"short": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 0.2623967160034226,
|
||||
"values": [
|
||||
0.32573997599683935,
|
||||
0.2600282569983392,
|
||||
0.2623967160034226
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 7.0642776130043785,
|
||||
"values": [
|
||||
7.360314821999054,
|
||||
7.060739258995454,
|
||||
7.0642776130043785
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 31.315723764990278,
|
||||
"values": [
|
||||
30.279974628892248,
|
||||
31.32107217692348,
|
||||
31.315723764990278
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
214,
|
||||
214,
|
||||
214
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
111,
|
||||
111,
|
||||
111
|
||||
]
|
||||
},
|
||||
"prefill-8192": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 3.7701075279983343,
|
||||
"values": [
|
||||
3.921214998998039,
|
||||
3.7701075279983343,
|
||||
3.768740819999948
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 5.670499198000471,
|
||||
"values": [
|
||||
6.191493427999376,
|
||||
5.596617848997994,
|
||||
5.670499198000471
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 35.516432222041566,
|
||||
"values": [
|
||||
33.338899955373805,
|
||||
35.516432222041566,
|
||||
37.32072215199369
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
74,
|
||||
63,
|
||||
69
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
66,
|
||||
55,
|
||||
61
|
||||
]
|
||||
},
|
||||
"reuse-8192": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 0.8737782960015466,
|
||||
"values": [
|
||||
0.8737782960015466,
|
||||
0.8690901810041396,
|
||||
0.8816422219970264
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 2.8319971849996364,
|
||||
"values": [
|
||||
2.8319971849996364,
|
||||
3.0074559670028975,
|
||||
2.711807350999152
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 35.510793077788634,
|
||||
"values": [
|
||||
34.055335354489465,
|
||||
35.510793077788634,
|
||||
35.522341967132114
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
65,
|
||||
74,
|
||||
63
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
57,
|
||||
66,
|
||||
55
|
||||
]
|
||||
},
|
||||
"prefill-32768": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 15.375977513001999,
|
||||
"values": [
|
||||
15.511759164000978,
|
||||
15.357147006005107,
|
||||
15.375977513001999
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 17.268514374001825,
|
||||
"values": [
|
||||
17.691939716001798,
|
||||
17.239325528003974,
|
||||
17.268514374001825
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 37.54862656190945,
|
||||
"values": [
|
||||
32.87688689931954,
|
||||
37.728914829613814,
|
||||
37.54862656190945
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
70,
|
||||
69,
|
||||
69
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
62,
|
||||
61,
|
||||
61
|
||||
]
|
||||
},
|
||||
"reuse-32768": {
|
||||
"n": 3,
|
||||
"correct": 3,
|
||||
"ttft_s": {
|
||||
"median": 1.3270910260034725,
|
||||
"values": [
|
||||
1.325252010996337,
|
||||
1.3272808809997514,
|
||||
1.3270910260034725
|
||||
]
|
||||
},
|
||||
"elapsed_s": {
|
||||
"median": 3.293290553992847,
|
||||
"values": [
|
||||
3.293290553992847,
|
||||
3.4540864679947845,
|
||||
3.2060768180017476
|
||||
]
|
||||
},
|
||||
"decode_tps_approx": {
|
||||
"median": 36.2184980346298,
|
||||
"values": [
|
||||
36.2184980346298,
|
||||
33.71980234098872,
|
||||
37.82214005937634
|
||||
]
|
||||
},
|
||||
"output_tokens": [
|
||||
69,
|
||||
70,
|
||||
69
|
||||
],
|
||||
"reasoning_tokens": [
|
||||
61,
|
||||
62,
|
||||
61
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
{"label": "eager", "time": 1789663943.2948034, "event": "REQUEST_START", "name": "warmup", "profile": false, "prompt_tokens_preflight": 51}
|
||||
{"label": "eager", "time": 1789663945.101216, "event": "RESULT", "name": "warmup", "profile": false, "correct": true, "content": "\n\n323", "finish_reason": "stop", "usage": {"prompt_tokens": 51, "total_tokens": 83, "completion_tokens": 32, "completion_tokens_details": {"reasoning_tokens": 26}}, "max_tokens": 2048, "prompt_sha256": "4181e1a44a0bed10d00c74e822377bd75a6c6c9a1ea458f5170b8193e9fce959", "output_sha256": "4477e82dcd9f3f8da1dd56be073d72e4ba20121d6ce6fbc97ee54258fb4b2959", "ttft_s": 0.8800418169994373, "first_content_s": 1.7162362779999967, "elapsed_s": 1.800662048997765, "decode_tps_approx": 37.07271627093361, "metrics_delta": {"vllm:prefix_cache_queries_total": 51.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789663945.1120813, "event": "REQUEST_START", "name": "short-0", "profile": false, "prompt_tokens_preflight": 65}
|
||||
{"label": "eager", "time": 1789663952.799862, "event": "RESULT", "name": "short-0", "profile": false, "correct": true, "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "finish_reason": "stop", "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "max_tokens": 512, "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "ttft_s": 0.33354272600263357, "first_content_s": 3.9721969300007913, "elapsed_s": 7.68168595500174, "decode_tps_approx": 28.987645227728436, "metrics_delta": {"vllm:prefix_cache_queries_total": 65.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789663952.8174412, "event": "REQUEST_START", "name": "prefill-8192-0", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "eager", "time": 1789663958.6588385, "event": "RESULT", "name": "prefill-8192-0", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8236, "completion_tokens": 63, "completion_tokens_details": {"reasoning_tokens": 55}}, "max_tokens": 2048, "prompt_sha256": "f59f163e1c7bb8a79d08df830d94d6d2e98fec457e9de75e3c09f6fca7ab0813", "output_sha256": "09d17d110b06ec13d532382c41ea29955b7edfa8dd4bd34163a08fa746e56610", "ttft_s": 4.006232126004761, "first_content_s": 5.657178637004108, "elapsed_s": 5.835497587999271, "decode_tps_approx": 35.584837627199924, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789663958.6799748, "event": "REQUEST_START", "name": "reuse-8192-0", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "eager", "time": 1789663961.4358463, "event": "RESULT", "name": "reuse-8192-0", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8236, "completion_tokens": 63, "completion_tokens_details": {"reasoning_tokens": 55}}, "max_tokens": 2048, "prompt_sha256": "f59f163e1c7bb8a79d08df830d94d6d2e98fec457e9de75e3c09f6fca7ab0813", "output_sha256": "09d17d110b06ec13d532382c41ea29955b7edfa8dd4bd34163a08fa746e56610", "ttft_s": 0.8856951920024585, "first_content_s": 2.5918558570047026, "elapsed_s": 2.7496982970042154, "decode_tps_approx": 34.74998178667343, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 6400.0}}
|
||||
{"label": "eager", "time": 1789663961.4779031, "event": "REQUEST_START", "name": "prefill-32768-0", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "eager", "time": 1789663979.3212829, "event": "RESULT", "name": "prefill-32768-0", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32819, "completion_tokens": 74, "completion_tokens_details": {"reasoning_tokens": 66}}, "max_tokens": 2048, "prompt_sha256": "605f7b492252f50eaf60e8d128dde3f2d2e1b50dd26f0b45db88dac00e36a5ca", "output_sha256": "997501e1996b1c37af707622b1c4caf8dba93113e868f28210bd47ba700b5804", "ttft_s": 15.595506220997777, "first_content_s": 17.658189941001183, "elapsed_s": 17.837196191001567, "decode_tps_approx": 33.795316059955795, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789663979.3644226, "event": "REQUEST_START", "name": "reuse-32768-0", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "eager", "time": 1789663982.9494953, "event": "RESULT", "name": "reuse-32768-0", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32815, "completion_tokens": 70, "completion_tokens_details": {"reasoning_tokens": 62}}, "max_tokens": 2048, "prompt_sha256": "605f7b492252f50eaf60e8d128dde3f2d2e1b50dd26f0b45db88dac00e36a5ca", "output_sha256": "7eec57291da7ee95ae6d9214f8ed940a405e64beb9f49a4693387362aac3e14c", "ttft_s": 1.368702627994935, "first_content_s": 3.42049944599421, "elapsed_s": 3.579059300995141, "decode_tps_approx": 32.419700549347965, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 30400.0}}
|
||||
{"label": "eager", "time": 1789663982.957122, "event": "REQUEST_START", "name": "short-1", "profile": false, "prompt_tokens_preflight": 65}
|
||||
{"label": "eager", "time": 1789663990.3364515, "event": "RESULT", "name": "short-1", "profile": false, "correct": true, "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "finish_reason": "stop", "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "max_tokens": 512, "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "ttft_s": 0.2627224419993581, "first_content_s": 3.7929709769960027, "elapsed_s": 7.373869219998596, "decode_tps_approx": 29.953694756255302, "metrics_delta": {"vllm:prefix_cache_queries_total": 65.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789663990.3527281, "event": "REQUEST_START", "name": "prefill-8192-1", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "eager", "time": 1789663995.9852123, "event": "RESULT", "name": "prefill-8192-1", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8236, "completion_tokens": 63, "completion_tokens_details": {"reasoning_tokens": 55}}, "max_tokens": 2048, "prompt_sha256": "6c13deb2d4ddee54db95d7ebc8a1b87d24ed8a4112c33644c18b6b5d61ad7a16", "output_sha256": "09d17d110b06ec13d532382c41ea29955b7edfa8dd4bd34163a08fa746e56610", "ttft_s": 3.7754420729979756, "first_content_s": 5.466695101997175, "elapsed_s": 5.626387860997056, "decode_tps_approx": 35.041074313706375, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789663996.0020921, "event": "REQUEST_START", "name": "reuse-8192-1", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "eager", "time": 1789663998.84421, "event": "RESULT", "name": "reuse-8192-1", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8242, "completion_tokens": 69, "completion_tokens_details": {"reasoning_tokens": 61}}, "max_tokens": 2048, "prompt_sha256": "6c13deb2d4ddee54db95d7ebc8a1b87d24ed8a4112c33644c18b6b5d61ad7a16", "output_sha256": "f72eba21a095aa0a43356b25cb120346b064d7f0a253a63cee7579d2c3422b82", "ttft_s": 0.8737719170021592, "first_content_s": 2.675332922997768, "elapsed_s": 2.835690722997242, "decode_tps_approx": 36.047200027797416, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 6400.0}}
|
||||
{"label": "eager", "time": 1789663998.8886485, "event": "REQUEST_START", "name": "prefill-32768-1", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "eager", "time": 1789664016.1901195, "event": "RESULT", "name": "prefill-32768-1", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32812, "completion_tokens": 67, "completion_tokens_details": {"reasoning_tokens": 59}}, "max_tokens": 2048, "prompt_sha256": "0a5e4ba33ff264d180b8a56d89184c0fefad59a96fdd348d655bd54fe5b5ed45", "output_sha256": "ca2e8f89b165bee277a12cac7bb1cbc8ce34eef2ee29ecd1a8a46aa71c502cfa", "ttft_s": 15.403379977004079, "first_content_s": 17.130364165001083, "elapsed_s": 17.295454844999767, "decode_tps_approx": 36.438073002765556, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789664016.2312467, "event": "REQUEST_START", "name": "reuse-32768-1", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "eager", "time": 1789664019.7841623, "event": "RESULT", "name": "reuse-32768-1", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32815, "completion_tokens": 70, "completion_tokens_details": {"reasoning_tokens": 62}}, "max_tokens": 2048, "prompt_sha256": "0a5e4ba33ff264d180b8a56d89184c0fefad59a96fdd348d655bd54fe5b5ed45", "output_sha256": "7eec57291da7ee95ae6d9214f8ed940a405e64beb9f49a4693387362aac3e14c", "ttft_s": 1.367758443004277, "first_content_s": 3.3833834170000046, "elapsed_s": 3.5468969909998123, "decode_tps_approx": 32.877578996597116, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 30400.0}}
|
||||
{"label": "eager", "time": 1789664019.7920012, "event": "REQUEST_START", "name": "short-2", "profile": false, "prompt_tokens_preflight": 65}
|
||||
{"label": "eager", "time": 1789664027.1539514, "event": "RESULT", "name": "short-2", "profile": false, "correct": true, "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "finish_reason": "stop", "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "max_tokens": 512, "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "ttft_s": 0.26176839799882146, "first_content_s": 3.792491918997257, "elapsed_s": 7.35648784199293, "decode_tps_approx": 30.02293954286349, "metrics_delta": {"vllm:prefix_cache_queries_total": 65.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789664027.17096, "event": "REQUEST_START", "name": "prefill-8192-2", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "eager", "time": 1789664032.9165242, "event": "RESULT", "name": "prefill-8192-2", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8235, "completion_tokens": 62, "completion_tokens_details": {"reasoning_tokens": 54}}, "max_tokens": 2048, "prompt_sha256": "b49f6dbbf80f49ad860a60b0a7c58dfc2122e38bb8493ae155790f22305d8f7a", "output_sha256": "0eb9ce304c0547f0814d0590d36cb2144dcc242a2c550f3ba9f8a70b2e22b203", "ttft_s": 3.7875278329956927, "first_content_s": 5.574596539998311, "elapsed_s": 5.739436170995759, "decode_tps_approx": 32.64277985613848, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789664032.9353032, "event": "REQUEST_START", "name": "reuse-8192-2", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "eager", "time": 1789664035.7520719, "event": "RESULT", "name": "reuse-8192-2", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8242, "completion_tokens": 69, "completion_tokens_details": {"reasoning_tokens": 61}}, "max_tokens": 2048, "prompt_sha256": "b49f6dbbf80f49ad860a60b0a7c58dfc2122e38bb8493ae155790f22305d8f7a", "output_sha256": "f72eba21a095aa0a43356b25cb120346b064d7f0a253a63cee7579d2c3422b82", "ttft_s": 0.8803304789980757, "first_content_s": 2.651041837001685, "elapsed_s": 2.811309409997193, "decode_tps_approx": 36.74879746387076, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 6400.0}}
|
||||
{"label": "eager", "time": 1789664035.7953794, "event": "REQUEST_START", "name": "prefill-32768-2", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "eager", "time": 1789664053.153125, "event": "RESULT", "name": "prefill-32768-2", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32814, "completion_tokens": 69, "completion_tokens_details": {"reasoning_tokens": 61}}, "max_tokens": 2048, "prompt_sha256": "d8796cea8608150048ce4f253e451ba96da5416ccfea4a6560dc2f6bb2412004", "output_sha256": "8b503330d6b70b8845bab80540601fc48c8f840d54d07f38d80250c8167de89f", "ttft_s": 15.41581593700539, "first_content_s": 17.18937433300016, "elapsed_s": 17.35202403000585, "decode_tps_approx": 36.713832650404036, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789664053.1986973, "event": "REQUEST_START", "name": "reuse-32768-2", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "eager", "time": 1789664056.5784104, "event": "RESULT", "name": "reuse-32768-2", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32814, "completion_tokens": 69, "completion_tokens_details": {"reasoning_tokens": 61}}, "max_tokens": 2048, "prompt_sha256": "d8796cea8608150048ce4f253e451ba96da5416ccfea4a6560dc2f6bb2412004", "output_sha256": "8b503330d6b70b8845bab80540601fc48c8f840d54d07f38d80250c8167de89f", "ttft_s": 1.357998088002205, "first_content_s": 3.2952792900032364, "elapsed_s": 3.3747847780032316, "decode_tps_approx": 33.71865583479496, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 30400.0}}
|
||||
{"label": "eager", "time": 1789664056.5835583, "event": "REQUEST_START", "name": "trace-short", "profile": true, "prompt_tokens_preflight": 65}
|
||||
{"label": "eager", "time": 1789664067.1052518, "event": "RESULT", "name": "trace-short", "profile": true, "correct": true, "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "finish_reason": "stop", "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "max_tokens": 512, "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "ttft_s": 0.28372384200338274, "first_content_s": 3.9164422460016795, "elapsed_s": 10.166879840006004, "decode_tps_approx": 29.24182266761644, "metrics_delta": {"vllm:prefix_cache_queries_total": 65.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789664067.137696, "event": "REQUEST_START", "name": "trace-prefill-32768", "profile": true, "prompt_tokens_preflight": 32742}
|
||||
{"label": "eager", "time": 1789664086.9158142, "event": "RESULT", "name": "trace-prefill-32768", "profile": true, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32742, "total_tokens": 32809, "completion_tokens": 67, "completion_tokens_details": {"reasoning_tokens": 59}}, "max_tokens": 2048, "prompt_sha256": "464e9da0976ca56ddece006a237605bca8b60d3dd0aab538abbc0aaa1696815c", "output_sha256": "ca2e8f89b165bee277a12cac7bb1cbc8ce34eef2ee29ecd1a8a46aa71c502cfa", "ttft_s": 15.503820653997536, "first_content_s": 17.303895275996183, "elapsed_s": 19.650969089998398, "decode_tps_approx": 35.058139901273435, "metrics_delta": {"vllm:prefix_cache_queries_total": 32742.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "eager", "time": 1789664086.9506161, "event": "REQUEST_START", "name": "trace-reuse-32768", "profile": true, "prompt_tokens_preflight": 32742}
|
||||
{"label": "eager", "time": 1789664093.2510297, "event": "RESULT", "name": "trace-reuse-32768", "profile": true, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32742, "total_tokens": 32810, "completion_tokens": 68, "completion_tokens_details": {"reasoning_tokens": 60}}, "max_tokens": 2048, "prompt_sha256": "464e9da0976ca56ddece006a237605bca8b60d3dd0aab538abbc0aaa1696815c", "output_sha256": "ce351e2ffc5a1b2641f170a3fa56c81034b4d81226c524d10936eb65c7eb3926", "ttft_s": 1.385693889002141, "first_content_s": 3.091133230998821, "elapsed_s": 6.173271200997988, "decode_tps_approx": 37.499826794181516, "metrics_delta": {"vllm:prefix_cache_queries_total": 32742.0, "vllm:prefix_cache_hits_total": 30400.0}}
|
||||
{"label": "eager", "time": 1789664093.251096, "event": "PROFILE_SUITE_PASS"}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,823 @@
|
||||
{
|
||||
"file": "trace.2.sqlite",
|
||||
"gpu_activity_span_s": 17.45799776,
|
||||
"gpu_busy_union_s": 16.999974145,
|
||||
"gpu_idle_within_span_s": 0.4580236150000019,
|
||||
"kernel_count": 107899,
|
||||
"graph_node_kernel_count": 0,
|
||||
"api": {
|
||||
"cudaStreamSynchronize_v3020": [
|
||||
92,
|
||||
6.388727359999995
|
||||
],
|
||||
"cuLaunchKernelEx": [
|
||||
30313,
|
||||
2.942097295999861
|
||||
],
|
||||
"cudaLaunchKernelExC_v11060": [
|
||||
27262,
|
||||
2.8157289279998983
|
||||
],
|
||||
"cudaLaunchKernel_v7000": [
|
||||
34403,
|
||||
2.1073674239999285
|
||||
],
|
||||
"cuLaunchKernel": [
|
||||
15921,
|
||||
0.18051391999999974
|
||||
],
|
||||
"cudaMemcpyAsync_v3020": [
|
||||
2359,
|
||||
0.02657009600000002
|
||||
],
|
||||
"cudaEventSynchronize_v3020": [
|
||||
46,
|
||||
0.01590248
|
||||
],
|
||||
"cudaEventRecordWithFlags_v11010": [
|
||||
3413,
|
||||
0.006630880000000002
|
||||
],
|
||||
"cudaMemsetAsync_v3020": [
|
||||
736,
|
||||
0.0055193599999999975
|
||||
],
|
||||
"cuTensorMapEncodeTiled": [
|
||||
27416,
|
||||
0.004886224000000271
|
||||
],
|
||||
"cuKernelGetFunction": [
|
||||
15921,
|
||||
0.0038959359999997194
|
||||
],
|
||||
"cudaGetDriverEntryPointByVersion_v12050": [
|
||||
26496,
|
||||
0.0037329280000005572
|
||||
],
|
||||
"cuKernelGetName": [
|
||||
35159,
|
||||
0.0033410560000001516
|
||||
],
|
||||
"cudaStreamWaitEvent_v3020": [
|
||||
2588,
|
||||
0.0021758880000000004
|
||||
],
|
||||
"cudaEventQuery_v3020": [
|
||||
1494,
|
||||
0.0019392639999999957
|
||||
],
|
||||
"cudaStreamIsCapturing_v10000": [
|
||||
6960,
|
||||
0.0014392959999999955
|
||||
],
|
||||
"cudaEventCreateWithFlags_v3020": [
|
||||
92,
|
||||
0.000234256
|
||||
],
|
||||
"cudaEventDestroy_v3020": [
|
||||
92,
|
||||
0.00018534400000000007
|
||||
]
|
||||
},
|
||||
"nvtx": {
|
||||
"PLE_lookup_tokens=1600": {
|
||||
"count": 20,
|
||||
"sum_s": 5.58014968,
|
||||
"union_s": 5.58014968,
|
||||
"gpu_overlap_s": 5.318215141
|
||||
},
|
||||
"PLE_wait_for_GPU": {
|
||||
"count": 46,
|
||||
"sum_s": 6.389229296,
|
||||
"union_s": 6.389229296,
|
||||
"gpu_overlap_s": 6.341070402
|
||||
},
|
||||
"PLE_CPU_dedup": {
|
||||
"count": 46,
|
||||
"sum_s": 0.01914024,
|
||||
"union_s": 0.01914024,
|
||||
"gpu_overlap_s": 0.0
|
||||
},
|
||||
"PLE_CPU_gather": {
|
||||
"count": 46,
|
||||
"sum_s": 0.275209776,
|
||||
"union_s": 0.275209776,
|
||||
"gpu_overlap_s": 0.0
|
||||
},
|
||||
"PLE_lookup_tokens=742": {
|
||||
"count": 1,
|
||||
"sum_s": 0.29896912,
|
||||
"union_s": 0.29896912,
|
||||
"gpu_overlap_s": 0.278459202
|
||||
},
|
||||
"PLE_lookup_tokens=3": {
|
||||
"count": 25,
|
||||
"sum_s": 0.820145456,
|
||||
"union_s": 0.820145456,
|
||||
"gpu_overlap_s": 0.746778411
|
||||
}
|
||||
},
|
||||
"dispatch": {
|
||||
"CG_DISPATCH:actual=1600:mode=NONE:padded=1600": 40,
|
||||
"CG_DISPATCH:actual=1:mode=NONE:padded=1": 46,
|
||||
"CG_DISPATCH:actual=742:mode=NONE:padded=742": 2,
|
||||
"CG_DISPATCH:actual=3:mode=NONE:padded=3": 50
|
||||
},
|
||||
"dispatch_timeline": [
|
||||
{
|
||||
"start_ns": 45828672,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 415343408,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 441920176,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 467464272,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 1117406208,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 1146479600,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1170640896,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 1839789152,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 1870939456,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1896334032,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 2568046544,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 2598479856,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2624039184,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 3305750304,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 3335311984,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3361130160,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 4046256096,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 4076056912,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 4101620656,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 4786128624,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 4816276720,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 4842056960,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 5522623936,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 5552300720,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 5576921440,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 6259346144,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 6290320240,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 6315131568,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 7002476032,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 7032926192,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 7058327584,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 7753655264,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 7785387856,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 7811813408,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 8504434640,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 8535191696,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 8561506848,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 9253677696,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 9285233312,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 9311217936,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 10009614064,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 10040506016,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 10066581360,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 10757969120,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 10789570400,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 10815237984,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 11519444752,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 11550823200,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 11575788896,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 12278226144,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 12309497040,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 12335515504,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 13034051120,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 13066055552,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 13091963328,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 13790163952,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 13821175296,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 13846090352,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 14557613040,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 14589908560,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 14615422640,
|
||||
"name": "CG_DISPATCH:actual=742:mode=NONE:padded=742"
|
||||
},
|
||||
{
|
||||
"start_ns": 15203740016,
|
||||
"name": "CG_DISPATCH:actual=742:mode=NONE:padded=742"
|
||||
},
|
||||
{
|
||||
"start_ns": 15220760336,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15234841296,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15491074784,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15493643856,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15497135184,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15564983824,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15566982016,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15570203408,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15644290848,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15646296832,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15649329520,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15729121168,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15731019920,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15734338320,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15813356656,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15815902416,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15819264448,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15894042800,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15895960816,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15898832688,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15982235440,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 15984929392,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15988404656,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16064077552,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16066769024,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16070591120,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16140542304,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16142492784,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16145392256,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16230150416,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16232897584,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16236422000,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16305893904,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16307780160,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16310441504,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16394138896,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16396838944,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16400267168,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16476712080,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16479446496,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16482904144,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16559135840,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16561788800,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16565117088,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16632650032,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16634497504,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16637262864,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16722767040,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16725243152,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16728652272,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16807455696,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16810202752,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16813561920,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16894260000,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16898134688,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16903894256,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16982500864,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 16985299840,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16988503344,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17054985488,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17056901872,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17059848448,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17143682800,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17146548688,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17150240800,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17226658560,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17229352608,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17233024096,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17306935664,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17308968096,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17311992480,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17388634448,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17391419008,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17394878992,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17463051952,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 17464913120,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
}
|
||||
],
|
||||
"top_kernels": [
|
||||
[
|
||||
"device_kernel",
|
||||
[
|
||||
4416,
|
||||
5.785133960999994
|
||||
]
|
||||
],
|
||||
[
|
||||
"Kernel2",
|
||||
[
|
||||
15921,
|
||||
1.6569721249999942
|
||||
]
|
||||
],
|
||||
[
|
||||
"_qsa_sparse_paged_gqa_splitk_kernel",
|
||||
[
|
||||
644,
|
||||
1.481612926999999
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x208x64_2_32x104x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
1000,
|
||||
1.3622323199999982
|
||||
]
|
||||
],
|
||||
[
|
||||
"_hc_combine_norm_kernel",
|
||||
[
|
||||
4646,
|
||||
0.8981812170000064
|
||||
]
|
||||
],
|
||||
[
|
||||
"kernel",
|
||||
[
|
||||
1510,
|
||||
0.8489803810000001
|
||||
]
|
||||
],
|
||||
[
|
||||
"_hc_gate_mix_kernel",
|
||||
[
|
||||
4738,
|
||||
0.6790835519999859
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x240x64_2_32x120x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
980,
|
||||
0.5484992310000006
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x128x64_3_32x64x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
2000,
|
||||
0.484882112
|
||||
]
|
||||
],
|
||||
[
|
||||
"finalizeMoeRoutingKernel",
|
||||
[
|
||||
2208,
|
||||
0.4100567680000026
|
||||
]
|
||||
],
|
||||
[
|
||||
"_causal_conv1d_fwd_kernel",
|
||||
[
|
||||
756,
|
||||
0.2423829440000001
|
||||
]
|
||||
],
|
||||
[
|
||||
"_fused_post_conv_kernel",
|
||||
[
|
||||
756,
|
||||
0.23299334399999985
|
||||
]
|
||||
],
|
||||
[
|
||||
"doActivationKernel",
|
||||
[
|
||||
2208,
|
||||
0.2290338470000007
|
||||
]
|
||||
],
|
||||
[
|
||||
"elementwise_kernel",
|
||||
[
|
||||
6060,
|
||||
0.22898364799999965
|
||||
]
|
||||
],
|
||||
[
|
||||
"kernel_cutlass_kernel_flashinfergdn_kernelsdelta_rule_dsldelta_rule_sm120_FullyFusedDeltaRuleSm120_object_at__tensorptrf32gmemalign16o1_tensorptrf32gmemalign16o1_CopyAtom_ThrID10_TVLayout_0",
|
||||
[
|
||||
756,
|
||||
0.19385532799999977
|
||||
]
|
||||
],
|
||||
[
|
||||
"vectorized_elementwise_kernel",
|
||||
[
|
||||
10976,
|
||||
0.18249009600000335
|
||||
]
|
||||
],
|
||||
[
|
||||
"expandInputRowsKernel",
|
||||
[
|
||||
2208,
|
||||
0.17563680399999979
|
||||
]
|
||||
],
|
||||
[
|
||||
"layer_norm_fwd_kernel",
|
||||
[
|
||||
756,
|
||||
0.17546492700000002
|
||||
]
|
||||
],
|
||||
[
|
||||
"sm120_fp8_fp4_gemm_1d1d_impl",
|
||||
[
|
||||
184,
|
||||
0.16484399999999996
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x192x64_2_32x96x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
980,
|
||||
0.13451315200000008
|
||||
]
|
||||
]
|
||||
],
|
||||
"notes": "GPU busy is interval union, not utilization counter. CPU/API/NVTX times overlap GPU work and each other; do not sum them as exclusive costs."
|
||||
}
|
||||
@@ -0,0 +1,583 @@
|
||||
{
|
||||
"file": "trace.3.sqlite",
|
||||
"gpu_activity_span_s": 3.245924224,
|
||||
"gpu_busy_union_s": 3.090907949,
|
||||
"gpu_idle_within_span_s": 0.15501627499999993,
|
||||
"kernel_count": 58496,
|
||||
"graph_node_kernel_count": 0,
|
||||
"api": {
|
||||
"cudaStreamSynchronize_v3020": [
|
||||
52,
|
||||
1.0780331839999997
|
||||
],
|
||||
"cudaLaunchKernelExC_v11060": [
|
||||
16696,
|
||||
0.3360265120000019
|
||||
],
|
||||
"cuLaunchKernelEx": [
|
||||
13138,
|
||||
0.22730212800000002
|
||||
],
|
||||
"cudaLaunchKernel_v7000": [
|
||||
16229,
|
||||
0.21477230400000039
|
||||
],
|
||||
"cuLaunchKernel": [
|
||||
12433,
|
||||
0.05239439999999998
|
||||
],
|
||||
"cudaMemcpyAsync_v3020": [
|
||||
939,
|
||||
0.00983091199999999
|
||||
],
|
||||
"cudaEventSynchronize_v3020": [
|
||||
26,
|
||||
0.0089596
|
||||
],
|
||||
"cudaEventRecordWithFlags_v11010": [
|
||||
2877,
|
||||
0.004890783999999996
|
||||
],
|
||||
"cuKernelGetFunction": [
|
||||
12433,
|
||||
0.0027621279999998907
|
||||
],
|
||||
"cuTensorMapEncodeTiled": [
|
||||
15496,
|
||||
0.002756256000000075
|
||||
],
|
||||
"cudaMemsetAsync_v3020": [
|
||||
416,
|
||||
0.0023939039999999988
|
||||
],
|
||||
"cudaGetDriverEntryPointByVersion_v12050": [
|
||||
14976,
|
||||
0.001993504000000201
|
||||
],
|
||||
"cudaStreamWaitEvent_v3020": [
|
||||
2430,
|
||||
0.0018610560000000033
|
||||
],
|
||||
"cuKernelGetName": [
|
||||
16301,
|
||||
0.0014150560000001082
|
||||
],
|
||||
"cudaStreamIsCapturing_v10000": [
|
||||
5810,
|
||||
0.0011180959999999769
|
||||
],
|
||||
"cudaEventQuery_v3020": [
|
||||
798,
|
||||
0.001018544
|
||||
],
|
||||
"cudaEventCreateWithFlags_v3020": [
|
||||
52,
|
||||
0.000134192
|
||||
],
|
||||
"cudaEventDestroy_v3020": [
|
||||
52,
|
||||
0.00010068799999999997
|
||||
]
|
||||
},
|
||||
"nvtx": {
|
||||
"PLE_lookup_tokens=1600": {
|
||||
"count": 1,
|
||||
"sum_s": 0.018957376,
|
||||
"union_s": 0.018957376,
|
||||
"gpu_overlap_s": 0.011426528
|
||||
},
|
||||
"PLE_wait_for_GPU": {
|
||||
"count": 26,
|
||||
"sum_s": 1.078286144,
|
||||
"union_s": 1.078286144,
|
||||
"gpu_overlap_s": 1.058767637
|
||||
},
|
||||
"PLE_CPU_dedup": {
|
||||
"count": 26,
|
||||
"sum_s": 0.002243088,
|
||||
"union_s": 0.002243088,
|
||||
"gpu_overlap_s": 0.0
|
||||
},
|
||||
"PLE_CPU_gather": {
|
||||
"count": 26,
|
||||
"sum_s": 0.062812192,
|
||||
"union_s": 0.062812192,
|
||||
"gpu_overlap_s": 0.0
|
||||
},
|
||||
"PLE_lookup_tokens=742": {
|
||||
"count": 1,
|
||||
"sum_s": 0.292337536,
|
||||
"union_s": 0.292337536,
|
||||
"gpu_overlap_s": 0.279791319
|
||||
},
|
||||
"PLE_lookup_tokens=3": {
|
||||
"count": 24,
|
||||
"sum_s": 0.842259408,
|
||||
"union_s": 0.842259408,
|
||||
"gpu_overlap_s": 0.771610318
|
||||
}
|
||||
},
|
||||
"dispatch": {
|
||||
"CG_DISPATCH:actual=1600:mode=NONE:padded=1600": 2,
|
||||
"CG_DISPATCH:actual=1:mode=NONE:padded=1": 26,
|
||||
"CG_DISPATCH:actual=742:mode=NONE:padded=742": 2,
|
||||
"CG_DISPATCH:actual=3:mode=NONE:padded=3": 48
|
||||
},
|
||||
"dispatch_timeline": [
|
||||
{
|
||||
"start_ns": 46439136,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 440098864,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 472554240,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 498029616,
|
||||
"name": "CG_DISPATCH:actual=742:mode=NONE:padded=742"
|
||||
},
|
||||
{
|
||||
"start_ns": 1080640368,
|
||||
"name": "CG_DISPATCH:actual=742:mode=NONE:padded=742"
|
||||
},
|
||||
{
|
||||
"start_ns": 1097925088,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1109689376,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1374741120,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1377837856,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1381724192,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1449909984,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1452632352,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1456060352,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1537442544,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1540106512,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1543362000,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1622817104,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1625489424,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1628952064,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1697539568,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1699380000,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1702138000,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1777756128,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1779549280,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1782099440,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1865886816,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1868546304,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1872017632,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1939645984,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 1941472672,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1944250112,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2021839712,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2023852576,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2026597104,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2103480432,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2105282848,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2107900064,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2193835440,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2196580896,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2199840944,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2282745504,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2285692352,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2289777504,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2362513824,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2365324800,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2368818832,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2436200000,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2438187600,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2441020928,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2516737184,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2518684848,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2521416768,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2596916928,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2598750864,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2601295488,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2678072112,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2679938368,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2682459200,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2756745440,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2758542912,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2761074960,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2838330208,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2840177632,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2842655760,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2918308832,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 2920073040,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2922520640,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 3006922960,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 3008938128,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3011760224,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 3094064288,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 3096710352,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3100043552,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 3174526160,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 3177165152,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3180415120,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 3257416576,
|
||||
"name": "CG_DISPATCH:actual=3:mode=NONE:padded=3"
|
||||
},
|
||||
{
|
||||
"start_ns": 3260041984,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
}
|
||||
],
|
||||
"top_kernels": [
|
||||
[
|
||||
"Kernel2",
|
||||
[
|
||||
12433,
|
||||
1.083779682999996
|
||||
]
|
||||
],
|
||||
[
|
||||
"device_kernel",
|
||||
[
|
||||
2496,
|
||||
0.8852412400000004
|
||||
]
|
||||
],
|
||||
[
|
||||
"kernel",
|
||||
[
|
||||
360,
|
||||
0.38886832
|
||||
]
|
||||
],
|
||||
[
|
||||
"_qsa_sparse_paged_gqa_splitk_kernel",
|
||||
[
|
||||
364,
|
||||
0.12547817500000008
|
||||
]
|
||||
],
|
||||
[
|
||||
"_hc_combine_norm_kernel",
|
||||
[
|
||||
2626,
|
||||
0.06954325899999982
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x208x64_2_32x104x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
50,
|
||||
0.068864416
|
||||
]
|
||||
],
|
||||
[
|
||||
"_hc_gate_mix_kernel",
|
||||
[
|
||||
2678,
|
||||
0.04872185600000148
|
||||
]
|
||||
],
|
||||
[
|
||||
"finalizeMoeRoutingKernel",
|
||||
[
|
||||
1248,
|
||||
0.03866553600000007
|
||||
]
|
||||
],
|
||||
[
|
||||
"sm120_fp8_fp4_gemm_1d1d_impl",
|
||||
[
|
||||
104,
|
||||
0.038346688000000004
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_192x160x64_2_48x80x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
51,
|
||||
0.034701248
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x240x64_2_32x120x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
49,
|
||||
0.02691910399999999
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x128x64_3_32x64x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
100,
|
||||
0.024421632000000002
|
||||
]
|
||||
],
|
||||
[
|
||||
"gdn_decode_post_conv_mtp_kernel",
|
||||
[
|
||||
864,
|
||||
0.023706272000000014
|
||||
]
|
||||
],
|
||||
[
|
||||
"expandInputRowsKernel",
|
||||
[
|
||||
1248,
|
||||
0.022278243999999982
|
||||
]
|
||||
],
|
||||
[
|
||||
"doActivationKernel",
|
||||
[
|
||||
1248,
|
||||
0.02122110900000002
|
||||
]
|
||||
],
|
||||
[
|
||||
"elementwise_kernel",
|
||||
[
|
||||
2004,
|
||||
0.020444831999999843
|
||||
]
|
||||
],
|
||||
[
|
||||
"vectorized_elementwise_kernel",
|
||||
[
|
||||
5307,
|
||||
0.018513366999999715
|
||||
]
|
||||
],
|
||||
[
|
||||
"_causal_conv1d_fwd_kernel",
|
||||
[
|
||||
72,
|
||||
0.017434495999999997
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x224x64_2_64x56x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
50,
|
||||
0.016634912
|
||||
]
|
||||
],
|
||||
[
|
||||
"_fused_post_conv_kernel",
|
||||
[
|
||||
72,
|
||||
0.015161888000000002
|
||||
]
|
||||
]
|
||||
],
|
||||
"notes": "GPU busy is interval union, not utilization counter. CPU/API/NVTX times overlap GPU work and each other; do not sum them as exclusive costs."
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"context_limit": 131072,
|
||||
"batch_token_limit": 2048,
|
||||
"observed_attention_block_size": 1600,
|
||||
"graph_capture_sizes": [
|
||||
1,
|
||||
2,
|
||||
4,
|
||||
8,
|
||||
12
|
||||
],
|
||||
"nsys_version": "NVIDIA Nsight Systems version 2025.3.2.474-253236389321v0",
|
||||
"trace_cases": {
|
||||
"1": "short",
|
||||
"2": "fresh-32768",
|
||||
"3": "reused-32768"
|
||||
},
|
||||
"files": {
|
||||
"graph/trace.2.nsys-rep": {
|
||||
"bytes": 7688024,
|
||||
"sha256": "443a6e052586d3f4bfe7c6f1492500e495374e8e52e6de5766a437e53d73d146"
|
||||
},
|
||||
"graph/trace.3.nsys-rep": {
|
||||
"bytes": 4115977,
|
||||
"sha256": "15e7602b8dd8d92e9f295fcc1a24df4a6245a36339555c83f04dda32c7d5dc86"
|
||||
},
|
||||
"graph/trace.1.nsys-rep": {
|
||||
"bytes": 9244219,
|
||||
"sha256": "4eceb2f5241678ac387064369ccfb485d741dec140d0a1ace46866da0d70e687"
|
||||
},
|
||||
"eager/trace.2.nsys-rep": {
|
||||
"bytes": 8540396,
|
||||
"sha256": "543a9c1a5698de7e2ce7fe9e25dab546d64236e0f6f2014648223f664ff967a1"
|
||||
},
|
||||
"eager/trace.3.nsys-rep": {
|
||||
"bytes": 4635497,
|
||||
"sha256": "af251e99de398302611f691f39073e1a48ae766bd9473931b2437c46e2e42227"
|
||||
},
|
||||
"eager/trace.1.nsys-rep": {
|
||||
"bytes": 15842503,
|
||||
"sha256": "0390fdffffaeee0b78809d64cbf8696e8301d5730de36c2d67da4e1aa18bc5d0"
|
||||
},
|
||||
"vllm_ple_mmap.original.py": {
|
||||
"bytes": 40227,
|
||||
"sha256": "02adb76b789f9fccf472e922e3be7b4b2a95962ae7d7c3eb99bfeac6052dccad"
|
||||
},
|
||||
"cudagraph_utils.original.py": {
|
||||
"bytes": 46652,
|
||||
"sha256": "e123e808a8b6a616f69cc97f8b0d25d77f0eb35d8ebc91c4cdbc505f649927c0"
|
||||
},
|
||||
"vllm_ple_mmap.profile.py": {
|
||||
"bytes": 40773,
|
||||
"sha256": "2dff4fe71a5f2f5a8579af09eebf798d49da0d1554a56819ab475b86aebe17e4"
|
||||
},
|
||||
"cudagraph_utils.profile.py": {
|
||||
"bytes": 47119,
|
||||
"sha256": "ef6dccbd27c6b109691a2ab2b9a74cc08c1d3055498ff91638f78af1fe5faf66"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
{"label": "graph", "time": 1789664773.0911076, "event": "REQUEST_START", "name": "warmup", "profile": false, "prompt_tokens_preflight": 51}
|
||||
{"label": "graph", "time": 1789664774.6022928, "event": "RESULT", "name": "warmup", "profile": false, "correct": true, "content": "\n\n323", "finish_reason": "stop", "usage": {"prompt_tokens": 51, "total_tokens": 83, "completion_tokens": 32, "completion_tokens_details": {"reasoning_tokens": 26}}, "max_tokens": 2048, "prompt_sha256": "4181e1a44a0bed10d00c74e822377bd75a6c6c9a1ea458f5170b8193e9fce959", "output_sha256": "4477e82dcd9f3f8da1dd56be073d72e4ba20121d6ce6fbc97ee54258fb4b2959", "ttft_s": 0.6364480190022732, "first_content_s": 1.4175457309975172, "elapsed_s": 1.5047177029991872, "decode_tps_approx": 39.6877362766987, "metrics_delta": {"vllm:prefix_cache_queries_total": 51.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664774.614013, "event": "REQUEST_START", "name": "short-0", "profile": false, "prompt_tokens_preflight": 65}
|
||||
{"label": "graph", "time": 1789664781.980255, "event": "RESULT", "name": "short-0", "profile": false, "correct": true, "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "finish_reason": "stop", "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "max_tokens": 512, "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "ttft_s": 0.32573997599683935, "first_content_s": 3.8834300599992275, "elapsed_s": 7.360314821999054, "decode_tps_approx": 30.279974628892248, "metrics_delta": {"vllm:prefix_cache_queries_total": 65.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664781.9975877, "event": "REQUEST_START", "name": "prefill-8192-0", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "graph", "time": 1789664788.1946163, "event": "RESULT", "name": "prefill-8192-0", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8247, "completion_tokens": 74, "completion_tokens_details": {"reasoning_tokens": 66}}, "max_tokens": 2048, "prompt_sha256": "f59f163e1c7bb8a79d08df830d94d6d2e98fec457e9de75e3c09f6fca7ab0813", "output_sha256": "c0d6109d7f5d6c6f5486e269f63ea0338562a90ab0ceeede5f228b020e701eea", "ttft_s": 3.921214998998039, "first_content_s": 6.03858627199952, "elapsed_s": 6.191493427999376, "decode_tps_approx": 33.338899955373805, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664788.2102215, "event": "REQUEST_START", "name": "reuse-8192-0", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "graph", "time": 1789664791.047786, "event": "RESULT", "name": "reuse-8192-0", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8238, "completion_tokens": 65, "completion_tokens_details": {"reasoning_tokens": 57}}, "max_tokens": 2048, "prompt_sha256": "f59f163e1c7bb8a79d08df830d94d6d2e98fec457e9de75e3c09f6fca7ab0813", "output_sha256": "50530d055152bcaa11d1779a66b456a8ad5175a5343fb48f268c163f397d0089", "ttft_s": 0.8737782960015466, "first_content_s": 2.671151395996276, "elapsed_s": 2.8319971849996364, "decode_tps_approx": 34.055335354489465, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 6400.0}}
|
||||
{"label": "graph", "time": 1789664791.096218, "event": "REQUEST_START", "name": "prefill-32768-0", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "graph", "time": 1789664808.7936926, "event": "RESULT", "name": "prefill-32768-0", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32815, "completion_tokens": 70, "completion_tokens_details": {"reasoning_tokens": 62}}, "max_tokens": 2048, "prompt_sha256": "605f7b492252f50eaf60e8d128dde3f2d2e1b50dd26f0b45db88dac00e36a5ca", "output_sha256": "7eec57291da7ee95ae6d9214f8ed940a405e64beb9f49a4693387362aac3e14c", "ttft_s": 15.511759164000978, "first_content_s": 17.52370634300314, "elapsed_s": 17.691939716001798, "decode_tps_approx": 32.87688689931954, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664808.8368387, "event": "REQUEST_START", "name": "reuse-32768-0", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "graph", "time": 1789664812.1356192, "event": "RESULT", "name": "reuse-32768-0", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32814, "completion_tokens": 69, "completion_tokens_details": {"reasoning_tokens": 61}}, "max_tokens": 2048, "prompt_sha256": "605f7b492252f50eaf60e8d128dde3f2d2e1b50dd26f0b45db88dac00e36a5ca", "output_sha256": "8b503330d6b70b8845bab80540601fc48c8f840d54d07f38d80250c8167de89f", "ttft_s": 1.325252010996337, "first_content_s": 3.1243166819986072, "elapsed_s": 3.293290553992847, "decode_tps_approx": 36.2184980346298, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 30400.0}}
|
||||
{"label": "graph", "time": 1789664812.1429493, "event": "REQUEST_START", "name": "short-1", "profile": false, "prompt_tokens_preflight": 65}
|
||||
{"label": "graph", "time": 1789664819.2091334, "event": "RESULT", "name": "short-1", "profile": false, "correct": true, "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "finish_reason": "stop", "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "max_tokens": 512, "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "ttft_s": 0.2600282569983392, "first_content_s": 3.7012813049950637, "elapsed_s": 7.060739258995454, "decode_tps_approx": 31.32107217692348, "metrics_delta": {"vllm:prefix_cache_queries_total": 65.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664819.2247505, "event": "REQUEST_START", "name": "prefill-8192-1", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "graph", "time": 1789664824.82708, "event": "RESULT", "name": "prefill-8192-1", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8236, "completion_tokens": 63, "completion_tokens_details": {"reasoning_tokens": 55}}, "max_tokens": 2048, "prompt_sha256": "6c13deb2d4ddee54db95d7ebc8a1b87d24ed8a4112c33644c18b6b5d61ad7a16", "output_sha256": "09d17d110b06ec13d532382c41ea29955b7edfa8dd4bd34163a08fa746e56610", "ttft_s": 3.7701075279983343, "first_content_s": 5.436177673000202, "elapsed_s": 5.596617848997994, "decode_tps_approx": 35.516432222041566, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664824.8427408, "event": "REQUEST_START", "name": "reuse-8192-1", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "graph", "time": 1789664827.8556995, "event": "RESULT", "name": "reuse-8192-1", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8247, "completion_tokens": 74, "completion_tokens_details": {"reasoning_tokens": 66}}, "max_tokens": 2048, "prompt_sha256": "6c13deb2d4ddee54db95d7ebc8a1b87d24ed8a4112c33644c18b6b5d61ad7a16", "output_sha256": "c0d6109d7f5d6c6f5486e269f63ea0338562a90ab0ceeede5f228b020e701eea", "ttft_s": 0.8690901810041396, "first_content_s": 2.8428866180038312, "elapsed_s": 3.0074559670028975, "decode_tps_approx": 35.510793077788634, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 6400.0}}
|
||||
{"label": "graph", "time": 1789664827.8988104, "event": "REQUEST_START", "name": "prefill-32768-1", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "graph", "time": 1789664845.1446679, "event": "RESULT", "name": "prefill-32768-1", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32814, "completion_tokens": 69, "completion_tokens_details": {"reasoning_tokens": 61}}, "max_tokens": 2048, "prompt_sha256": "0a5e4ba33ff264d180b8a56d89184c0fefad59a96fdd348d655bd54fe5b5ed45", "output_sha256": "8b503330d6b70b8845bab80540601fc48c8f840d54d07f38d80250c8167de89f", "ttft_s": 15.357147006005107, "first_content_s": 17.08089536800253, "elapsed_s": 17.239325528003974, "decode_tps_approx": 37.728914829613814, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664845.1907535, "event": "REQUEST_START", "name": "reuse-32768-1", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "graph", "time": 1789664848.6512141, "event": "RESULT", "name": "reuse-32768-1", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32815, "completion_tokens": 70, "completion_tokens_details": {"reasoning_tokens": 62}}, "max_tokens": 2048, "prompt_sha256": "0a5e4ba33ff264d180b8a56d89184c0fefad59a96fdd348d655bd54fe5b5ed45", "output_sha256": "7eec57291da7ee95ae6d9214f8ed940a405e64beb9f49a4693387362aac3e14c", "ttft_s": 1.3272808809997514, "first_content_s": 3.296925946000556, "elapsed_s": 3.4540864679947845, "decode_tps_approx": 33.71980234098872, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 30400.0}}
|
||||
{"label": "graph", "time": 1789664848.660781, "event": "REQUEST_START", "name": "short-2", "profile": false, "prompt_tokens_preflight": 65}
|
||||
{"label": "graph", "time": 1789664855.73135, "event": "RESULT", "name": "short-2", "profile": false, "correct": true, "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "finish_reason": "stop", "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "max_tokens": 512, "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "ttft_s": 0.2623967160034226, "first_content_s": 3.7147180649990332, "elapsed_s": 7.0642776130043785, "decode_tps_approx": 31.315723764990278, "metrics_delta": {"vllm:prefix_cache_queries_total": 65.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664855.7496016, "event": "REQUEST_START", "name": "prefill-8192-2", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "graph", "time": 1789664861.4250884, "event": "RESULT", "name": "prefill-8192-2", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8242, "completion_tokens": 69, "completion_tokens_details": {"reasoning_tokens": 61}}, "max_tokens": 2048, "prompt_sha256": "b49f6dbbf80f49ad860a60b0a7c58dfc2122e38bb8493ae155790f22305d8f7a", "output_sha256": "f72eba21a095aa0a43356b25cb120346b064d7f0a253a63cee7579d2c3422b82", "ttft_s": 3.768740819999948, "first_content_s": 5.512005377000605, "elapsed_s": 5.670499198000471, "decode_tps_approx": 37.32072215199369, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664861.4389272, "event": "REQUEST_START", "name": "reuse-8192-2", "profile": false, "prompt_tokens_preflight": 8173}
|
||||
{"label": "graph", "time": 1789664864.1561685, "event": "RESULT", "name": "reuse-8192-2", "profile": false, "correct": true, "content": "\n\n21573", "finish_reason": "stop", "usage": {"prompt_tokens": 8173, "total_tokens": 8236, "completion_tokens": 63, "completion_tokens_details": {"reasoning_tokens": 55}}, "max_tokens": 2048, "prompt_sha256": "b49f6dbbf80f49ad860a60b0a7c58dfc2122e38bb8493ae155790f22305d8f7a", "output_sha256": "09d17d110b06ec13d532382c41ea29955b7edfa8dd4bd34163a08fa746e56610", "ttft_s": 0.8816422219970264, "first_content_s": 2.543782171997009, "elapsed_s": 2.711807350999152, "decode_tps_approx": 35.522341967132114, "metrics_delta": {"vllm:prefix_cache_queries_total": 8173.0, "vllm:prefix_cache_hits_total": 6400.0}}
|
||||
{"label": "graph", "time": 1789664864.196416, "event": "REQUEST_START", "name": "prefill-32768-2", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "graph", "time": 1789664881.472875, "event": "RESULT", "name": "prefill-32768-2", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32814, "completion_tokens": 69, "completion_tokens_details": {"reasoning_tokens": 61}}, "max_tokens": 2048, "prompt_sha256": "d8796cea8608150048ce4f253e451ba96da5416ccfea4a6560dc2f6bb2412004", "output_sha256": "8b503330d6b70b8845bab80540601fc48c8f840d54d07f38d80250c8167de89f", "ttft_s": 15.375977513001999, "first_content_s": 17.108021717001975, "elapsed_s": 17.268514374001825, "decode_tps_approx": 37.54862656190945, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664881.5214229, "event": "REQUEST_START", "name": "reuse-32768-2", "profile": false, "prompt_tokens_preflight": 32745}
|
||||
{"label": "graph", "time": 1789664884.7351754, "event": "RESULT", "name": "reuse-32768-2", "profile": false, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32745, "total_tokens": 32814, "completion_tokens": 69, "completion_tokens_details": {"reasoning_tokens": 61}}, "max_tokens": 2048, "prompt_sha256": "d8796cea8608150048ce4f253e451ba96da5416ccfea4a6560dc2f6bb2412004", "output_sha256": "8b503330d6b70b8845bab80540601fc48c8f840d54d07f38d80250c8167de89f", "ttft_s": 1.3270910260034725, "first_content_s": 3.045836498000426, "elapsed_s": 3.2060768180017476, "decode_tps_approx": 37.82214005937634, "metrics_delta": {"vllm:prefix_cache_queries_total": 32745.0, "vllm:prefix_cache_hits_total": 30400.0}}
|
||||
{"label": "graph", "time": 1789664884.7446704, "event": "REQUEST_START", "name": "trace-short", "profile": true, "prompt_tokens_preflight": 65}
|
||||
{"label": "graph", "time": 1789664894.3422408, "event": "RESULT", "name": "trace-short", "profile": true, "correct": true, "content": "\n\n数据库索引类似于书籍的目录，它通过B+树等数据结构将数据按特定字段有序组织，使查询时能快速定位目标记录，避免全表扫描，从而大幅提升读取效率。然而，索引对写入操作有负面影响：每次插入、更新或删除数据时，数据库都需要同步维护索引结构，这会增加额外的I/O和计算开销，导致写入性能下降。因此，索引是\"以空间换时间\"的典型策略，需根据实际读写比例合理设计。", "finish_reason": "stop", "usage": {"prompt_tokens": 65, "total_tokens": 279, "completion_tokens": 214, "completion_tokens_details": {"reasoning_tokens": 111}}, "max_tokens": 512, "prompt_sha256": "40e7d1df3e96ad760d661cfaff312f5d732cb2edd5eab555b4b5c43cea4512ca", "output_sha256": "126232af3bc1889efc9c0dad67f2d47fc60d3c4ff56e699a83f1576c3763c274", "ttft_s": 0.23192091599776177, "first_content_s": 3.732794447001652, "elapsed_s": 9.19204898099997, "decode_tps_approx": 30.944518883030874, "metrics_delta": {"vllm:prefix_cache_queries_total": 65.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664894.3764791, "event": "REQUEST_START", "name": "trace-prefill-32768", "profile": true, "prompt_tokens_preflight": 32742}
|
||||
{"label": "graph", "time": 1789664914.0692568, "event": "RESULT", "name": "trace-prefill-32768", "profile": true, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32742, "total_tokens": 32812, "completion_tokens": 70, "completion_tokens_details": {"reasoning_tokens": 62}}, "max_tokens": 2048, "prompt_sha256": "464e9da0976ca56ddece006a237605bca8b60d3dd0aab538abbc0aaa1696815c", "output_sha256": "7eec57291da7ee95ae6d9214f8ed940a405e64beb9f49a4693387362aac3e14c", "ttft_s": 15.522943222997128, "first_content_s": 17.421024231000047, "elapsed_s": 19.56250263000402, "decode_tps_approx": 34.818013845987785, "metrics_delta": {"vllm:prefix_cache_queries_total": 32742.0, "vllm:prefix_cache_hits_total": 0.0}}
|
||||
{"label": "graph", "time": 1789664914.1044817, "event": "REQUEST_START", "name": "trace-reuse-32768", "profile": true, "prompt_tokens_preflight": 32742}
|
||||
{"label": "graph", "time": 1789664920.2165241, "event": "RESULT", "name": "trace-reuse-32768", "profile": true, "correct": true, "content": "\n\n90812", "finish_reason": "stop", "usage": {"prompt_tokens": 32742, "total_tokens": 32816, "completion_tokens": 74, "completion_tokens_details": {"reasoning_tokens": 66}}, "max_tokens": 2048, "prompt_sha256": "464e9da0976ca56ddece006a237605bca8b60d3dd0aab538abbc0aaa1696815c", "output_sha256": "997501e1996b1c37af707622b1c4caf8dba93113e868f28210bd47ba700b5804", "ttft_s": 1.3475809320007102, "first_content_s": 3.358952419999696, "elapsed_s": 5.983986291001202, "decode_tps_approx": 34.79999075012843, "metrics_delta": {"vllm:prefix_cache_queries_total": 32742.0, "vllm:prefix_cache_hits_total": 30400.0}}
|
||||
{"label": "graph", "time": 1789664920.2165873, "event": "PROFILE_SUITE_PASS"}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,852 @@
|
||||
{
|
||||
"file": "trace.2.sqlite",
|
||||
"gpu_activity_span_s": 17.613957408,
|
||||
"gpu_busy_union_s": 17.173015684,
|
||||
"gpu_idle_within_span_s": 0.44094172400000176,
|
||||
"kernel_count": 111055,
|
||||
"graph_node_kernel_count": 49140,
|
||||
"api": {
|
||||
"cudaStreamSynchronize_v3020": [
|
||||
96,
|
||||
7.291123311999998
|
||||
],
|
||||
"cuLaunchKernelEx": [
|
||||
22540,
|
||||
2.9715556959999576
|
||||
],
|
||||
"cudaLaunchKernelExC_v11060": [
|
||||
11197,
|
||||
2.7721689919999997
|
||||
],
|
||||
"cudaLaunchKernel_v7000": [
|
||||
24666,
|
||||
2.074785632000026
|
||||
],
|
||||
"cuLaunchKernel": [
|
||||
3512,
|
||||
0.1402584800000001
|
||||
],
|
||||
"cudaGraphLaunch_v10000": [
|
||||
1431,
|
||||
0.13174172799999972
|
||||
],
|
||||
"cudaEventSynchronize_v3020": [
|
||||
48,
|
||||
0.042627568
|
||||
],
|
||||
"cudaMemcpyAsync_v3020": [
|
||||
2398,
|
||||
0.02786942400000001
|
||||
],
|
||||
"cudaMemsetAsync_v3020": [
|
||||
768,
|
||||
0.005161456000000002
|
||||
],
|
||||
"cuKernelGetName": [
|
||||
25422,
|
||||
0.002872591999999989
|
||||
],
|
||||
"cudaEventQuery_v3020": [
|
||||
1557,
|
||||
0.0023473760000000004
|
||||
],
|
||||
"cuTensorMapEncodeTiled": [
|
||||
12786,
|
||||
0.002259360000000043
|
||||
],
|
||||
"cudaEventRecordWithFlags_v11010": [
|
||||
1003,
|
||||
0.001927552000000001
|
||||
],
|
||||
"cudaGetDriverEntryPointByVersion_v12050": [
|
||||
12096,
|
||||
0.0017841920000000863
|
||||
],
|
||||
"cuKernelGetFunction": [
|
||||
3512,
|
||||
0.0013231039999999851
|
||||
],
|
||||
"cudaStreamIsCapturing_v10000": [
|
||||
2144,
|
||||
0.0006217759999999976
|
||||
],
|
||||
"cudaStreamWaitEvent_v3020": [
|
||||
144,
|
||||
0.00041838400000000016
|
||||
],
|
||||
"cudaEventCreateWithFlags_v3020": [
|
||||
96,
|
||||
0.00024513599999999997
|
||||
],
|
||||
"cudaEventDestroy_v3020": [
|
||||
96,
|
||||
0.00022665600000000007
|
||||
]
|
||||
},
|
||||
"nvtx": {
|
||||
"PLE_lookup_tokens=1600": {
|
||||
"count": 20,
|
||||
"sum_s": 5.607371088,
|
||||
"union_s": 5.607371088,
|
||||
"gpu_overlap_s": 5.336825187
|
||||
},
|
||||
"PLE_wait_for_GPU": {
|
||||
"count": 48,
|
||||
"sum_s": 7.291963728,
|
||||
"union_s": 7.291963728,
|
||||
"gpu_overlap_s": 7.233423306
|
||||
},
|
||||
"PLE_CPU_dedup": {
|
||||
"count": 48,
|
||||
"sum_s": 0.020901856,
|
||||
"union_s": 0.020901856,
|
||||
"gpu_overlap_s": 0.0
|
||||
},
|
||||
"PLE_CPU_gather": {
|
||||
"count": 48,
|
||||
"sum_s": 0.292111936,
|
||||
"union_s": 0.292111936,
|
||||
"gpu_overlap_s": 0.0
|
||||
},
|
||||
"PLE_lookup_tokens=742": {
|
||||
"count": 1,
|
||||
"sum_s": 0.29688864,
|
||||
"union_s": 0.29688864,
|
||||
"gpu_overlap_s": 0.282970556
|
||||
},
|
||||
"PLE_lookup_tokens=4": {
|
||||
"count": 27,
|
||||
"sum_s": 1.71834584,
|
||||
"union_s": 1.71834584,
|
||||
"gpu_overlap_s": 1.61634065
|
||||
}
|
||||
},
|
||||
"dispatch": {
|
||||
"CG_DISPATCH:actual=1600:mode=NONE:padded=1600": 40,
|
||||
"CG_DISPATCH:actual=1:mode=NONE:padded=1": 48,
|
||||
"CG_DISPATCH:actual=742:mode=NONE:padded=742": 2,
|
||||
"CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4": 27,
|
||||
"CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4": 27
|
||||
},
|
||||
"dispatch_timeline": [
|
||||
{
|
||||
"start_ns": 46743952,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 416451360,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 443082544,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 469310784,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 1124006496,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 1154683696,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1179774000,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 1845801456,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 1877195728,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1902001888,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 2578899920,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 2609916288,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2635831856,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 3318901952,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 3349740016,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3376088736,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 4054732640,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 4085130432,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 4110746784,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 4797770512,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 4829017936,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 4854209488,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 5551719760,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 5581307632,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 5607134000,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 6295334464,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 6325098224,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 6349994272,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 7042285248,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 7072015024,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 7097530960,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 7785043856,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 7816741056,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 7842952592,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 8537987536,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 8569536560,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 8595452192,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 9285771248,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 9317233072,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 9342219952,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 10060629616,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 10091559456,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 10116812048,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 10807242976,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 10837982160,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 10864521920,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 11562733424,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 11593737408,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 11619654832,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 12323127600,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 12354216560,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 12380982848,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 13081498400,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 13113506512,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 13138598000,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 13833930640,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 13865480864,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 13890978272,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 14599979088,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 14632423408,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 14657285504,
|
||||
"name": "CG_DISPATCH:actual=742:mode=NONE:padded=742"
|
||||
},
|
||||
{
|
||||
"start_ns": 15244687408,
|
||||
"name": "CG_DISPATCH:actual=742:mode=NONE:padded=742"
|
||||
},
|
||||
{
|
||||
"start_ns": 15261168608,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15272860704,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15510425328,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15511343584,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15516520016,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15591171712,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15592218224,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15597515056,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15670668032,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15671484256,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15675816752,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15748911968,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15749703280,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15753801520,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15829628848,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15830463168,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15834544656,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15911548208,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15912496096,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15917334144,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15989715552,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 15990467984,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 15994717936,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16067760256,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16068616608,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16072918640,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16151936192,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16153013376,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16158295088,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16231124064,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16232113808,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16237373136,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16308737952,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16309646016,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16314212128,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16388651696,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16389407904,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16393690448,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16470779408,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16471838880,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16476836288,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16549986896,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16550865424,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16555256672,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16632067984,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16632894512,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16637253408,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16709708944,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16710541136,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16714847184,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16789609392,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16790634928,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16795643776,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16869435120,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16870399200,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16875445984,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16952330656,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 16953311568,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 16958431376,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17031362304,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17032114688,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17036354400,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17111019568,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17112056224,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17117105808,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17188540768,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17189324992,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17193788208,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17269626256,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17270693728,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17276082128,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17347956512,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17348732480,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17352868096,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17427032704,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17427739648,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17431571536,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17508960384,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17509972320,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 17514870544,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17592660304,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 17593603200,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
}
|
||||
],
|
||||
"top_kernels": [
|
||||
[
|
||||
"device_kernel",
|
||||
[
|
||||
4608,
|
||||
5.806803458000012
|
||||
]
|
||||
],
|
||||
[
|
||||
"Kernel2",
|
||||
[
|
||||
16931,
|
||||
1.7378030570000023
|
||||
]
|
||||
],
|
||||
[
|
||||
"_qsa_sparse_paged_gqa_splitk_kernel",
|
||||
[
|
||||
672,
|
||||
1.4925525440000014
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x208x64_2_32x104x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
1000,
|
||||
1.3766302719999985
|
||||
]
|
||||
],
|
||||
[
|
||||
"_hc_combine_norm_kernel",
|
||||
[
|
||||
4848,
|
||||
0.9208744449999874
|
||||
]
|
||||
],
|
||||
[
|
||||
"kernel",
|
||||
[
|
||||
2853,
|
||||
0.8909366829999992
|
||||
]
|
||||
],
|
||||
[
|
||||
"_hc_gate_mix_kernel",
|
||||
[
|
||||
4944,
|
||||
0.6871558350000507
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x240x64_2_32x120x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
980,
|
||||
0.5473217280000005
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x128x64_3_32x64x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
2000,
|
||||
0.4623586880000001
|
||||
]
|
||||
],
|
||||
[
|
||||
"finalizeMoeRoutingKernel",
|
||||
[
|
||||
2304,
|
||||
0.4111630399999976
|
||||
]
|
||||
],
|
||||
[
|
||||
"_causal_conv1d_fwd_kernel",
|
||||
[
|
||||
756,
|
||||
0.24460185600000015
|
||||
]
|
||||
],
|
||||
[
|
||||
"doActivationKernel",
|
||||
[
|
||||
2304,
|
||||
0.23128571500000175
|
||||
]
|
||||
],
|
||||
[
|
||||
"_fused_post_conv_kernel",
|
||||
[
|
||||
756,
|
||||
0.23053734400000014
|
||||
]
|
||||
],
|
||||
[
|
||||
"elementwise_kernel",
|
||||
[
|
||||
6192,
|
||||
0.22792899099999814
|
||||
]
|
||||
],
|
||||
[
|
||||
"kernel_cutlass_kernel_flashinfergdn_kernelsdelta_rule_dsldelta_rule_sm120_FullyFusedDeltaRuleSm120_object_at__tensorptrf32gmemalign16o1_tensorptrf32gmemalign16o1_CopyAtom_ThrID10_TVLayout_0",
|
||||
[
|
||||
756,
|
||||
0.19290607999999995
|
||||
]
|
||||
],
|
||||
[
|
||||
"vectorized_elementwise_kernel",
|
||||
[
|
||||
11370,
|
||||
0.18636042799999758
|
||||
]
|
||||
],
|
||||
[
|
||||
"layer_norm_fwd_kernel",
|
||||
[
|
||||
756,
|
||||
0.18163276799999997
|
||||
]
|
||||
],
|
||||
[
|
||||
"sm120_fp8_fp4_gemm_1d1d_impl",
|
||||
[
|
||||
192,
|
||||
0.17354873599999998
|
||||
]
|
||||
],
|
||||
[
|
||||
"expandInputRowsKernel",
|
||||
[
|
||||
2304,
|
||||
0.1723688820000004
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x192x64_2_32x96x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
980,
|
||||
0.13571574399999997
|
||||
]
|
||||
]
|
||||
],
|
||||
"notes": "GPU busy is interval union, not utilization counter. CPU/API/NVTX times overlap GPU work and each other; do not sum them as exclusive costs."
|
||||
}
|
||||
@@ -0,0 +1,636 @@
|
||||
{
|
||||
"file": "trace.3.sqlite",
|
||||
"gpu_activity_span_s": 3.549492992,
|
||||
"gpu_busy_union_s": 3.382135698,
|
||||
"gpu_idle_within_span_s": 0.16735729399999988,
|
||||
"kernel_count": 66056,
|
||||
"graph_node_kernel_count": 50960,
|
||||
"api": {
|
||||
"cudaStreamSynchronize_v3020": [
|
||||
60,
|
||||
2.063916384
|
||||
],
|
||||
"cudaLaunchKernelExC_v11060": [
|
||||
1284,
|
||||
0.28009695999999934
|
||||
],
|
||||
"cuLaunchKernelEx": [
|
||||
5990,
|
||||
0.2070799999999992
|
||||
],
|
||||
"cudaLaunchKernel_v7000": [
|
||||
7285,
|
||||
0.1831771039999994
|
||||
],
|
||||
"cudaGraphLaunch_v10000": [
|
||||
1484,
|
||||
0.119649248
|
||||
],
|
||||
"cudaEventSynchronize_v3020": [
|
||||
30,
|
||||
0.040196352
|
||||
],
|
||||
"cuLaunchKernel": [
|
||||
537,
|
||||
0.01219283200000001
|
||||
],
|
||||
"cudaMemcpyAsync_v3020": [
|
||||
1043,
|
||||
0.010008064000000006
|
||||
],
|
||||
"cudaMemsetAsync_v3020": [
|
||||
480,
|
||||
0.0025478400000000017
|
||||
],
|
||||
"cudaEventQuery_v3020": [
|
||||
921,
|
||||
0.0011545439999999998
|
||||
],
|
||||
"cuKernelGetName": [
|
||||
7357,
|
||||
0.00088769599999998
|
||||
],
|
||||
"cudaEventRecordWithFlags_v11010": [
|
||||
605,
|
||||
0.0008087840000000007
|
||||
],
|
||||
"cudaStreamIsCapturing_v10000": [
|
||||
1274,
|
||||
0.00032420800000000107
|
||||
],
|
||||
"cuTensorMapEncodeTiled": [
|
||||
1472,
|
||||
0.0003082879999999999
|
||||
],
|
||||
"cuKernelGetFunction": [
|
||||
537,
|
||||
0.0002906560000000001
|
||||
],
|
||||
"cudaGetDriverEntryPointByVersion_v12050": [
|
||||
1152,
|
||||
0.00016971199999999957
|
||||
],
|
||||
"cudaStreamWaitEvent_v3020": [
|
||||
90,
|
||||
0.000155392
|
||||
],
|
||||
"cudaEventCreateWithFlags_v3020": [
|
||||
60,
|
||||
0.00014443199999999998
|
||||
],
|
||||
"cudaEventDestroy_v3020": [
|
||||
60,
|
||||
8.001600000000003e-05
|
||||
]
|
||||
},
|
||||
"nvtx": {
|
||||
"PLE_lookup_tokens=1600": {
|
||||
"count": 1,
|
||||
"sum_s": 0.019309872,
|
||||
"union_s": 0.019309872,
|
||||
"gpu_overlap_s": 0.010934608
|
||||
},
|
||||
"PLE_wait_for_GPU": {
|
||||
"count": 30,
|
||||
"sum_s": 2.064303616,
|
||||
"union_s": 2.064303616,
|
||||
"gpu_overlap_s": 2.035614378
|
||||
},
|
||||
"PLE_CPU_dedup": {
|
||||
"count": 30,
|
||||
"sum_s": 0.00226768,
|
||||
"union_s": 0.00226768,
|
||||
"gpu_overlap_s": 0.0
|
||||
},
|
||||
"PLE_CPU_gather": {
|
||||
"count": 30,
|
||||
"sum_s": 0.109426208,
|
||||
"union_s": 0.109426208,
|
||||
"gpu_overlap_s": 0.0
|
||||
},
|
||||
"PLE_lookup_tokens=742": {
|
||||
"count": 1,
|
||||
"sum_s": 0.293855216,
|
||||
"union_s": 0.293855216,
|
||||
"gpu_overlap_s": 0.27970459
|
||||
},
|
||||
"PLE_lookup_tokens=4": {
|
||||
"count": 28,
|
||||
"sum_s": 1.874469744,
|
||||
"union_s": 1.874469744,
|
||||
"gpu_overlap_s": 1.748828475
|
||||
}
|
||||
},
|
||||
"dispatch": {
|
||||
"CG_DISPATCH:actual=1600:mode=NONE:padded=1600": 2,
|
||||
"CG_DISPATCH:actual=1:mode=NONE:padded=1": 30,
|
||||
"CG_DISPATCH:actual=742:mode=NONE:padded=742": 2,
|
||||
"CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4": 28,
|
||||
"CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4": 28
|
||||
},
|
||||
"dispatch_timeline": [
|
||||
{
|
||||
"start_ns": 47667312,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 442471520,
|
||||
"name": "CG_DISPATCH:actual=1600:mode=NONE:padded=1600"
|
||||
},
|
||||
{
|
||||
"start_ns": 473807600,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 498595616,
|
||||
"name": "CG_DISPATCH:actual=742:mode=NONE:padded=742"
|
||||
},
|
||||
{
|
||||
"start_ns": 1085215856,
|
||||
"name": "CG_DISPATCH:actual=742:mode=NONE:padded=742"
|
||||
},
|
||||
{
|
||||
"start_ns": 1101603472,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1113435424,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1335575776,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1336548688,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1340993280,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1412886784,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1413839472,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1418416416,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1496932144,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1497849376,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1502437168,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1574235248,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1574893632,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1578152608,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1659975184,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1660978304,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1666040464,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1739695136,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1740582992,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1745368080,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1819362544,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1820259120,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1825002016,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1898444080,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1899320480,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1903978080,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1973992688,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 1974642976,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 1977460352,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2058854000,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2059781888,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2064550128,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2140294400,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2141166416,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2145782752,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2215803760,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2216437328,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2219274592,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2295308880,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2295905872,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2298529488,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2373456000,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2374001712,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2376400016,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2451495056,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2452018624,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2454446720,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2534005728,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2534498160,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2536805632,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2617391424,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2618055024,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2620944928,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2698279104,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2699135648,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2704056256,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2781484592,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2782408128,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2787209600,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2872938352,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2873856272,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2878412464,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2955704400,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 2956657904,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 2961494272,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3046465232,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3047422016,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3052108288,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3123689648,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3124649664,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3129308048,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3199193712,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3199819264,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3202578560,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3285661472,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3286559472,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3291397856,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3363509456,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3364085808,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3366681840,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3447715488,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3448689632,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
},
|
||||
{
|
||||
"start_ns": 3453541344,
|
||||
"name": "CG_DISPATCH:actual=3:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3529683168,
|
||||
"name": "CG_DISPATCH:actual=4:mode=PIECEWISE:padded=4"
|
||||
},
|
||||
{
|
||||
"start_ns": 3530662560,
|
||||
"name": "CG_DISPATCH:actual=1:mode=NONE:padded=1"
|
||||
}
|
||||
],
|
||||
"top_kernels": [
|
||||
[
|
||||
"Kernel2",
|
||||
[
|
||||
14453,
|
||||
1.2634071650000012
|
||||
]
|
||||
],
|
||||
[
|
||||
"device_kernel",
|
||||
[
|
||||
2880,
|
||||
0.9274175719999982
|
||||
]
|
||||
],
|
||||
[
|
||||
"kernel",
|
||||
[
|
||||
1772,
|
||||
0.4575362240000001
|
||||
]
|
||||
],
|
||||
[
|
||||
"_qsa_sparse_paged_gqa_splitk_kernel",
|
||||
[
|
||||
420,
|
||||
0.13202313600000007
|
||||
]
|
||||
],
|
||||
[
|
||||
"_hc_combine_norm_kernel",
|
||||
[
|
||||
3030,
|
||||
0.07136511900000105
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x208x64_2_32x104x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
50,
|
||||
0.06789817599999999
|
||||
]
|
||||
],
|
||||
[
|
||||
"sm120_fp8_fp4_gemm_1d1d_impl",
|
||||
[
|
||||
120,
|
||||
0.05055315199999998
|
||||
]
|
||||
],
|
||||
[
|
||||
"_hc_gate_mix_kernel",
|
||||
[
|
||||
3090,
|
||||
0.049511773999999786
|
||||
]
|
||||
],
|
||||
[
|
||||
"finalizeMoeRoutingKernel",
|
||||
[
|
||||
1440,
|
||||
0.04024281600000026
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_192x160x64_2_48x80x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
51,
|
||||
0.035206783999999984
|
||||
]
|
||||
],
|
||||
[
|
||||
"gdn_decode_post_conv_mtp_kernel",
|
||||
[
|
||||
1008,
|
||||
0.027670688000000006
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x240x64_2_32x120x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
49,
|
||||
0.027358240000000006
|
||||
]
|
||||
],
|
||||
[
|
||||
"elementwise_kernel",
|
||||
[
|
||||
2268,
|
||||
0.02362060799999982
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x128x64_3_32x64x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
100,
|
||||
0.023191840000000002
|
||||
]
|
||||
],
|
||||
[
|
||||
"vectorized_elementwise_kernel",
|
||||
[
|
||||
6095,
|
||||
0.02246828099999996
|
||||
]
|
||||
],
|
||||
[
|
||||
"doActivationKernel",
|
||||
[
|
||||
1440,
|
||||
0.02160813899999983
|
||||
]
|
||||
],
|
||||
[
|
||||
"expandInputRowsKernel",
|
||||
[
|
||||
1440,
|
||||
0.020183961000000014
|
||||
]
|
||||
],
|
||||
[
|
||||
"_causal_conv1d_fwd_kernel",
|
||||
[
|
||||
72,
|
||||
0.017587615999999997
|
||||
]
|
||||
],
|
||||
[
|
||||
"nvjet_sm121_tst_mma_128x224x64_2_64x56x64_tmaAB_bz_TNNN",
|
||||
[
|
||||
50,
|
||||
0.016199904
|
||||
]
|
||||
],
|
||||
[
|
||||
"_fused_post_conv_kernel",
|
||||
[
|
||||
72,
|
||||
0.014694784
|
||||
]
|
||||
]
|
||||
],
|
||||
"notes": "GPU busy is interval union, not utilization counter. CPU/API/NVTX times overlap GPU work and each other; do not sum them as exclusive costs."
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{"models": {"object": "list", "data": [{"id": "qwen3.8-flash-next", "object": "model", "created": 1789665662, "owned_by": "vllm", "root": "/root/.cache/huggingface/hub/models--nvidia--Qwen3.8-Flash-Next-NVFP4/snapshots/fc694b54fb0174e0913e6adf86691ef85a4ead47", "parent": null, "max_model_len": 262144, "permission": [{"id": "modelperm-b7d422ef1689806e", "object": "model_permission", "created": 1789665662, "allow_create_engine": false, "allow_sampling": true, "allow_logprobs": true, "allow_search_indices": false, "allow_view": true, "allow_fine_tuning": false, "organization": "*", "group": null, "is_blocking": false}]}]}}
|
||||
{"test": "arithmetic", "seconds": 2.43, "content": "\n\n323", "finish_reason": "stop", "usage": {"prompt_tokens": 53, "total_tokens": 109, "completion_tokens": 56, "prompt_tokens_details": null, "completion_tokens_details": {"reasoning_tokens": 50}}}
|
||||
{"test": "chinese", "seconds": 5.48, "content": "\n\n太阳光包含各种颜色的光（红、橙、黄、绿、蓝、紫等）。当阳光进入大气层时，会与空气中的氮、氧等小分子发生散射。\n\n短波长的光（蓝、紫）比长波长的光（红、橙）更容易被散射，这种现象称为**瑞利散射**。蓝光被散射到四面八方，充满整个天空，所以我们抬头看到的天空就是蓝色的。\n\n（紫光其实散射更强，但人眼对紫光不敏感，且部分紫光被高层大气吸收，所以天空呈现蓝色而非紫色。）", "finish_reason": "stop", "usage": {"prompt_tokens": 49, "total_tokens": 199, "completion_tokens": 150, "prompt_tokens_details": null, "completion_tokens_details": {"reasoning_tokens": 31}}}
|
||||
{"test": "tool_call", "seconds": 2.0665973159993882, "message": {"role": "assistant", "content": null, "refusal": null, "annotations": null, "audio": null, "function_call": null, "tool_calls": [{"id": "chatcmpl-tool-9e731f967ef9d3d6", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"上海\"}"}}], "reasoning": "用户要求我调用 get_weather 工具查询上海的天气，并且明确说不要自行编造。我需要直接调用这个工具。\n"}}
|
||||
TOOL_TEST_PASS
|
||||
SMOKE_TEST_PASS
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"time": "2026-09-18T01:21:40.503543+08:00",
|
||||
"image": "local/qwen38-flash-spark:prefix-eager-0bfc7a15",
|
||||
"health": "healthy",
|
||||
"restart_policy": "unless-stopped",
|
||||
"compose_unchanged": true,
|
||||
"profile_mounts": 0,
|
||||
"profiler_arg_present": false,
|
||||
"host_oom_kills": 25,
|
||||
"available_GiB": 18.851730346679688,
|
||||
"system_failed_units": 0,
|
||||
"user_failed_units": 0,
|
||||
"smoke_pass": true
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user