cyy/go1_pro_deploy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

adapt gym 5history

Browse Source
This commit is contained in:
cyy_mac
2026-07-28 16:36:33 +08:00
parent 6c4dd6ed14
commit 071c576bfe
14 changed files with 1782 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files

103
deploy_45dim_rl_gym/bpu_quantization/README.md
View File

@@ -1,8 +1,8 @@
# RDK X5 BPU 量化流程
这个目录用于把 `../policy_robotlab_15000.onnx` 转成 RDK X5 可运行的
Horizon runtime `.bin`。量化在 Mac 上用 CPU Docker 完成，板端只做离线
`hrt_model_exec` 验证，暂时不要直接接入实机控制。
这个目录用于把 Gym/RobotLab 的 ONNX 策略转成 RDK X5 可运行的 Horizon
runtime `.bin`。量化在 Mac 上用 CPU Docker 完成，板端可以做离线测速和实机
部署测试。
参考资料：
@@ -10,14 +10,103 @@ Horizon runtime `.bin`。量化在 Mac 上用 CPU Docker 完成，板端只做
- D-Robotics 论坛 WTW/Go2/X5 流程：`https://forum.d-robotics.cc/t/topic/28338`
官方工具链对 ONNX 的关键限制是：`ir_version <= 7`、`opset10/11`、固定
4 维输入，且 N 维只能为 1。因此这里不能直接拿原始 RobotLab ONNX 编译，
需要先降级 opset，再把 `[1, 450]` 输入包成固定 4D `NCHW`：
`[1, 1, 1, 450]`。
4 维输入，且 N 维只能为 1。因此这里不能直接拿原始 ONNX 编译，需要先裁剪
actions-only 输出、降级 opset，再把 `[1, D]` 输入包成固定 4D `NCHW`：
Gym 5 帧是 `[1, 1, 1, 225]`，RobotLab 10 帧是 `[1, 1, 1, 450]`。
## 当前状态
新增 Gym 5 帧策略的一键量化入口，默认目标是：
- 原始模型：`../policy_35k.onnx`
- 原始输入：`obs [1, 225]`
- BPU 编译输入：`obs_4d [1, 1, 1, 225]`
- BPU 输出：`actions [1, 12, 1, 1]`
- 默认输出：`mapper_output_35k_gemm/policy_35k_int16_gemm.bin`
- 默认校准数据：`calibration_data_35k_gym_fast64/`
本机 Docker 已完成一次默认量化：
- 浮点 4D/Gemm 图等价对比：64 个真实样本上 `max_abs_diff = 7.15e-7`
- `hb_mapper makertbin` 输出：`actions` cosine `0.998524`，L1 `0.014313`，L2 `0.005103`，Chebyshev `0.040004`
- 编译估计延迟：`463.9 us`
- 产物大小：`2.0M`
- 产物路径：`deploy_45dim_rl_gym/bpu_quantization/mapper_output_35k_gemm/policy_35k_int16_gemm.bin`
一键量化命令：
```bash
cd /Users/chenyouyuan/cyy_ws/deploy_go1_pro/deploy_45dim_rl_gym/bpu_quantization
./quantize_policy_x5.sh
```
切换其他 Gym 轮次时直接指定模型和 round：
```bash
./quantize_policy_x5.sh --policy ../policy_30k.onnx --round 30k
./quantize_policy_x5.sh --policy ../policy_25k.onnx --round 25k
./quantize_policy_x5.sh --policy ../policy_15k.onnx --round 15k
```
脚本默认只抽 64 个真实 RL 样本做校准和最多 64 个样本做浮点等价对比，避免
之前 512 样本和 batch 回退导致的量化流程过慢。需要更稳的校准时再手动加大：
```bash
./quantize_policy_x5.sh --samples 128 --compare-limit 128
```
Gym BPU 部署入口支持快速切换轮次：
```bash
cd /root/go1_pro_deploy
PYTHONPATH=/root/go1_pro_deploy python3 deploy_45dim_rl_gym/deploy_go1_rlgym_bpu_x5_fastcpp.py \
--bpu-round 35k \
--kill-sport \
--enable-rl \
--log-dir logs \
--kp 32 --kd 1.0 \
--kp-cal 20 --kd-cal 1.0 \
--power-factor 9 \
--position-protect-limit 0.0 \
--action-clip 6.5 \
--action-trip-limit 8.0 \
--action-hard-trip-limit 16.0 \
--max-target-step 0.0 \
--max-roll-deg 50 \
--max-pitch-deg 50 \
--swap-vy-yaw \
--rc-vx-scale 0.5 \
--rc-vy-scale 0.5 \
--rc-wz-scale 1.0 \
--log-timing
```
也可以直接指定 bin：
```bash
PYTHONPATH=/root/go1_pro_deploy python3 deploy_45dim_rl_gym/deploy_go1_rlgym_bpu_x5_fastcpp.py \
--bpu-model deploy_45dim_rl_gym/bpu_quantization/mapper_output_35k_gemm/policy_35k_int16_gemm.bin \
--infer-check --max-steps 1000 --log-timing
```
板端离线测速：
```bash
cd /root/go1_pro_deploy
PYTHONPATH=/root/go1_pro_deploy python3 deploy_45dim_rl_gym/bpu_deploy_x5/test_bpu_policy.py \
--bpu-model deploy_45dim_rl_gym/bpu_quantization/mapper_output_35k_gemm/policy_35k_int16_gemm.bin \
--input-bin deploy_45dim_rl_gym/bpu_quantization/calibration_data_35k_gym_fast64/00000.bin \
--repeat 1000
cd /root/go1_pro_deploy/deploy_45dim_rl_gym/bpu_deploy_x5/cpp
./bpu_dnn_bench \
/root/go1_pro_deploy/deploy_45dim_rl_gym/bpu_quantization/mapper_output_35k_gemm/policy_35k_int16_gemm.bin \
/root/go1_pro_deploy/deploy_45dim_rl_gym/bpu_quantization/calibration_data_35k_gym_fast64/00000.bin \
1000
```
已经完成 `policy_robotlab_15000.onnx` 和 `policy_robotlab_6500.onnx` 的 int16
量化。当前 BPU 部署默认使用 6500 版本：
量化。RobotLab BPU 部署默认仍使用 6500 版本：
- 原始模型：`../policy_robotlab_6500.onnx`
- 原始输入：`obs [1, 450]`

15
deploy_45dim_rl_gym/bpu_quantization/compare_4d_onnx.py
View File

@@ -8,11 +8,17 @@ import numpy as np
import onnxruntime as ort
def load_samples(calibration_dir, limit):
def load_samples(calibration_dir, limit, flat_dim):
paths = sorted(Path(calibration_dir).glob("*.bin"))[:limit]
if not paths:
raise FileNotFoundError(f"No calibration .bin files found in {calibration_dir}")
return [np.fromfile(path, dtype=np.float32).reshape(1, 450) for path in paths]
samples = []
for path in paths:
sample = np.fromfile(path, dtype=np.float32)
if sample.size != flat_dim:
raise ValueError(f"{path} has {sample.size} float32 values, expected {flat_dim}")
samples.append(sample.reshape(1, flat_dim))
return samples
def main():
@@ -20,6 +26,7 @@ def main():
parser.add_argument("--flat-onnx", type=Path, required=True)
parser.add_argument("--bpu4d-onnx", type=Path, required=True)
parser.add_argument("--calibration-dir", type=Path, default=Path("calibration_data"))
parser.add_argument("--flat-dim", type=int, default=450)
parser.add_argument("--limit", type=int, default=64)
args = parser.parse_args()
@@ -30,9 +37,9 @@ def main():
max_abs = 0.0
max_mean_abs = 0.0
for sample in load_samples(args.calibration_dir, args.limit):
for sample in load_samples(args.calibration_dir, args.limit, args.flat_dim):
out_flat = flat.run(None, {flat_input: sample})[0]
out_wrapped = wrapped.run(None, {wrapped_input: sample.reshape(1, 1, 1, 450)})[0]
out_wrapped = wrapped.run(None, {wrapped_input: sample.reshape(1, 1, 1, args.flat_dim)})[0]
diff = np.abs(out_flat - out_wrapped)
max_abs = max(max_abs, float(diff.max()))
max_mean_abs = max(max_mean_abs, float(diff.mean()))

40
deploy_45dim_rl_gym/bpu_quantization/keep_actions_output.py Normal file
View File

@@ -0,0 +1,40 @@
#!/usr/bin/env python3
"""Keep only the actions output in a policy ONNX graph."""
import argparse
from pathlib import Path
import onnx
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--input", type=Path, required=True)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--output-name", default="actions")
args = parser.parse_args()
model = onnx.load(str(args.input))
outputs = list(model.graph.output)
if not outputs:
raise ValueError("ONNX graph has no outputs")
selected = None
for output in outputs:
if output.name == args.output_name:
selected = output
break
if selected is None:
selected = outputs[0]
print(f"[WARN] output {args.output_name!r} not found; keeping first output {selected.name!r}")
del model.graph.output[:]
model.graph.output.append(selected)
args.output.parent.mkdir(parents=True, exist_ok=True)
onnx.save(model, str(args.output))
print(f"Wrote actions-only ONNX: {args.output}")
if __name__ == "__main__":
main()

53
deploy_45dim_rl_gym/bpu_quantization/make_calibration_data.py
View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Build float32 BPU calibration inputs from recorded RobotLab deployment logs."""
"""Build float32 BPU calibration inputs from recorded deployment logs."""
import argparse
import json
@@ -11,16 +11,14 @@ import numpy as np
NUM_OBS = 45
HISTORY_LEN = 10
ONNX_INPUT_DIM = NUM_OBS * HISTORY_LEN
TERM_DIMS = (3, 3, 3, 12, 12, 12)
HERE = Path(__file__).resolve().parent
DEFAULT_LOG_ROOT = HERE.parents[1] / "logs"
def build_onnx_input(history):
def build_policy_input(history, history_len):
frames = list(history)
while len(frames) < HISTORY_LEN:
while len(frames) < history_len:
frames.insert(0, np.zeros(NUM_OBS, dtype=np.float32))
chunks = []
@@ -41,14 +39,14 @@ def reservoir_add(samples, value, seen, max_samples, rng):
samples[replace_index] = value
def collect_samples(log_paths, max_samples, seed):
def collect_samples(log_paths, max_samples, seed, history_len):
rng = random.Random(seed)
samples = []
seen = 0
usable_runs = []
for steps_path in log_paths:
history = deque(maxlen=HISTORY_LEN)
history = deque(maxlen=history_len)
run_seen = 0
was_rl = False
@@ -71,13 +69,13 @@ def collect_samples(log_paths, max_samples, seed):
continue
history.append(obs)
onnx_input = build_onnx_input(history)
if not np.all(np.isfinite(onnx_input)):
policy_input = build_policy_input(history, history_len)
if not np.all(np.isfinite(policy_input)):
continue
seen += 1
run_seen += 1
reservoir_add(samples, onnx_input, seen, max_samples, rng)
reservoir_add(samples, policy_input, seen, max_samples, rng)
if run_seen:
usable_runs.append({"steps": str(steps_path), "samples": run_seen})
@@ -87,14 +85,18 @@ def collect_samples(log_paths, max_samples, seed):
def main():
parser = argparse.ArgumentParser(
description="Create BPU float32 calibration .bin files from RobotLab JSONL logs."
description="Create BPU float32 calibration .bin files from JSONL deployment logs."
)
parser.add_argument(
"--logs-root",
type=Path,
default=DEFAULT_LOG_ROOT,
help="Directory containing robotlab_go1_deploy_*/steps.jsonl.",
help="Directory containing <log-prefix>_*/steps.jsonl.",
)
parser.add_argument("--log-prefix", default="robotlab_go1_deploy",
help="Run directory prefix below --logs-root")
parser.add_argument("--history-len", type=int, default=10,
help="Number of 45-dim observations to stack by term")
parser.add_argument(
"--output-dir",
type=Path,
@@ -102,23 +104,28 @@ def main():
help="Output directory for raw float32 feature-map .bin files.",
)
parser.add_argument("--max-samples", type=int, default=512)
parser.add_argument("--min-samples", type=int, default=32)
parser.add_argument("--seed", type=int, default=20260727)
parser.add_argument("--overwrite", action="store_true")
args = parser.parse_args()
if args.max_samples < 32:
raise ValueError("--max-samples must be at least 32")
if args.max_samples < 1:
raise ValueError("--max-samples must be positive")
if args.history_len < 1:
raise ValueError("--history-len must be positive")
log_paths = sorted(args.logs_root.glob("robotlab_go1_deploy_*/steps.jsonl"))
log_paths = sorted(args.logs_root.glob(f"{args.log_prefix}_*/steps.jsonl"))
if not log_paths:
raise FileNotFoundError(f"No RobotLab step logs found below {args.logs_root}")
raise FileNotFoundError(
f"No step logs found for prefix {args.log_prefix!r} below {args.logs_root}"
)
samples, total_seen, usable_runs = collect_samples(
log_paths, args.max_samples, args.seed
log_paths, args.max_samples, args.seed, args.history_len
)
if len(samples) < 32:
if len(samples) < args.min_samples:
raise RuntimeError(
f"Only {len(samples)} valid RL inputs found; need at least 32 calibration samples."
f"Only {len(samples)} valid RL inputs found; need at least {args.min_samples} calibration samples."
)
output_dir = args.output_dir.resolve()
@@ -134,13 +141,15 @@ def main():
for index, sample in enumerate(samples):
sample.astype(np.float32, copy=False).tofile(output_dir / f"{index:05d}.bin")
policy_input_dim = NUM_OBS * args.history_len
metadata = {
"format": "raw float32 feature-map",
"flat_shape": [1, ONNX_INPUT_DIM],
"mapper_shape": [1, 1, 1, ONNX_INPUT_DIM],
"history_len": HISTORY_LEN,
"flat_shape": [1, policy_input_dim],
"mapper_shape": [1, 1, 1, policy_input_dim],
"history_len": args.history_len,
"num_obs": NUM_OBS,
"term_dims": list(TERM_DIMS),
"log_prefix": args.log_prefix,
"selected_samples": len(samples),
"candidate_rl_inputs": total_seen,
"seed": args.seed,

199
deploy_45dim_rl_gym/bpu_quantization/quantize_policy_x5.sh Executable file
View File

@@ -0,0 +1,199 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
POLICY="../policy_35k.onnx"
ROUND="35k"
NAME=""
HISTORY_LEN=5
FLAT_DIM=""
SAMPLES=64
MIN_SAMPLES=32
LOG_PREFIX="rlgym_go1_deploy"
DOCKER_IMAGE="openexplorer/ai_toolchain_ubuntu_20_x5_cpu:v1.2.8"
COMPARE_LIMIT=64
RUN_CHECKER=1
usage() {
cat <<'EOF'
Usage:
./quantize_policy_x5.sh [options]
Default: quantize Gym policy_35k.onnx as 5-frame/225-dim int16 Gemm BPU model.
Options:
--policy PATH ONNX policy path, relative to this directory or absolute
--round NAME round label used in output paths, e.g. 15k/25k/30k/35k
--name NAME model basename; default is policy filename without .onnx
--history-len N observation history length; Gym=5, RobotLab=10
--flat-dim N flat input dim; default 45 * history-len
--samples N calibration sample count; default 64 for faster mapping
--min-samples N minimum valid samples required; default 32
--log-prefix PREFIX log dir prefix below logs/, default rlgym_go1_deploy
--docker-image IMAGE D-Robotics CPU toolchain image
--compare-limit N float ONNX equivalence sample count, default 64
--skip-checker skip hb_mapper checker before makertbin
EOF
}
while [[ $# -gt 0 ]]; do
case "$1" in
--policy) POLICY="$2"; shift 2 ;;
--round) ROUND="$2"; shift 2 ;;
--name) NAME="$2"; shift 2 ;;
--history-len) HISTORY_LEN="$2"; shift 2 ;;
--flat-dim) FLAT_DIM="$2"; shift 2 ;;
--samples) SAMPLES="$2"; shift 2 ;;
--min-samples) MIN_SAMPLES="$2"; shift 2 ;;
--log-prefix) LOG_PREFIX="$2"; shift 2 ;;
--docker-image) DOCKER_IMAGE="$2"; shift 2 ;;
--compare-limit) COMPARE_LIMIT="$2"; shift 2 ;;
--skip-checker) RUN_CHECKER=0; shift ;;
-h|--help) usage; exit 0 ;;
*) echo "Unknown argument: $1" >&2; usage >&2; exit 2 ;;
esac
done
if [[ -z "${FLAT_DIM}" ]]; then
FLAT_DIM=$((45 * HISTORY_LEN))
fi
if [[ "${POLICY}" = /* ]]; then
POLICY_ABS="${POLICY}"
else
POLICY_ABS="${SCRIPT_DIR}/${POLICY}"
fi
POLICY_ABS="$(cd "$(dirname "${POLICY_ABS}")" && pwd)/$(basename "${POLICY_ABS}")"
if [[ ! -f "${POLICY_ABS}" ]]; then
echo "Policy not found: ${POLICY_ABS}" >&2
exit 2
fi
case "${POLICY_ABS}" in
"${REPO_ROOT}"/*) POLICY_REL="${POLICY_ABS#${REPO_ROOT}/}" ;;
*) echo "Policy must be inside repo root ${REPO_ROOT}: ${POLICY_ABS}" >&2; exit 2 ;;
esac
if [[ -z "${NAME}" ]]; then
NAME="$(basename "${POLICY_ABS}" .onnx)"
fi
CAL_DIR="calibration_data_${ROUND}_gym_fast${SAMPLES}"
OUTPUT_DIR="mapper_output_${ROUND}_gemm"
OUTPUT_PREFIX="${NAME}_int16_gemm"
YAML_FILE="${OUTPUT_PREFIX}.yaml"
echo "[INFO] repo : ${REPO_ROOT}"
echo "[INFO] policy : ${POLICY_REL}"
echo "[INFO] name/round : ${NAME} / ${ROUND}"
echo "[INFO] history/shape : ${HISTORY_LEN} / 1x1x1x${FLAT_DIM}"
echo "[INFO] calibration : ${CAL_DIR} (${SAMPLES} samples, prefix ${LOG_PREFIX})"
echo "[INFO] output : ${OUTPUT_DIR}/${OUTPUT_PREFIX}.bin"
docker run --rm --platform linux/amd64 \
-e POLICY_REL="${POLICY_REL}" \
-e NAME="${NAME}" \
-e HISTORY_LEN="${HISTORY_LEN}" \
-e FLAT_DIM="${FLAT_DIM}" \
-e SAMPLES="${SAMPLES}" \
-e MIN_SAMPLES="${MIN_SAMPLES}" \
-e LOG_PREFIX="${LOG_PREFIX}" \
-e CAL_DIR="${CAL_DIR}" \
-e OUTPUT_DIR="${OUTPUT_DIR}" \
-e OUTPUT_PREFIX="${OUTPUT_PREFIX}" \
-e YAML_FILE="${YAML_FILE}" \
-e COMPARE_LIMIT="${COMPARE_LIMIT}" \
-e RUN_CHECKER="${RUN_CHECKER}" \
-v "${REPO_ROOT}:/workspace/deploy_go1_pro" \
"${DOCKER_IMAGE}" \
bash -lc '
set -euo pipefail
cd /workspace/deploy_go1_pro/deploy_45dim_rl_gym/bpu_quantization
POLICY="/workspace/deploy_go1_pro/${POLICY_REL}"
ACTIONS_ONNX="${NAME}_actions.onnx"
OPSET_ONNX="${NAME}_opset11.onnx"
BPU4D_ONNX="${NAME}_bpu4d.onnx"
GEMM_ONNX="${NAME}_bpu4d_gemm.onnx"
python3 make_calibration_data.py \
--logs-root ../../logs \
--log-prefix "${LOG_PREFIX}" \
--history-len "${HISTORY_LEN}" \
--output-dir "${CAL_DIR}" \
--max-samples "${SAMPLES}" \
--min-samples "${MIN_SAMPLES}" \
--overwrite
python3 keep_actions_output.py \
--input "${POLICY}" \
--output "${ACTIONS_ONNX}"
python3 downgrade_policy_to_opset11.py \
--input "${ACTIONS_ONNX}" \
--output "${OPSET_ONNX}"
python3 make_bpu_4d_onnx.py \
--input "${OPSET_ONNX}" \
--output "${BPU4D_ONNX}" \
--flat-dim "${FLAT_DIM}"
python3 replace_group_conv_with_gemm.py \
--input "${BPU4D_ONNX}" \
--output "${GEMM_ONNX}"
python3 compare_4d_onnx.py \
--flat-onnx "${ACTIONS_ONNX}" \
--bpu4d-onnx "${GEMM_ONNX}" \
--calibration-dir "${CAL_DIR}" \
--flat-dim "${FLAT_DIM}" \
--limit "${COMPARE_LIMIT}"
cat > "${YAML_FILE}" <<YAML
model_parameters:
onnx_model: "./${GEMM_ONNX}"
march: "bayes-e"
layer_out_dump: false
working_dir: "${OUTPUT_DIR}"
output_model_file_prefix: "${OUTPUT_PREFIX}"
input_parameters:
input_name: "obs_4d"
input_shape: "1x1x1x${FLAT_DIM}"
input_type_rt: "featuremap"
input_layout_rt: "NCHW"
input_type_train: "featuremap"
input_layout_train: "NCHW"
norm_type: "no_preprocess"
calibration_parameters:
cal_data_dir: "./${CAL_DIR}"
cal_data_type: "float32"
calibration_type: "default"
optimization: "set_all_nodes_int16"
per_channel: true
compiler_parameters:
compile_mode: "latency"
debug: false
optimize_level: "O3"
YAML
if [[ "${RUN_CHECKER}" = "1" ]]; then
hb_mapper checker \
--model "${GEMM_ONNX}" \
--model-type onnx \
--march bayes-e \
--input-shape obs_4d "1x1x1x${FLAT_DIM}"
fi
hb_mapper makertbin \
--config "${YAML_FILE}" \
--model-type onnx
ls -lh "${OUTPUT_DIR}/${OUTPUT_PREFIX}.bin"
'
echo "[INFO] Done: deploy_45dim_rl_gym/bpu_quantization/${OUTPUT_DIR}/${OUTPUT_PREFIX}.bin"

12
deploy_45dim_rl_gym/bpu_quantization/replace_group_conv_with_gemm.py
View File

@@ -48,7 +48,17 @@ def replace_node(model):
conv = next((node for node in nodes if node.name == CONV_NAME), None)
if conv is None:
raise ValueError(f"Cannot find node {CONV_NAME!r}")
candidates = [
node for node in nodes
if node.op_type == "Conv"
and int(attr_value(node, "group", 1)) > 1
and attr_value(node, "kernel_shape") == [1]
]
if len(candidates) == 1:
conv = candidates[0]
print(f"[WARN] {CONV_NAME!r} not found; using grouped Conv {conv.name!r}")
if conv is None:
raise ValueError(f"Cannot find unique grouped 1x1 Conv node; fixed name {CONV_NAME!r} not found")
if conv.op_type != "Conv":
raise ValueError(f"{CONV_NAME!r} is {conv.op_type}, expected Conv")