1
0
forked from zbw/yiliao2026

feat: add mtran ROS2 translation service

This commit is contained in:
2026-07-30 19:34:49 +08:00
parent 0c09a97e7f
commit 8de4e20046
400 changed files with 137442 additions and 0 deletions

View File

@@ -0,0 +1,595 @@
# MTranServer ROS2 English-to-Chinese Service Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build a ROS2 Humble C++ service under `migrate_ws` that accepts one English string, translates it through a launch-managed persistent MTranServer, and returns Simplified Chinese.
**Architecture:** A C++ `mtran_bridge_node` exposes `/translate_en_to_zh` and sends fixed `en -> zh-Hans` JSON requests to a loopback MTranServer sidecar. ROS2 launch owns both processes; the bridge performs startup health checking, model warm-up, serialized request handling, and periodic keep-warm calls.
**Tech Stack:** ROS2 Humble, Ubuntu 22.04, C++17, ament_cmake, rosidl, libcurl, nlohmann/json, GoogleTest, Python ROS2 launch, Bun-compiled MTranServer.
## Global Constraints
- Every new or generated migration file must remain under `/home/hikos/MTranServer/migrate_ws`.
- The service name is `/translate_en_to_zh`.
- The service request contains only `string text`.
- The response contains `bool success`, `string translation`, and `string error`.
- Translation direction is fixed to `en -> zh-Hans`; HTML, language detection, topics, batching, and retries are excluded.
- Input is limited to 512 UTF-8 code points.
- Translation execution is serialized with `rclcpp::executors::SingleThreadedExecutor`.
- The current machine uses a native MTranServer build; ARM64 is documented but not built in this iteration.
- MTranServer uses a 86,400-second idle timeout and the bridge sends a keep-warm translation every 43,200 seconds.
- Runtime translation must work offline after the executable and `en_zh-Hans` model are staged.
---
## File Map
- `migrate_ws/.gitignore`: excludes colcon output, local tools, runtime binary, temporary runtime build tree, and model data.
- `migrate_ws/src/mtran_ros2/package.xml`: ROS2 and system dependency metadata.
- `migrate_ws/src/mtran_ros2/CMakeLists.txt`: service generation, C++ targets, install rules, and tests.
- `migrate_ws/src/mtran_ros2/srv/TranslateEnglishToChinese.srv`: public ROS2 interface.
- `migrate_ws/src/mtran_ros2/include/mtran_ros2/http_translation_client.hpp`: transport-independent client contract and libcurl client declaration.
- `migrate_ws/src/mtran_ros2/src/http_translation_client.cpp`: fixed HTTP payload, libcurl transport, JSON parsing, and error mapping.
- `migrate_ws/src/mtran_ros2/include/mtran_ros2/translator_node.hpp`: node state and service declaration.
- `migrate_ws/src/mtran_ros2/src/translator_node.cpp`: readiness state machine, validation, service callback, recovery, and keep-warm.
- `migrate_ws/src/mtran_ros2/src/main.cpp`: ROS2 initialization and single-threaded spin.
- `migrate_ws/src/mtran_ros2/config/translator.yaml`: production parameter defaults.
- `migrate_ws/src/mtran_ros2/launch/translator.launch.py`: sidecar and bridge lifecycle.
- `migrate_ws/src/mtran_ros2/scripts/build_mtranserver.sh`: native sidecar staging entirely below `migrate_ws`.
- `migrate_ws/src/mtran_ros2/test/test_http_translation_client.cpp`: injected-transport unit tests.
- `migrate_ws/src/mtran_ros2/test/test_translator_node.cpp`: fake-client ROS2 service tests.
- `migrate_ws/README.md`: dependency, local build, launch, smoke-test, offline-model, and ARM64 instructions.
---
### Task 1: Create the ROS2 Package and Service Contract
**Files:**
- Create: `migrate_ws/.gitignore`
- Create: `migrate_ws/src/mtran_ros2/package.xml`
- Create: `migrate_ws/src/mtran_ros2/CMakeLists.txt`
- Create: `migrate_ws/src/mtran_ros2/srv/TranslateEnglishToChinese.srv`
**Interfaces:**
- Consumes: ROS2 Humble `ament_cmake` and `rosidl_default_generators`.
- Produces: `mtran_ros2::srv::TranslateEnglishToChinese` with request field `text` and response fields `success`, `translation`, and `error`.
- [ ] **Step 1: Record generated-file exclusions**
```gitignore
build/
install/
log/
.tools/
.runtime-build/
runtime/bin/mtranserver
models/**
!models/.gitkeep
```
- [ ] **Step 2: Write the service definition**
```srv
string text
---
bool success
string translation
string error
```
- [ ] **Step 3: Add the minimal package manifest**
Declare `ament_cmake`, `rosidl_default_generators`, `rclcpp`, `rosidl_default_runtime`, `launch`, `launch_ros`, libcurl, nlohmann/json, and test dependencies. Add membership in `rosidl_interface_packages`.
- [ ] **Step 4: Add interface generation to CMake**
```cmake
cmake_minimum_required(VERSION 3.8)
project(mtran_ros2)
find_package(ament_cmake REQUIRED)
find_package(rosidl_default_generators REQUIRED)
rosidl_generate_interfaces(${PROJECT_NAME}
"srv/TranslateEnglishToChinese.srv"
)
ament_export_dependencies(rosidl_default_runtime)
ament_package()
```
- [ ] **Step 5: Build to verify service generation**
Run:
```bash
source /opt/ros/humble/setup.bash
cd /home/hikos/MTranServer/migrate_ws
colcon build --packages-select mtran_ros2
```
Expected: exit code 0 and generated service headers below `build/mtran_ros2/rosidl_generator_cpp`.
- [ ] **Step 6: Inspect the generated interface**
Run:
```bash
source /home/hikos/MTranServer/migrate_ws/install/setup.bash
ros2 interface show mtran_ros2/srv/TranslateEnglishToChinese
```
Expected: the exact four fields from the `.srv` definition.
- [ ] **Step 7: Commit the package contract**
```bash
git add migrate_ws/.gitignore migrate_ws/src/mtran_ros2/package.xml \
migrate_ws/src/mtran_ros2/CMakeLists.txt \
migrate_ws/src/mtran_ros2/srv/TranslateEnglishToChinese.srv
git commit -m "feat: add ROS2 translation service contract"
```
---
### Task 2: Implement the Fixed-Direction HTTP Client
**Files:**
- Create: `migrate_ws/src/mtran_ros2/include/mtran_ros2/http_translation_client.hpp`
- Create: `migrate_ws/src/mtran_ros2/src/http_translation_client.cpp`
- Create: `migrate_ws/src/mtran_ros2/test/test_http_translation_client.cpp`
- Modify: `migrate_ws/src/mtran_ros2/CMakeLists.txt`
- Modify: `migrate_ws/src/mtran_ros2/package.xml`
**Interfaces:**
- Consumes: `server_url` and `request_timeout_ms` configuration.
- Produces: `TranslationClient::health()` and `TranslationClient::translate(const std::string&)`.
Use these exact public types:
```cpp
enum class ClientError {
kNone,
kUnavailable,
kTimeout,
kUpstream,
kInvalidResponse,
};
struct TranslationResult {
bool success;
std::string translation;
ClientError error;
};
struct HttpResponse {
int transport_code;
long status_code;
std::string body;
};
using HttpExecutor = std::function<HttpResponse(
const std::string & method,
const std::string & url,
const std::string & body,
long timeout_ms)>;
class TranslationClient {
public:
virtual ~TranslationClient() = default;
virtual bool health() = 0;
virtual TranslationResult translate(const std::string & text) = 0;
};
class HttpTranslationClient final : public TranslationClient {
public:
HttpTranslationClient(
std::string server_url,
long timeout_ms,
HttpExecutor executor = {});
bool health() override;
TranslationResult translate(const std::string & text) override;
};
```
- [ ] **Step 1: Write failing payload and parsing tests**
Inject an `HttpExecutor` lambda that captures method, URL, body, and timeout. Assert that `translate("Hello")` sends `POST`, targets `/translate`, uses the configured timeout, and emits:
```json
{"from":"en","html":false,"text":"Hello","to":"zh-Hans"}
```
Also assert that HTTP 200 with `{"result":"你好"}` returns `success=true`, `translation="你好"`, and `ClientError::kNone`.
- [ ] **Step 2: Write failing error-mapping tests**
Add separate tests for:
```cpp
HttpResponse{CURLE_COULDNT_CONNECT, 0, ""} // kUnavailable
HttpResponse{CURLE_OPERATION_TIMEDOUT, 0, ""} // kTimeout
HttpResponse{CURLE_OK, 500, "failure"} // kUpstream
HttpResponse{CURLE_OK, 200, "not-json"} // kInvalidResponse
HttpResponse{CURLE_OK, 200, R"({"result":7})"} // kInvalidResponse
```
For every failure, assert `success=false` and `translation.empty()`.
- [ ] **Step 3: Run the client test to verify it fails**
Run:
```bash
source /opt/ros/humble/setup.bash
cd /home/hikos/MTranServer/migrate_ws
colcon test --packages-select mtran_ros2 --ctest-args -R test_http_translation_client --output-on-failure
```
Expected: test target or referenced client types are missing.
- [ ] **Step 4: Implement `HttpTranslationClient`**
Use nlohmann/json to serialize the fixed request and parse `result`. Normalize `server_url` by removing trailing slashes. The default executor initializes libcurl once with `std::call_once`, writes response bytes through a callback, sets `CURLOPT_CONNECTTIMEOUT_MS`, `CURLOPT_TIMEOUT_MS`, JSON content type, and `CURLOPT_NOSIGNAL=1L`.
Map libcurl connection and name-resolution failures to `kUnavailable`, `CURLE_OPERATION_TIMEDOUT` to `kTimeout`, other transport/non-200 failures to `kUpstream`, and response-shape failures to `kInvalidResponse`.
- [ ] **Step 5: Add the client library and test targets**
```cmake
find_package(CURL REQUIRED)
find_package(nlohmann_json REQUIRED)
find_package(rclcpp REQUIRED)
add_library(mtran_core
src/http_translation_client.cpp
)
target_compile_features(mtran_core PUBLIC cxx_std_17)
target_include_directories(mtran_core PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>)
target_link_libraries(mtran_core PUBLIC
CURL::libcurl
nlohmann_json::nlohmann_json)
```
Under `BUILD_TESTING`, add `ament_cmake_gtest`, create `test_http_translation_client`, and link it to `mtran_core`.
- [ ] **Step 6: Run the client tests**
Run:
```bash
source /opt/ros/humble/setup.bash
cd /home/hikos/MTranServer/migrate_ws
colcon build --packages-select mtran_ros2 --cmake-args -DBUILD_TESTING=ON
colcon test --packages-select mtran_ros2 --ctest-args -R test_http_translation_client --output-on-failure
colcon test-result --verbose
```
Expected: all client tests pass.
- [ ] **Step 7: Commit the HTTP client**
```bash
git add migrate_ws/src/mtran_ros2
git commit -m "feat: add fixed-direction MTran HTTP client"
```
---
### Task 3: Implement the ROS2 Service Node and Readiness State Machine
**Files:**
- Create: `migrate_ws/src/mtran_ros2/include/mtran_ros2/translator_node.hpp`
- Create: `migrate_ws/src/mtran_ros2/src/translator_node.cpp`
- Create: `migrate_ws/src/mtran_ros2/src/main.cpp`
- Create: `migrate_ws/src/mtran_ros2/test/test_translator_node.cpp`
- Modify: `migrate_ws/src/mtran_ros2/CMakeLists.txt`
**Interfaces:**
- Consumes: `std::shared_ptr<TranslationClient>` and generated `TranslateEnglishToChinese` service type.
- Produces: `/translate_en_to_zh`, startup/recovery health polling, warm-up, validation, and keep-warm behavior.
Use this constructor boundary:
```cpp
class TranslatorNode : public rclcpp::Node {
public:
explicit TranslatorNode(
const rclcpp::NodeOptions & options = rclcpp::NodeOptions(),
std::shared_ptr<TranslationClient> client = nullptr);
};
```
- [ ] **Step 1: Write the fake client and failing validation tests**
Create a thread-safe fake implementing `TranslationClient`. Configure node parameter overrides `health_retry_ms=1`, `keep_warm_interval_s=3600`, and `max_input_characters=512`. Spin the node until `/translate_en_to_zh` is available.
Assert:
```text
"" and " " -> INVALID_INPUT
512 ASCII characters -> client called and success returned
513 ASCII characters -> INPUT_TOO_LONG and client not called
```
- [ ] **Step 2: Write failing response-mapping tests**
For each fake client result, assert these exact service responses:
```text
kNone -> success=true, translation set, error empty
kUnavailable -> success=false, translation empty, NOT_READY
kTimeout -> success=false, translation empty, TIMEOUT
kUpstream -> success=false, translation empty, UPSTREAM_ERROR
kInvalidResponse -> success=false, translation empty, INVALID_RESPONSE
```
- [ ] **Step 3: Write failing readiness and recovery tests**
Verify that the service is not created while `health()` is false. Then switch health to true and make the warm-up translation succeed; verify the service becomes available. Simulate `kUnavailable` during a service call, verify `NOT_READY`, restore the fake, and verify health polling plus warm-up return the node to successful service operation.
- [ ] **Step 4: Run node tests to verify failure**
Run:
```bash
source /opt/ros/humble/setup.bash
cd /home/hikos/MTranServer/migrate_ws
colcon test --packages-select mtran_ros2 --ctest-args -R test_translator_node --output-on-failure
```
Expected: `TranslatorNode` and executable are missing.
- [ ] **Step 5: Implement startup, service, and recovery**
Declare the six parameters from the design. When no client is injected, construct `HttpTranslationClient`. Start a health timer immediately. A successful health call followed by `translate("Hello.")` sets `ready_=true`, creates the service on first startup, cancels health polling, and starts keep-warm scheduling.
Count UTF-8 characters by counting bytes that are not continuation bytes (`(byte & 0xC0) != 0x80`). Validate before checking readiness. Serialize all callbacks through the single-threaded executor.
If a service or keep-warm call returns `kUnavailable`, set `ready_=false` and restart health polling. Do not destroy the already-advertised service after initial startup; requests during recovery return `NOT_READY`.
- [ ] **Step 6: Implement `main.cpp`**
```cpp
int main(int argc, char ** argv) {
rclcpp::init(argc, argv);
auto node = std::make_shared<mtran_ros2::TranslatorNode>();
rclcpp::executors::SingleThreadedExecutor executor;
executor.add_node(node);
executor.spin();
rclcpp::shutdown();
return 0;
}
```
- [ ] **Step 7: Add node targets and generated typesupport linkage**
Add `translator_node.cpp` to `mtran_core`, obtain the generated C++ typesupport target with `rosidl_get_typesupport_target`, and link it. Add `mtran_bridge_node` from `main.cpp`, link `mtran_core`, and install the library, executable, and headers.
- [ ] **Step 8: Run node and client tests**
Run:
```bash
source /opt/ros/humble/setup.bash
cd /home/hikos/MTranServer/migrate_ws
colcon build --packages-select mtran_ros2 --cmake-args -DBUILD_TESTING=ON
colcon test --packages-select mtran_ros2 --event-handlers console_direct+
colcon test-result --verbose
```
Expected: all tests pass with no failed test cases.
- [ ] **Step 9: Commit the ROS2 node**
```bash
git add migrate_ws/src/mtran_ros2
git commit -m "feat: expose persistent English to Chinese ROS2 service"
```
---
### Task 4: Add Native Runtime Staging and ROS2 Launch
**Files:**
- Create: `migrate_ws/src/mtran_ros2/config/translator.yaml`
- Create: `migrate_ws/src/mtran_ros2/launch/translator.launch.py`
- Create: `migrate_ws/src/mtran_ros2/scripts/build_mtranserver.sh`
- Create: `migrate_ws/models/.gitkeep`
- Modify: `migrate_ws/src/mtran_ros2/CMakeLists.txt`
**Interfaces:**
- Consumes: repository MTranServer source, local npm/node, launch arguments, staged executable, and staged model directory.
- Produces: native `migrate_ws/runtime/bin/mtranserver` and one launch entry point that owns sidecar and bridge lifecycles.
- [ ] **Step 1: Write the production parameter file**
```yaml
mtran_bridge_node:
ros__parameters:
server_url: "http://127.0.0.1:8989"
request_timeout_ms: 30000
startup_timeout_ms: 60000
health_retry_ms: 500
keep_warm_interval_s: 43200
max_input_characters: 512
```
- [ ] **Step 2: Write the runtime staging script**
The script resolves the repository root from its own path, installs Bun locally with npm under `migrate_ws/.tools` when no usable local Bun exists, copies the MTranServer source into `migrate_ws/.runtime-build/mtranserver` while excluding `.git`, `migrate_ws`, `node_modules`, and generated outputs, runs `bun install --frozen-lockfile`, runs `bun run build --single`, and installs the executable at `migrate_ws/runtime/bin/mtranserver` with mode 0755.
The script must use `set -euo pipefail`, must not write to `/usr/local`, and must not modify the repository's root `dist/` directory.
- [ ] **Step 3: Run the staging script**
Run:
```bash
cd /home/hikos/MTranServer
./migrate_ws/src/mtran_ros2/scripts/build_mtranserver.sh
file migrate_ws/runtime/bin/mtranserver
```
Expected on the current machine: an executable x86-64 ELF MTranServer binary.
- [ ] **Step 4: Write the launch file**
Declare arguments `server_executable`, `model_dir`, `config_dir`, `params_file`, `host`, and `port`. Defaults resolve from the source workspace for local development and may be overridden for installed deployment. Start MTranServer with `ExecuteProcess` using:
```text
MT_HOST=<host>
MT_PORT=<port>
MT_ENABLE_UI=false
MT_OFFLINE=true
MT_CHECK_UPDATE=false
MT_CONFIG_DIR=<config_dir>
MT_MODEL_DIR=<model_dir>
MT_WORKER_IDLE_TIMEOUT=86400
MT_CACHE_SIZE=100
```
Enable sidecar respawn with a 2-second delay. Start `mtran_bridge_node` with the parameter file and a `server_url` override derived from host and port.
- [ ] **Step 5: Add launch syntax test and install rules**
Run:
```bash
python3 -m py_compile migrate_ws/src/mtran_ros2/launch/translator.launch.py
```
Expected: exit code 0. Install `launch/`, `config/`, and `scripts/` into the package share/lib destinations through CMake.
- [ ] **Step 6: Verify launch description without starting translation**
Run:
```bash
source /opt/ros/humble/setup.bash
cd /home/hikos/MTranServer/migrate_ws
colcon build --packages-select mtran_ros2
source install/setup.bash
ros2 launch mtran_ros2 translator.launch.py --show-args
```
Expected: all six launch arguments are listed with concrete defaults.
- [ ] **Step 7: Commit runtime and launch support**
```bash
git add migrate_ws/.gitignore migrate_ws/models/.gitkeep \
migrate_ws/src/mtran_ros2/config \
migrate_ws/src/mtran_ros2/launch \
migrate_ws/src/mtran_ros2/scripts \
migrate_ws/src/mtran_ros2/CMakeLists.txt
git commit -m "feat: launch persistent MTran sidecar"
```
---
### Task 5: Document, Stage the Model, and Verify End to End
**Files:**
- Create: `migrate_ws/README.md`
- Modify only if verification finds a defect: files below `migrate_ws/src/mtran_ros2/`
**Interfaces:**
- Consumes: built native sidecar, ROS2 package, and `en_zh-Hans` model files.
- Produces: reproducible local and ARM64 instructions plus evidence that automated and available runtime checks pass.
- [ ] **Step 1: Write the README**
Document:
- Architecture and fixed service contract
- Ubuntu 22.04/ROS2 Humble dependencies
- Native runtime staging command
- How to pre-download/copy only `en_zh-Hans` model files
- `colcon build`, launch, and service-call commands
- Error-code table
- Offline operation and model residency behavior
- x64 native build behavior
- ARM64 native build and `bun-linux-arm64` cross-target commands
- How to override executable and model paths at launch
- [ ] **Step 2: Run formatting and static build checks**
Run:
```bash
source /opt/ros/humble/setup.bash
cd /home/hikos/MTranServer/migrate_ws
colcon build --packages-select mtran_ros2 --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTING=ON
colcon test --packages-select mtran_ros2 --event-handlers console_direct+
colcon test-result --verbose
python3 -m py_compile src/mtran_ros2/launch/translator.launch.py
```
Expected: build and tests exit 0; Python compilation produces no output.
- [ ] **Step 3: Check the staged runtime and model prerequisites**
Run:
```bash
test -x /home/hikos/MTranServer/migrate_ws/runtime/bin/mtranserver
find /home/hikos/MTranServer/migrate_ws/models/en_zh-Hans -maxdepth 1 -type f -print
```
Expected: executable check passes and the model directory contains model, vocabulary, and lexical files required by MTranServer.
- [ ] **Step 4: Start launch and execute the smoke test**
Terminal 1:
```bash
source /opt/ros/humble/setup.bash
source /home/hikos/MTranServer/migrate_ws/install/setup.bash
ros2 launch mtran_ros2 translator.launch.py
```
Terminal 2:
```bash
source /opt/ros/humble/setup.bash
source /home/hikos/MTranServer/migrate_ws/install/setup.bash
ros2 service call /translate_en_to_zh \
mtran_ros2/srv/TranslateEnglishToChinese \
"{text: 'The robot has completed the inspection.'}"
```
Expected: `success: true`, non-empty Chinese `translation`, and empty `error`.
- [ ] **Step 5: Verify offline restart**
Restart the launch with `MT_OFFLINE=true` already enforced by the launch file and no network dependency. Repeat the same service request and expect the same response shape without model download attempts.
- [ ] **Step 6: Inspect repository scope**
Run:
```bash
cd /home/hikos/MTranServer
git status --short
git diff --check
find migrate_ws -maxdepth 4 -type f -print | sort
```
Expected: authored migration changes are only under `migrate_ws`; generated runtime, models, tools, and colcon directories are ignored.
- [ ] **Step 7: Commit documentation and final corrections**
```bash
git add migrate_ws/README.md migrate_ws/src/mtran_ros2 migrate_ws/.gitignore migrate_ws/models/.gitkeep
git commit -m "docs: add ROS2 translation deployment guide"
```
- [ ] **Step 8: Run final verification from a clean build directory**
Move the existing generated `build`, `install`, and `log` directories to a temporary directory under `/tmp`, then rerun the complete build and test commands from Step 2. Do not delete user data or model files.
Expected: a clean configuration, build, and test pass.