GPS fix + log directory redesign + dashcamd binary

GPS: fix AT response parsing (strip mmcli `response: '...'` wrapper),
fix capnp field names (horizontalAccuracy, hasFix), set system clock
directly from first GPS fix when time is invalid, kill system gpsd on
startup.

Logging: replace module-level log dir creation with init_log_dir()
called from manager_init(). Active session always at /data/log2/current
(real dir until time resolves, then symlink to timestamped dir). Add
LogDirInitialized param. Redirect both stdout+stderr for all processes.

Also: thorough process cleanup in launch scripts, dashcamd binary,
CLAUDE.md updates for GPS/telemetry/bench/logging docs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-13 04:51:33 +00:00
parent f969214a3d
commit 4f16a8953a
11 changed files with 324 additions and 48 deletions

View File

@@ -14,8 +14,13 @@ ClearPilot is a custom fork of **FrogPilot** (itself a fork of comma.ai's openpi
- **ClearPilot service**: Node.js service at `selfdrive/clearpilot/` with behavior scripts for lane change and longitudinal control
- **Native dashcamd**: C++ process capturing raw camera frames via VisionIPC with OMX H.264 hardware encoding
- **Standstill power saving**: model inference throttled to 1fps and fan quieted when car is stopped
- **Clean offroad UI**: grid launcher replacing stock home screen
- **ClearPilot menu**: sidebar settings panel replacing stock home screen (Home, Dashcam, Debug panels)
- **Status window**: live system stats (temp, fan, storage, RAM, WiFi, VPN, GPS, telemetry status)
- **Debug button (LFA)**: steering wheel button repurposed for screen toggle and future UI actions
- **Telemetry system**: diff-based CSV logger via ZMQ IPC, toggleable from Debug panel
- **Bench mode**: `--bench` flag for onroad UI testing without car connected
- **GPS**: custom AT-command based GPS daemon (`system/clearpilot/gpsd.py`) replacing broken qcomgpsd diag interface
- **OpenVPN tunnel**: auto-connecting VPN to expose device on remote network for SSH access
See `GOALS.md` for feature roadmap.
@@ -72,10 +77,11 @@ su - comma -c "bash /data/openpilot/build_only.sh"
su - comma -c "bash /data/openpilot/launch_openpilot.sh"
# 4. Review the aggregate session log for errors
cat /data/log2/$(ls -t /data/log2/ | head -1)/session.log
cat /data/log2/current/session.log
# 5. Check per-process stderr logs if needed
ls /data/log2/$(ls -t /data/log2/ | head -1)/
# 5. Check per-process stdout/stderr logs if needed
ls /data/log2/current/
cat /data/log2/current/gpsd.log
```
### Adding New Params
@@ -128,14 +134,19 @@ su - comma -c "PYTHONPATH=/data/openpilot python3 -m selfdrive.clearpilot.bench_
The UI has a SIGSEGV/SIGABRT crash handler (`selfdrive/ui/main.cc`) that prints a stack trace to stderr, captured in the per-process log:
```bash
# Check for crash traces
grep -A 30 "CRASH" /data/log2/$(ls -t /data/log2/ | head -1)/ui.log
# Check for crash traces (use /data/log2/current which is always the active session)
grep -A 30 "CRASH" /data/log2/current/ui.log
# Resolve addresses to source lines
addr2line -e /data/openpilot/selfdrive/ui/ui -f 0xADDRESS
# bench_cmd dump detects crash loops automatically:
# if UI process uptime < 5s, it reports "likely crash looping"
# Check per-process logs
ls /data/log2/current/
cat /data/log2/current/session.log
cat /data/log2/current/gpsd.log
```
### UI Introspection RPC
@@ -150,9 +161,9 @@ The UI process runs a ZMQ REP server at `ipc:///tmp/clearpilot_ui_rpc`. Send `"d
- `launch_openpilot.sh --bench` sets `BENCH_MODE=1` env var
- `manager.py` reads `BENCH_MODE`, blocks real car processes, enables `bench_onroad` process
- `bench_onroad.py` publishes fake `deviceState`, `pandaStates`, `carState`, `controlsState` at correct frequencies
- The UI receives these messages identically to real car data
- Blocked processes in bench mode: pandad, thermald, controlsd, radard, plannerd, calibrationd, torqued, paramsd, locationd, sensord, ubloxd, pigeond, dmonitoringmodeld, dmonitoringd, modeld, soundd, camerad, loggerd, micd, dashcamd
- `bench_onroad.py` publishes fake `pandaStates` (ignition=true), `carState`, `controlsState` — thermald reads the fake pandaStates to determine ignition and publishes `deviceState.started=true` on its own
- thermald and camerad run normally in bench mode (thermald manages CPU cores needed for camerad)
- Blocked processes in bench mode: pandad, controlsd, radard, plannerd, calibrationd, torqued, paramsd, locationd, sensord, ubloxd, pigeond, dmonitoringmodeld, dmonitoringd, modeld, soundd, loggerd, micd, dashcamd
### Key Files
@@ -174,12 +185,17 @@ The UI process runs a ZMQ REP server at `ipc:///tmp/clearpilot_ui_rpc`. Send `"d
## Session Logging
Per-process stderr and an aggregate event log are captured in `/data/log2/{session}/`.
Per-process stderr and an aggregate event log are captured in `/data/log2/current/`.
### Log Directory
- Created at manager import time with timestamp: `/data/log2/YYYY-MM-DD-HH-MM-SS/`
- If system clock is invalid (cold boot, no WiFi, RTC stuck at 1970): uses `/data/log2/boot-{monotonic}/`, renamed to real timestamp once GPS/NTP resolves the time
- `/data/log2/current/` is always the active session directory
- `init_log_dir()` is called once from `manager_init()` — creates a fresh `/data/log2/current/` real directory
- If a previous `current/` real directory exists (unresolved session), it's renamed using its mtime timestamp
- If a previous `current` symlink exists, it's removed
- Once system time is valid (GPS/NTP), the real directory is renamed to `/data/log2/YYYY-MM-DD-HH-MM-SS/` and `current` becomes a symlink to it
- `LogDirInitialized` param: `"0"` until time resolves, then `"1"`
- Open file handles survive the rename (same inode, same filesystem)
- Session directories older than 30 days are deleted on manager startup
### Per-Process Logs
@@ -424,8 +440,25 @@ Power On
### GPS
- `ubloxd` + `pigeond` for u-blox GPS hardware
- `qcomgpsd`, `ugpsd`, `navd` currently **commented out** in process_config
- Device has **no u-blox chip** (`/dev/ttyHS0` does not exist) — `ubloxd`/`pigeond` never start
- GPS hardware is a **Quectel EC25 LTE modem** (USB, `lsusb: 2c7c:0125`) with built-in GPS
- GPS is accessed via AT commands through `mmcli`: `mmcli -m any --command='AT+QGPSLOC=2'`
- **`qcomgpsd`** (original openpilot process) uses the modem's diag interface which is broken on this device — the diag recv loop blocks forever after setup. Commented out.
- **`system/clearpilot/gpsd.py`** is the replacement — polls GPS via AT commands at 1Hz, publishes `gpsLocation` cereal messages
- GPS data flows: `gpsd``gpsLocation``locationd``liveLocationKalman``timed` (sets system clock)
- `locationd` checks `UbloxAvailable` param (false on this device) to subscribe to `gpsLocation` instead of `gpsLocationExternal`
- `mmcli` returns `response: '...'` wrapper — `at_cmd()` strips it before parsing (fixed)
- GPS antenna power must be enabled via GPIO: `gpio_set(GPIO.GNSS_PWR_EN, True)`
- System `/usr/sbin/gpsd` daemon may respawn and interfere — should be disabled or killed
### Telemetry
- **Client**: `selfdrive/clearpilot/telemetry.py``tlog(group, data)` sends JSON over ZMQ PUSH
- **Collector**: `selfdrive/clearpilot/telemetryd.py` — diffs against previous state, writes changed values to CSV
- **Toggle**: `TelemetryEnabled` param, controlled from Debug panel in ClearPilot menu
- **Auto-disable**: disabled on every manager start; disabled if `/data` free < 5GB
- **Hyundai CAN-FD data**: logged from `selfdrive/car/hyundai/carstate.py` `update_canfd()` — groups: `car`, `cruise`, `speed_limit`, `buttons`
- **CSV location**: `/data/log2/current/telemetry.csv` (or session directory)
### Key Dependencies