diff --git a/CLAUDE.md b/CLAUDE.md index b1af24e..d5d0067 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,13 +12,12 @@ ClearPilot is a custom fork of **FrogPilot** (itself a fork of comma.ai's openpi - **Driver monitoring timeouts**: modified safety timeouts for the driver monitoring system - **Custom driving models**: `duck-amigo.thneed`, `farmville.onnx`, `wd-40.thneed` in `selfdrive/clearpilot/models/` - **ClearPilot service**: Node.js service at `selfdrive/clearpilot/` with behavior scripts for lane change and longitudinal control +- **Native dashcamd**: C++ process capturing raw camera frames via VisionIPC with OMX H.264 hardware encoding +- **Standstill power saving**: model inference throttled to 1fps and fan quieted when car is stopped +- **Clean offroad UI**: grid launcher replacing stock home screen +- **Debug button (LFA)**: steering wheel button repurposed for screen toggle and future UI actions -### Short-Term Goals - -- ~~Fix the dashcam/screen recorder feature~~ (done — see Dashcam section below) -- Fix GPS tracking feature -- Add a safe-speed-exceeded chime -- Implement interactions for the "debug function button" +See `GOALS.md` for feature roadmap. ## Working Rules @@ -58,67 +57,142 @@ chown -R comma:comma /data/openpilot ### Testing Changes -To restart the full openpilot stack after making changes: +The launch script now self-cleans — it kills other instances of itself, `launch_chffrplus.sh`, and `manager.py` before starting. No need to manually kill first. ```bash -# Fix ownership first (we edit as root, openpilot runs as comma) +# Fix ownership (we edit as root, openpilot runs as comma) chown -R comma:comma /data/openpilot -# Kill existing stack — use pgrep/kill to avoid pkill matching our own shell -kill $(pgrep -f 'python.*manager') 2>/dev/null; sleep 2 +# Remove prebuilt to force recompilation (it is recreated after each successful build) +rm -f /data/openpilot/prebuilt -# Must use a login shell as comma — sudo -u won't set up the right Python/env +# Must use a login shell as comma — sudo -u won't set up the right Python/env (3.11 via pyenv) su - comma -c "bash /data/openpilot/launch_openpilot.sh" ``` -Note: `pkill -f manager` or `pkill -f launch_openpilot` will match the invoking shell's own command line and kill the launch process itself. Use `pgrep`+`kill` or run the kill as a separate step before launching. +### Adding New Params -**Adding new params**: The params system uses a C++ whitelist. Adding a new param name in `manager.py` alone will crash with `UnknownKeyName`. You must also register the key in `common/params.cc` in the key list (alphabetically, with `PERSISTENT` or `CLEAR_ON_*` flag), then rebuild. The rebuild will recompile `params.cc` -> `libcommon.a` and re-link all binaries that use it. +The params system uses a C++ whitelist. Adding a new param name in `manager.py` alone will crash with `UnknownKeyName`. You must: -The `prebuilt` marker file skips compilation. It is **recreated automatically** after each successful build, so you must remove it every time you want to recompile: -```bash -rm -f /data/openpilot/prebuilt -``` +1. Register the key in `common/params.cc` (alphabetically, with `PERSISTENT` or `CLEAR_ON_*` flag) +2. Add the default value in `selfdrive/manager/manager.py` in `manager_init()` +3. Remove `prebuilt`, `common/params.o`, and `common/libcommon.a` to force rebuild -## Dashcam / Screen Recorder +### Building Native (C++) Processes + +- SCons is the build system. Static libraries (`common`, `messaging`, `cereal`, `visionipc`) must be imported as SCons objects, not `-l` flags +- The `--as-needed` linker flag can cause link order issues with static libs — disable it in your SConscript if needed +- OMX encoder object (`omx_encoder.o`) is compiled by the UI build — reference the pre-built `.o` file rather than recompiling (avoids "two environments" scons error) +- `prebuilt` is recreated after every successful build — always remove it before rebuilding + +## Dashcam (dashcamd) ### Architecture -The dashcam is the FrogPilot `ScreenRecorder` — it captures the onroad UI screen (with overlays) and encodes to MP4 using the Qualcomm OMX H.264 hardware encoder. +`dashcamd` is a native C++ process that captures raw camera frames directly from `camerad` via VisionIPC and encodes to MP4 using the Qualcomm OMX H.264 hardware encoder. This replaces the earlier FrogPilot screen recorder approach (`QWidget::grab()` -> OMX). - **Codec**: H.264 AVC (hardware accelerated via `OMX.qcom.video.encoder.avc`) -- **Resolution**: 1440x720 (downscaled from 2160x1080) -- **Bitrate**: 2 Mbps -- **Container**: MP4 +- **Resolution**: 1928x1208 (full camera resolution, no downscaling) +- **Bitrate**: 4 Mbps +- **Container**: MP4 (remuxed via libavformat) - **Segment length**: 3 minutes per file - **Save path**: `/data/media/0/videos/YYYYMMDD-HHMMSS.mp4` +- **Standstill timeout**: suspends recording after 10 minutes of standstill, resumes when car moves +- **Storage**: ~90 MB per 3-minute segment, ~43 hours of footage in 78 GB free space +- **Storage device**: WDC SDINDDH4-128G UFS 2.1 — automotive grade, ~384 TB write endurance, no concern for continuous writes -### Changes Made (2026-04-11) +### Key Differences from Old Screen Recorder -1. **Disabled comma training data video** (`selfdrive/manager/process_config.py`): commented out `encoderd` and `stream_encoderd`. CAN/sensor logs (`rlog`/`qlog`) are still recorded by `loggerd`. - -2. **Re-enabled screen recorder** (`selfdrive/ui/qt/onroad.cc`): uncommented the `QTimer` that feeds frames to the encoder at UI_FREQ rate. - -3. **Auto-start recording** (`selfdrive/frogpilot/screenrecorder/screenrecorder.cc`): modified `update_screen()` to automatically start recording when the car is on (`scene.started`) and stop when off. No button press needed. - -4. **Hidden UI elements**: the record button is constructed but never made visible or added to layout. Recording is invisible to the driver. - -5. **Debug flag** (`ScreenRecorderDebug` param): when set to `"1"`, recording starts even without a car connected. Used for bench testing. Read via `scene.screen_recorder_debug` in `ui.h`/`ui.cc`. - -6. **Deleter updated** (`system/loggerd/deleter.py`): free space threshold raised from 5 GB to 9 GB. Oldest videos in `/data/media/0/videos/` are deleted first before falling back to log segments. +| | Old (screen recorder) | New (dashcamd) | +|---|---|---| +| Source | `QWidget::grab()` screen capture | Raw NV12 from VisionIPC | +| Resolution | 1440x720 | 1928x1208 | +| Works with screen off | No (needs visible widget) | Yes (independent of UI) | +| Process type | Part of UI process | Standalone native process | +| Encoder input | RGBA -> NV12 conversion | NV12 direct (added `encode_frame_nv12`) | ### Key Files | File | Role | |------|------| -| `selfdrive/frogpilot/screenrecorder/screenrecorder.cc` | Screen capture and auto-start/stop logic | -| `selfdrive/frogpilot/screenrecorder/omx_encoder.cc` | OMX H.264 hardware encoder wrapper | -| `selfdrive/ui/qt/onroad.cc` | Timer that drives frame capture | -| `selfdrive/ui/ui.h` | `screen_recorder_debug` scene flag | -| `selfdrive/ui/ui.cc` | Reads `ScreenRecorderDebug` param | -| `selfdrive/manager/manager.py` | Default params | -| `selfdrive/manager/process_config.py` | encoderd disabled here | -| `system/loggerd/deleter.py` | Storage rotation (9 GB threshold, videos + logs) | +| `selfdrive/clearpilot/dashcamd.cc` | Main dashcam process — VisionIPC -> OMX encoder | +| `selfdrive/clearpilot/SConscript` | Build config for dashcamd | +| `selfdrive/frogpilot/screenrecorder/omx_encoder.cc` | OMX encoder (added `encode_frame_nv12` method) | +| `selfdrive/frogpilot/screenrecorder/omx_encoder.h` | Encoder header | +| `selfdrive/manager/process_config.py` | dashcamd registered as NativeProcess, encoderd disabled | +| `system/loggerd/deleter.py` | Storage rotation (9 GB threshold, oldest videos deleted first) | + +### Params + +- `DashcamDebug` — when `"1"`, dashcamd runs even without car connected (for bench testing) +- `IsDriverViewEnabled` — must be `"1"` to start camerad on bench (no car ignition) + +## Standstill Power Saving + +When `carState.standstill` is true: + +- **modeld**: skips GPU inference on 19/20 frames (1fps vs 20fps), reports 0 frame drops to avoid triggering `modeldLagging` in controlsd +- **dmonitoringmodeld**: same 1fps throttle, added `carState` subscription +- **Fan controller**: uses offroad clamps (0-30%) instead of onroad (30-100%) at standstill; thermal protection still active via feedforward if temp > 60°C + +### Key Files + +| File | Role | +|------|------| +| `selfdrive/modeld/modeld.py` | Standstill frame skip logic | +| `selfdrive/modeld/dmonitoringmodeld.py` | Standstill frame skip logic | +| `selfdrive/thermald/fan_controller.py` | Standstill-aware fan clamps | +| `selfdrive/thermald/thermald.py` | Passes standstill to fan controller via carState | + +## Debug Function Button (LFA/LKAS Steering Wheel Button) + +The Hyundai Tucson's LFA (Lane Following Assist) steering wheel button is repurposed as a general-purpose UI control button. It has no driving function in ClearPilot since lateral control is disabled. + +### Signal Chain + +``` +Steering wheel LFA button press + -> CAN-FD message: cruise_btns_msg_canfd["LFA_BTN"] + [selfdrive/car/hyundai/carstate.py:332-339] + -> Edge detection: lkas_enabled vs lkas_previously_enabled + -> create_button_events() -> ButtonEvent(type=FrogPilotButtonType.lkas) + [selfdrive/car/hyundai/interface.py:168] + -> controlsd.update_clearpilot_events(CS) + [selfdrive/controls/controlsd.py:1235-1239] + -> events.add(EventName.clpDebug) + -> controlsd.clearpilot_state_control(CC, CS) + [selfdrive/controls/controlsd.py:1241-1258] + -> Toggles ScreenDisaplayMode param (0=on, 1=off) in /dev/shm/params + -> UI reads ScreenDisaplayMode in drawHud() + [selfdrive/ui/qt/onroad.cc:390-403] + -> mode=1 and no alert: Hardware::set_display_power(false) + -> mode=0 or alert visible: Hardware::set_display_power(true) +``` + +### Current Behavior + +- Each press toggles the display on/off instantly (debug alert suppressed) +- `ScreenDisaplayMode` is in-memory params (`/dev/shm/params`), resets on reboot +- `max_display_mode = 1` — currently only two states (on/off); can be extended for future modes + +### Key Files + +| File | Role | +|------|------| +| `selfdrive/car/hyundai/carstate.py` | Reads LFA_BTN from CAN-FD | +| `selfdrive/car/hyundai/interface.py` | Creates ButtonEvent with FrogPilotButtonType.lkas | +| `selfdrive/controls/controlsd.py` | Fires clpDebug event, toggles ScreenDisaplayMode | +| `selfdrive/controls/lib/events.py` | clpDebug event definition (alert suppressed) | +| `selfdrive/ui/qt/onroad.cc` | Reads ScreenDisaplayMode, controls display power | + +## Offroad UI + +The offroad home screen (`selfdrive/ui/qt/home.cc`) was replaced with a clean grid launcher. Stock FrogPilot widgets (date, version, update/alert notifications) were removed. + +- **Settings button**: opens the original comma/FrogPilot settings (backdoor to all original settings) +- **Dashcam button**: placeholder for future dashcam footage viewer +- Tapping the splash screen (ReadyWindow) goes directly to the grid launcher (no sidebar) +- Sidebar with metrics (TEMP, VEHICLE, CONNECT) is hidden but still accessible via settings path ## Device: comma 3x @@ -126,12 +200,14 @@ The dashcam is the FrogPilot `ScreenRecorder` — it captures the onroad UI scre - Qualcomm Snapdragon SoC (aarch64) - Serial: comma-3889765b +- Storage: WDC SDINDDH4-128G, 128 GB UFS 2.1 - Connects to the car via comma panda (CAN bus interface) ### Operating System - **Ubuntu 20.04.6 LTS (Focal Fossa)** on aarch64 - **Kernel**: 4.9.103+ (custom comma.ai PREEMPT build, Feb 2024) — very old, vendor-patched Qualcomm kernel +- **Python**: 3.11.4 via pyenv at `/usr/local/pyenv/versions/3.11.4/` (system python is 3.8, do not use) - **AGNOS version**: 9.7 (comma's custom OS layer on top of Ubuntu) - **Display server**: Weston (Wayland compositor) on tty1 - **SELinux**: mounted (enforcement status varies) @@ -155,23 +231,16 @@ The dashcam is the FrogPilot `ScreenRecorder` — it captures the onroad UI scre | `/cache` | /dev/sda11 | ext4 | Android-style cache partition | | `/dsp` | /dev/sde26 | ext4 | **Read-only** Qualcomm DSP firmware | | `/firmware` | /dev/sde4 | vfat | **Read-only** firmware blobs | -| `/data/media`| NVMe | auto | 69 GB dashcam video storage | -### Unusual Packages / Services +### Hardware Encoding -- `vnstat` — network traffic monitor -- `cdsprpcd`, `qseecomd` — Qualcomm DSP and secure execution daemons -- `tftp_server` — TFTP server running (Qualcomm firmware access) -- armhf multiarch libraries present (32-bit binary support) -- Reverse SSH tunnel running via screen (`/data/brian/reverse_ssh.sh`) -- `phantom_touch_logger.py`, `power_drop_monitor.py` — comma hardware monitors +- **OMX**: `OMX.qcom.video.encoder.avc` (H.264) and `OMX.qcom.video.encoder.hevc` — used by dashcamd and screen recorder +- **V4L2**: Qualcomm VIDC at `/dev/v4l/by-path/platform-aa00000.qcom_vidc-video-index1` — used by encoderd (now disabled). Not accessible from ffmpeg due to permission/driver issues +- **ffmpeg**: v4.2.2, has `h264_v4l2m2m` and `h264_omx` listed but neither works from ffmpeg subprocess (OMX port issues, V4L2 device not found). Use OMX directly via the C++ encoder -### Large Files on Device +### Fan Control -- `/data/media` — ~69 GB (dashcam video segments) -- `/data/scons_cache` — ~1.9 GB (build cache) -- `/data/safe_staging` — ~1.5 GB (OTA update staging) -- Model files in repo: ~238 MB total (see models section below) +Software-controlled via `thermald` -> `fan_controller.py` -> panda USB -> PWM. Target temp 70°C, PI+feedforward controller. See Standstill Power Saving section for standstill-aware clamps. ## Boot Sequence @@ -181,13 +250,16 @@ Power On -> /usr/comma/comma.sh (waits for Weston, handles factory reset) -> /data/continue.sh (exec bridge to openpilot) -> /data/openpilot/launch_openpilot.sh - -> Sources launch_env.sh (thread counts, AGNOS_VERSION) - -> Runs agnos_init (marks boot slot, GPU perms, checks OS update) - -> Sets PYTHONPATH, symlinks /data/pythonpath - -> Runs build.py if no `prebuilt` marker - -> Launches selfdrive/manager/manager.py - -> manager_init() sets default params - -> ensure_running() loop starts all managed processes + -> Kills other instances of itself and manager.py + -> Runs on_start.sh (logo, reverse SSH) + -> exec launch_chffrplus.sh + -> Sources launch_env.sh (thread counts, AGNOS_VERSION) + -> Runs agnos_init (marks boot slot, GPU perms, checks OS update) + -> Sets PYTHONPATH, symlinks /data/pythonpath + -> Runs build.py if no `prebuilt` marker + -> Launches selfdrive/manager/manager.py + -> manager_init() sets default params + -> ensure_running() loop starts all managed processes ``` ## Openpilot Architecture @@ -198,73 +270,43 @@ Power On ### Always-Running Processes (offroad + onroad) -- `thermald` — thermal management, high CPU (~5.6%) +- `thermald` — thermal management and fan control - `pandad` — panda CAN bus interface - `ui` — Qt-based onroad/offroad UI -- `deleter` — storage cleanup +- `deleter` — storage cleanup (9 GB threshold) - `statsd`, `timed`, `logmessaged`, `tombstoned` — telemetry/logging - `manage_athenad` — comma cloud connectivity +- `fleet_manager`, `frogpilot_process` — FrogPilot additions ### Onroad-Only Processes (when driving) - `controlsd` — main vehicle control loop - `plannerd` — path planning - `radard` — radar processing -- `modeld` — driving model inference -- `dmonitoringmodeld` — driver monitoring model -- `locationd` — positioning/localization -- `calibrationd` — camera calibration -- `paramsd`, `torqued` — parameter estimation +- `modeld` — driving model inference (throttled to 1fps at standstill) +- `dmonitoringmodeld` — driver monitoring model (throttled to 1fps at standstill) +- `locationd`, `calibrationd`, `paramsd`, `torqued` — localization and calibration - `sensord` — IMU/sensor data - `soundd` — alert sounds - `camerad` — camera capture -- `loggerd`, `encoderd` — video logging/encoding -- `boardd` — board communication +- `loggerd` — CAN/sensor log recording (video encoding disabled) + +### ClearPilot Processes + +- `dashcamd` — raw camera dashcam recording (runs onroad or with DashcamDebug flag) ### GPS - `ubloxd` + `pigeond` for u-blox GPS hardware - `qcomgpsd`, `ugpsd`, `navd` currently **commented out** in process_config -### FrogPilot Additions - -- `selfdrive/frogpilot/frogpilot_process.py` — FrogPilot main process -- `selfdrive/frogpilot/controls/` — custom planner, control libraries -- `selfdrive/frogpilot/fleetmanager/` — fleet management web UI -- `selfdrive/frogpilot/screenrecorder/` — C++ OMX-based screen recorder (dashcam) -- `selfdrive/frogpilot/ui/qt/` — custom UI widgets and settings panels -- `selfdrive/frogpilot/assets/` — custom assets - -### ClearPilot Additions - -- `selfdrive/clearpilot/clearpilot.js` — Node.js service -- `selfdrive/clearpilot/behavior/` — lane change, longitudinal control mode scripts -- `selfdrive/clearpilot/models/` — custom driving models (duck-amigo, farmville, wd-40) -- `selfdrive/clearpilot/manager/api.js` — manager API -- `selfdrive/clearpilot/theme/`, `selfdrive/clearpilot/resource/` — theming and assets - -### Car Interface (Hyundai) - -- `selfdrive/car/hyundai/interface.py` — main car interface -- `selfdrive/car/hyundai/carcontroller.py` — actuator commands -- `selfdrive/car/hyundai/carstate.py` — vehicle state parsing -- `selfdrive/car/hyundai/radar_interface.py` — radar data -- `selfdrive/car/hyundai/hyundaicanfd.py` — CAN-FD message definitions (HDA2 uses CAN-FD) -- `selfdrive/car/hyundai/values.py`, `fingerprints.py` — car-specific constants - -### UI Code - -- `selfdrive/ui/` — Qt/C++ based -- `selfdrive/ui/qt/onroad.cc` — main driving screen (contains uiDebug publish, screen recorder integration) -- `selfdrive/ui/qt/home.cc` — home/offroad screen -- `selfdrive/ui/qt/sidebar.cc` — sidebar -- `selfdrive/ui/ui.cc`, `ui.h` — UI state management - ### Key Dependencies -- **Python 3.11** with: numpy, casadi, onnx/onnxruntime, pycapnp, pyzmq, sentry-sdk, sympy, Cython +- **Python 3.11** (via pyenv) with: numpy, casadi, onnx/onnxruntime, pycapnp, pyzmq, sentry-sdk, sympy, Cython - **capnp (Cap'n Proto)** — IPC message serialization between all processes - **ZeroMQ** — IPC transport layer -- **Qt 5** — UI framework -- **OpenMAX (OMX)** — hardware video encoding (screen recorder) +- **Qt 5** — UI framework (with WebEngine available but not used for rotation reasons) +- **OpenMAX (OMX)** — hardware video encoding +- **libavformat** — MP4 container muxing +- **libyuv** — color space conversion - **SCons** — build system for native C++ components diff --git a/GOALS.md b/GOALS.md new file mode 100644 index 0000000..fb80758 --- /dev/null +++ b/GOALS.md @@ -0,0 +1,46 @@ +# ClearPilot — Feature Goals + +Each goal below will be discussed in detail before implementation. Background and specifics to be provided by Brian when we get to each item. + +## Dashcam + +- [ ] **Dashcam viewer** — on-screen Qt widget for browsing and playing back recorded footage from the offroad home screen (Dashcam button already placed). See `DASHCAM_PROJECT.md` for architecture notes. +- [ ] **Dashcam uploader** — upload footage to a remote server or cloud storage +- [ ] **GPS + speed overlay on dashcam footage** — embed GPS coordinates and vehicle speed into the video stream or as metadata +- [ ] **Dashcam footage archive** — option to mark a trip's footage as archived (kept in storage but hidden from the main UI) +- [ ] **Suspend dashcam / route recording** — on-screen setting to pause recording for a trip and delete footage of the current trip so far (privacy mode) + +## Driving Alerts & Safety + +- [ ] **Chirp on speed change and over speed limit** — audible alert when speed limit changes or vehicle exceeds the limit +- [ ] **Warn on traffic standstill ahead** — alert when approaching cars ahead too quickly (closing distance warning) +- [ ] **On-screen current speed limit** — display the speed limit as reported by the car's CAN data +- [ ] **Fix on-screen speed value for MPH** — correct the speed display to use the proper value for MPH conversion + +## Cruise Control & Driving State + +- [ ] **Auto-set speed via CAN bus** — simulate pressing speed up/down buttons on CAN bus to automatically set cruise to the correct speed via UI interaction (stretch goal) +- [ ] **Fix engaged-while-braking state sync** — if openpilot is engaged while brakes are pressed, the system thinks it's active but the car isn't actually in cruise control, causing a sync error that should be recoverable +- [ ] **Fix resume-cruise lateral mode bug** — pressing resume cruise control (rather than enable) causes the car to enter a state where it thinks it's in always-on lateral mode and disables driver monitoring +- [ ] **Lateral assist in turn lanes** — keep lateral assist active when using turn signal in a turn lane (at low/turning speeds) as opposed to cruising speeds, where lateral currently disengages + +## Driver Monitoring + +- [ ] **Disable driver monitoring on demand** — add a technique to disable DM for 2 minutes via a button or gesture +- [ ] **Hands-on-wheel mode** — add a setting requiring the driver to keep hands on the wheel (configurable via settings) + +## UI Modes & Display + +- [ ] **Curves-only UI mode** — show only on-screen curve/path depictions without the camera feed (reduces visual clutter, saves processing) +- [ ] **Night mode auto-display-off** — turn off display if device starts at night (determine sunset for current location/datetime or use ambient light levels from car sensors) +- [ ] **Update boot/offroad splash logos** — replace the red pac-man ghost with a blue pac-man ghost for boot and offroad full-screen splash +- [ ] **Dashcam-only reboot mode** — on-screen option to reboot into a mode that provides no driver assist (reverts to stock OEM lane assist) but keeps dashcam running. For lending the car to friends who shouldn't use comma's driving features. + +## Connectivity & Data + +- [ ] **Offroad weather report screen** — weather display accessible from the offroad home grid (depends on internet connection) +- [ ] **Fix GPS tracking** — GPS tracking feature currently broken, processes commented out in process_config + +## Vehicle-Specific (Hyundai Tucson HDA2) + +- [ ] **Warn if panoramic roof is open** — if possible to intercept window/roof state from CAN bus, warn when the top roof is left open when the car is turned off diff --git a/common/params.cc b/common/params.cc index 9e216ef..da74e4d 100755 --- a/common/params.cc +++ b/common/params.cc @@ -109,6 +109,7 @@ std::unordered_map keys = { {"ControlsReady", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION}, {"CurrentBootlog", PERSISTENT}, {"CurrentRoute", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION}, + {"DashcamDebug", PERSISTENT}, {"DisableLogging", CLEAR_ON_MANAGER_START | CLEAR_ON_ONROAD_TRANSITION}, {"DisablePowerDown", PERSISTENT}, {"DisableUpdates", PERSISTENT}, diff --git a/launch_openpilot.sh b/launch_openpilot.sh index 8c2a08f..875b3aa 100755 --- a/launch_openpilot.sh +++ b/launch_openpilot.sh @@ -1,5 +1,15 @@ #!/usr/bin/bash +# Kill other instances of this script and any running manager +for pid in $(pgrep -f 'launch_openpilot.sh' | grep -v $$); do + kill "$pid" 2>/dev/null +done +for pid in $(pgrep -f 'launch_chffrplus.sh' | grep -v $$); do + kill "$pid" 2>/dev/null +done +pkill -f 'python.*manager.py' 2>/dev/null +sleep 1 + bash /data/openpilot/system/clearpilot/on_start.sh exec ./launch_chffrplus.sh diff --git a/selfdrive/SConscript b/selfdrive/SConscript index f9d1c88..a03a014 100755 --- a/selfdrive/SConscript +++ b/selfdrive/SConscript @@ -3,4 +3,5 @@ SConscript(['controls/lib/lateral_mpc_lib/SConscript']) SConscript(['controls/lib/longitudinal_mpc_lib/SConscript']) SConscript(['locationd/SConscript']) SConscript(['modeld/SConscript']) -SConscript(['ui/SConscript']) \ No newline at end of file +SConscript(['ui/SConscript']) +SConscript(['clearpilot/SConscript']) \ No newline at end of file diff --git a/selfdrive/clearpilot/SConscript b/selfdrive/clearpilot/SConscript new file mode 100644 index 0000000..2a58a7e --- /dev/null +++ b/selfdrive/clearpilot/SConscript @@ -0,0 +1,16 @@ +Import('env', 'arch', 'common', 'messaging', 'visionipc', 'cereal') + +clearpilot_env = env.Clone() +clearpilot_env['CPPPATH'] += ['#selfdrive/frogpilot/screenrecorder/openmax/include/'] +# Disable --as-needed so static lib ordering doesn't matter +clearpilot_env['LINKFLAGS'] = [f for f in clearpilot_env.get('LINKFLAGS', []) if f != '-Wl,--as-needed'] + +if arch == "larch64": + omx_obj = File('#selfdrive/frogpilot/screenrecorder/omx_encoder.o') + clearpilot_env.Program( + 'dashcamd', + ['dashcamd.cc', omx_obj], + LIBS=[common, 'json11', cereal, visionipc, messaging, + 'zmq', 'capnp', 'kj', 'm', 'OpenCL', 'ssl', 'crypto', 'pthread', + 'OmxCore', 'avformat', 'avcodec', 'avutil', 'yuv'] + ) diff --git a/selfdrive/clearpilot/dashcamd.cc b/selfdrive/clearpilot/dashcamd.cc new file mode 100644 index 0000000..12b0df5 --- /dev/null +++ b/selfdrive/clearpilot/dashcamd.cc @@ -0,0 +1,140 @@ +/* + * CLEARPILOT dashcamd — records raw camera footage to MP4 using OMX H.264 hardware encoder. + * + * Connects to camerad via VisionIPC, receives NV12 frames, and feeds them directly + * to the Qualcomm OMX encoder. Produces 3-minute MP4 segments in /data/media/0/videos/. + * + * Suspends recording after 10 minutes of standstill, resumes when car moves. + */ + +#include +#include +#include +#include +#include +#include + +#include "cereal/messaging/messaging.h" +#include "cereal/visionipc/visionipc_client.h" +#include "common/params.h" +#include "common/timing.h" +#include "common/swaglog.h" +#include "common/util.h" +#include "selfdrive/frogpilot/screenrecorder/omx_encoder.h" + +const std::string VIDEOS_DIR = "/data/media/0/videos"; +const int SEGMENT_SECONDS = 180; // 3 minutes +const int CAMERA_FPS = 20; +const int FRAMES_PER_SEGMENT = SEGMENT_SECONDS * CAMERA_FPS; +const int BITRATE = 4 * 1024 * 1024; // 4 Mbps +const double STANDSTILL_TIMEOUT_SECONDS = 600.0; // 10 minutes + +ExitHandler do_exit; + +static std::string make_filename() { + char buf[64]; + time_t t = time(NULL); + struct tm tm = *localtime(&t); + snprintf(buf, sizeof(buf), "%04d%02d%02d-%02d%02d%02d.mp4", + tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, + tm.tm_hour, tm.tm_min, tm.tm_sec); + return std::string(buf); +} + +int main(int argc, char *argv[]) { + setpriority(PRIO_PROCESS, 0, -10); + + // Ensure output directory exists + mkdir(VIDEOS_DIR.c_str(), 0755); + + LOGW("dashcamd: connecting to camerad road stream"); + + VisionIpcClient vipc("camerad", VISION_STREAM_ROAD, false); + while (!do_exit && !vipc.connect(false)) { + usleep(100000); + } + if (do_exit) return 0; + + int width = vipc.buffers[0].width; + int height = vipc.buffers[0].height; + int y_stride = vipc.buffers[0].stride; + int uv_stride = y_stride; + + LOGW("dashcamd: connected %dx%d, stride=%d", width, height, y_stride); + + // Subscribe to carState for standstill detection + SubMaster sm({"carState"}); + + // Create encoder — H.264, no downscale, with MP4 remuxing (h265=false) + OmxEncoder encoder(VIDEOS_DIR.c_str(), width, height, CAMERA_FPS, BITRATE, false, false); + + int frame_count = FRAMES_PER_SEGMENT; // force new segment on first frame + uint64_t start_ts = 0; + bool recording = false; + bool suspended = false; + double standstill_start = 0.0; + + while (!do_exit) { + VisionBuf *buf = vipc.recv(); + if (buf == nullptr) continue; + + // Check standstill state + sm.update(0); + bool is_standstill = sm.valid("carState") && sm["carState"].getCarState().getStandstill(); + double now = nanos_since_boot() / 1e9; + + if (is_standstill) { + if (standstill_start == 0.0) { + standstill_start = now; + } + // Suspend after 10 minutes of continuous standstill + if (!suspended && (now - standstill_start) >= STANDSTILL_TIMEOUT_SECONDS) { + LOGW("dashcamd: suspending — standstill for 10 minutes"); + if (recording) { + encoder.encoder_close(); + recording = false; + frame_count = FRAMES_PER_SEGMENT; + } + suspended = true; + } + } else { + standstill_start = 0.0; + if (suspended) { + LOGW("dashcamd: resuming — car moving"); + suspended = false; + } + } + + if (suspended) continue; + + // Start new segment if needed + if (frame_count >= FRAMES_PER_SEGMENT) { + if (recording) { + encoder.encoder_close(); + } + + std::string filename = make_filename(); + LOGW("dashcamd: opening segment %s", filename.c_str()); + encoder.encoder_open(filename.c_str()); + frame_count = 0; + start_ts = nanos_since_boot(); + recording = true; + } + + uint64_t ts = nanos_since_boot() - start_ts; + + // Feed NV12 frame directly to OMX encoder + uint8_t *y_ptr = buf->y; + uint8_t *uv_ptr = buf->uv; + encoder.encode_frame_nv12(y_ptr, y_stride, uv_ptr, uv_stride, width, height, ts); + + frame_count++; + } + + if (recording) { + encoder.encoder_close(); + } + LOGW("dashcamd: stopped"); + + return 0; +} diff --git a/selfdrive/clearpilot/dashcamd.py b/selfdrive/clearpilot/dashcamd.py new file mode 100644 index 0000000..2e9765a --- /dev/null +++ b/selfdrive/clearpilot/dashcamd.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python3 +""" +CLEARPILOT dashcamd — records raw camera footage to MP4 using hardware H.264 encoder. + +Connects directly to camerad via VisionIPC, receives NV12 frames, and pipes them +to ffmpeg's h264_v4l2m2m encoder. Produces 3-minute MP4 segments in /data/media/0/videos/. + +This replaces the FrogPilot screen recorder approach (QWidget::grab -> OMX) with a +direct camera capture that works regardless of UI state (screen off, alternate modes, etc). +""" +import os +import time +import subprocess +import signal +from pathlib import Path +from datetime import datetime + +from cereal.visionipc import VisionIpcClient, VisionStreamType +from openpilot.common.params import Params +from openpilot.common.swaglog import cloudlog +from openpilot.selfdrive import sentry + +PROCESS_NAME = "selfdrive.clearpilot.dashcamd" +VIDEOS_DIR = "/data/media/0/videos" +SEGMENT_SECONDS = 180 # 3 minutes +CAMERA_FPS = 20 +FRAMES_PER_SEGMENT = SEGMENT_SECONDS * CAMERA_FPS + + +def make_filename(): + return datetime.now().strftime("%Y%m%d-%H%M%S") + ".mp4" + + +def open_encoder(width, height, filepath): + """Start an ffmpeg subprocess that accepts raw NV12 on stdin and writes MP4.""" + cmd = [ + "ffmpeg", "-y", "-nostdin", "-loglevel", "error", + "-f", "rawvideo", + "-pix_fmt", "nv12", + "-s", f"{width}x{height}", + "-r", str(CAMERA_FPS), + "-i", "pipe:0", + "-c:v", "h264_v4l2m2m", + "-b:v", "4M", + "-f", "mp4", + "-movflags", "+faststart", + filepath, + ] + return subprocess.Popen(cmd, stdin=subprocess.PIPE) + + +def main(): + sentry.set_tag("daemon", PROCESS_NAME) + cloudlog.bind(daemon=PROCESS_NAME) + + os.makedirs(VIDEOS_DIR, exist_ok=True) + + params = Params() + + # Connect to camerad road stream + cloudlog.info("dashcamd: connecting to camerad road stream") + vipc = VisionIpcClient("camerad", VisionStreamType.VISION_STREAM_ROAD, False) + while not vipc.connect(False): + time.sleep(0.1) + + width, height = vipc.width, vipc.height + # NV12 frame: Y plane (w*h) + UV plane (w*h/2) + frame_size = width * height * 3 // 2 + cloudlog.info(f"dashcamd: connected, {width}x{height}, frame_size={frame_size}") + + frame_count = 0 + encoder = None + lock_path = None + + try: + while True: + buf = vipc.recv() + if buf is None: + continue + + # Start new segment if needed + if encoder is None or frame_count >= FRAMES_PER_SEGMENT: + # Close previous segment + if encoder is not None: + encoder.stdin.close() + encoder.wait() + if lock_path and os.path.exists(lock_path): + os.remove(lock_path) + cloudlog.info(f"dashcamd: closed segment, {frame_count} frames") + + # Open new segment + filename = make_filename() + filepath = os.path.join(VIDEOS_DIR, filename) + lock_path = filepath + ".lock" + Path(lock_path).touch() + + cloudlog.info(f"dashcamd: opening segment {filename}") + encoder = open_encoder(width, height, filepath) + frame_count = 0 + + # Write raw NV12 frame to ffmpeg stdin + try: + encoder.stdin.write(buf.data[:frame_size]) + frame_count += 1 + except BrokenPipeError: + cloudlog.error("dashcamd: encoder pipe broken, restarting segment") + encoder = None + + except (KeyboardInterrupt, SystemExit): + pass + finally: + if encoder is not None: + encoder.stdin.close() + encoder.wait() + if lock_path and os.path.exists(lock_path): + os.remove(lock_path) + cloudlog.info("dashcamd: stopped") + + +if __name__ == "__main__": + main() diff --git a/selfdrive/controls/controlsd.py b/selfdrive/controls/controlsd.py index 8da37b2..09c25ce 100755 --- a/selfdrive/controls/controlsd.py +++ b/selfdrive/controls/controlsd.py @@ -1233,7 +1233,7 @@ class Controls: self.frogpilot_variables.use_ev_tables = self.params.get_bool("EVTable") def update_clearpilot_events(self, CS): - if (len(CS.buttonEvents) > 0): + if (len(CS.buttonEvents) > 0): print (CS.buttonEvents) if any(be.pressed and be.type == FrogPilotButtonType.lkas for be in CS.buttonEvents): self.events.add(EventName.clpDebug) diff --git a/selfdrive/controls/lib/events.py b/selfdrive/controls/lib/events.py index 6d23406..d0805fc 100755 --- a/selfdrive/controls/lib/events.py +++ b/selfdrive/controls/lib/events.py @@ -778,8 +778,8 @@ EVENTS: dict[int, dict[str, Alert | AlertCallbackType]] = { ET.SOFT_DISABLE: soft_disable_alert("Sensor Data Invalid"), }, + # CLEARPILOT: alert suppressed — event still fires for screen toggle and future actions EventName.clpDebug: { - ET.PERMANENT: clp_debug_notice, }, EventName.noGps: { diff --git a/selfdrive/frogpilot/screenrecorder/omx_encoder.cc b/selfdrive/frogpilot/screenrecorder/omx_encoder.cc index 332e541..4a037fe 100755 --- a/selfdrive/frogpilot/screenrecorder/omx_encoder.cc +++ b/selfdrive/frogpilot/screenrecorder/omx_encoder.cc @@ -565,6 +565,60 @@ int OmxEncoder::encode_frame_rgba(const uint8_t *ptr, int in_width, int in_heigh return ret; } +// CLEARPILOT: encode raw NV12 frames directly (no RGBA conversion needed) +int OmxEncoder::encode_frame_nv12(const uint8_t *y_ptr, int y_stride, const uint8_t *uv_ptr, int uv_stride, + int in_width, int in_height, uint64_t ts) { + if (!this->is_open) { + return -1; + } + + OMX_BUFFERHEADERTYPE* in_buf = nullptr; + while (!this->free_in.try_pop(in_buf, 20)) { + if (do_exit) { + return -1; + } + } + + int ret = this->counter; + + uint8_t *in_buf_ptr = in_buf->pBuffer; + int venus_y_stride = VENUS_Y_STRIDE(COLOR_FMT_NV12, this->width); + int venus_uv_stride = VENUS_UV_STRIDE(COLOR_FMT_NV12, this->width); + uint8_t *dst_y = in_buf_ptr; + uint8_t *dst_uv = in_buf_ptr + (venus_y_stride * VENUS_Y_SCANLINES(COLOR_FMT_NV12, this->height)); + + // Copy Y plane row by row (source stride may differ from VENUS stride) + for (int row = 0; row < in_height; row++) { + memcpy(dst_y + row * venus_y_stride, y_ptr + row * y_stride, in_width); + } + // Copy UV plane row by row + int uv_height = in_height / 2; + for (int row = 0; row < uv_height; row++) { + memcpy(dst_uv + row * venus_uv_stride, uv_ptr + row * uv_stride, in_width); + } + + in_buf->nFilledLen = VENUS_BUFFER_SIZE(COLOR_FMT_NV12, this->width, this->height); + in_buf->nFlags = OMX_BUFFERFLAG_ENDOFFRAME; + in_buf->nOffset = 0; + in_buf->nTimeStamp = ts / 1000LL; + this->last_t = in_buf->nTimeStamp; + + OMX_CHECK(OMX_EmptyThisBuffer(this->handle, in_buf)); + + while (true) { + OMX_BUFFERHEADERTYPE *out_buf; + if (!this->done_out.try_pop(out_buf)) { + break; + } + handle_out_buf(this, out_buf); + } + + this->dirty = true; + this->counter++; + + return ret; +} + void OmxEncoder::encoder_open(const char* filename) { int err; diff --git a/selfdrive/frogpilot/screenrecorder/omx_encoder.h b/selfdrive/frogpilot/screenrecorder/omx_encoder.h index 5cb630c..d425e38 100755 --- a/selfdrive/frogpilot/screenrecorder/omx_encoder.h +++ b/selfdrive/frogpilot/screenrecorder/omx_encoder.h @@ -19,6 +19,8 @@ public: ~OmxEncoder(); int encode_frame_rgba(const uint8_t *ptr, int in_width, int in_height, uint64_t ts); + int encode_frame_nv12(const uint8_t *y_ptr, int y_stride, const uint8_t *uv_ptr, int uv_stride, + int in_width, int in_height, uint64_t ts); void encoder_open(const char* filename); void encoder_close(); diff --git a/selfdrive/manager/manager.py b/selfdrive/manager/manager.py index 7983e4d..8ee32e6 100755 --- a/selfdrive/manager/manager.py +++ b/selfdrive/manager/manager.py @@ -135,6 +135,7 @@ def manager_init(frogpilot_functions) -> None: ("DisableOpenpilotLongitudinal", "0"), ("DisableVTSCSmoothing", "0"), ("DisengageVolume", "100"), + ("DashcamDebug", "1"), ("DragonPilotTune", "0"), ("DriverCamera", "0"), ("DynamicPathWidth", "0"), diff --git a/selfdrive/manager/process_config.py b/selfdrive/manager/process_config.py index 948646c..6fe78cb 100755 --- a/selfdrive/manager/process_config.py +++ b/selfdrive/manager/process_config.py @@ -51,6 +51,10 @@ def allow_uploads(started, params, CP: car.CarParams) -> bool: allow_uploads = not (params.get_bool("DeviceManagement") and params.get_bool("NoUploads")) return allow_uploads +# ClearPilot functions +def dashcam_should_run(started, params, CP: car.CarParams) -> bool: + return started or params.get_bool("DashcamDebug") + procs = [ DaemonProcess("manage_athenad", "selfdrive.athena.manage_athenad", "AthenadPid"), @@ -102,6 +106,9 @@ procs = [ # FrogPilot processes PythonProcess("fleet_manager", "selfdrive.frogpilot.fleetmanager.fleet_manager", always_run), PythonProcess("frogpilot_process", "selfdrive.frogpilot.frogpilot_process", always_run), + + # ClearPilot processes + NativeProcess("dashcamd", "selfdrive/clearpilot", ["./dashcamd"], dashcam_should_run), ] managed_processes = {p.name: p for p in procs} \ No newline at end of file diff --git a/selfdrive/ui/qt/onroad.cc b/selfdrive/ui/qt/onroad.cc index 783f3a6..a9903bc 100755 --- a/selfdrive/ui/qt/onroad.cc +++ b/selfdrive/ui/qt/onroad.cc @@ -919,14 +919,14 @@ void AnnotatedCameraWidget::initializeFrogPilotWidgets() { animationFrameIndex = (animationFrameIndex + 1) % totalFrames; }); - // CLEARPILOT: screen recorder timer — feeds frames to the OMX encoder - QTimer *record_timer = new QTimer(this); - connect(record_timer, &QTimer::timeout, this, [this]() { - if (recorder_btn) { - recorder_btn->update_screen(); - } - }); - record_timer->start(1000 / UI_FREQ); + // CLEARPILOT: screen recorder disabled — replaced by dedicated dashcamd process + // QTimer *record_timer = new QTimer(this); + // connect(record_timer, &QTimer::timeout, this, [this]() { + // if (recorder_btn) { + // recorder_btn->update_screen(); + // } + // }); + // record_timer->start(1000 / UI_FREQ); } void AnnotatedCameraWidget::updateFrogPilotWidgets() {