diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..b1af24e --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,270 @@ +# ClearPilot — CLAUDE.md + +## Project Overview + +ClearPilot is a custom fork of **FrogPilot** (itself a fork of comma.ai's openpilot), based on a 2024 release. It is purpose-built for Brian Hanson's **Hyundai Tucson** (HDA2 equipped). The vehicle's HDA2 system has specific quirks around how it synchronizes driving state with openpilot that require careful handling. + +### Key Customizations in This Fork + +- **UI changes** to the onroad driving interface +- **Lane change behavior**: brief disengage when turn signal is active during lane changes +- **Lateral control disabled**: the car's own radar cruise control handles lateral; openpilot handles longitudinal only +- **Driver monitoring timeouts**: modified safety timeouts for the driver monitoring system +- **Custom driving models**: `duck-amigo.thneed`, `farmville.onnx`, `wd-40.thneed` in `selfdrive/clearpilot/models/` +- **ClearPilot service**: Node.js service at `selfdrive/clearpilot/` with behavior scripts for lane change and longitudinal control + +### Short-Term Goals + +- ~~Fix the dashcam/screen recorder feature~~ (done — see Dashcam section below) +- Fix GPS tracking feature +- Add a safe-speed-exceeded chime +- Implement interactions for the "debug function button" + +## Working Rules + +### CRITICAL: Change Control + +This is self-driving software. All changes must be deliberate and well-understood. + +- **NEVER make changes outside of what is explicitly requested** +- **Always explain proposed changes first** — describe the change, the logic, and the architecture; let Brian review and approve before writing any code +- **Brian is an expert on this software** — do not override his judgment or assume responsibility for changes he doesn't understand +- **Every line must be understood** — work slowly and deliberately +- **Test everything thoroughly** — Brian must always be in the loop +- **Do not refactor, clean up, or "improve" code beyond the specific request** + +### File Ownership + +We operate as `root` on this device, but openpilot runs as the `comma` user (uid=1000, gid=1000). After any code changes that touch multiple files or before testing: + +```bash +chown -R comma:comma /data/openpilot +``` + +### Git + +- Remote: `git@git.internal.hanson.xyz:brianhansonxyz/comma.git` +- Branch: `clearpilot` +- Large model files are tracked in git (intentional — this is a backup) + +### Samba Share + +- Share name: `openpilot` (e.g. `\\comma-3889765b\openpilot`) +- Path: `/data/openpilot` +- Username: `comma` +- Password: `i-like-to-drive-cars` +- Runs as `comma:comma` via force user/group — files created over SMB are owned correctly +- Enabled at boot (`smbd` + `nmbd`) + +### Testing Changes + +To restart the full openpilot stack after making changes: + +```bash +# Fix ownership first (we edit as root, openpilot runs as comma) +chown -R comma:comma /data/openpilot + +# Kill existing stack — use pgrep/kill to avoid pkill matching our own shell +kill $(pgrep -f 'python.*manager') 2>/dev/null; sleep 2 + +# Must use a login shell as comma — sudo -u won't set up the right Python/env +su - comma -c "bash /data/openpilot/launch_openpilot.sh" +``` + +Note: `pkill -f manager` or `pkill -f launch_openpilot` will match the invoking shell's own command line and kill the launch process itself. Use `pgrep`+`kill` or run the kill as a separate step before launching. + +**Adding new params**: The params system uses a C++ whitelist. Adding a new param name in `manager.py` alone will crash with `UnknownKeyName`. You must also register the key in `common/params.cc` in the key list (alphabetically, with `PERSISTENT` or `CLEAR_ON_*` flag), then rebuild. The rebuild will recompile `params.cc` -> `libcommon.a` and re-link all binaries that use it. + +The `prebuilt` marker file skips compilation. It is **recreated automatically** after each successful build, so you must remove it every time you want to recompile: +```bash +rm -f /data/openpilot/prebuilt +``` + +## Dashcam / Screen Recorder + +### Architecture + +The dashcam is the FrogPilot `ScreenRecorder` — it captures the onroad UI screen (with overlays) and encodes to MP4 using the Qualcomm OMX H.264 hardware encoder. + +- **Codec**: H.264 AVC (hardware accelerated via `OMX.qcom.video.encoder.avc`) +- **Resolution**: 1440x720 (downscaled from 2160x1080) +- **Bitrate**: 2 Mbps +- **Container**: MP4 +- **Segment length**: 3 minutes per file +- **Save path**: `/data/media/0/videos/YYYYMMDD-HHMMSS.mp4` + +### Changes Made (2026-04-11) + +1. **Disabled comma training data video** (`selfdrive/manager/process_config.py`): commented out `encoderd` and `stream_encoderd`. CAN/sensor logs (`rlog`/`qlog`) are still recorded by `loggerd`. + +2. **Re-enabled screen recorder** (`selfdrive/ui/qt/onroad.cc`): uncommented the `QTimer` that feeds frames to the encoder at UI_FREQ rate. + +3. **Auto-start recording** (`selfdrive/frogpilot/screenrecorder/screenrecorder.cc`): modified `update_screen()` to automatically start recording when the car is on (`scene.started`) and stop when off. No button press needed. + +4. **Hidden UI elements**: the record button is constructed but never made visible or added to layout. Recording is invisible to the driver. + +5. **Debug flag** (`ScreenRecorderDebug` param): when set to `"1"`, recording starts even without a car connected. Used for bench testing. Read via `scene.screen_recorder_debug` in `ui.h`/`ui.cc`. + +6. **Deleter updated** (`system/loggerd/deleter.py`): free space threshold raised from 5 GB to 9 GB. Oldest videos in `/data/media/0/videos/` are deleted first before falling back to log segments. + +### Key Files + +| File | Role | +|------|------| +| `selfdrive/frogpilot/screenrecorder/screenrecorder.cc` | Screen capture and auto-start/stop logic | +| `selfdrive/frogpilot/screenrecorder/omx_encoder.cc` | OMX H.264 hardware encoder wrapper | +| `selfdrive/ui/qt/onroad.cc` | Timer that drives frame capture | +| `selfdrive/ui/ui.h` | `screen_recorder_debug` scene flag | +| `selfdrive/ui/ui.cc` | Reads `ScreenRecorderDebug` param | +| `selfdrive/manager/manager.py` | Default params | +| `selfdrive/manager/process_config.py` | encoderd disabled here | +| `system/loggerd/deleter.py` | Storage rotation (9 GB threshold, videos + logs) | + +## Device: comma 3x + +### Hardware + +- Qualcomm Snapdragon SoC (aarch64) +- Serial: comma-3889765b +- Connects to the car via comma panda (CAN bus interface) + +### Operating System + +- **Ubuntu 20.04.6 LTS (Focal Fossa)** on aarch64 +- **Kernel**: 4.9.103+ (custom comma.ai PREEMPT build, Feb 2024) — very old, vendor-patched Qualcomm kernel +- **AGNOS version**: 9.7 (comma's custom OS layer on top of Ubuntu) +- **Display server**: Weston (Wayland compositor) on tty1 +- **SELinux**: mounted (enforcement status varies) + +### Users + +- `comma` (uid=1000) — the user openpilot runs as; member of root, sudo, disk, gpu, gpio groups +- `root` — what we SSH in as; files must be chowned back to comma before running openpilot + +### Filesystem / Mount Quirks + +| Mount | Device | Type | Notes | +|-------------|-------------|---------|-------| +| `/` | /dev/sda7 | ext4 | Root filesystem, read-write | +| `/data` | /dev/sda12 | ext4 | **Persistent**. Openpilot lives here. Survives reboots. | +| `/home` | overlay | overlayfs | **VOLATILE** — upperdir on tmpfs, changes lost on reboot | +| `/tmp` | tmpfs | tmpfs | Volatile, 150 MB | +| `/var` | tmpfs | tmpfs | Volatile, 128 MB (fstab) / 1.5 GB (actual) | +| `/systemrw` | /dev/sda10 | ext4 | Writable system overlay, noexec | +| `/persist` | /dev/sda2 | ext4 | Persistent config/certs, noexec | +| `/cache` | /dev/sda11 | ext4 | Android-style cache partition | +| `/dsp` | /dev/sde26 | ext4 | **Read-only** Qualcomm DSP firmware | +| `/firmware` | /dev/sde4 | vfat | **Read-only** firmware blobs | +| `/data/media`| NVMe | auto | 69 GB dashcam video storage | + +### Unusual Packages / Services + +- `vnstat` — network traffic monitor +- `cdsprpcd`, `qseecomd` — Qualcomm DSP and secure execution daemons +- `tftp_server` — TFTP server running (Qualcomm firmware access) +- armhf multiarch libraries present (32-bit binary support) +- Reverse SSH tunnel running via screen (`/data/brian/reverse_ssh.sh`) +- `phantom_touch_logger.py`, `power_drop_monitor.py` — comma hardware monitors + +### Large Files on Device + +- `/data/media` — ~69 GB (dashcam video segments) +- `/data/scons_cache` — ~1.9 GB (build cache) +- `/data/safe_staging` — ~1.5 GB (OTA update staging) +- Model files in repo: ~238 MB total (see models section below) + +## Boot Sequence + +``` +Power On + -> systemd: comma.service (runs as comma user) + -> /usr/comma/comma.sh (waits for Weston, handles factory reset) + -> /data/continue.sh (exec bridge to openpilot) + -> /data/openpilot/launch_openpilot.sh + -> Sources launch_env.sh (thread counts, AGNOS_VERSION) + -> Runs agnos_init (marks boot slot, GPU perms, checks OS update) + -> Sets PYTHONPATH, symlinks /data/pythonpath + -> Runs build.py if no `prebuilt` marker + -> Launches selfdrive/manager/manager.py + -> manager_init() sets default params + -> ensure_running() loop starts all managed processes +``` + +## Openpilot Architecture + +### Process Manager + +`selfdrive/manager/manager.py` orchestrates all processes defined in `selfdrive/manager/process_config.py`. + +### Always-Running Processes (offroad + onroad) + +- `thermald` — thermal management, high CPU (~5.6%) +- `pandad` — panda CAN bus interface +- `ui` — Qt-based onroad/offroad UI +- `deleter` — storage cleanup +- `statsd`, `timed`, `logmessaged`, `tombstoned` — telemetry/logging +- `manage_athenad` — comma cloud connectivity + +### Onroad-Only Processes (when driving) + +- `controlsd` — main vehicle control loop +- `plannerd` — path planning +- `radard` — radar processing +- `modeld` — driving model inference +- `dmonitoringmodeld` — driver monitoring model +- `locationd` — positioning/localization +- `calibrationd` — camera calibration +- `paramsd`, `torqued` — parameter estimation +- `sensord` — IMU/sensor data +- `soundd` — alert sounds +- `camerad` — camera capture +- `loggerd`, `encoderd` — video logging/encoding +- `boardd` — board communication + +### GPS + +- `ubloxd` + `pigeond` for u-blox GPS hardware +- `qcomgpsd`, `ugpsd`, `navd` currently **commented out** in process_config + +### FrogPilot Additions + +- `selfdrive/frogpilot/frogpilot_process.py` — FrogPilot main process +- `selfdrive/frogpilot/controls/` — custom planner, control libraries +- `selfdrive/frogpilot/fleetmanager/` — fleet management web UI +- `selfdrive/frogpilot/screenrecorder/` — C++ OMX-based screen recorder (dashcam) +- `selfdrive/frogpilot/ui/qt/` — custom UI widgets and settings panels +- `selfdrive/frogpilot/assets/` — custom assets + +### ClearPilot Additions + +- `selfdrive/clearpilot/clearpilot.js` — Node.js service +- `selfdrive/clearpilot/behavior/` — lane change, longitudinal control mode scripts +- `selfdrive/clearpilot/models/` — custom driving models (duck-amigo, farmville, wd-40) +- `selfdrive/clearpilot/manager/api.js` — manager API +- `selfdrive/clearpilot/theme/`, `selfdrive/clearpilot/resource/` — theming and assets + +### Car Interface (Hyundai) + +- `selfdrive/car/hyundai/interface.py` — main car interface +- `selfdrive/car/hyundai/carcontroller.py` — actuator commands +- `selfdrive/car/hyundai/carstate.py` — vehicle state parsing +- `selfdrive/car/hyundai/radar_interface.py` — radar data +- `selfdrive/car/hyundai/hyundaicanfd.py` — CAN-FD message definitions (HDA2 uses CAN-FD) +- `selfdrive/car/hyundai/values.py`, `fingerprints.py` — car-specific constants + +### UI Code + +- `selfdrive/ui/` — Qt/C++ based +- `selfdrive/ui/qt/onroad.cc` — main driving screen (contains uiDebug publish, screen recorder integration) +- `selfdrive/ui/qt/home.cc` — home/offroad screen +- `selfdrive/ui/qt/sidebar.cc` — sidebar +- `selfdrive/ui/ui.cc`, `ui.h` — UI state management + +### Key Dependencies + +- **Python 3.11** with: numpy, casadi, onnx/onnxruntime, pycapnp, pyzmq, sentry-sdk, sympy, Cython +- **capnp (Cap'n Proto)** — IPC message serialization between all processes +- **ZeroMQ** — IPC transport layer +- **Qt 5** — UI framework +- **OpenMAX (OMX)** — hardware video encoding (screen recorder) +- **SCons** — build system for native C++ components diff --git a/common/params.cc b/common/params.cc index 3c58d55..9e216ef 100755 --- a/common/params.cc +++ b/common/params.cc @@ -416,6 +416,7 @@ std::unordered_map keys = { {"ScreenBrightnessOnroad", PERSISTENT}, {"ScreenManagement", PERSISTENT}, {"ScreenRecorder", PERSISTENT}, + {"ScreenRecorderDebug", PERSISTENT}, {"ScreenTimeout", PERSISTENT}, {"ScreenTimeoutOnroad", PERSISTENT}, {"SearchInput", PERSISTENT}, diff --git a/selfdrive/frogpilot/screenrecorder/screenrecorder.cc b/selfdrive/frogpilot/screenrecorder/screenrecorder.cc index 21f3265..01f19ed 100755 --- a/selfdrive/frogpilot/screenrecorder/screenrecorder.cc +++ b/selfdrive/frogpilot/screenrecorder/screenrecorder.cc @@ -132,13 +132,20 @@ void ScreenRecorder::stop() { } void ScreenRecorder::update_screen() { - if (!uiState()->scene.started) { + bool car_on = uiState()->scene.started || uiState()->scene.screen_recorder_debug; + + if (!car_on) { if (recording) { stop(); } return; } - if (!recording) return; + + // CLEARPILOT: auto-start recording when car is on (or debug flag set) + if (!recording) { + start(); + return; + } if (milliseconds() - started > 1000 * 60 * 3) { stop(); diff --git a/selfdrive/manager/manager.py b/selfdrive/manager/manager.py index b1901dc..7983e4d 100755 --- a/selfdrive/manager/manager.py +++ b/selfdrive/manager/manager.py @@ -229,6 +229,7 @@ def manager_init(frogpilot_functions) -> None: ("ScreenBrightnessOnroad", "101"), ("ScreenManagement", "1"), ("ScreenRecorder", "1"), + ("ScreenRecorderDebug", "1"), ("ScreenTimeout", "30"), ("ScreenTimeoutOnroad", "30"), ("SearchInput", "0"), diff --git a/selfdrive/manager/process_config.py b/selfdrive/manager/process_config.py index 59c6181..948646c 100755 --- a/selfdrive/manager/process_config.py +++ b/selfdrive/manager/process_config.py @@ -62,8 +62,9 @@ procs = [ PythonProcess("timed", "system.timed", always_run, enabled=not PC), PythonProcess("dmonitoringmodeld", "selfdrive.modeld.dmonitoringmodeld", driverview, enabled=(not PC or WEBCAM)), - NativeProcess("encoderd", "system/loggerd", ["./encoderd"], allow_logging), - NativeProcess("stream_encoderd", "system/loggerd", ["./encoderd", "--stream"], notcar), + # CLEARPILOT: disabled video encoding (camera .hevc files) — CAN/sensor logs still recorded via loggerd + # NativeProcess("encoderd", "system/loggerd", ["./encoderd"], allow_logging), + # NativeProcess("stream_encoderd", "system/loggerd", ["./encoderd", "--stream"], notcar), NativeProcess("loggerd", "system/loggerd", ["./loggerd"], allow_logging), NativeProcess("modeld", "selfdrive/modeld", ["./modeld"], only_onroad), #NativeProcess("mapsd", "selfdrive/navd", ["./mapsd"], only_onroad), diff --git a/selfdrive/ui/qt/onroad.cc b/selfdrive/ui/qt/onroad.cc index 5b89787..783f3a6 100755 --- a/selfdrive/ui/qt/onroad.cc +++ b/selfdrive/ui/qt/onroad.cc @@ -919,14 +919,14 @@ void AnnotatedCameraWidget::initializeFrogPilotWidgets() { animationFrameIndex = (animationFrameIndex + 1) % totalFrames; }); - // Initialize the timer for the screen recorder - // QTimer *record_timer = new QTimer(this); - // connect(record_timer, &QTimer::timeout, this, [this]() { - // if (recorder_btn) { - // recorder_btn->update_screen(); - // } - // }); - // record_timer->start(1000 / UI_FREQ); + // CLEARPILOT: screen recorder timer — feeds frames to the OMX encoder + QTimer *record_timer = new QTimer(this); + connect(record_timer, &QTimer::timeout, this, [this]() { + if (recorder_btn) { + recorder_btn->update_screen(); + } + }); + record_timer->start(1000 / UI_FREQ); } void AnnotatedCameraWidget::updateFrogPilotWidgets() { @@ -1016,7 +1016,7 @@ void AnnotatedCameraWidget::paintFrogPilotWidgets(QPainter &p) { drawSLCConfirmation(p); } - // recorder_btn->setVisible(scene.screen_recorder && !mapOpen); + // CLEARPILOT: screen recorder runs invisibly, no UI button shown recorder_btn->setVisible(false); } diff --git a/selfdrive/ui/ui.cc b/selfdrive/ui/ui.cc index 93c95e0..67cccde 100755 --- a/selfdrive/ui/ui.cc +++ b/selfdrive/ui/ui.cc @@ -391,6 +391,7 @@ void ui_update_frogpilot_params(UIState *s) { scene.screen_brightness = screen_management ? params.getInt("ScreenBrightness") : 101; scene.screen_brightness_onroad = screen_management ? params.getInt("ScreenBrightnessOnroad") : 101; scene.screen_recorder = screen_management && params.getBool("ScreenRecorder"); + scene.screen_recorder_debug = params.getBool("ScreenRecorderDebug"); scene.screen_timeout = screen_management ? params.getInt("ScreenTimeout") : 120; scene.screen_timeout_onroad = screen_management ? params.getInt("ScreenTimeoutOnroad") : 10; scene.standby_mode = screen_management && params.getBool("StandbyMode"); diff --git a/selfdrive/ui/ui.h b/selfdrive/ui/ui.h index 78dc059..85368c3 100755 --- a/selfdrive/ui/ui.h +++ b/selfdrive/ui/ui.h @@ -231,6 +231,7 @@ typedef struct UIScene { bool road_name_ui; bool rotating_wheel; bool screen_recorder; + bool screen_recorder_debug; bool show_aol_status_bar; bool show_cem_status_bar; bool show_jerk; diff --git a/system/loggerd/deleter.py b/system/loggerd/deleter.py index 2f0b96c..5dcec7b 100755 --- a/system/loggerd/deleter.py +++ b/system/loggerd/deleter.py @@ -8,11 +8,15 @@ from openpilot.system.loggerd.config import get_available_bytes, get_available_p from openpilot.system.loggerd.uploader import listdir_by_creation from openpilot.system.loggerd.xattr_cache import getxattr -MIN_BYTES = 5 * 1024 * 1024 * 1024 +# CLEARPILOT: increased from 5 GB to 9 GB to reserve space for screen recordings +MIN_BYTES = 9 * 1024 * 1024 * 1024 MIN_PERCENT = 10 DELETE_LAST = ['boot', 'crash'] +# CLEARPILOT: screen recorder video directory +VIDEOS_DIR = '/data/media/0/videos' + PRESERVE_ATTR_NAME = 'user.preserve' PRESERVE_ATTR_VALUE = b'1' PRESERVE_COUNT = 5 @@ -44,12 +48,37 @@ def get_preserved_segments(dirs_by_creation: list[str]) -> list[str]: return preserved +def delete_oldest_video(): + """CLEARPILOT: delete the oldest screen recording MP4 when disk space is low.""" + try: + if not os.path.isdir(VIDEOS_DIR): + return False + videos = sorted( + (f for f in os.listdir(VIDEOS_DIR) if f.endswith('.mp4')), + key=lambda f: os.path.getctime(os.path.join(VIDEOS_DIR, f)) + ) + if not videos: + return False + delete_path = os.path.join(VIDEOS_DIR, videos[0]) + cloudlog.info(f"deleting video {delete_path}") + os.remove(delete_path) + return True + except OSError: + cloudlog.exception(f"issue deleting video from {VIDEOS_DIR}") + return False + + def deleter_thread(exit_event): while not exit_event.is_set(): out_of_bytes = get_available_bytes(default=MIN_BYTES + 1) < MIN_BYTES out_of_percent = get_available_percent(default=MIN_PERCENT + 1) < MIN_PERCENT if out_of_percent or out_of_bytes: + # CLEARPILOT: try deleting oldest video first, then fall back to log segments + if delete_oldest_video(): + exit_event.wait(.1) + continue + dirs = listdir_by_creation(Paths.log_root()) # skip deleting most recent N preserved segments (and their prior segment)