clearpilot/CLAUDE.md

ClearPilot — CLAUDE.md

Project Overview

ClearPilot is a custom fork of FrogPilot (itself a fork of comma.ai's openpilot), based on a 2024 release. It is purpose-built for Brian Hanson's Hyundai Tucson (HDA2 equipped). The vehicle's HDA2 system has specific quirks around how it synchronizes driving state with openpilot that require careful handling.

Key Customizations in This Fork

• UI changes to the onroad driving interface
• Lane change behavior: brief disengage when turn signal is active during lane changes
• Lateral control disabled: the car's own radar cruise control handles lateral; openpilot handles longitudinal only
• Driver monitoring timeouts: modified safety timeouts for the driver monitoring system
• Custom driving models: duck-amigo.thneed, farmville.onnx, wd-40.thneed in selfdrive/clearpilot/models/
• ClearPilot service: Node.js service at selfdrive/clearpilot/ with behavior scripts for lane change and longitudinal control
• Native dashcamd: C++ process capturing raw camera frames via VisionIPC with OMX H.264 hardware encoding
• Standstill power saving: model inference throttled to 1fps and fan quieted when car is stopped
• Clean offroad UI: grid launcher replacing stock home screen
• Debug button (LFA): steering wheel button repurposed for screen toggle and future UI actions

See GOALS.md for feature roadmap.

Working Rules

CRITICAL: Change Control

This is self-driving software. All changes must be deliberate and well-understood.

• NEVER make changes outside of what is explicitly requested
• Always explain proposed changes first — describe the change, the logic, and the architecture; let Brian review and approve before writing any code
• Brian is an expert on this software — do not override his judgment or assume responsibility for changes he doesn't understand
• Every line must be understood — work slowly and deliberately
• Test everything thoroughly — Brian must always be in the loop
• Do not refactor, clean up, or "improve" code beyond the specific request

File Ownership

We operate as root on this device, but openpilot runs as the comma user (uid=1000, gid=1000). After any code changes that touch multiple files or before testing:

chown -R comma:comma /data/openpilot

Git

• Remote: git@git.internal.hanson.xyz:brianhansonxyz/comma.git
• Branch: clearpilot
• Large model files are tracked in git (intentional — this is a backup)

Samba Share

• Share name: openpilot (e.g. \\comma-3889765b\openpilot)
• Path: /data/openpilot
• Username: comma
• Password: i-like-to-drive-cars
• Runs as comma:comma via force user/group — files created over SMB are owned correctly
• Enabled at boot (smbd + nmbd)

Testing Changes

The launch script now self-cleans — it kills other instances of itself, launch_chffrplus.sh, and manager.py before starting. No need to manually kill first.

# Fix ownership (we edit as root, openpilot runs as comma)
chown -R comma:comma /data/openpilot

# Remove prebuilt to force recompilation (it is recreated after each successful build)
rm -f /data/openpilot/prebuilt

# Must use a login shell as comma — sudo -u won't set up the right Python/env (3.11 via pyenv)
su - comma -c "bash /data/openpilot/launch_openpilot.sh"

Adding New Params

The params system uses a C++ whitelist. Adding a new param name in manager.py alone will crash with UnknownKeyName. You must:

1. Register the key in common/params.cc (alphabetically, with PERSISTENT or CLEAR_ON_* flag)
2. Add the default value in selfdrive/manager/manager.py in manager_init()
3. Remove prebuilt, common/params.o, and common/libcommon.a to force rebuild

Building Native (C++) Processes

• SCons is the build system. Static libraries (common, messaging, cereal, visionipc) must be imported as SCons objects, not -l flags
• The --as-needed linker flag can cause link order issues with static libs — disable it in your SConscript if needed
• OMX encoder object (omx_encoder.o) is compiled by the UI build — reference the pre-built .o file rather than recompiling (avoids "two environments" scons error)
• prebuilt is recreated after every successful build — always remove it before rebuilding

Dashcam (dashcamd)

Architecture

dashcamd is a native C++ process that captures raw camera frames directly from camerad via VisionIPC and encodes to MP4 using the Qualcomm OMX H.264 hardware encoder. This replaces the earlier FrogPilot screen recorder approach (QWidget::grab() -> OMX).

• Codec: H.264 AVC (hardware accelerated via OMX.qcom.video.encoder.avc)
• Resolution: 1928x1208 (full camera resolution, no downscaling)
• Bitrate: 4 Mbps
• Container: MP4 (remuxed via libavformat)
• Segment length: 3 minutes per file
• Save path: /data/media/0/videos/YYYYMMDD-HHMMSS.mp4
• Standstill timeout: suspends recording after 10 minutes of standstill, resumes when car moves
• Storage: ~90 MB per 3-minute segment, ~43 hours of footage in 78 GB free space
• Storage device: WDC SDINDDH4-128G UFS 2.1 — automotive grade, ~384 TB write endurance, no concern for continuous writes

Key Differences from Old Screen Recorder

	Old (screen recorder)	New (dashcamd)
Source	QWidget::grab() screen capture	Raw NV12 from VisionIPC
Resolution	1440x720	1928x1208
Works with screen off	No (needs visible widget)	Yes (independent of UI)
Process type	Part of UI process	Standalone native process
Encoder input	RGBA -> NV12 conversion	NV12 direct (added encode_frame_nv12)

Key Files

File	Role
selfdrive/clearpilot/dashcamd.cc	Main dashcam process — VisionIPC -> OMX encoder
selfdrive/clearpilot/SConscript	Build config for dashcamd
selfdrive/frogpilot/screenrecorder/omx_encoder.cc	OMX encoder (added encode_frame_nv12 method)
selfdrive/frogpilot/screenrecorder/omx_encoder.h	Encoder header
selfdrive/manager/process_config.py	dashcamd registered as NativeProcess, encoderd disabled
system/loggerd/deleter.py	Storage rotation (9 GB threshold, oldest videos deleted first)

Params

• DashcamDebug — when "1", dashcamd runs even without car connected (for bench testing)
• IsDriverViewEnabled — must be "1" to start camerad on bench (no car ignition)

Standstill Power Saving

When carState.standstill is true:

• modeld: skips GPU inference on 19/20 frames (1fps vs 20fps), reports 0 frame drops to avoid triggering modeldLagging in controlsd
• dmonitoringmodeld: same 1fps throttle, added carState subscription
• Fan controller: uses offroad clamps (0-30%) instead of onroad (30-100%) at standstill; thermal protection still active via feedforward if temp > 60°C

Key Files

File	Role
selfdrive/modeld/modeld.py	Standstill frame skip logic
selfdrive/modeld/dmonitoringmodeld.py	Standstill frame skip logic
selfdrive/thermald/fan_controller.py	Standstill-aware fan clamps
selfdrive/thermald/thermald.py	Passes standstill to fan controller via carState

Debug Function Button (LFA/LKAS Steering Wheel Button)

The Hyundai Tucson's LFA (Lane Following Assist) steering wheel button is repurposed as a general-purpose UI control button. It has no driving function in ClearPilot since lateral control is disabled.

Signal Chain

Steering wheel LFA button press
  -> CAN-FD message: cruise_btns_msg_canfd["LFA_BTN"]
     [selfdrive/car/hyundai/carstate.py:332-339]
  -> Edge detection: lkas_enabled vs lkas_previously_enabled
  -> create_button_events() -> ButtonEvent(type=FrogPilotButtonType.lkas)
     [selfdrive/car/hyundai/interface.py:168]
  -> controlsd.update_clearpilot_events(CS)
     [selfdrive/controls/controlsd.py:1235-1239]
     -> events.add(EventName.clpDebug)
  -> controlsd.clearpilot_state_control(CC, CS)
     [selfdrive/controls/controlsd.py:1241-1258]
     -> Toggles ScreenDisaplayMode param (0=on, 1=off) in /dev/shm/params
  -> UI reads ScreenDisaplayMode in drawHud()
     [selfdrive/ui/qt/onroad.cc:390-403]
     -> mode=1 and no alert: Hardware::set_display_power(false)
     -> mode=0 or alert visible: Hardware::set_display_power(true)

Current Behavior

• Each press toggles the display on/off instantly (debug alert suppressed)
• ScreenDisaplayMode is in-memory params (/dev/shm/params), resets on reboot
• max_display_mode = 1 — currently only two states (on/off); can be extended for future modes

Key Files

File	Role
selfdrive/car/hyundai/carstate.py	Reads LFA_BTN from CAN-FD
selfdrive/car/hyundai/interface.py	Creates ButtonEvent with FrogPilotButtonType.lkas
selfdrive/controls/controlsd.py	Fires clpDebug event, toggles ScreenDisaplayMode
selfdrive/controls/lib/events.py	clpDebug event definition (alert suppressed)
selfdrive/ui/qt/onroad.cc	Reads ScreenDisaplayMode, controls display power

Offroad UI

The offroad home screen (selfdrive/ui/qt/home.cc) was replaced with a clean grid launcher. Stock FrogPilot widgets (date, version, update/alert notifications) were removed.

• Settings button: opens the original comma/FrogPilot settings (backdoor to all original settings)
• Dashcam button: placeholder for future dashcam footage viewer
• Tapping the splash screen (ReadyWindow) goes directly to the grid launcher (no sidebar)
• Sidebar with metrics (TEMP, VEHICLE, CONNECT) is hidden but still accessible via settings path

Device: comma 3x

Hardware

• Qualcomm Snapdragon SoC (aarch64)
• Serial: comma-3889765b
• Storage: WDC SDINDDH4-128G, 128 GB UFS 2.1
• Connects to the car via comma panda (CAN bus interface)

Operating System

• Ubuntu 20.04.6 LTS (Focal Fossa) on aarch64
• Kernel: 4.9.103+ (custom comma.ai PREEMPT build, Feb 2024) — very old, vendor-patched Qualcomm kernel
• Python: 3.11.4 via pyenv at /usr/local/pyenv/versions/3.11.4/ (system python is 3.8, do not use)
• AGNOS version: 9.7 (comma's custom OS layer on top of Ubuntu)
• Display server: Weston (Wayland compositor) on tty1
• SELinux: mounted (enforcement status varies)

Users

• comma (uid=1000) — the user openpilot runs as; member of root, sudo, disk, gpu, gpio groups
• root — what we SSH in as; files must be chowned back to comma before running openpilot

Filesystem / Mount Quirks

Mount	Device	Type	Notes
/	/dev/sda7	ext4	Root filesystem, read-write
/data	/dev/sda12	ext4	Persistent. Openpilot lives here. Survives reboots.
/home	overlay	overlayfs	VOLATILE — upperdir on tmpfs, changes lost on reboot
/tmp	tmpfs	tmpfs	Volatile, 150 MB
/var	tmpfs	tmpfs	Volatile, 128 MB (fstab) / 1.5 GB (actual)
/systemrw	/dev/sda10	ext4	Writable system overlay, noexec
/persist	/dev/sda2	ext4	Persistent config/certs, noexec
/cache	/dev/sda11	ext4	Android-style cache partition
/dsp	/dev/sde26	ext4	Read-only Qualcomm DSP firmware
/firmware	/dev/sde4	vfat	Read-only firmware blobs

Hardware Encoding

• OMX: OMX.qcom.video.encoder.avc (H.264) and OMX.qcom.video.encoder.hevc — used by dashcamd and screen recorder
• V4L2: Qualcomm VIDC at /dev/v4l/by-path/platform-aa00000.qcom_vidc-video-index1 — used by encoderd (now disabled). Not accessible from ffmpeg due to permission/driver issues
• ffmpeg: v4.2.2, has h264_v4l2m2m and h264_omx listed but neither works from ffmpeg subprocess (OMX port issues, V4L2 device not found). Use OMX directly via the C++ encoder

Fan Control

Software-controlled via thermald -> fan_controller.py -> panda USB -> PWM. Target temp 70°C, PI+feedforward controller. See Standstill Power Saving section for standstill-aware clamps.

Boot Sequence

Power On
  -> systemd: comma.service (runs as comma user)
    -> /usr/comma/comma.sh (waits for Weston, handles factory reset)
      -> /data/continue.sh (exec bridge to openpilot)
        -> /data/openpilot/launch_openpilot.sh
          -> Kills other instances of itself and manager.py
          -> Runs on_start.sh (logo, reverse SSH)
          -> exec launch_chffrplus.sh
            -> Sources launch_env.sh (thread counts, AGNOS_VERSION)
            -> Runs agnos_init (marks boot slot, GPU perms, checks OS update)
            -> Sets PYTHONPATH, symlinks /data/pythonpath
            -> Runs build.py if no `prebuilt` marker
            -> Launches selfdrive/manager/manager.py
              -> manager_init() sets default params
              -> ensure_running() loop starts all managed processes

Openpilot Architecture

Process Manager

selfdrive/manager/manager.py orchestrates all processes defined in selfdrive/manager/process_config.py.

Always-Running Processes (offroad + onroad)

• thermald — thermal management and fan control
• pandad — panda CAN bus interface
• ui — Qt-based onroad/offroad UI
• deleter — storage cleanup (9 GB threshold)
• statsd, timed, logmessaged, tombstoned — telemetry/logging
• manage_athenad — comma cloud connectivity
• fleet_manager, frogpilot_process — FrogPilot additions

Onroad-Only Processes (when driving)

• controlsd — main vehicle control loop
• plannerd — path planning
• radard — radar processing
• modeld — driving model inference (throttled to 1fps at standstill)
• dmonitoringmodeld — driver monitoring model (throttled to 1fps at standstill)
• locationd, calibrationd, paramsd, torqued — localization and calibration
• sensord — IMU/sensor data
• soundd — alert sounds
• camerad — camera capture
• loggerd — CAN/sensor log recording (video encoding disabled)

ClearPilot Processes

• dashcamd — raw camera dashcam recording (runs onroad or with DashcamDebug flag)

GPS

• ubloxd + pigeond for u-blox GPS hardware
• qcomgpsd, ugpsd, navd currently commented out in process_config

Key Dependencies

• Python 3.11 (via pyenv) with: numpy, casadi, onnx/onnxruntime, pycapnp, pyzmq, sentry-sdk, sympy, Cython
• capnp (Cap'n Proto) — IPC message serialization between all processes
• ZeroMQ — IPC transport layer
• Qt 5 — UI framework (with WebEngine available but not used for rotation reasons)
• OpenMAX (OMX) — hardware video encoding
• libavformat — MP4 container muxing
• libyuv — color space conversion
• SCons — build system for native C++ components