Snapshot: MLS sync fixes, image refresh, plugin/theme updates

MLS plugin fixes from this session:
- Fix silent insert failures: location column NOT NULL was rejecting wpdb->insert calls,
  causing ~18k new properties since Dec 2025 to be lost. Inserts now build raw SQL
  with ST_PointFromText so the spatial column is populated atomically.
- Auto-refresh expired media URLs in MLS_Media_Handler::fetch_and_cache(), guarded by
  a property-level GET_LOCK so concurrent fetches share one API refresh.
- Normalize WP_Error to null in mls_get_property_image() so callers can rely on the
  documented string|null contract.
- Support comma-separated property_type filters in MLS_Query and MLS_Cluster so the
  homepage "View All Commercial" link (?property_type=Commercial+Sale,Land,Farm)
  actually filters correctly.
- Incremental sync now looks back 10 minutes past the latest modification timestamp
  as a safety margin against missed records.
- Smart sync exits silently (info-level, not warning) when a full sync is in progress.

Operational:
- New cron: weekly full sync Sundays at 3 AM (/usr/local/bin/mls-full-sync).
- New cron: hourly 2GB cap on mls-thumbnails/ and cache/transformed-images/
  (/usr/local/bin/mls-image-cache-cap).
- Logrotate config for wp-content/debug.log (2-day retention, daily rotation,
  delaycompress).

Repo policy:
- CLAUDE.md updated with explicit "commit everything except build artifacts" policy.
- .gitignore: untrack runtime image caches and debug.log rotations.

Other modifications in this snapshot are pre-existing in-flight theme/plugin/db_content_updates work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
root
2026-04-29 15:32:23 +00:00
parent 57b752f54e
commit b6df4dbb92
5385 changed files with 838580 additions and 2416 deletions
+61 -3
View File
@@ -22,10 +22,12 @@ WordPress plugin for syncing MLS Grid API data (NorthStar MLS) into a local data
## Features
- Syncs Active and Pending property listings from MLS Grid API
- **HomeProz Listing Persistence**: Sold HomeProz listings are retained for historical viewing
- Automatic incremental updates via replication
- On-demand image fetching and local caching
- **Persistent Image Cache**: HomeProz listing images are permanently cached
- Automatic WebP conversion for cached images
- Disk space garbage collection for image cache
- Disk space garbage collection for image cache (excludes HomeProz images)
- Self-healing sync with automatic error recovery
- Rate limit compliance (MLS Grid limits enforced)
- Resume capability for interrupted syncs
@@ -462,6 +464,30 @@ Per MLS Grid rules, media URLs cannot be used directly on websites. Images must
- Automatic re-fetch if cache cleared
- Works with MLS Grid's URL expiration
### HomeProz Persistent Cache
HomeProz listings receive special treatment to preserve images even after properties are sold:
**How it works:**
1. HomeProz listings are identified by `ListOfficeMlsId` matching the configured HomeProz office ID
2. During sync, ALL images for HomeProz listings are automatically downloaded
3. Images are stored in the persistent cache: `wp-content/uploads/mls-listings-persistent/`
4. The persistent cache is NEVER subject to garbage collection
5. Images remain available indefinitely, even after the listing is sold and removed from MLS
**Cache Directories:**
| Directory | Purpose | Garbage Collected |
|-----------|---------|-------------------|
| `mls-listings/` | Standard cache (non-HomeProz) | Yes |
| `mls-listings-persistent/` | HomeProz listings | No |
**Benefits:**
- Sold HomeProz listings can be displayed on a "Sold Homes" page
- No loss of images when listings are removed from MLS feed
- Historical record of HomeProz sales preserved
### Pre-caching Images
To pre-cache images for specific listings:
@@ -482,6 +508,8 @@ Shows total media records, cached count, and uncached count.
The plugin includes automatic garbage collection to prevent disk space from filling up with cached MLS images.
**Important:** Garbage collection ONLY affects the standard cache (`mls-listings/`). The persistent cache (`mls-listings-persistent/`) containing HomeProz listing images is NEVER touched.
### Enabling Garbage Collection
Add to `wp-config.php`:
@@ -497,11 +525,12 @@ If `MLS_GC_DISK_THRESHOLD` is not defined, garbage collection is disabled.
1. After each sync (`wp mls run`), the plugin checks free disk space on the volume hosting MLS images
2. If free space is below the threshold, cleanup begins
3. Directories older than 24 hours are deleted, oldest first
3. Directories older than 24 hours are deleted from the standard cache, oldest first
4. Cleanup stops when:
- Free space reaches 5GB, OR
- 2GB has been deleted in this run
5. Directories modified within the last 24 hours are never deleted (protects recently accessed images)
6. HomeProz images in the persistent cache are NEVER deleted
### Behavior Summary
@@ -552,6 +581,33 @@ define('MLS_GC_DISK_THRESHOLD', 10 * 1024 * 1024 * 1024);
When a deleted image is requested again, it is automatically re-fetched from MLS Grid and cached. This is the normal on-demand fetching behavior - garbage collection simply clears old cached files to free disk space.
## HomeProz Listing Persistence
HomeProz listings (properties listed by the configured HomeProz office) receive special handling to preserve them even after they are sold.
### Retention Rules
| Listing Type | Active | Pending | Sold | Other Statuses |
|--------------|--------|---------|------|----------------|
| HomeProz | Retained | Retained | Retained | Deleted |
| Non-HomeProz | Retained | Retained | Deleted | Deleted |
### How It Works
1. **Identification**: Listings are identified as HomeProz by matching `ListOfficeMlsId` to `MLS_HOMEPROZ_OFFICE_ID` (configured in plugin constants)
2. **Sync Behavior**: HomeProz listings with Active, Pending, or Sold status are kept in the database
3. **Image Caching**: All images for HomeProz listings are automatically downloaded during sync
4. **Persistent Storage**: Images are stored in `mls-listings-persistent/` which is never garbage collected
5. **Future Use**: Sold HomeProz listings can be displayed on a "Sold Homes" page
### Configuration
The HomeProz office ID is defined in the main plugin file:
```php
define('MLS_HOMEPROZ_OFFICE_ID', 'NST253235');
```
## Sync Strategy
### Initial Import (Full Sync)
@@ -568,8 +624,10 @@ When a deleted image is requested again, it is automatically re-fetched from MLS
- No filter on status (need to detect changes)
- For each record:
- If `MlgCanView = false`: DELETE from local DB
- If `StandardStatus` not Active/Pending: DELETE from local DB
- If HomeProz listing: DELETE only if status not Active/Pending/Sold
- If non-HomeProz listing: DELETE if status not Active/Pending
- Otherwise: INSERT or UPDATE
- HomeProz images are auto-downloaded during sync
### Why This Approach?