Snapshot: MLS sync fixes, image refresh, plugin/theme updates
MLS plugin fixes from this session: - Fix silent insert failures: location column NOT NULL was rejecting wpdb->insert calls, causing ~18k new properties since Dec 2025 to be lost. Inserts now build raw SQL with ST_PointFromText so the spatial column is populated atomically. - Auto-refresh expired media URLs in MLS_Media_Handler::fetch_and_cache(), guarded by a property-level GET_LOCK so concurrent fetches share one API refresh. - Normalize WP_Error to null in mls_get_property_image() so callers can rely on the documented string|null contract. - Support comma-separated property_type filters in MLS_Query and MLS_Cluster so the homepage "View All Commercial" link (?property_type=Commercial+Sale,Land,Farm) actually filters correctly. - Incremental sync now looks back 10 minutes past the latest modification timestamp as a safety margin against missed records. - Smart sync exits silently (info-level, not warning) when a full sync is in progress. Operational: - New cron: weekly full sync Sundays at 3 AM (/usr/local/bin/mls-full-sync). - New cron: hourly 2GB cap on mls-thumbnails/ and cache/transformed-images/ (/usr/local/bin/mls-image-cache-cap). - Logrotate config for wp-content/debug.log (2-day retention, daily rotation, delaycompress). Repo policy: - CLAUDE.md updated with explicit "commit everything except build artifacts" policy. - .gitignore: untrack runtime image caches and debug.log rotations. Other modifications in this snapshot are pre-existing in-flight theme/plugin/db_content_updates work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -22,10 +22,12 @@ WordPress plugin for syncing MLS Grid API data (NorthStar MLS) into a local data
|
||||
## Features
|
||||
|
||||
- Syncs Active and Pending property listings from MLS Grid API
|
||||
- **HomeProz Listing Persistence**: Sold HomeProz listings are retained for historical viewing
|
||||
- Automatic incremental updates via replication
|
||||
- On-demand image fetching and local caching
|
||||
- **Persistent Image Cache**: HomeProz listing images are permanently cached
|
||||
- Automatic WebP conversion for cached images
|
||||
- Disk space garbage collection for image cache
|
||||
- Disk space garbage collection for image cache (excludes HomeProz images)
|
||||
- Self-healing sync with automatic error recovery
|
||||
- Rate limit compliance (MLS Grid limits enforced)
|
||||
- Resume capability for interrupted syncs
|
||||
@@ -462,6 +464,30 @@ Per MLS Grid rules, media URLs cannot be used directly on websites. Images must
|
||||
- Automatic re-fetch if cache cleared
|
||||
- Works with MLS Grid's URL expiration
|
||||
|
||||
### HomeProz Persistent Cache
|
||||
|
||||
HomeProz listings receive special treatment to preserve images even after properties are sold:
|
||||
|
||||
**How it works:**
|
||||
|
||||
1. HomeProz listings are identified by `ListOfficeMlsId` matching the configured HomeProz office ID
|
||||
2. During sync, ALL images for HomeProz listings are automatically downloaded
|
||||
3. Images are stored in the persistent cache: `wp-content/uploads/mls-listings-persistent/`
|
||||
4. The persistent cache is NEVER subject to garbage collection
|
||||
5. Images remain available indefinitely, even after the listing is sold and removed from MLS
|
||||
|
||||
**Cache Directories:**
|
||||
|
||||
| Directory | Purpose | Garbage Collected |
|
||||
|-----------|---------|-------------------|
|
||||
| `mls-listings/` | Standard cache (non-HomeProz) | Yes |
|
||||
| `mls-listings-persistent/` | HomeProz listings | No |
|
||||
|
||||
**Benefits:**
|
||||
- Sold HomeProz listings can be displayed on a "Sold Homes" page
|
||||
- No loss of images when listings are removed from MLS feed
|
||||
- Historical record of HomeProz sales preserved
|
||||
|
||||
### Pre-caching Images
|
||||
|
||||
To pre-cache images for specific listings:
|
||||
@@ -482,6 +508,8 @@ Shows total media records, cached count, and uncached count.
|
||||
|
||||
The plugin includes automatic garbage collection to prevent disk space from filling up with cached MLS images.
|
||||
|
||||
**Important:** Garbage collection ONLY affects the standard cache (`mls-listings/`). The persistent cache (`mls-listings-persistent/`) containing HomeProz listing images is NEVER touched.
|
||||
|
||||
### Enabling Garbage Collection
|
||||
|
||||
Add to `wp-config.php`:
|
||||
@@ -497,11 +525,12 @@ If `MLS_GC_DISK_THRESHOLD` is not defined, garbage collection is disabled.
|
||||
|
||||
1. After each sync (`wp mls run`), the plugin checks free disk space on the volume hosting MLS images
|
||||
2. If free space is below the threshold, cleanup begins
|
||||
3. Directories older than 24 hours are deleted, oldest first
|
||||
3. Directories older than 24 hours are deleted from the standard cache, oldest first
|
||||
4. Cleanup stops when:
|
||||
- Free space reaches 5GB, OR
|
||||
- 2GB has been deleted in this run
|
||||
5. Directories modified within the last 24 hours are never deleted (protects recently accessed images)
|
||||
6. HomeProz images in the persistent cache are NEVER deleted
|
||||
|
||||
### Behavior Summary
|
||||
|
||||
@@ -552,6 +581,33 @@ define('MLS_GC_DISK_THRESHOLD', 10 * 1024 * 1024 * 1024);
|
||||
|
||||
When a deleted image is requested again, it is automatically re-fetched from MLS Grid and cached. This is the normal on-demand fetching behavior - garbage collection simply clears old cached files to free disk space.
|
||||
|
||||
## HomeProz Listing Persistence
|
||||
|
||||
HomeProz listings (properties listed by the configured HomeProz office) receive special handling to preserve them even after they are sold.
|
||||
|
||||
### Retention Rules
|
||||
|
||||
| Listing Type | Active | Pending | Sold | Other Statuses |
|
||||
|--------------|--------|---------|------|----------------|
|
||||
| HomeProz | Retained | Retained | Retained | Deleted |
|
||||
| Non-HomeProz | Retained | Retained | Deleted | Deleted |
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Identification**: Listings are identified as HomeProz by matching `ListOfficeMlsId` to `MLS_HOMEPROZ_OFFICE_ID` (configured in plugin constants)
|
||||
2. **Sync Behavior**: HomeProz listings with Active, Pending, or Sold status are kept in the database
|
||||
3. **Image Caching**: All images for HomeProz listings are automatically downloaded during sync
|
||||
4. **Persistent Storage**: Images are stored in `mls-listings-persistent/` which is never garbage collected
|
||||
5. **Future Use**: Sold HomeProz listings can be displayed on a "Sold Homes" page
|
||||
|
||||
### Configuration
|
||||
|
||||
The HomeProz office ID is defined in the main plugin file:
|
||||
|
||||
```php
|
||||
define('MLS_HOMEPROZ_OFFICE_ID', 'NST253235');
|
||||
```
|
||||
|
||||
## Sync Strategy
|
||||
|
||||
### Initial Import (Full Sync)
|
||||
@@ -568,8 +624,10 @@ When a deleted image is requested again, it is automatically re-fetched from MLS
|
||||
- No filter on status (need to detect changes)
|
||||
- For each record:
|
||||
- If `MlgCanView = false`: DELETE from local DB
|
||||
- If `StandardStatus` not Active/Pending: DELETE from local DB
|
||||
- If HomeProz listing: DELETE only if status not Active/Pending/Sold
|
||||
- If non-HomeProz listing: DELETE if status not Active/Pending
|
||||
- Otherwise: INSERT or UPDATE
|
||||
- HomeProz images are auto-downloaded during sync
|
||||
|
||||
### Why This Approach?
|
||||
|
||||
|
||||
Reference in New Issue
Block a user