Add WebP conversion and garbage collection to MLS plugin
Image handling improvements: - Convert PNG and images >500KB to WebP format - Resize images wider than 1600px maintaining aspect ratio - Check for .webp version before falling back to original - WebP quality set to 80 (equivalent to JPEG 90%) Garbage collection for disk space management: - New MLS_Garbage_Collector class runs after each sync - Only active when MLS_GC_DISK_THRESHOLD defined in wp-config - Deletes image directories older than 24 hours, oldest first - Stops when free space reaches 5GB or 2GB deleted per run - Protects recently accessed images from deletion Documentation: - Added Garbage Collection section to README - Updated Features list and File Structure Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Hanson.xyz Dev
@@ -14,6 +14,7 @@ WordPress plugin for syncing MLS Grid API data (NorthStar MLS) into a local data
|
||||
- [Public API](#public-api)
|
||||
- [Database Schema](#database-schema)
|
||||
- [Media Handling](#media-handling)
|
||||
- [Garbage Collection](#garbage-collection)
|
||||
- [Sync Strategy](#sync-strategy)
|
||||
- [Error Recovery](#error-recovery)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
@@ -23,6 +24,8 @@ WordPress plugin for syncing MLS Grid API data (NorthStar MLS) into a local data
|
||||
- Syncs Active and Pending property listings from MLS Grid API
|
||||
- Automatic incremental updates via replication
|
||||
- On-demand image fetching and local caching
|
||||
- Automatic WebP conversion for cached images
|
||||
- Disk space garbage collection for image cache
|
||||
- Self-healing sync with automatic error recovery
|
||||
- Rate limit compliance (MLS Grid limits enforced)
|
||||
- Resume capability for interrupted syncs
|
||||
@@ -56,6 +59,17 @@ define('MLSGRID_API_URL', 'https://api.mlsgrid.com/v2');
|
||||
define('MLSGRID_ACCESS_TOKEN', 'your-access-token-here');
|
||||
```
|
||||
|
||||
### Image Garbage Collection (Optional)
|
||||
|
||||
To enable automatic cleanup of old cached images when disk space is low, add to `wp-config.php`:
|
||||
|
||||
```php
|
||||
// Enable garbage collection when free space drops below 5GB
|
||||
define('MLS_GC_DISK_THRESHOLD', 5 * 1024 * 1024 * 1024); // 5GB in bytes
|
||||
```
|
||||
|
||||
See [Garbage Collection](#garbage-collection) for details.
|
||||
|
||||
### WordPress Admin Settings
|
||||
|
||||
Navigate to **Settings > MLS Settings** to configure:
|
||||
@@ -464,6 +478,80 @@ wp mls media status
|
||||
|
||||
Shows total media records, cached count, and uncached count.
|
||||
|
||||
## Garbage Collection
|
||||
|
||||
The plugin includes automatic garbage collection to prevent disk space from filling up with cached MLS images.
|
||||
|
||||
### Enabling Garbage Collection
|
||||
|
||||
Add to `wp-config.php`:
|
||||
|
||||
```php
|
||||
// Enable garbage collection when free space drops below 5GB
|
||||
define('MLS_GC_DISK_THRESHOLD', 5 * 1024 * 1024 * 1024); // 5GB in bytes
|
||||
```
|
||||
|
||||
If `MLS_GC_DISK_THRESHOLD` is not defined, garbage collection is disabled.
|
||||
|
||||
### How It Works
|
||||
|
||||
1. After each sync (`wp mls run`), the plugin checks free disk space on the volume hosting MLS images
|
||||
2. If free space is below the threshold, cleanup begins
|
||||
3. Directories older than 24 hours are deleted, oldest first
|
||||
4. Cleanup stops when:
|
||||
- Free space reaches 5GB, OR
|
||||
- 2GB has been deleted in this run
|
||||
5. Directories modified within the last 24 hours are never deleted (protects recently accessed images)
|
||||
|
||||
### Behavior Summary
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Threshold trigger | Configurable via `MLS_GC_DISK_THRESHOLD` |
|
||||
| Target free space | 5GB |
|
||||
| Max delete per run | 2GB |
|
||||
| Minimum directory age | 24 hours |
|
||||
| Runs automatically | After every sync |
|
||||
|
||||
### CLI Output
|
||||
|
||||
During sync, garbage collection status is shown:
|
||||
|
||||
```
|
||||
Garbage Collection:
|
||||
Disk space OK: 12.45 GB free (threshold: 5.00 GB)
|
||||
```
|
||||
|
||||
Or if cleanup occurs:
|
||||
|
||||
```
|
||||
Garbage Collection:
|
||||
Disk space low: 3.21 GB free (threshold: 5.00 GB). Starting cleanup...
|
||||
Deleted: NST123456 (45.23 MB)
|
||||
Deleted: NST789012 (38.91 MB)
|
||||
...
|
||||
Cleanup complete: Deleted 42 directories (1.89 GB). Free space now: 5.10 GB
|
||||
```
|
||||
|
||||
### Recommended Threshold
|
||||
|
||||
For most installations, 5GB is a good threshold:
|
||||
|
||||
```php
|
||||
define('MLS_GC_DISK_THRESHOLD', 5 * 1024 * 1024 * 1024);
|
||||
```
|
||||
|
||||
For servers with limited disk space, you may want a higher threshold to trigger cleanup earlier:
|
||||
|
||||
```php
|
||||
// Trigger cleanup when below 10GB
|
||||
define('MLS_GC_DISK_THRESHOLD', 10 * 1024 * 1024 * 1024);
|
||||
```
|
||||
|
||||
### Image Regeneration
|
||||
|
||||
When a deleted image is requested again, it is automatically re-fetched from MLS Grid and cached. This is the normal on-demand fetching behavior - garbage collection simply clears old cached files to free disk space.
|
||||
|
||||
## Sync Strategy
|
||||
|
||||
### Initial Import (Full Sync)
|
||||
@@ -610,16 +698,17 @@ mls-by-hansonxyz/
|
||||
├── cli/
|
||||
│ └── class-mls-cli.php # WP-CLI commands
|
||||
├── includes/
|
||||
│ ├── class-mls-activator.php # Plugin activation
|
||||
│ ├── class-mls-api-client.php # MLS Grid API communication
|
||||
│ ├── class-mls-db.php # Database operations
|
||||
│ ├── class-mls-deactivator.php # Plugin deactivation
|
||||
│ ├── class-mls-logger.php # Event logging
|
||||
│ ├── class-mls-media-handler.php # On-demand image caching
|
||||
│ ├── class-mls-options.php # Configuration management
|
||||
│ ├── class-mls-query.php # Public query API
|
||||
│ ├── class-mls-rate-limiter.php # Rate limit compliance
|
||||
│ └── class-mls-sync-engine.php # Sync orchestration
|
||||
│ ├── class-mls-activator.php # Plugin activation
|
||||
│ ├── class-mls-api-client.php # MLS Grid API communication
|
||||
│ ├── class-mls-db.php # Database operations
|
||||
│ ├── class-mls-deactivator.php # Plugin deactivation
|
||||
│ ├── class-mls-garbage-collector.php # Disk space management
|
||||
│ ├── class-mls-logger.php # Event logging
|
||||
│ ├── class-mls-media-handler.php # On-demand image caching
|
||||
│ ├── class-mls-options.php # Configuration management
|
||||
│ ├── class-mls-query.php # Public query API
|
||||
│ ├── class-mls-rate-limiter.php # Rate limit compliance
|
||||
│ └── class-mls-sync-engine.php # Sync orchestration
|
||||
└── docs/
|
||||
├── API.md # MLS Grid API reference
|
||||
├── CLAUDE.md # AI assistant context
|
||||
|
||||
Reference in New Issue
Block a user