Hanson.xyz Dev
5e4ebfb99e
- Add --verbose flag to sync commands for detailed API request/response output - Add progress indicators (.=#xPpE|) for compact sync output - Implement exponential backoff (1s, 2s, 4s, 8s, 16s) for media downloads - Log failed media downloads to wp-content/uploads/mls-missing-media.log - Add 'wp mls cache missing' command to view/clear the log - Retry on rate limit (429) and server errors (5xx) - Update documentation with new features Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
166 lines
4.3 KiB
Markdown
166 lines
4.3 KiB
Markdown
# MLS by HansonXyz Plugin
|
|
|
|
WordPress plugin for syncing MLS Grid API data (NorthStar MLS) into local database.
|
|
|
|
## Development Rules
|
|
|
|
1. **No emojis** - nowhere in code, commits, docs, or conversation
|
|
2. **PHP 7.4+** compatible code
|
|
3. **WordPress Coding Standards**
|
|
4. Follow patterns from existing HomeProz theme
|
|
|
|
## Quick Reference
|
|
|
|
### Database Tables
|
|
|
|
All tables use `{$wpdb->prefix}mls_` prefix:
|
|
|
|
| Table | Purpose |
|
|
|-------|---------|
|
|
| `mls_properties` | Listing data |
|
|
| `mls_media` | Media files |
|
|
| `mls_sync_state` | Sync progress tracking |
|
|
| `mls_rate_limits` | API usage tracking |
|
|
| `mls_sync_log` | Debug logging |
|
|
|
|
### API Configuration
|
|
|
|
Credentials in wp-config.php:
|
|
```php
|
|
define('MLSGRID_API_URL', 'https://api.mlsgrid.com/v2');
|
|
define('MLSGRID_ACCESS_TOKEN', 'your-token-here');
|
|
```
|
|
|
|
### MLS Grid API Rate Limits
|
|
|
|
MUST comply with these limits:
|
|
- 2 requests/second
|
|
- 7,200 requests/hour
|
|
- 40,000 requests/day
|
|
- 4GB data/hour
|
|
|
|
### Key Files
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `includes/class-mls-api-client.php` | API communication, auth, gzip |
|
|
| `includes/class-mls-sync-engine.php` | Sync orchestration |
|
|
| `includes/class-mls-media-handler.php` | Media download/storage |
|
|
| `includes/class-mls-query.php` | Public query API |
|
|
| `includes/class-mls-rate-limiter.php` | Rate limit compliance |
|
|
| `cli/class-mls-cli.php` | WP-CLI commands |
|
|
|
|
### WP-CLI Commands
|
|
|
|
```bash
|
|
# Test connectivity
|
|
wp mls test connection
|
|
wp mls test auth
|
|
|
|
# Show status
|
|
wp mls status
|
|
wp mls status rate-limits
|
|
|
|
# Run sync (use --verbose for detailed output)
|
|
wp mls sync full [--dry-run] [--limit=N] [--verbose]
|
|
wp mls sync incremental [--dry-run] [--verbose]
|
|
wp mls sync media [--limit=N] [--verbose]
|
|
wp mls sync resume --id=<sync_id>
|
|
|
|
# Statistics
|
|
wp mls stats
|
|
|
|
# Cache management
|
|
wp mls cache clear --confirm
|
|
wp mls cache cleanup
|
|
wp mls cache missing # View failed media downloads
|
|
wp mls cache missing --limit=20 # View first 20 entries
|
|
wp mls cache missing --clear # Clear the log
|
|
```
|
|
|
|
### Progress Output
|
|
|
|
Without --verbose (compact mode):
|
|
- `.` = new property created
|
|
- `#` = property updated
|
|
- `x` = property deleted
|
|
- `-` = skipped (dry-run)
|
|
- `P` = photo downloaded
|
|
- `p` = photo skipped (already exists)
|
|
- `E` = photo error
|
|
- `|` = page complete
|
|
|
|
With --verbose: Full timestamped output showing API requests, responses, and individual item status.
|
|
|
|
### Missing Media Log
|
|
|
|
Failed media downloads are logged to: `wp-content/uploads/mls-missing-media.log`
|
|
|
|
Format: `[timestamp] listing_key | media_key | error | url`
|
|
|
|
Media downloads use exponential backoff (1s, 2s, 4s, 8s, 16s) for rate limit (429) and server errors (5xx).
|
|
|
|
### Public API Functions
|
|
|
|
Available for themes/plugins:
|
|
|
|
```php
|
|
// Get properties with filters
|
|
$properties = mls_get_properties([
|
|
'status' => 'Active',
|
|
'city' => 'Albert Lea',
|
|
'min_price' => 100000,
|
|
'limit' => 20,
|
|
]);
|
|
|
|
// Get single property
|
|
$property = mls_get_property('NST123456');
|
|
|
|
// Get media
|
|
$media = mls_get_property_media('NST123456');
|
|
$image_url = mls_get_property_image('NST123456');
|
|
|
|
// Get distinct values
|
|
$cities = mls_get_cities('Active');
|
|
|
|
// Check data availability
|
|
if (mls_is_available()) { ... }
|
|
```
|
|
|
|
### Sync Strategy
|
|
|
|
1. **Initial Import**: Full sync downloads all viewable properties
|
|
2. **Incremental**: Uses ModificationTimestamp to fetch only changes
|
|
3. **Delete Handling**: MlgCanView=false triggers local deletion
|
|
4. **Media**: Downloads to wp-content/uploads/mls-listings/
|
|
5. **Recovery**: Stores last_next_link for resume on failure
|
|
|
|
### Testing After Changes
|
|
|
|
```bash
|
|
wp mls test connection
|
|
wp mls test auth
|
|
wp mls sync full --dry-run --limit=10
|
|
wp mls stats
|
|
```
|
|
|
|
### Property Data Mapping
|
|
|
|
Key fields from API to database:
|
|
|
|
| API Field | DB Column |
|
|
|-----------|-----------|
|
|
| ListingKey | listing_key |
|
|
| ListingId | listing_id |
|
|
| ListPrice | list_price |
|
|
| StandardStatus | standard_status |
|
|
| BedroomsTotal | bedrooms_total |
|
|
| BathroomsTotalInteger | bathrooms_total |
|
|
| LivingArea | living_area |
|
|
| City | city |
|
|
| ModificationTimestamp | modification_timestamp |
|
|
| PhotosChangeTimestamp | photos_change_timestamp |
|
|
| MlgCanView | mlg_can_view |
|
|
|
|
Full API response stored in `raw_data` column as JSON.
|