117 lines
3.1 KiB
Markdown
117 lines
3.1 KiB
Markdown
# Database Date/Time Fix Script
|
|
|
|
This script fixes date and time format issues in the `OptionsFlow_monthly` and `options_flow` tables.
|
|
|
|
## What It Fixes
|
|
|
|
1. **CreatedDate** - Normalizes all dates to `YYYY-MM-DD` format
|
|
- Converts `MM/DD/YYYY` → `YYYY-MM-DD`
|
|
- Converts `M/D/YYYY` → `YYYY-MM-DD`
|
|
- Keeps already valid `YYYY-MM-DD` dates
|
|
|
|
2. **CreatedTime** - Normalizes all times to `HH:MM:SS AM/PM` format with consistent spacing
|
|
- Adds space before AM/PM: `"3:59:59PM"` → `"3:59:59 PM"`
|
|
- Fixes missing seconds: `"3:59 PM"` → `"3:59:00 PM"`
|
|
- Fixes missing colon: `"359PM"` → `"3:59:00 PM"`
|
|
- Fixes missing hour: `":59 AM"` → `"9:59:00 AM"`
|
|
|
|
3. **ExpirationDate** - Normalizes to `YYYY-MM-DD` format
|
|
- Same normalization as CreatedDate
|
|
|
|
4. **options_flow table** - Fixes invalid timestamps
|
|
- Sets NULL for dates outside valid range
|
|
- Updates invalid `created_at` to current time
|
|
|
|
## Usage
|
|
|
|
### Option 1: Run SQL File Directly
|
|
|
|
```bash
|
|
# Connect to your PostgreSQL database
|
|
psql -h localhost -U postgres -d institutional_trader
|
|
|
|
# Run the fix script
|
|
\i backend/database/fix_all_dates_and_times.sql
|
|
```
|
|
|
|
### Option 2: Use Node.js Script
|
|
|
|
```bash
|
|
# Analyze current state (no changes)
|
|
node backend/scripts/fixDatabaseDates.js analyze
|
|
|
|
# Run fixes
|
|
node backend/scripts/fixDatabaseDates.js fix
|
|
|
|
# Verify fixes
|
|
node backend/scripts/fixDatabaseDates.js verify
|
|
|
|
# Run all (analyze, fix, verify)
|
|
node backend/scripts/fixDatabaseDates.js all
|
|
```
|
|
|
|
### Option 3: Use in Supabase SQL Editor
|
|
|
|
1. Open Supabase Dashboard → SQL Editor
|
|
2. Copy contents of `fix_all_dates_and_times.sql`
|
|
3. Paste and run in SQL Editor
|
|
|
|
## What to Expect
|
|
|
|
### Before Fix:
|
|
```
|
|
CreatedDate: "12/17/2025", "2025-12-17", "12-17-2025"
|
|
CreatedTime: "3:59:59PM", "359PM", "3:59 PM", ":59 AM"
|
|
```
|
|
|
|
### After Fix:
|
|
```
|
|
CreatedDate: "2025-12-17" (all normalized)
|
|
CreatedTime: "3:59:59 PM" (all with space, consistent format)
|
|
```
|
|
|
|
## Safety
|
|
|
|
- **Read-only analysis first**: Run `analyze` command to see what will be changed
|
|
- **Non-destructive**: Script only updates format, doesn't delete data
|
|
- **Idempotent**: Safe to run multiple times (won't change already-correct data)
|
|
- **Verification**: Includes verification queries to check results
|
|
|
|
## Database Requirements
|
|
|
|
- PostgreSQL 12+ recommended
|
|
- Connection to `OptionsFlow_monthly` table
|
|
- Sufficient permissions for UPDATE operations
|
|
|
|
## Troubleshooting
|
|
|
|
### Permission Errors
|
|
```sql
|
|
-- Grant necessary permissions
|
|
GRANT UPDATE ON "OptionsFlow_monthly" TO your_user;
|
|
```
|
|
|
|
### Timeout Errors
|
|
```sql
|
|
-- For large tables, increase timeout
|
|
SET statement_timeout = '10min';
|
|
```
|
|
|
|
### Check Specific Records
|
|
```sql
|
|
-- See problematic entries
|
|
SELECT "CreatedDate", "CreatedTime", "Symbol"
|
|
FROM "OptionsFlow_monthly"
|
|
WHERE "CreatedTime"::text !~ '^\d{1,2}:\d{2}(:\d{2})?\s+[AP]M$'
|
|
AND "CreatedTime" IS NOT NULL
|
|
LIMIT 100;
|
|
```
|
|
|
|
## Notes
|
|
|
|
- Script preserves NULL values (doesn't change them)
|
|
- Ambiguous formats (like "12 AM" - could be hour or minute) default to hour format
|
|
- Large tables may take several minutes to update
|
|
- Recommended to backup database before running on production
|
|
|