institutional-trader/README/TROUBLESHOOTING_SQL.md

2.1 KiB

SQL Query Troubleshooting

Common Issues

Issue: invalid value "09" for "AM"

Problem: PostgreSQL's HH12 format doesn't accept leading zeros like "09 AM". It expects "9 AM".

Solution: The SQL query has been updated to handle this by:

  1. Normalizing times like "09 AM" to "9:00 AM" before parsing
  2. Adding regex replacements to remove leading zeros
  3. Handling edge cases for times without colons

If you still see this error:

  • Check your data format in OptionsFlow_monthly.CreatedTime
  • The query now handles: "09 AM", "9 AM", "09:00 AM", "9:00 AM", etc.
  • If you have other formats, you may need to add additional normalization

Issue: Python Service Unavailable

This is normal if:

  • Python service is not running
  • Python service is on a different port
  • Network/firewall issues

The system will automatically:

  • Fall back to SQL query
  • Continue working normally
  • Log the fallback for monitoring

To fix:

  1. Start Python service: cd backend/python_service && uvicorn main:app --port 8010
  2. Check PYTHON_SERVICE_URL in .env matches the service URL
  3. Verify network connectivity

Issue: Date Parsing Errors

Common causes:

  • Invalid date formats in database
  • Missing date/time values
  • Timezone issues

Solutions:

  • The query handles multiple date formats (YYYY-MM-DD, MM/DD/YYYY)
  • Handles multiple time formats (12-hour with AM/PM, 24-hour)
  • Returns NULL for invalid dates (won't crash)

Data Validation

To check your data format:

SELECT 
  "CreatedDate",
  "CreatedTime",
  COUNT(*) as count
FROM "OptionsFlow_monthly"
GROUP BY "CreatedDate", "CreatedTime"
ORDER BY count DESC
LIMIT 20;

This will show you the most common date/time formats in your data.

Testing SQL Query

Test the query directly in PostgreSQL:

-- Test with a small date range
SELECT * FROM (
  -- Your full query here with date parameters
) subquery
LIMIT 10;

Performance

If the SQL query is slow:

  1. Check indexes on OptionsFlow_monthly
  2. Verify date range is reasonable
  3. Consider using Python service (faster for complex processing)