institutional-trader/backend/python_service/scripts/README.md

Python Service Scripts

Utility scripts for the Python options flow processing service.

Validation Script

validate_against_sql.py

Compares Python processing output with SQL query output to ensure correctness.

Usage:

cd backend/python_service
python scripts/validate_against_sql.py

What it does:

1. Runs the original SQL query
2. Runs the Python processing pipeline
3. Compares outputs column by column
4. Generates a detailed comparison report
5. Saves report to validation_report.json

Output:

• Console report showing differences
• JSON file with detailed comparison
• Highlights missing columns, value differences, etc.

Example output:

============================================================
PYTHON vs SQL OUTPUT COMPARISON
============================================================

Row Counts:
  Python: 150
  SQL:    150
  Match:  ✅

📊 Column Differences:
  Column: premium_num (numeric)
    Max difference: 0.000001
    Mean difference: 0.000000
    Different rows: 0

✅ No significant differences found!

Running Scripts

All scripts should be run from the backend/python_service directory:

cd backend/python_service

# Activate virtual environment
source venv/bin/activate  # Linux/Mac
# or
venv\Scripts\activate  # Windows

# Run script
python scripts/validate_against_sql.py

Environment

Scripts use the same environment variables as the main service:

• Database connection (from .env)
• Same logging configuration
• Same error handling