# BlackBox Stocks API Integration Guide

This guide explains how to sync options flow data from BlackBox Stocks API to your database.

## Setup

### 1. Get Your API Token

1. Log in to your BlackBox Stocks account
2. Navigate to API settings or developer section
3. Copy your Bearer token (JWT)

### 2. Configure Environment Variable

Add the following to your `.env` file in the `backend` directory:

```env
BLACKBOX_API_TOKEN=your_bearer_token_here
```

**Important**: Never commit your API token to version control. The `.env` file should be in `.gitignore`.

## Usage

### Basic Sync (Today's Data)

Fetch and sync today's flow data:

```bash
cd backend
node scripts/syncBlackBoxFlow.js
```

### Sync Specific Date Range

```bash
node scripts/syncBlackBoxFlow.js --start-date 2024-01-15 --end-date 2024-01-16
```

### Limit Number of Records

```bash
node scripts/syncBlackBoxFlow.js --count 100
# or
node scripts/syncBlackBoxFlow.js --limit 100
```

### Filter by Symbol

```bash
node scripts/syncBlackBoxFlow.js --symbol AAPL
```

### Combine Options

```bash
node scripts/syncBlackBoxFlow.js --start-date 2024-01-15 --end-date 2024-01-15 --limit 500
```

## How It Works

1. **Fetches Data**: The script calls the BlackBox Stocks API endpoint:
   - `POST https://api.blackboxstocks.com/api/v2/options/getFlowMobile`
   - Uses Bearer token authentication

2. **Maps Fields**: The API response is mapped to your database schema (`OptionsFlow_monthly` table):
   - Handles various field name variations from the API
   - Converts dates and times to appropriate formats
   - Maps all 22 database columns

3. **Inserts Records**: Records are inserted in batches of 100:
   - Skips duplicates (if database supports `ON CONFLICT`)
   - Provides progress updates during insertion

## API Response Mapping

The service automatically maps common field name variations:

| Database Column | API Field Variations |
|----------------|---------------------|
| `Symbol` | `symbol`, `Symbol`, `ticker`, `Ticker`, `underlying` |
| `CallPut` | `callPut`, `CallPut`, `optionType`, `OptionType`, `putCall` |
| `Strike` | `strike`, `Strike`, `strikePrice`, `StrikePrice` |
| `Premium` | `premium`, `Premium`, `totalPremium`, `TotalPremium`, `notional` |
| `Volume` | `volume`, `Volume`, `vol`, `Vol`, `contracts`, `Contracts` |
| ... and more |

## Troubleshooting

### Error: "BLACKBOX_API_TOKEN not found"

**Solution**: Add `BLACKBOX_API_TOKEN` to your `.env` file.

### Error: "BlackBox API error: 401 Unauthorized"

**Solution**: 
- Verify your Bearer token is correct
- Check if your token has expired
- Ensure you're using the full token (including `Bearer ` prefix is added automatically)

### Error: "BlackBox API error: 403 Forbidden"

**Solution**: 
- Check your API subscription/plan
- Verify you have access to the `/api/v2/options/getFlowMobile` endpoint

### No Records Returned

**Possible Causes**:
- No flow data for the specified date range
- API rate limiting
- Invalid date format

**Solution**: 
- Try a different date range
- Check the API response structure (the script logs a sample record)
- Verify your API plan includes access to flow data

### Database Insert Errors

**Solution**: 
- Ensure the `OptionsFlow_monthly` table exists
- Check database connection settings
- Verify you have write permissions

## Testing the Connection

You can test the API connection without syncing:

```javascript
import { testBlackBoxConnection } from './src/services/blackboxApiService.js';

const result = await testBlackBoxConnection();
console.log(result);
```

## Automated Syncing

### Using Cron (Linux/Mac)

Add to your crontab to sync every hour:

```bash
0 * * * * cd /path/to/institutional_trader/backend && node scripts/syncBlackBoxFlow.js
```

### Using Task Scheduler (Windows)

1. Open Task Scheduler
2. Create a new task
3. Set trigger (e.g., every hour)
4. Set action: `node scripts/syncBlackBoxFlow.js`
5. Set working directory to `backend` folder

### Using PM2 (Node.js Process Manager)

```bash
# Install PM2
npm install -g pm2

# Create ecosystem file
cat > ecosystem.config.js << EOF
module.exports = {
  apps: [{
    name: 'blackbox-sync',
    script: 'scripts/syncBlackBoxFlow.js',
    cwd: './backend',
    cron_restart: '0 * * * *',  // Every hour
    autorestart: false,
    watch: false
  }]
}
EOF

# Start with PM2
pm2 start ecosystem.config.js
```

## API Endpoint Details

- **URL**: `https://api.blackboxstocks.com/api/v2/options/getFlowMobile`
- **Method**: `POST`
- **Headers**:
  - `Content-Type: application/json`
  - `Accept: application/json`
  - `Authorization: Bearer <your_token>`
- **Body**:
  ```json
  {
    "historical": false,
    "symbol": "",
    "strike": 0,
    "count": 300,
    "filter": 2198487171967,
    "filters": {
      "optionsDate": {
        "start": "2025-12-11T15:36:02.432Z",
        "end": "2025-12-11T15:36:02.432Z"
      },
      "expireOptionsDate": {
        "start": "2025-12-11T15:36:02.432Z",
        "end": "2025-12-11T15:36:02.432Z"
      },
      "optionsFlowPuts": true,
      "optionsFlowCalls": true,
      "optionsFlowStock": true,
      "optionsFlowEtf": true,
      // ... many more filter options
    },
    "fromDate": "2025-12-11T15:36:02.432Z",
    "toDate": "2025-12-11T15:36:02.432Z"
  }
  ```

The service automatically builds this payload structure. You can customize filters by passing a `filters` object in the options.

## Database Schema

The data is inserted into the `OptionsFlow_monthly` table with the following columns:

- `CreatedDate` (TEXT)
- `CreatedTime` (TEXT)
- `Symbol` (TEXT)
- `Type` (TEXT)
- `Volume` (TEXT)
- `Price` (TEXT)
- `Side` (TEXT)
- `CallPut` (TEXT)
- `Strike` (TEXT)
- `Spot` (TEXT)
- `Premium` (TEXT)
- `ExpirationDate` (TEXT)
- `Color` (TEXT)
- `ImpliedVolatility` (TEXT)
- `Dte` (TEXT)
- `ER` (TEXT)
- `StockEtf` (TEXT)
- `Sector` (TEXT)
- `Uoa` (TEXT)
- `Weekly` (TEXT)
- `MktCap` (TEXT)
- `OI` (TEXT)

## Next Steps

After syncing data:

1. **Verify Data**: Check that records were inserted:
   ```sql
   SELECT COUNT(*) FROM "OptionsFlow_monthly";
   SELECT * FROM "OptionsFlow_monthly" ORDER BY "CreatedDate" DESC LIMIT 10;
   ```

2. **Use in Queries**: The synced data will automatically be available in:
   - `/api/options/flow` endpoint
   - Scanner queries
   - Flow analysis features

3. **Set Up Automation**: Configure automated syncing to keep data fresh

## Support

If you encounter issues:

1. Check the console output for detailed error messages
2. Verify your API token is valid
3. Test the API connection using the test function
4. Review the sample API record structure logged by the script