institutional-trader/README/BLACKBOX_SYNC_GUIDE.md

BlackBox Stocks API Integration Guide

This guide explains how to sync options flow data from BlackBox Stocks API to your database.

Setup

1. Get Your API Token

1. Log in to your BlackBox Stocks account
2. Navigate to API settings or developer section
3. Copy your Bearer token (JWT)

2. Configure Environment Variable

Add the following to your .env file in the backend directory:

BLACKBOX_API_TOKEN=your_bearer_token_here

Important: Never commit your API token to version control. The .env file should be in .gitignore.

Usage

Basic Sync (Today's Data)

Fetch and sync today's flow data:

cd backend
node scripts/syncBlackBoxFlow.js

Sync Specific Date Range

node scripts/syncBlackBoxFlow.js --start-date 2024-01-15 --end-date 2024-01-16

Limit Number of Records

node scripts/syncBlackBoxFlow.js --count 100
# or
node scripts/syncBlackBoxFlow.js --limit 100

Filter by Symbol

node scripts/syncBlackBoxFlow.js --symbol AAPL

Combine Options

node scripts/syncBlackBoxFlow.js --start-date 2024-01-15 --end-date 2024-01-15 --limit 500

How It Works

1. Fetches Data: The script calls the BlackBox Stocks API endpoint:

   • POST https://api.blackboxstocks.com/api/v2/options/getFlowMobile
   • Uses Bearer token authentication

2. Maps Fields: The API response is mapped to your database schema (OptionsFlow_monthly table):

   • Handles various field name variations from the API
   • Converts dates and times to appropriate formats
   • Maps all 22 database columns

3. Inserts Records: Records are inserted in batches of 100:

   • Skips duplicates (if database supports ON CONFLICT)
   • Provides progress updates during insertion

API Response Mapping

The service automatically maps common field name variations:

Database Column	API Field Variations
Symbol	symbol, Symbol, ticker, Ticker, underlying
CallPut	callPut, CallPut, optionType, OptionType, putCall
Strike	strike, Strike, strikePrice, StrikePrice
Premium	premium, Premium, totalPremium, TotalPremium, notional
Volume	volume, Volume, vol, Vol, contracts, Contracts
... and more

Troubleshooting

Error: "BLACKBOX_API_TOKEN not found"

Solution: Add BLACKBOX_API_TOKEN to your .env file.

Error: "BlackBox API error: 401 Unauthorized"

Solution:

• Verify your Bearer token is correct
• Check if your token has expired
• Ensure you're using the full token (including Bearer prefix is added automatically)

Error: "BlackBox API error: 403 Forbidden"

Solution:

• Check your API subscription/plan
• Verify you have access to the /api/v2/options/getFlowMobile endpoint

No Records Returned

Possible Causes:

• No flow data for the specified date range
• API rate limiting
• Invalid date format

Solution:

• Try a different date range
• Check the API response structure (the script logs a sample record)
• Verify your API plan includes access to flow data

Database Insert Errors

Solution:

• Ensure the OptionsFlow_monthly table exists
• Check database connection settings
• Verify you have write permissions

Testing the Connection

You can test the API connection without syncing:

import { testBlackBoxConnection } from './src/services/blackboxApiService.js';

const result = await testBlackBoxConnection();
console.log(result);

Automated Syncing

Using Cron (Linux/Mac)

Add to your crontab to sync every hour:

0 * * * * cd /path/to/institutional_trader/backend && node scripts/syncBlackBoxFlow.js

Using Task Scheduler (Windows)

1. Open Task Scheduler
2. Create a new task
3. Set trigger (e.g., every hour)
4. Set action: node scripts/syncBlackBoxFlow.js
5. Set working directory to backend folder

Using PM2 (Node.js Process Manager)

# Install PM2
npm install -g pm2

# Create ecosystem file
cat > ecosystem.config.js << EOF
module.exports = {
  apps: [{
    name: 'blackbox-sync',
    script: 'scripts/syncBlackBoxFlow.js',
    cwd: './backend',
    cron_restart: '0 * * * *',  // Every hour
    autorestart: false,
    watch: false
  }]
}
EOF

# Start with PM2
pm2 start ecosystem.config.js

API Endpoint Details

• URL: https://api.blackboxstocks.com/api/v2/options/getFlowMobile
• Method: POST
• Headers:
  • Content-Type: application/json
  • Accept: application/json
  • Authorization: Bearer <your_token>
• Body:

  {
    "historical": false,
    "symbol": "",
    "strike": 0,
    "count": 300,
    "filter": 2198487171967,
    "filters": {
      "optionsDate": {
        "start": "2025-12-11T15:36:02.432Z",
        "end": "2025-12-11T15:36:02.432Z"
      },
      "expireOptionsDate": {
        "start": "2025-12-11T15:36:02.432Z",
        "end": "2025-12-11T15:36:02.432Z"
      },
      "optionsFlowPuts": true,
      "optionsFlowCalls": true,
      "optionsFlowStock": true,
      "optionsFlowEtf": true,
      // ... many more filter options
    },
    "fromDate": "2025-12-11T15:36:02.432Z",
    "toDate": "2025-12-11T15:36:02.432Z"
  }
  

The service automatically builds this payload structure. You can customize filters by passing a filters object in the options.

Database Schema

The data is inserted into the OptionsFlow_monthly table with the following columns:

• CreatedDate (TEXT)
• CreatedTime (TEXT)
• Symbol (TEXT)
• Type (TEXT)
• Volume (TEXT)
• Price (TEXT)
• Side (TEXT)
• CallPut (TEXT)
• Strike (TEXT)
• Spot (TEXT)
• Premium (TEXT)
• ExpirationDate (TEXT)
• Color (TEXT)
• ImpliedVolatility (TEXT)
• Dte (TEXT)
• ER (TEXT)
• StockEtf (TEXT)
• Sector (TEXT)
• Uoa (TEXT)
• Weekly (TEXT)
• MktCap (TEXT)
• OI (TEXT)

Next Steps

After syncing data:

1. Verify Data: Check that records were inserted:

   SELECT COUNT(*) FROM "OptionsFlow_monthly";
   SELECT * FROM "OptionsFlow_monthly" ORDER BY "CreatedDate" DESC LIMIT 10;
   

2. Use in Queries: The synced data will automatically be available in:

   • /api/options/flow endpoint
   • Scanner queries
   • Flow analysis features

3. Set Up Automation: Configure automated syncing to keep data fresh

Support

If you encounter issues:

1. Check the console output for detailed error messages
2. Verify your API token is valid
3. Test the API connection using the test function
4. Review the sample API record structure logged by the script