270 lines
6.5 KiB
Markdown
270 lines
6.5 KiB
Markdown
# BlackBox Stocks API Integration Guide
|
|
|
|
This guide explains how to sync options flow data from BlackBox Stocks API to your database.
|
|
|
|
## Setup
|
|
|
|
### 1. Get Your API Token
|
|
|
|
1. Log in to your BlackBox Stocks account
|
|
2. Navigate to API settings or developer section
|
|
3. Copy your Bearer token (JWT)
|
|
|
|
### 2. Configure Environment Variable
|
|
|
|
Add the following to your `.env` file in the `backend` directory:
|
|
|
|
```env
|
|
BLACKBOX_API_TOKEN=your_bearer_token_here
|
|
```
|
|
|
|
**Important**: Never commit your API token to version control. The `.env` file should be in `.gitignore`.
|
|
|
|
## Usage
|
|
|
|
### Basic Sync (Today's Data)
|
|
|
|
Fetch and sync today's flow data:
|
|
|
|
```bash
|
|
cd backend
|
|
node scripts/syncBlackBoxFlow.js
|
|
```
|
|
|
|
### Sync Specific Date Range
|
|
|
|
```bash
|
|
node scripts/syncBlackBoxFlow.js --start-date 2024-01-15 --end-date 2024-01-16
|
|
```
|
|
|
|
### Limit Number of Records
|
|
|
|
```bash
|
|
node scripts/syncBlackBoxFlow.js --count 100
|
|
# or
|
|
node scripts/syncBlackBoxFlow.js --limit 100
|
|
```
|
|
|
|
### Filter by Symbol
|
|
|
|
```bash
|
|
node scripts/syncBlackBoxFlow.js --symbol AAPL
|
|
```
|
|
|
|
### Combine Options
|
|
|
|
```bash
|
|
node scripts/syncBlackBoxFlow.js --start-date 2024-01-15 --end-date 2024-01-15 --limit 500
|
|
```
|
|
|
|
## How It Works
|
|
|
|
1. **Fetches Data**: The script calls the BlackBox Stocks API endpoint:
|
|
- `POST https://api.blackboxstocks.com/api/v2/options/getFlowMobile`
|
|
- Uses Bearer token authentication
|
|
|
|
2. **Maps Fields**: The API response is mapped to your database schema (`OptionsFlow_monthly` table):
|
|
- Handles various field name variations from the API
|
|
- Converts dates and times to appropriate formats
|
|
- Maps all 22 database columns
|
|
|
|
3. **Inserts Records**: Records are inserted in batches of 100:
|
|
- Skips duplicates (if database supports `ON CONFLICT`)
|
|
- Provides progress updates during insertion
|
|
|
|
## API Response Mapping
|
|
|
|
The service automatically maps common field name variations:
|
|
|
|
| Database Column | API Field Variations |
|
|
|----------------|---------------------|
|
|
| `Symbol` | `symbol`, `Symbol`, `ticker`, `Ticker`, `underlying` |
|
|
| `CallPut` | `callPut`, `CallPut`, `optionType`, `OptionType`, `putCall` |
|
|
| `Strike` | `strike`, `Strike`, `strikePrice`, `StrikePrice` |
|
|
| `Premium` | `premium`, `Premium`, `totalPremium`, `TotalPremium`, `notional` |
|
|
| `Volume` | `volume`, `Volume`, `vol`, `Vol`, `contracts`, `Contracts` |
|
|
| ... and more |
|
|
|
|
## Troubleshooting
|
|
|
|
### Error: "BLACKBOX_API_TOKEN not found"
|
|
|
|
**Solution**: Add `BLACKBOX_API_TOKEN` to your `.env` file.
|
|
|
|
### Error: "BlackBox API error: 401 Unauthorized"
|
|
|
|
**Solution**:
|
|
- Verify your Bearer token is correct
|
|
- Check if your token has expired
|
|
- Ensure you're using the full token (including `Bearer ` prefix is added automatically)
|
|
|
|
### Error: "BlackBox API error: 403 Forbidden"
|
|
|
|
**Solution**:
|
|
- Check your API subscription/plan
|
|
- Verify you have access to the `/api/v2/options/getFlowMobile` endpoint
|
|
|
|
### No Records Returned
|
|
|
|
**Possible Causes**:
|
|
- No flow data for the specified date range
|
|
- API rate limiting
|
|
- Invalid date format
|
|
|
|
**Solution**:
|
|
- Try a different date range
|
|
- Check the API response structure (the script logs a sample record)
|
|
- Verify your API plan includes access to flow data
|
|
|
|
### Database Insert Errors
|
|
|
|
**Solution**:
|
|
- Ensure the `OptionsFlow_monthly` table exists
|
|
- Check database connection settings
|
|
- Verify you have write permissions
|
|
|
|
## Testing the Connection
|
|
|
|
You can test the API connection without syncing:
|
|
|
|
```javascript
|
|
import { testBlackBoxConnection } from './src/services/blackboxApiService.js';
|
|
|
|
const result = await testBlackBoxConnection();
|
|
console.log(result);
|
|
```
|
|
|
|
## Automated Syncing
|
|
|
|
### Using Cron (Linux/Mac)
|
|
|
|
Add to your crontab to sync every hour:
|
|
|
|
```bash
|
|
0 * * * * cd /path/to/institutional_trader/backend && node scripts/syncBlackBoxFlow.js
|
|
```
|
|
|
|
### Using Task Scheduler (Windows)
|
|
|
|
1. Open Task Scheduler
|
|
2. Create a new task
|
|
3. Set trigger (e.g., every hour)
|
|
4. Set action: `node scripts/syncBlackBoxFlow.js`
|
|
5. Set working directory to `backend` folder
|
|
|
|
### Using PM2 (Node.js Process Manager)
|
|
|
|
```bash
|
|
# Install PM2
|
|
npm install -g pm2
|
|
|
|
# Create ecosystem file
|
|
cat > ecosystem.config.js << EOF
|
|
module.exports = {
|
|
apps: [{
|
|
name: 'blackbox-sync',
|
|
script: 'scripts/syncBlackBoxFlow.js',
|
|
cwd: './backend',
|
|
cron_restart: '0 * * * *', // Every hour
|
|
autorestart: false,
|
|
watch: false
|
|
}]
|
|
}
|
|
EOF
|
|
|
|
# Start with PM2
|
|
pm2 start ecosystem.config.js
|
|
```
|
|
|
|
## API Endpoint Details
|
|
|
|
- **URL**: `https://api.blackboxstocks.com/api/v2/options/getFlowMobile`
|
|
- **Method**: `POST`
|
|
- **Headers**:
|
|
- `Content-Type: application/json`
|
|
- `Accept: application/json`
|
|
- `Authorization: Bearer <your_token>`
|
|
- **Body**:
|
|
```json
|
|
{
|
|
"historical": false,
|
|
"symbol": "",
|
|
"strike": 0,
|
|
"count": 300,
|
|
"filter": 2198487171967,
|
|
"filters": {
|
|
"optionsDate": {
|
|
"start": "2025-12-11T15:36:02.432Z",
|
|
"end": "2025-12-11T15:36:02.432Z"
|
|
},
|
|
"expireOptionsDate": {
|
|
"start": "2025-12-11T15:36:02.432Z",
|
|
"end": "2025-12-11T15:36:02.432Z"
|
|
},
|
|
"optionsFlowPuts": true,
|
|
"optionsFlowCalls": true,
|
|
"optionsFlowStock": true,
|
|
"optionsFlowEtf": true,
|
|
// ... many more filter options
|
|
},
|
|
"fromDate": "2025-12-11T15:36:02.432Z",
|
|
"toDate": "2025-12-11T15:36:02.432Z"
|
|
}
|
|
```
|
|
|
|
The service automatically builds this payload structure. You can customize filters by passing a `filters` object in the options.
|
|
|
|
## Database Schema
|
|
|
|
The data is inserted into the `OptionsFlow_monthly` table with the following columns:
|
|
|
|
- `CreatedDate` (TEXT)
|
|
- `CreatedTime` (TEXT)
|
|
- `Symbol` (TEXT)
|
|
- `Type` (TEXT)
|
|
- `Volume` (TEXT)
|
|
- `Price` (TEXT)
|
|
- `Side` (TEXT)
|
|
- `CallPut` (TEXT)
|
|
- `Strike` (TEXT)
|
|
- `Spot` (TEXT)
|
|
- `Premium` (TEXT)
|
|
- `ExpirationDate` (TEXT)
|
|
- `Color` (TEXT)
|
|
- `ImpliedVolatility` (TEXT)
|
|
- `Dte` (TEXT)
|
|
- `ER` (TEXT)
|
|
- `StockEtf` (TEXT)
|
|
- `Sector` (TEXT)
|
|
- `Uoa` (TEXT)
|
|
- `Weekly` (TEXT)
|
|
- `MktCap` (TEXT)
|
|
- `OI` (TEXT)
|
|
|
|
## Next Steps
|
|
|
|
After syncing data:
|
|
|
|
1. **Verify Data**: Check that records were inserted:
|
|
```sql
|
|
SELECT COUNT(*) FROM "OptionsFlow_monthly";
|
|
SELECT * FROM "OptionsFlow_monthly" ORDER BY "CreatedDate" DESC LIMIT 10;
|
|
```
|
|
|
|
2. **Use in Queries**: The synced data will automatically be available in:
|
|
- `/api/options/flow` endpoint
|
|
- Scanner queries
|
|
- Flow analysis features
|
|
|
|
3. **Set Up Automation**: Configure automated syncing to keep data fresh
|
|
|
|
## Support
|
|
|
|
If you encounter issues:
|
|
|
|
1. Check the console output for detailed error messages
|
|
2. Verify your API token is valid
|
|
3. Test the API connection using the test function
|
|
4. Review the sample API record structure logged by the script
|
|
|