institutional-trader/README/CHEDDARFLOW_SYNC_GUIDE.md

CheddarFlow Live Option Flow Integration Guide

This guide explains how to use the CheddarFlow WebSocket service to receive live option flow data.

Quick Start

1. Get authentication tokens from browser DevTools (see Authentication section below)
2. Add to .env file:

   CHEDDARFLOW_SESSION=your_session_token
   CHEDDARFLOW_TOKEN=your_jwt_token
   

3. Find the GraphQL subscription query from WebSocket messages (see GraphQL Subscription Query section)
4. Set the query in .env:

   CHEDDARFLOW_SUBSCRIPTION_QUERY="subscription { ... }"
   

5. Start the server - the service will automatically connect

Overview

The CheddarFlow WebSocket service connects to wss://app.cheddarflow.com/api/ws using the GraphQL Transport WebSocket protocol (graphql-transport-ws) to receive real-time option flow data.

Setup

1. Enable the Service

The service is enabled by default. To disable it, add to your .env file:

ENABLE_CHEDDARFLOW=false

2. Database Storage

By default, received data is automatically saved to the OptionsFlow_monthly table. To disable database storage:

CHEDDARFLOW_SAVE_TO_DB=false

3. Authentication (Required)

CheddarFlow requires authentication. You have two options:

Option 1: Cookies (Recommended - Easiest)

Get cookies from your browser after logging into CheddarFlow:

1. Open CheddarFlow dashboard in your browser
2. Open DevTools (F12) → Application tab → Cookies
3. Copy all cookies for app.cheddarflow.com or dash.cheddarflow.com
4. Format them as: cookie1=value1; cookie2=value2; cookie3=value3

Add to .env:

CHEDDARFLOW_COOKIES=your_cookie_string_here

Example:

CHEDDARFLOW_COOKIES=auth0_session=xxx; auth0_csrf=yyy; other_cookie=zzz

Option 2: Session + Token

If you prefer using session and token separately:

# Session token
CHEDDARFLOW_SESSION=your_session_token_here

# JWT token
CHEDDARFLOW_TOKEN=your_jwt_token_here

How to get your authentication:

Method 1: Cookies (Easiest - Recommended)

1. Open CheddarFlow dashboard in your browser (while logged in)
2. Open DevTools (F12) → Application tab → Cookies
3. Select app.cheddarflow.com or dash.cheddarflow.com
4. Copy all cookie name-value pairs
5. Format as: name1=value1; name2=value2; name3=value3
6. Add to .env as CHEDDARFLOW_COOKIES

Important: Cookies may include session tokens that expire. Refresh them if connection fails.

Method 2: From WebSocket Messages

1. Open CheddarFlow dashboard in your browser
2. Open DevTools (F12) → Network tab
3. Filter by "WS" (WebSocket)
4. Connect to the WebSocket (refresh page or navigate to a page that uses live data)
5. Look for the first message with type: "connection_init"
6. Copy the session and token values from the payload object

Example payload structure:

{
  "type": "connection_init",
  "payload": {
    "session": "C7vnUHLdb9TORyCGmdOtb",
    "token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImdLSEdIb1UxWTAxaE1JcmwzLUpSdiJ9..."
  }
}

Method 2: From GraphQL API Response

1. Open CheddarFlow dashboard in your browser
2. Open DevTools (F12) → Network tab
3. Filter by "Fetch/XHR"
4. Look for a POST request to https://app.cheddarflow.com/api
5. Find the request with operationName: "GetUserSessionAndMarketStatus"
6. Copy the token from data.session.token in the response
7. For session, check cookies or use the WebSocket method above

Method 3: Programmatic Fetch (Advanced)

You can use the helper function to fetch tokens programmatically, but this requires authentication cookies:

import { fetchCheddarFlowSession } from './src/services/cheddarflowWebSocketService.js';

// Get cookies from browser DevTools → Application → Cookies
const cookies = 'your_cookie_string_here';

const sessionData = await fetchCheddarFlowSession({
  cookie: cookies
});

console.log('Token:', sessionData.token);

Note: These tokens may expire. If the connection fails, refresh your tokens from the browser DevTools.

4. Configuration Options

Add these optional settings to your .env file:

# Enable/disable CheddarFlow service (default: true)
ENABLE_CHEDDARFLOW=true

# Save received data to database (default: true)
CHEDDARFLOW_SAVE_TO_DB=true

# WebSocket URL (default: wss://app.cheddarflow.com/api/ws)
CHEDDARFLOW_WS_URL=wss://app.cheddarflow.com/api/ws

# Origin header (default: https://dash.cheddarflow.com)
CHEDDARFLOW_ORIGIN=https://dash.cheddarflow.com

# Feed ID and Topic for options feed subscription (required)
CHEDDARFLOW_FEED_ID=1612407366
CHEDDARFLOW_FEED_TOPIC=1612407366

How It Works

1. Connection: The service automatically connects to CheddarFlow's WebSocket on server startup
2. GraphQL Protocol: Uses the graphql-transport-ws protocol for GraphQL subscriptions
3. Data Mapping: Incoming data is automatically mapped to the OptionsFlow_monthly schema
4. Database Storage: Records are inserted into the database in real-time
5. Auto-Reconnect: Automatically reconnects if the connection is lost

GraphQL Subscription Query

The service uses the OnOptionsFeedEvents subscription which provides real-time option flow data. The subscription is already configured and ready to use.

Feed ID Configuration

The subscription requires a feed ID and topic (usually the same value). This appears to be a view/feed identifier in CheddarFlow.

To find your feed ID:

1. Open CheddarFlow dashboard in browser DevTools
2. Go to Network tab → Filter by "WS" (WebSocket)
3. Look for a subscribe message with operationName: "OnOptionsFeedEvents"
4. Copy the id and topic values from payload.variables

Example:

{
  "type": "subscribe",
  "payload": {
    "variables": {
      "id": "1612407366",
      "topic": "1612407366"
    }
  }
}

Add to .env file:

CHEDDARFLOW_FEED_ID=1612407366
CHEDDARFLOW_FEED_TOPIC=1612407366

If not set, the service defaults to 1612407366 (you may need to update this with your own feed ID).

Data Format

CheddarFlow sends option flow data in a tabular format within the lastProcessedItems field:

{
  "lastProcessedItems": "{\"ins\":[[\"headers\"],[[\"data\"]]]}"
}

The service automatically:

1. Parses the JSON string from lastProcessedItems
2. Extracts the data rows (after the header row)
3. Maps each row to the database schema
4. Inserts records into OptionsFlow_monthly table

Data Fields

The subscription provides these fields (in order):

• idx, id, contract_id, timestamp, symbol, expiry, strike, put_call, side, buy_sell, spot, size, price, premium, sweep_block_split, volume, open_int, conds

These are automatically mapped to the OptionsFlow_monthly schema.

Data Mapping

The service automatically maps CheddarFlow data fields to the OptionsFlow_monthly schema:

Database Column	CheddarFlow Field Variations
Symbol	symbol, Symbol, ticker, Ticker
CallPut	callPut, CallPut, optionType, OptionType
Strike	strike, Strike, strikePrice, StrikePrice
Premium	premium, Premium, totalPremium, TotalPremium
Volume	volume, Volume, vol, Vol, contracts
Spot	spot, Spot, underlyingPrice, stockPrice
Side	side, Side, tradeSide, direction
ExpirationDate	expirationDate, ExpirationDate, expiry, expiration
... and more	See mapCheddarFlowToDatabase() function

Monitoring

Health Check

Check service status via the health endpoint:

curl http://localhost:3010/health

Response includes cheddarFlow status:

{
  "cheddarFlow": {
    "running": true,
    "connected": true,
    "subscriptionId": "sub_1234567890",
    "reconnectAttempts": 0,
    "status": "connected"
  }
}

Logs

The service logs connection status and data processing:

📡 CheddarFlow: Connecting to wss://app.cheddarflow.com/api/ws...
📡 CheddarFlow: WebSocket opened
📡 CheddarFlow: Connection acknowledged
📡 CheddarFlow: Subscription started
📡 CheddarFlow: Inserted 10 records

Troubleshooting

Service Not Connecting

Symptoms: Logs show connection attempts but no connection

Solutions:

1. Check your network connection
2. Verify the WebSocket URL is correct
3. Check if CheddarFlow requires authentication (may need to add auth headers)
4. Review firewall/proxy settings

No Data Received

Symptoms: Connected but no data flowing

Solutions:

1. Verify the GraphQL subscription query is correct
2. Check if CheddarFlow requires authentication tokens
3. Inspect WebSocket messages to see what CheddarFlow is sending
4. Check if there's actual flow data available (market hours, etc.)

Database Insert Errors

Symptoms: Connection works but records aren't being saved

Solutions:

1. Verify database connection is working
2. Check that OptionsFlow_monthly table exists
3. Review database logs for constraint violations
4. Ensure CHEDDARFLOW_SAVE_TO_DB is not set to false

Authentication Required

CheddarFlow requires session and token in the connection_init payload. The service automatically includes these if set in environment variables:

CHEDDARFLOW_SESSION=your_session_token
CHEDDARFLOW_TOKEN=your_jwt_token

If these are missing, you'll see a warning in the logs and the connection will likely fail.

To get fresh tokens:

1. Open CheddarFlow dashboard in browser
2. DevTools → Network → WS filter
3. Find connection_init message
4. Copy session and token from payload
5. Update your .env file

Token expiration: JWT tokens typically expire after some time. If connection fails, refresh your tokens from the browser.

Manual Testing

You can test the service manually:

import { startCheddarFlowService, stopCheddarFlowService, getCheddarFlowStatus } from './src/services/cheddarflowWebSocketService.js';

// Start service
const client = startCheddarFlowService({
  enableDatabaseInsert: true,
  onData: (mappedRecord, rawData) => {
    console.log('Received flow:', mappedRecord);
  },
  onError: (error) => {
    console.error('Error:', error);
  }
});

// Check status
setTimeout(() => {
  const status = getCheddarFlowStatus();
  console.log('Status:', status);
}, 5000);

// Stop service (when done)
// stopCheddarFlowService();

Next Steps

1. Determine GraphQL Query: Find the actual subscription query from CheddarFlow
2. Update Subscription: Replace the placeholder query in startSubscription()
3. Add Authentication: If required, add authentication tokens
4. Customize Mapping: Adjust field mappings if CheddarFlow uses different field names
5. Test: Verify data is flowing and being saved correctly

API Reference

Functions

• startCheddarFlowService(options) - Start the WebSocket service
• stopCheddarFlowService() - Stop the service
• getCheddarFlowStatus() - Get current service status
• mapCheddarFlowToDatabase(data) - Map CheddarFlow data to database schema

Options

{
  url: 'wss://app.cheddarflow.com/api/ws',  // WebSocket URL
  origin: 'https://dash.cheddarflow.com',   // Origin header
  enableDatabaseInsert: true,                // Save to database
  autoReconnect: true,                       // Auto-reconnect on disconnect
  reconnectInterval: 5000,                   // Reconnect delay (ms)
  maxReconnectAttempts: 10,                  // Max reconnect attempts
  onData: (mappedRecord, rawData) => {},     // Data callback
  onError: (error) => {}                     // Error callback
}

Support

If you encounter issues:

1. Check the console logs for detailed error messages
2. Verify the WebSocket connection in browser DevTools
3. Test the GraphQL subscription query manually
4. Review the data mapping function for field name mismatches