35 KiB

Raw Blame History

🚀 Proxmox Deployment Guide

Complete guide for deploying the Institutional Trader platform on a Proxmox server.

Prerequisites
Proxmox VM/Container Setup
System Dependencies
Application Setup
Database Configuration
Systemd Services
Nginx Configuration
DNS Configuration
SSL Certificates
Database Access for Friend
Monitoring & Maintenance
Troubleshooting

1. Prerequisites

Proxmox server with network access
Domain name pointing to your server's IP (for SSL)
SSH access to the Proxmox host
Basic knowledge of Linux command line

2. Proxmox VM/Container Setup

Option A: Ubuntu LXC Container (Recommended - Lightweight)

Create LXC Container:
- In Proxmox Web UI, go to your node
- Click "Create CT" (Container)
- Choose Ubuntu 22.04 or 24.04 template
- Configure:
  - Hostname: institutional-trader
  - Password: Set root password
  - CPU: 2-4 cores
  - Memory: 4-8 GB RAM
  - Disk: 20-50 GB
  - Network: Bridge with static IP (recommended) or DHCP

Start Container:

# In Proxmox shell or via Web UI
pct start <container-id>

Access Container:

# From Proxmox host
pct enter <container-id>

# Or SSH if you configured it
ssh root@<container-ip>

Option B: Ubuntu VM (More Isolation)

Create VM:
- In Proxmox Web UI, go to your node
- Click "Create VM"
- Choose Ubuntu 22.04 or 24.04 ISO
- Configure:
  - CPU: 2-4 cores
  - Memory: 4-8 GB RAM
  - Disk: 50-100 GB
  - Network: Bridge adapter
- Install Ubuntu Server (minimal installation)
Access VM:
```
ssh <user>@<vm-ip>
```

3. System Dependencies

Run these commands on your Ubuntu container/VM:

# Update system
apt update && apt upgrade -y

# Install Node.js 20.x (LTS)
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt install -y nodejs

# Install Python 3.11+ and pip
apt install -y python3 python3-pip python3-venv

# Install PostgreSQL client (if using local DB)
apt install -y postgresql-client

# Install Nginx
apt install -y nginx

# Install Git
apt install -y git

# Install build tools (for native modules)
apt install -y build-essential

# Install PM2 (process manager - optional but recommended)
npm install -g pm2

# Install Uvicorn (for Python service)
pip3 install uvicorn

# Verify installations
node --version  # Should be v20.x.x
python3 --version  # Should be 3.11+
nginx -v

4. Application Setup

4.1 Clone and Prepare Repository

# Create application directory
mkdir -p /opt/institutional_trader
cd /opt/institutional_trader

# Clone your repository (or upload files)
# Option 1: If using Git
git clone <your-repo-url> .

# Option 2: If uploading files manually
# Use SCP or Proxmox file browser to upload files

4.2 Backend Setup

cd /opt/institutional_trader/backend

# Install dependencies
npm install --production

# Create production .env file
cp env.production.example .env
nano .env

Backend .env configuration:

NODE_ENV=production
PORT=3000

# Supabase Configuration (if using Supabase)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_ANON_KEY=your_anon_key
SUPABASE_SERVICE_KEY=your_service_role_key

# OR Direct PostgreSQL Connection (if using local/remote PostgreSQL)
DATABASE_URL=postgresql://postgres:password@localhost:5432/institutional_trader

# CORS Configuration
CORS_ORIGIN=https://yourdomain.com

# Security
JWT_SECRET=your_super_secret_jwt_key_min_32_chars_long
RATE_LIMIT_WINDOW=15
RATE_LIMIT_MAX=100

# Python Service (if using)
PYTHON_SERVICE_URL=http://localhost:8010

4.3 Python Service Setup

cd /opt/institutional_trader/backend/python_service

# Create virtual environment
python3 -m venv venv

# Activate virtual environment
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Deactivate (we'll use systemd to manage this)
deactivate

4.4 Frontend Setup

cd /opt/institutional_trader/frontend

# Install dependencies
npm install

# Create production .env file
cp env.production.example .env.production
nano .env.production

Frontend .env.production configuration:

⚠️ Critical: The frontend must have a .env.production file with the correct API URL, otherwise it will try to connect to localhost:3010 which won't work in production.

VITE_API_URL=https://api.yourdomain.com
VITE_WS_URL=wss://api.yourdomain.com
VITE_SUPABASE_URL=https://your-project.supabase.co
VITE_SUPABASE_ANON_KEY=your_anon_key

Build frontend:

npm run build

This creates a dist folder with production-ready files.

5. Database Configuration

Option A: Using Supabase (Cloud Database - Recommended)

Set up Supabase:
- Go to supabase.com
- Create a new project
- Get your connection details from Settings > API
- Run database schema scripts from backend/database/ in Supabase SQL Editor
Connection String:
- Use DATABASE_URL or SUPABASE_URL + keys in backend .env
- See Database Access for Friend section for sharing access

Option B: Local PostgreSQL (Self-Hosted)

# Install PostgreSQL
apt install -y postgresql postgresql-contrib

# Start PostgreSQL
systemctl start postgresql
systemctl enable postgresql

# Create database and user
sudo -u postgres psql

# In PostgreSQL prompt:
CREATE DATABASE institutional_trader;
CREATE USER trader_user WITH PASSWORD 'secure_password_here';
GRANT ALL PRIVILEGES ON DATABASE institutional_trader TO trader_user;
\q

# Import schema
cd /opt/institutional_trader/backend/database
sudo -u postgres psql institutional_trader < schema.sql
sudo -u postgres psql institutional_trader < missing_tables.sql
# Import other SQL files as needed

Update backend .env:

USE_LOCAL_DB=true
LOCAL_DB_HOST=localhost
LOCAL_DB_PORT=5432
LOCAL_DB_USER=trader_user
LOCAL_DB_PASSWORD=secure_password_here
LOCAL_DB_NAME=institutional_trader

6. Systemd Services

6.1 Backend Service

Create systemd service for Node.js backend:

sudo nano /etc/systemd/system/institutional-trader-backend.service

Add:

[Unit]
Description=Institutional Trader Backend API
After=network.target postgresql.service

[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend
Environment=NODE_ENV=production
EnvironmentFile=/opt/institutional_trader/backend/.env
ExecStart=/usr/bin/node src/server.js
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-backend
sudo systemctl start institutional-trader-backend
sudo systemctl status institutional-trader-backend

6.2 Python Service

Create systemd service for Python service:

sudo nano /etc/systemd/system/institutional-trader-python.service

Add:

[Unit]
Description=Institutional Trader Python Service
After=network.target postgresql.service

[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend/python_service
Environment="PATH=/opt/institutional_trader/backend/python_service/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
EnvironmentFile=/opt/institutional_trader/backend/python_service/.env
ExecStart=/opt/institutional_trader/backend/python_service/venv/bin/uvicorn main:app --host 0.0.0.0 --port 8010
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Note: The EnvironmentFile directive is optional - the service will also load .env automatically via load_dotenv() in the code. However, adding it explicitly ensures environment variables are available even if the code path changes.

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-python
sudo systemctl start institutional-trader-python
sudo systemctl status institutional-trader-python

6.3 View Logs

# Backend logs
sudo journalctl -u institutional-trader-backend -f

# Python service logs
sudo journalctl -u institutional-trader-python -f

# All logs
sudo journalctl -u institutional-trader-* -f

7. Nginx Configuration

7.1 Create Nginx Configuration

sudo nano /etc/nginx/sites-available/institutional-trader

Copy and customize the configuration (replace yourdomain.com with your domain):

# Backend API Server
server {
    listen 443 ssl http2;
    server_name api.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    # SSL Configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # API Routes
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
    }

    # WebSocket Support
    location /ws {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # WebSocket specific timeouts
        proxy_read_timeout 86400;
    }

    # Health Check
    location /health {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }
}

# Frontend Server
server {
    listen 443 ssl http2;
    server_name yourdomain.com www.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    # SSL Configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    root /opt/institutional_trader/frontend/dist;
    index index.html;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # Gzip Compression
    gzip on;
    gzip_vary on;
    gzip_min_length 1024;
    gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml+rss application/json;

    location / {
        try_files $uri $uri/ /index.html;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }

    # Static assets caching
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }
}

# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com api.yourdomain.com;
    return 301 https://$server_name$request_uri;
}

7.2 Multi-Container Setup (Nginx on Separate Container)

If you're running nginx on a separate container/VM that proxies to your application containers:

Example configuration for proxying to another Proxmox container:

server {
    listen 443 ssl http2;
    server_name traderideas.deepteklabs.com;

    ssl_certificate /etc/letsencrypt/live/traderideas.deepteklabs.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/traderideas.deepteklabs.com/privkey.pem;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header X-Content-Type-Options "nosniff" always;

    # Backend API Routes
    location /api/ {
        proxy_pass http://192.168.8.151:3000/api/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
        proxy_buffering off;
    }

    # WebSocket Support (if using WebSockets)
    location /ws {
        proxy_pass http://192.168.8.151:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 86400;  # 24 hours for long-lived connections
    }

    # Health Check
    location /health {
        proxy_pass http://192.168.8.151:3000/health;
        access_log off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }

    # Frontend (served from application container)
    location / {
        proxy_pass http://192.168.8.151:8080/;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_hide_header Cache-Control;
        add_header Cache-Control "no-cache, no-store, must-revalidate" always;
    }

    # Static assets with caching
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot|otf|map)$ {
        proxy_pass http://192.168.8.151:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        expires 1y;
        add_header Cache-Control "public, immutable";
        access_log off;
    }
}

Important notes for multi-container setup:

Replace 192.168.8.151 with your application container's IP
Ensure containers can communicate (same network/bridge)
Port 3000 = Backend API, Port 8080 = Frontend (adjust as needed)
Test connectivity: curl http://192.168.8.151:3000/health from nginx container

7.3 Enable Site

# Create symlink
sudo ln -s /etc/nginx/sites-available/institutional-trader /etc/nginx/sites-enabled/

# Test configuration
sudo nginx -t

# Reload Nginx
sudo systemctl reload nginx

8. DNS Configuration

⚠️ Important: Before setting up SSL certificates, you must configure DNS records pointing to your server.

8.1 Find Your Server's Public IP Address

# On your server, find the public IP
curl ifconfig.me
# Or
hostname -I
# Or check in Proxmox Web UI: Datacenter → Your Node → Network

8.2 Configure DNS A Records

Go to your domain registrar's DNS management panel (where you registered traderideas.deepteklabs.com) and add these A records:

Type	Name	Value	TTL
A	`@` (or blank)	`<your-server-ip>`	3600
A	`www`	`<your-server-ip>`	3600
A	`api`	`<your-server-ip>`	3600

Example for traderideas.deepteklabs.com:

traderideas.deepteklabs.com → <your-server-ip> (A record with name @ or blank)
www.traderideas.deepteklabs.com → <your-server-ip> (A record with name www)
api.traderideas.deepteklabs.com → <your-server-ip> (A record with name api)

8.3 Verify DNS Propagation

Wait 5-15 minutes for DNS to propagate, then verify:

# Check if DNS records are resolving
dig traderideas.deepteklabs.com +short
dig www.traderideas.deepteklabs.com +short
dig api.traderideas.deepteklabs.com +short

# All should return your server's IP address

# Alternative: Use nslookup
nslookup traderideas.deepteklabs.com
nslookup www.traderideas.deepteklabs.com
nslookup api.traderideas.deepteklabs.com

# Or use online tools:
# - https://dnschecker.org
# - https://www.whatsmydns.net

Common DNS Providers:

Cloudflare: Dashboard → DNS → Records → Add record
Namecheap: Domain List → Manage → Advanced DNS → Add New Record
GoDaddy: DNS Management → Add Record
Google Domains: DNS → Custom Records → Add Record

8.4 Test Domain Accessibility

Once DNS is propagated, test that domains are reachable:

# From your server
curl -I http://traderideas.deepteklabs.com
curl -I http://www.traderideas.deepteklabs.com
curl -I http://api.traderideas.deepteklabs.com

# Should return HTTP responses (even if 502/503, that's OK - means DNS is working)

8.5 Cloudflare Proxy (Orange Cloud)

If your domain uses Cloudflare proxy (orange cloud icon):

When DNS resolves to Cloudflare IPs (like 104.21.x.x or 172.67.x.x), your domain is behind Cloudflare proxy. This affects SSL certificate setup:

Check if using Cloudflare proxy:

dig traderideas.deepteklabs.com +short
# If returns Cloudflare IPs (104.x.x.x, 172.x.x.x), proxy is enabled

Options for SSL with Cloudflare:

Use DNS Challenge (Recommended) - Works with proxy enabled
Temporarily Disable Proxy - Get certs, then re-enable proxy
Use Cloudflare SSL - Let Cloudflare handle SSL (easiest)

See Section 9.2 for detailed instructions.

⚠️ Only proceed to SSL setup once DNS records are working!

9. SSL Certificates

9.1 Install Certbot

sudo apt install -y certbot python3-certbot-nginx

9.2 Obtain SSL Certificate

⚠️ Prerequisites:

DNS A records must be configured and propagated (see Section 8)
Domains must resolve to your server's IP (or Cloudflare if using proxy)
Port 80 must be accessible from the internet (for HTTP challenge) OR DNS API access (for DNS challenge)

⚠️ Important: If your nginx config already references SSL certificates that don't exist, you need to fix this first.

Check if using Cloudflare proxy:

dig traderideas.deepteklabs.com +short
# If returns Cloudflare IPs (104.x.x.x, 172.x.x.x), you're using Cloudflare proxy

If using Cloudflare Proxy, use Option D (DNS Challenge) or Option E (Cloudflare SSL)

Option A: Use Standalone Mode (Works if NOT using Cloudflare proxy)

# Stop nginx temporarily
sudo systemctl stop nginx

# Get certificates using standalone mode
sudo certbot certonly --standalone -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com

# Start nginx again
sudo systemctl start nginx

# Now certbot can update the nginx config
sudo certbot --nginx -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com

Option B: Temporarily Comment Out SSL in Nginx Config

# Edit nginx config
sudo nano /etc/nginx/sites-available/institutional-trader

# Temporarily comment out SSL lines:
# - Comment out: listen 443 ssl http2;
# - Comment out: ssl_certificate and ssl_certificate_key lines
# - Change: listen 443 ssl http2; to: listen 80;

# Example temporary config (HTTP only):
# server {
#     listen 80;
#     server_name traderideas.deepteklabs.com;
#     # ssl_certificate /etc/letsencrypt/live/traderideas.deepteklabs.com/fullchain.pem;
#     # ssl_certificate_key /etc/letsencrypt/live/traderideas.deepteklabs.com/privkey.pem;
#     ...
# }

# Test and reload
sudo nginx -t
sudo systemctl reload nginx

# Now get certificates (certbot will automatically update the config)
sudo certbot --nginx -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com

# Certbot will automatically:
# - Get the certificates
# - Update your nginx config to use them
# - Add SSL redirects

Option C: Use Certbot with Nginx Plugin (If config is clean and NOT using Cloudflare proxy)

# Replace with your actual domain
sudo certbot --nginx -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com

Follow the prompts:

Enter your email address
Agree to terms
Choose whether to redirect HTTP to HTTPS (recommended: Yes)

Option D: DNS Challenge (Recommended for Cloudflare proxy)

If your domain is behind Cloudflare proxy, use DNS challenge:

# Install Cloudflare plugin for certbot
sudo apt install -y python3-pip
sudo pip3 install certbot-dns-cloudflare

# Create Cloudflare API token directory
sudo mkdir -p /etc/letsencrypt
sudo chmod 700 /etc/letsencrypt

# Create Cloudflare credentials file
sudo nano /etc/letsencrypt/cloudflare.ini

Add your Cloudflare API token:

# Cloudflare API token (get from: Cloudflare Dashboard → My Profile → API Tokens → Create Token)
# Permissions needed: Zone:Zone:Read, Zone:DNS:Edit
dns_cloudflare_api_token = YOUR_CLOUDFLARE_API_TOKEN

# Secure the credentials file
sudo chmod 600 /etc/letsencrypt/cloudflare.ini

# Get certificates using DNS challenge
sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini \
  -d traderideas.deepteklabs.com \
  -d www.traderideas.deepteklabs.com \
  -d api.traderideas.deepteklabs.com

# Update nginx config manually (certbot won't auto-update with DNS challenge)
sudo nano /etc/nginx/sites-available/institutional-trader
# Update ssl_certificate paths to:
# ssl_certificate /etc/letsencrypt/live/traderideas.deepteklabs.com/fullchain.pem;
# ssl_certificate_key /etc/letsencrypt/live/traderideas.deepteklabs.com/privkey.pem;

sudo nginx -t
sudo systemctl reload nginx

Option E: Use Cloudflare SSL (Easiest with Cloudflare proxy)

If using Cloudflare proxy, you can let Cloudflare handle SSL:

In Cloudflare Dashboard:
- Go to SSL/TLS → Overview
- Set encryption mode to "Full" or "Full (strict)"
- Cloudflare will automatically provide SSL certificates

On your server, use self-signed or Cloudflare Origin Certificate:

# Generate self-signed certificate (Cloudflare will accept it)
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout /etc/ssl/private/nginx-selfsigned.key \
  -out /etc/ssl/certs/nginx-selfsigned.crt

# Update nginx config to use self-signed cert
sudo nano /etc/nginx/sites-available/institutional-trader
# Change to:
# ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
# ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;

sudo nginx -t
sudo systemctl reload nginx

Or get Cloudflare Origin Certificate:
- Cloudflare Dashboard → SSL/TLS → Origin Server → Create Certificate
- Copy certificate and key
- Save to /etc/ssl/certs/cloudflare-origin.crt and /etc/ssl/private/cloudflare-origin.key
- Update nginx config to use these files

9.3 Auto-Renewal

Certbot sets up auto-renewal automatically. Test it:

sudo certbot renew --dry-run

10. Database Access for Friend

Option A: Supabase (Recommended)

Method 1: Create New Supabase User (Best for Collaboration)

Invite via Supabase Dashboard:
- Go to your Supabase project
- Navigate to Settings > Team
- Click Invite Member
- Enter your friend's email
- Choose role (Developer or Admin)
- They'll receive an invitation email
Share Connection Details:
- Your friend will need:
  - SUPABASE_URL (same as yours)
  - SUPABASE_ANON_KEY (same as yours - safe to share)
  - Their own account credentials
For Backend Access:
- If they need service key access, you can:
  - Share SUPABASE_SERVICE_KEY (keep this secure!)
  - Or create a separate Supabase project for them
  - Or use Row Level Security (RLS) policies to limit access

Method 2: Create Database User (PostgreSQL User)

If using direct PostgreSQL connection:

-- Connect to Supabase via psql or Supabase SQL Editor
-- Create a new user
CREATE USER friend_user WITH PASSWORD 'secure_password_here';

-- Grant necessary permissions
GRANT CONNECT ON DATABASE postgres TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;

-- Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;

Connection String for Friend:

postgresql://friend_user:secure_password_here@db.your-project.supabase.co:5432/postgres

Option B: Local PostgreSQL

If using local PostgreSQL:

# Connect to PostgreSQL
sudo -u postgres psql

# Create user for friend
CREATE USER friend_user WITH PASSWORD 'secure_password_here';

# Grant permissions
GRANT CONNECT ON DATABASE institutional_trader TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;

# Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;

\q

Connection String for Friend:

postgresql://friend_user:secure_password_here@your-server-ip:5432/institutional_trader

Important: If your friend needs to connect from outside your network:

Configure PostgreSQL to accept remote connections
Set up firewall rules
Consider using SSH tunnel for security

Option C: SSH Tunnel (Most Secure)

Create an SSH tunnel for your friend:

# On your friend's machine
ssh -L 5432:localhost:5432 user@your-server-ip

# Then they can connect using:
# postgresql://friend_user:password@localhost:5432/institutional_trader

Option D: Read-Only Access (If Friend Only Needs to Read Data)

-- Create read-only user
CREATE USER friend_readonly WITH PASSWORD 'secure_password_here';
GRANT CONNECT ON DATABASE institutional_trader TO friend_readonly;
GRANT USAGE ON SCHEMA public TO friend_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO friend_readonly;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO friend_readonly;

11. Monitoring & Maintenance

10.1 Health Checks

# Check backend health
curl https://api.yourdomain.com/health

# Check services status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx

10.2 Logs

# Backend logs
sudo journalctl -u institutional-trader-backend -n 100 -f

# Python service logs
sudo journalctl -u institutional-trader-python -n 100 -f

# Nginx access logs
sudo tail -f /var/log/nginx/access.log

# Nginx error logs
sudo tail -f /var/log/nginx/error.log

10.3 Update Application

# Pull latest changes (if using Git)
cd /opt/institutional_trader
git pull

# Update backend
cd backend
npm install --production
sudo systemctl restart institutional-trader-backend

# Update Python service
cd python_service
source venv/bin/activate
pip install -r requirements.txt
deactivate
sudo systemctl restart institutional-trader-python

# Update frontend
cd ../frontend
npm install
npm run build
sudo systemctl reload nginx

10.4 Backup Database

For Supabase:

Use Supabase Dashboard > Database > Backups
Or use pg_dump with connection string

For Local PostgreSQL:

# Create backup
sudo -u postgres pg_dump institutional_trader > backup_$(date +%Y%m%d).sql

# Restore backup
sudo -u postgres psql institutional_trader < backup_20240101.sql

12. Troubleshooting

Git Pull Fails - DNS Resolution Error

Error: fatal: unable to access 'https://github.com/...': Could not resolve host: github.com

This means your container/VM cannot resolve DNS names.

Quick Fix:

# 1. Check current DNS configuration
cat /etc/resolv.conf

# 2. If empty or missing, add DNS servers
sudo nano /etc/resolv.conf

Add these lines:

nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 1.1.1.1

For LXC Containers (Persistent Fix):

# Edit container DNS configuration
# On Proxmox host, edit container config:
nano /etc/pve/lxc/<container-id>.conf

# Add DNS servers:
nameserver: 8.8.8.8
nameserver: 8.8.4.4

Or via Proxmox Web UI:

Go to your container → Options → DNS
Set DNS servers: 8.8.8.8, 8.8.4.4

For Ubuntu VMs/Containers (Systemd-resolved):

# Edit systemd-resolved config
sudo nano /etc/systemd/resolved.conf

Uncomment and set:

[Resolve]
DNS=8.8.8.8 8.8.4.4 1.1.1.1
FallbackDNS=1.1.1.1 8.8.8.8

If using Tailscale (DNS server 100.100.100.100):

If your /etc/resolv.conf shows Tailscale DNS (100.100.100.100) but it's timing out, add fallback DNS servers:

# Edit systemd-resolved config
sudo nano /etc/systemd/resolved.conf

Set:

[Resolve]
DNS=100.100.100.100 8.8.8.8 8.8.4.4 1.1.1.1
FallbackDNS=8.8.8.8 8.8.4.4 1.1.1.1

This keeps Tailscale DNS as primary (for Tailscale network access) but adds public DNS as fallback.

# Restart systemd-resolved
sudo systemctl restart systemd-resolved

# Test DNS
nslookup github.com
ping -c 2 github.com

Alternative: Use IP Address (Temporary Workaround)

If DNS still doesn't work, you can manually update git remote:

# Get GitHub IP
nslookup github.com 8.8.8.8

# Or use known GitHub IPs (may change)
# Update git remote to use IP (not recommended for long-term)
git remote set-url origin https://140.82.121.3/deepkoluguri/INSTITUTIONAL-FLOW-TRADING-PLATFORM.git

Verify DNS is Working:

# Test DNS resolution
nslookup github.com
dig github.com

# Test connectivity
ping -c 2 github.com
curl -I https://github.com

Backend Won't Start

# Check logs
sudo journalctl -u institutional-trader-backend -n 50

# Check if port is in use
sudo netstat -tlnp | grep 3000

# Test database connection
cd /opt/institutional_trader/backend
node -e "import('./src/db.js').then(m => m.testConnection())"

Python Service Won't Start

Step 1: Check detailed error logs

# View recent logs with full error messages
sudo journalctl -u institutional-trader-python -n 100 --no-pager

# Follow logs in real-time
sudo journalctl -u institutional-trader-python -f

Step 2: Verify systemd service file has correct port

# Check current service file
sudo cat /etc/systemd/system/institutional-trader-python.service

# If port is 8000, update it to 8010:
sudo nano /etc/systemd/system/institutional-trader-python.service
# Change: --port 8000 to --port 8010
# Then reload and restart:
sudo systemctl daemon-reload
sudo systemctl restart institutional-trader-python

Step 3: Test manually to see actual error

cd /opt/institutional_trader/backend/python_service
source venv/bin/activate

# Check if .env file exists and has correct DB config
cat .env

# Test startup manually
uvicorn main:app --host 0.0.0.0 --port 8010

Step 4: Common issues and fixes

Database connection timeout:

The service now has a 10-second timeout and will start even if DB is temporarily unavailable
Check database is reachable: psql -h <DB_HOST> -U <DB_USER> -d institutional_trader
Verify .env file in backend/python_service/ has correct database credentials

Port already in use:

# Check if port 8010 is in use
sudo netstat -tlnp | grep 8010
# Or
sudo lsof -i :8010

# Kill process if needed
sudo kill -9 <PID>

Missing dependencies:

cd /opt/institutional_trader/backend/python_service
source venv/bin/activate
pip install -r requirements.txt

Step 5: Verify backend .env has correct Python service URL

# Check backend .env
cat /opt/institutional_trader/backend/.env | grep PYTHON_SERVICE_URL

# Should be:
# PYTHON_SERVICE_URL=http://localhost:8010

# If not, update it:
nano /opt/institutional_trader/backend/.env
# Add or update: PYTHON_SERVICE_URL=http://localhost:8010
# Then restart backend:
sudo systemctl restart institutional-trader-backend

Nginx Errors

# Test configuration
sudo nginx -t

# Check error logs
sudo tail -f /var/log/nginx/error.log

# Reload configuration
sudo systemctl reload nginx

Database Connection Issues

# Test PostgreSQL connection (local)
psql -h localhost -U trader_user -d institutional_trader

# Test Supabase connection
psql "postgresql://postgres:password@db.project.supabase.co:5432/postgres"

Firewall Issues

# Check firewall status
sudo ufw status

# Allow ports (if using UFW)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 22/tcp  # SSH
sudo ufw enable

SSL Certificate Issues

# Check certificate status
sudo certbot certificates

# Renew certificate manually
sudo certbot renew

# Check certificate expiration
sudo certbot certificates | grep Expiry

Quick Reference Commands

# Start all services
sudo systemctl start institutional-trader-backend
sudo systemctl start institutional-trader-python

# Stop all services
sudo systemctl stop institutional-trader-backend
sudo systemctl stop institutional-trader-python

# Restart all services
sudo systemctl restart institutional-trader-backend
sudo systemctl restart institutional-trader-python

# Check status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx

# View logs
sudo journalctl -u institutional-trader-* -f

# Test health
curl https://api.yourdomain.com/health

Security Checklist

All .env files have secure passwords
JWT_SECRET is at least 32 characters and random
SSL certificates installed and auto-renewal enabled
Firewall configured (only necessary ports open)
Database users have appropriate permissions (principle of least privilege)
Regular backups configured
System updates applied regularly
SSH key authentication enabled (disable password auth)
Rate limiting enabled in backend
CORS origin set to production domain only

Next Steps

✅ Deploy application to Proxmox
✅ Configure database access for friend
✅ Set up monitoring/alerting (optional: UptimeRobot, Sentry)
✅ Configure automated backups
✅ Set up CI/CD pipeline (optional)

Your application should now be running on your Proxmox server! 🎉

35 KiB Raw Blame History