institutional-trader/PROXMOX_DEPLOYMENT.md

# 🚀 Proxmox Deployment Guide

Complete guide for deploying the Institutional Trader platform on a Proxmox server.

## Table of Contents

1. [Prerequisites](#1-prerequisites)
2. [Proxmox VM/Container Setup](#2-proxmox-vmcontainer-setup)
3. [System Dependencies](#3-system-dependencies)
4. [Application Setup](#4-application-setup)
5. [Database Configuration](#5-database-configuration)
6. [Systemd Services](#6-systemd-services)
7. [Nginx Configuration](#7-nginx-configuration)
8. [SSL Certificates](#8-ssl-certificates)
9. [Database Access for Friend](#9-database-access-for-friend)
10. [Monitoring & Maintenance](#10-monitoring--maintenance)
11. [Troubleshooting](#11-troubleshooting)

---

## 1. Prerequisites

- Proxmox server with network access
- Domain name pointing to your server's IP (for SSL)
- SSH access to the Proxmox host
- Basic knowledge of Linux command line

---

## 2. Proxmox VM/Container Setup

### Option A: Ubuntu LXC Container (Recommended - Lightweight)

1. **Create LXC Container:**
   - In Proxmox Web UI, go to your node
   - Click "Create CT" (Container)
   - Choose Ubuntu 22.04 or 24.04 template
   - Configure:
     - **Hostname**: `institutional-trader`
     - **Password**: Set root password
     - **CPU**: 2-4 cores
     - **Memory**: 4-8 GB RAM
     - **Disk**: 20-50 GB
     - **Network**: Bridge with static IP (recommended) or DHCP

2. **Start Container:**
   ```bash
   # In Proxmox shell or via Web UI
   pct start <container-id>
   ```

3. **Access Container:**
   ```bash
   # From Proxmox host
   pct enter <container-id>
   
   # Or SSH if you configured it
   ssh root@<container-ip>
   ```

### Option B: Ubuntu VM (More Isolation)

1. **Create VM:**
   - In Proxmox Web UI, go to your node
   - Click "Create VM"
   - Choose Ubuntu 22.04 or 24.04 ISO
   - Configure:
     - **CPU**: 2-4 cores
     - **Memory**: 4-8 GB RAM
     - **Disk**: 50-100 GB
     - **Network**: Bridge adapter
   - Install Ubuntu Server (minimal installation)

2. **Access VM:**
   ```bash
   ssh <user>@<vm-ip>
   ```

---

## 3. System Dependencies

Run these commands on your Ubuntu container/VM:

```bash
# Update system
apt update && apt upgrade -y

# Install Node.js 20.x (LTS)
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt install -y nodejs

# Install Python 3.11+ and pip
apt install -y python3 python3-pip python3-venv

# Install PostgreSQL client (if using local DB)
apt install -y postgresql-client

# Install Nginx
apt install -y nginx

# Install Git
apt install -y git

# Install build tools (for native modules)
apt install -y build-essential

# Install PM2 (process manager - optional but recommended)
npm install -g pm2

# Install Uvicorn (for Python service)
pip3 install uvicorn

# Verify installations
node --version  # Should be v20.x.x
python3 --version  # Should be 3.11+
nginx -v
```

---

## 4. Application Setup

### 4.1 Clone and Prepare Repository

```bash
# Create application directory
mkdir -p /opt/institutional_trader
cd /opt/institutional_trader

# Clone your repository (or upload files)
# Option 1: If using Git
git clone <your-repo-url> .

# Option 2: If uploading files manually
# Use SCP or Proxmox file browser to upload files
```

### 4.2 Backend Setup

```bash
cd /opt/institutional_trader/backend

# Install dependencies
npm install --production

# Create production .env file
cp env.production.example .env
nano .env
```

**Backend `.env` configuration:**
```env
NODE_ENV=production
PORT=3000

# Supabase Configuration (if using Supabase)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_ANON_KEY=your_anon_key
SUPABASE_SERVICE_KEY=your_service_role_key

# OR Direct PostgreSQL Connection (if using local/remote PostgreSQL)
DATABASE_URL=postgresql://postgres:password@localhost:5432/institutional_trader

# CORS Configuration
CORS_ORIGIN=https://yourdomain.com

# Security
JWT_SECRET=your_super_secret_jwt_key_min_32_chars_long
RATE_LIMIT_WINDOW=15
RATE_LIMIT_MAX=100

# Python Service (if using)
PYTHON_SERVICE_URL=http://localhost:8000
```

### 4.3 Python Service Setup

```bash
cd /opt/institutional_trader/backend/python_service

# Create virtual environment
python3 -m venv venv

# Activate virtual environment
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Deactivate (we'll use systemd to manage this)
deactivate
```

### 4.4 Frontend Setup

```bash
cd /opt/institutional_trader/frontend

# Install dependencies
npm install

# Create production .env file
cp env.production.example .env.production
nano .env.production
```

**Frontend `.env.production` configuration:**
```env
VITE_API_URL=https://api.yourdomain.com
VITE_WS_URL=wss://api.yourdomain.com
VITE_SUPABASE_URL=https://your-project.supabase.co
VITE_SUPABASE_ANON_KEY=your_anon_key
```

**Build frontend:**
```bash
npm run build
```

This creates a `dist` folder with production-ready files.

---

## 5. Database Configuration

### Option A: Using Supabase (Cloud Database - Recommended)

1. **Set up Supabase:**
   - Go to [supabase.com](https://supabase.com)
   - Create a new project
   - Get your connection details from Settings > API
   - Run database schema scripts from `backend/database/` in Supabase SQL Editor

2. **Connection String:**
   - Use `DATABASE_URL` or `SUPABASE_URL` + keys in backend `.env`
   - See [Database Access for Friend](#9-database-access-for-friend) section for sharing access

### Option B: Local PostgreSQL (Self-Hosted)

```bash
# Install PostgreSQL
apt install -y postgresql postgresql-contrib

# Start PostgreSQL
systemctl start postgresql
systemctl enable postgresql

# Create database and user
sudo -u postgres psql

# In PostgreSQL prompt:
CREATE DATABASE institutional_trader;
CREATE USER trader_user WITH PASSWORD 'secure_password_here';
GRANT ALL PRIVILEGES ON DATABASE institutional_trader TO trader_user;
\q

# Import schema
cd /opt/institutional_trader/backend/database
sudo -u postgres psql institutional_trader < schema.sql
sudo -u postgres psql institutional_trader < missing_tables.sql
# Import other SQL files as needed
```

Update backend `.env`:
```env
USE_LOCAL_DB=true
LOCAL_DB_HOST=localhost
LOCAL_DB_PORT=5432
LOCAL_DB_USER=trader_user
LOCAL_DB_PASSWORD=secure_password_here
LOCAL_DB_NAME=institutional_trader
```

---

## 6. Systemd Services

### 6.1 Backend Service

Create systemd service for Node.js backend:

```bash
sudo nano /etc/systemd/system/institutional-trader-backend.service
```

Add:
```ini
[Unit]
Description=Institutional Trader Backend API
After=network.target postgresql.service

[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend
Environment=NODE_ENV=production
EnvironmentFile=/opt/institutional_trader/backend/.env
ExecStart=/usr/bin/node src/server.js
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-backend
sudo systemctl start institutional-trader-backend
sudo systemctl status institutional-trader-backend
```

### 6.2 Python Service

Create systemd service for Python service:

```bash
sudo nano /etc/systemd/system/institutional-trader-python.service
```

Add:
```ini
[Unit]
Description=Institutional Trader Python Service
After=network.target postgresql.service

[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend/python_service
Environment="PATH=/opt/institutional_trader/backend/python_service/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
ExecStart=/opt/institutional_trader/backend/python_service/venv/bin/uvicorn main:app --host 0.0.0.0 --port 8000
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-python
sudo systemctl start institutional-trader-python
sudo systemctl status institutional-trader-python
```

### 6.3 View Logs

```bash
# Backend logs
sudo journalctl -u institutional-trader-backend -f

# Python service logs
sudo journalctl -u institutional-trader-python -f

# All logs
sudo journalctl -u institutional-trader-* -f
```

---

## 7. Nginx Configuration

### 7.1 Create Nginx Configuration

```bash
sudo nano /etc/nginx/sites-available/institutional-trader
```

Copy and customize the configuration (replace `yourdomain.com` with your domain):

```nginx
# Backend API Server
server {
    listen 443 ssl http2;
    server_name api.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    # SSL Configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # API Routes
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
    }

    # WebSocket Support
    location /ws {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # WebSocket specific timeouts
        proxy_read_timeout 86400;
    }

    # Health Check
    location /health {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }
}

# Frontend Server
server {
    listen 443 ssl http2;
    server_name yourdomain.com www.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    # SSL Configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    root /opt/institutional_trader/frontend/dist;
    index index.html;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # Gzip Compression
    gzip on;
    gzip_vary on;
    gzip_min_length 1024;
    gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml+rss application/json;

    location / {
        try_files $uri $uri/ /index.html;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }

    # Static assets caching
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }
}

# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com api.yourdomain.com;
    return 301 https://$server_name$request_uri;
}
```

### 7.2 Enable Site

```bash
# Create symlink
sudo ln -s /etc/nginx/sites-available/institutional-trader /etc/nginx/sites-enabled/

# Test configuration
sudo nginx -t

# Reload Nginx
sudo systemctl reload nginx
```

---

## 8. SSL Certificates

### 8.1 Install Certbot

```bash
sudo apt install -y certbot python3-certbot-nginx
```

### 8.2 Obtain SSL Certificate

```bash
# Replace with your actual domain
sudo certbot --nginx -d yourdomain.com -d www.yourdomain.com -d api.yourdomain.com
```

Follow the prompts:
- Enter your email address
- Agree to terms
- Choose whether to redirect HTTP to HTTPS (recommended: Yes)

### 8.3 Auto-Renewal

Certbot sets up auto-renewal automatically. Test it:

```bash
sudo certbot renew --dry-run
```

---

## 9. Database Access for Friend

### Option A: Supabase (Recommended)

#### Method 1: Create New Supabase User (Best for Collaboration)

1. **Invite via Supabase Dashboard:**
   - Go to your Supabase project
   - Navigate to **Settings > Team**
   - Click **Invite Member**
   - Enter your friend's email
   - Choose role (Developer or Admin)
   - They'll receive an invitation email

2. **Share Connection Details:**
   - Your friend will need:
     - `SUPABASE_URL` (same as yours)
     - `SUPABASE_ANON_KEY` (same as yours - safe to share)
     - Their own account credentials

3. **For Backend Access:**
   - If they need service key access, you can:
     - Share `SUPABASE_SERVICE_KEY` (keep this secure!)
     - Or create a separate Supabase project for them
     - Or use Row Level Security (RLS) policies to limit access

#### Method 2: Create Database User (PostgreSQL User)

If using direct PostgreSQL connection:

```sql
-- Connect to Supabase via psql or Supabase SQL Editor
-- Create a new user
CREATE USER friend_user WITH PASSWORD 'secure_password_here';

-- Grant necessary permissions
GRANT CONNECT ON DATABASE postgres TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;

-- Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;
```

**Connection String for Friend:**
```
postgresql://friend_user:secure_password_here@db.your-project.supabase.co:5432/postgres
```

### Option B: Local PostgreSQL

If using local PostgreSQL:

```bash
# Connect to PostgreSQL
sudo -u postgres psql

# Create user for friend
CREATE USER friend_user WITH PASSWORD 'secure_password_here';

# Grant permissions
GRANT CONNECT ON DATABASE institutional_trader TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;

# Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;

\q
```

**Connection String for Friend:**
```
postgresql://friend_user:secure_password_here@your-server-ip:5432/institutional_trader
```

**Important:** If your friend needs to connect from outside your network:
1. Configure PostgreSQL to accept remote connections
2. Set up firewall rules
3. Consider using SSH tunnel for security

### Option C: SSH Tunnel (Most Secure)

Create an SSH tunnel for your friend:

```bash
# On your friend's machine
ssh -L 5432:localhost:5432 user@your-server-ip

# Then they can connect using:
# postgresql://friend_user:password@localhost:5432/institutional_trader
```

### Option D: Read-Only Access (If Friend Only Needs to Read Data)

```sql
-- Create read-only user
CREATE USER friend_readonly WITH PASSWORD 'secure_password_here';
GRANT CONNECT ON DATABASE institutional_trader TO friend_readonly;
GRANT USAGE ON SCHEMA public TO friend_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO friend_readonly;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO friend_readonly;
```

---

## 10. Monitoring & Maintenance

### 10.1 Health Checks

```bash
# Check backend health
curl https://api.yourdomain.com/health

# Check services status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx
```

### 10.2 Logs

```bash
# Backend logs
sudo journalctl -u institutional-trader-backend -n 100 -f

# Python service logs
sudo journalctl -u institutional-trader-python -n 100 -f

# Nginx access logs
sudo tail -f /var/log/nginx/access.log

# Nginx error logs
sudo tail -f /var/log/nginx/error.log
```

### 10.3 Update Application

```bash
# Pull latest changes (if using Git)
cd /opt/institutional_trader
git pull

# Update backend
cd backend
npm install --production
sudo systemctl restart institutional-trader-backend

# Update Python service
cd python_service
source venv/bin/activate
pip install -r requirements.txt
deactivate
sudo systemctl restart institutional-trader-python

# Update frontend
cd ../frontend
npm install
npm run build
sudo systemctl reload nginx
```

### 10.4 Backup Database

**For Supabase:**
- Use Supabase Dashboard > Database > Backups
- Or use `pg_dump` with connection string

**For Local PostgreSQL:**
```bash
# Create backup
sudo -u postgres pg_dump institutional_trader > backup_$(date +%Y%m%d).sql

# Restore backup
sudo -u postgres psql institutional_trader < backup_20240101.sql
```

---

## 11. Troubleshooting

### Backend Won't Start

```bash
# Check logs
sudo journalctl -u institutional-trader-backend -n 50

# Check if port is in use
sudo netstat -tlnp | grep 3000

# Test database connection
cd /opt/institutional_trader/backend
node -e "import('./src/db.js').then(m => m.testConnection())"
```

### Python Service Won't Start

```bash
# Check logs
sudo journalctl -u institutional-trader-python -n 50

# Test manually
cd /opt/institutional_trader/backend/python_service
source venv/bin/activate
uvicorn main:app --host 0.0.0.0 --port 8000
```

### Nginx Errors

```bash
# Test configuration
sudo nginx -t

# Check error logs
sudo tail -f /var/log/nginx/error.log

# Reload configuration
sudo systemctl reload nginx
```

### Database Connection Issues

```bash
# Test PostgreSQL connection (local)
psql -h localhost -U trader_user -d institutional_trader

# Test Supabase connection
psql "postgresql://postgres:password@db.project.supabase.co:5432/postgres"
```

### Firewall Issues

```bash
# Check firewall status
sudo ufw status

# Allow ports (if using UFW)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 22/tcp  # SSH
sudo ufw enable
```

### SSL Certificate Issues

```bash
# Check certificate status
sudo certbot certificates

# Renew certificate manually
sudo certbot renew

# Check certificate expiration
sudo certbot certificates | grep Expiry
```

---

## Quick Reference Commands

```bash
# Start all services
sudo systemctl start institutional-trader-backend
sudo systemctl start institutional-trader-python

# Stop all services
sudo systemctl stop institutional-trader-backend
sudo systemctl stop institutional-trader-python

# Restart all services
sudo systemctl restart institutional-trader-backend
sudo systemctl restart institutional-trader-python

# Check status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx

# View logs
sudo journalctl -u institutional-trader-* -f

# Test health
curl https://api.yourdomain.com/health
```

---

## Security Checklist

- [ ] All `.env` files have secure passwords
- [ ] `JWT_SECRET` is at least 32 characters and random
- [ ] SSL certificates installed and auto-renewal enabled
- [ ] Firewall configured (only necessary ports open)
- [ ] Database users have appropriate permissions (principle of least privilege)
- [ ] Regular backups configured
- [ ] System updates applied regularly
- [ ] SSH key authentication enabled (disable password auth)
- [ ] Rate limiting enabled in backend
- [ ] CORS origin set to production domain only

---

## Next Steps

1. ✅ Deploy application to Proxmox
2. ✅ Configure database access for friend
3. ✅ Set up monitoring/alerting (optional: UptimeRobot, Sentry)
4. ✅ Configure automated backups
5. ✅ Set up CI/CD pipeline (optional)

Your application should now be running on your Proxmox server! 🎉