# 🚀 Proxmox Deployment Guide

Complete guide for deploying the Institutional Trader platform on a Proxmox server.

## Table of Contents

1. [Prerequisites](#1-prerequisites)
2. [Proxmox VM/Container Setup](#2-proxmox-vmcontainer-setup)
3. [System Dependencies](#3-system-dependencies)
4. [Application Setup](#4-application-setup)
5. [Database Configuration](#5-database-configuration)
6. [Systemd Services](#6-systemd-services)
7. [Nginx Configuration](#7-nginx-configuration)
8. [DNS Configuration](#8-dns-configuration)
9. [SSL Certificates](#9-ssl-certificates)
10. [Database Access for Friend](#10-database-access-for-friend)
11. [Monitoring & Maintenance](#11-monitoring--maintenance)
12. [Troubleshooting](#12-troubleshooting)

---

## 1. Prerequisites

- Proxmox server with network access
- Domain name pointing to your server's IP (for SSL)
- SSH access to the Proxmox host
- Basic knowledge of Linux command line

---

## 2. Proxmox VM/Container Setup

### Option A: Ubuntu LXC Container (Recommended - Lightweight)

1. **Create LXC Container:**
   - In Proxmox Web UI, go to your node
   - Click "Create CT" (Container)
   - Choose Ubuntu 22.04 or 24.04 template
   - Configure:
     - **Hostname**: `institutional-trader`
     - **Password**: Set root password
     - **CPU**: 2-4 cores
     - **Memory**: 4-8 GB RAM
     - **Disk**: 20-50 GB
     - **Network**: Bridge with static IP (recommended) or DHCP

2. **Start Container:**
   ```bash
   # In Proxmox shell or via Web UI
   pct start <container-id>
   ```

3. **Access Container:**
   ```bash
   # From Proxmox host
   pct enter <container-id>
   
   # Or SSH if you configured it
   ssh root@<container-ip>
   ```

### Option B: Ubuntu VM (More Isolation)

1. **Create VM:**
   - In Proxmox Web UI, go to your node
   - Click "Create VM"
   - Choose Ubuntu 22.04 or 24.04 ISO
   - Configure:
     - **CPU**: 2-4 cores
     - **Memory**: 4-8 GB RAM
     - **Disk**: 50-100 GB
     - **Network**: Bridge adapter
   - Install Ubuntu Server (minimal installation)

2. **Access VM:**
   ```bash
   ssh <user>@<vm-ip>
   ```

---

## 3. System Dependencies

Run these commands on your Ubuntu container/VM:

```bash
# Update system
apt update && apt upgrade -y

# Install Node.js 20.x (LTS)
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt install -y nodejs

# Install Python 3.11+ and pip
apt install -y python3 python3-pip python3-venv

# Install PostgreSQL client (if using local DB)
apt install -y postgresql-client

# Install Nginx
apt install -y nginx

# Install Git
apt install -y git

# Install build tools (for native modules)
apt install -y build-essential

# Install PM2 (process manager - optional but recommended)
npm install -g pm2

# Install Uvicorn (for Python service)
pip3 install uvicorn

# Verify installations
node --version  # Should be v20.x.x
python3 --version  # Should be 3.11+
nginx -v
```

---

## 4. Application Setup

### 4.1 Clone and Prepare Repository

```bash
# Create application directory
mkdir -p /opt/institutional_trader
cd /opt/institutional_trader

# Clone your repository (or upload files)
# Option 1: If using Git
git clone <your-repo-url> .

# Option 2: If uploading files manually
# Use SCP or Proxmox file browser to upload files
```

### 4.2 Backend Setup

```bash
cd /opt/institutional_trader/backend

# Install dependencies
npm install --production

# Create production .env file
cp env.production.example .env
nano .env
```

**Backend `.env` configuration:**
```env
NODE_ENV=production
PORT=3000

# Supabase Configuration (if using Supabase)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_ANON_KEY=your_anon_key
SUPABASE_SERVICE_KEY=your_service_role_key

# OR Direct PostgreSQL Connection (if using local/remote PostgreSQL)
DATABASE_URL=postgresql://postgres:password@localhost:5432/institutional_trader

# CORS Configuration
CORS_ORIGIN=https://yourdomain.com

# Security
JWT_SECRET=your_super_secret_jwt_key_min_32_chars_long
RATE_LIMIT_WINDOW=15
RATE_LIMIT_MAX=100

# Python Service (if using)
PYTHON_SERVICE_URL=http://localhost:8010
```

### 4.3 Python Service Setup

```bash
cd /opt/institutional_trader/backend/python_service

# Create virtual environment
python3 -m venv venv

# Activate virtual environment
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Deactivate (we'll use systemd to manage this)
deactivate
```

### 4.4 Frontend Setup

```bash
cd /opt/institutional_trader/frontend

# Install dependencies
npm install

# Create production .env file
cp env.production.example .env.production
nano .env.production
```

**Frontend `.env.production` configuration:**

**⚠️ Critical:** The frontend must have a `.env.production` file with the correct API URL, otherwise it will try to connect to `localhost:3010` which won't work in production.
```env
VITE_API_URL=https://api.yourdomain.com
VITE_WS_URL=wss://api.yourdomain.com
VITE_SUPABASE_URL=https://your-project.supabase.co
VITE_SUPABASE_ANON_KEY=your_anon_key
```

**Build frontend:**
```bash
npm run build
```

This creates a `dist` folder with production-ready files.

---

## 5. Database Configuration

### Option A: Using Supabase (Cloud Database - Recommended)

1. **Set up Supabase:**
   - Go to [supabase.com](https://supabase.com)
   - Create a new project
   - Get your connection details from Settings > API
   - Run database schema scripts from `backend/database/` in Supabase SQL Editor

2. **Connection String:**
   - Use `DATABASE_URL` or `SUPABASE_URL` + keys in backend `.env`
   - See [Database Access for Friend](#9-database-access-for-friend) section for sharing access

### Option B: Local PostgreSQL (Self-Hosted)

```bash
# Install PostgreSQL
apt install -y postgresql postgresql-contrib

# Start PostgreSQL
systemctl start postgresql
systemctl enable postgresql

# Create database and user
sudo -u postgres psql

# In PostgreSQL prompt:
CREATE DATABASE institutional_trader;
CREATE USER trader_user WITH PASSWORD 'secure_password_here';
GRANT ALL PRIVILEGES ON DATABASE institutional_trader TO trader_user;
\q

# Import schema
cd /opt/institutional_trader/backend/database
sudo -u postgres psql institutional_trader < schema.sql
sudo -u postgres psql institutional_trader < missing_tables.sql
# Import other SQL files as needed
```

Update backend `.env`:
```env
USE_LOCAL_DB=true
LOCAL_DB_HOST=localhost
LOCAL_DB_PORT=5432
LOCAL_DB_USER=trader_user
LOCAL_DB_PASSWORD=secure_password_here
LOCAL_DB_NAME=institutional_trader
```

---

## 6. Systemd Services

### 6.1 Backend Service

Create systemd service for Node.js backend:

```bash
sudo nano /etc/systemd/system/institutional-trader-backend.service
```

Add:
```ini
[Unit]
Description=Institutional Trader Backend API
After=network.target postgresql.service

[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend
Environment=NODE_ENV=production
EnvironmentFile=/opt/institutional_trader/backend/.env
ExecStart=/usr/bin/node src/server.js
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-backend
sudo systemctl start institutional-trader-backend
sudo systemctl status institutional-trader-backend
```

### 6.2 Python Service

Create systemd service for Python service:

```bash
sudo nano /etc/systemd/system/institutional-trader-python.service
```

Add:
```ini
[Unit]
Description=Institutional Trader Python Service
After=network.target postgresql.service

[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend/python_service
Environment="PATH=/opt/institutional_trader/backend/python_service/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
EnvironmentFile=/opt/institutional_trader/backend/python_service/.env
ExecStart=/opt/institutional_trader/backend/python_service/venv/bin/uvicorn main:app --host 0.0.0.0 --port 8010
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

**Note:** The `EnvironmentFile` directive is optional - the service will also load `.env` automatically via `load_dotenv()` in the code. However, adding it explicitly ensures environment variables are available even if the code path changes.

Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-python
sudo systemctl start institutional-trader-python
sudo systemctl status institutional-trader-python
```

### 6.3 View Logs

```bash
# Backend logs
sudo journalctl -u institutional-trader-backend -f

# Python service logs
sudo journalctl -u institutional-trader-python -f

# All logs
sudo journalctl -u institutional-trader-* -f
```

---

## 7. Nginx Configuration

### 7.1 Create Nginx Configuration

```bash
sudo nano /etc/nginx/sites-available/institutional-trader
```

Copy and customize the configuration (replace `yourdomain.com` with your domain):

```nginx
# Backend API Server
server {
    listen 443 ssl http2;
    server_name api.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    # SSL Configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # API Routes
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
    }

    # WebSocket Support
    location /ws {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # WebSocket specific timeouts
        proxy_read_timeout 86400;
    }

    # Health Check
    location /health {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }
}

# Frontend Server
server {
    listen 443 ssl http2;
    server_name yourdomain.com www.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    # SSL Configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    root /opt/institutional_trader/frontend/dist;
    index index.html;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # Gzip Compression
    gzip on;
    gzip_vary on;
    gzip_min_length 1024;
    gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml+rss application/json;

    location / {
        try_files $uri $uri/ /index.html;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }

    # Static assets caching
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }
}

# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com api.yourdomain.com;
    return 301 https://$server_name$request_uri;
}
```

### 7.2 Multi-Container Setup (Nginx on Separate Container)

If you're running nginx on a separate container/VM that proxies to your application containers:

**Example configuration for proxying to another Proxmox container:**

```nginx
server {
    listen 443 ssl http2;
    server_name traderideas.deepteklabs.com;

    ssl_certificate /etc/letsencrypt/live/traderideas.deepteklabs.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/traderideas.deepteklabs.com/privkey.pem;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header X-Content-Type-Options "nosniff" always;

    # Backend API Routes
    location /api/ {
        proxy_pass http://192.168.8.151:3000/api/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
        proxy_buffering off;
    }

    # WebSocket Support (if using WebSockets)
    location /ws {
        proxy_pass http://192.168.8.151:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 86400;  # 24 hours for long-lived connections
    }

    # Health Check
    location /health {
        proxy_pass http://192.168.8.151:3000/health;
        access_log off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }

    # Frontend (served from application container)
    location / {
        proxy_pass http://192.168.8.151:8080/;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_hide_header Cache-Control;
        add_header Cache-Control "no-cache, no-store, must-revalidate" always;
    }

    # Static assets with caching
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot|otf|map)$ {
        proxy_pass http://192.168.8.151:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        expires 1y;
        add_header Cache-Control "public, immutable";
        access_log off;
    }
}
```

**Important notes for multi-container setup:**
- Replace `192.168.8.151` with your application container's IP
- Ensure containers can communicate (same network/bridge)
- Port 3000 = Backend API, Port 8080 = Frontend (adjust as needed)
- Test connectivity: `curl http://192.168.8.151:3000/health` from nginx container

### 7.3 Enable Site

```bash
# Create symlink
sudo ln -s /etc/nginx/sites-available/institutional-trader /etc/nginx/sites-enabled/

# Test configuration
sudo nginx -t

# Reload Nginx
sudo systemctl reload nginx
```

---

## 8. DNS Configuration

**⚠️ Important:** Before setting up SSL certificates, you must configure DNS records pointing to your server.

### 8.1 Find Your Server's Public IP Address

```bash
# On your server, find the public IP
curl ifconfig.me
# Or
hostname -I
# Or check in Proxmox Web UI: Datacenter → Your Node → Network
```

### 8.2 Configure DNS A Records

Go to your domain registrar's DNS management panel (where you registered `traderideas.deepteklabs.com`) and add these A records:

| Type | Name | Value | TTL |
|------|------|-------|-----|
| A | `@` (or blank) | `<your-server-ip>` | 3600 |
| A | `www` | `<your-server-ip>` | 3600 |
| A | `api` | `<your-server-ip>` | 3600 |

**Example for `traderideas.deepteklabs.com`:**
- `traderideas.deepteklabs.com` → `<your-server-ip>` (A record with name `@` or blank)
- `www.traderideas.deepteklabs.com` → `<your-server-ip>` (A record with name `www`)
- `api.traderideas.deepteklabs.com` → `<your-server-ip>` (A record with name `api`)

### 8.3 Verify DNS Propagation

Wait 5-15 minutes for DNS to propagate, then verify:

```bash
# Check if DNS records are resolving
dig traderideas.deepteklabs.com +short
dig www.traderideas.deepteklabs.com +short
dig api.traderideas.deepteklabs.com +short

# All should return your server's IP address

# Alternative: Use nslookup
nslookup traderideas.deepteklabs.com
nslookup www.traderideas.deepteklabs.com
nslookup api.traderideas.deepteklabs.com

# Or use online tools:
# - https://dnschecker.org
# - https://www.whatsmydns.net
```

**Common DNS Providers:**
- **Cloudflare**: Dashboard → DNS → Records → Add record
- **Namecheap**: Domain List → Manage → Advanced DNS → Add New Record
- **GoDaddy**: DNS Management → Add Record
- **Google Domains**: DNS → Custom Records → Add Record

### 8.4 Test Domain Accessibility

Once DNS is propagated, test that domains are reachable:

```bash
# From your server
curl -I http://traderideas.deepteklabs.com
curl -I http://www.traderideas.deepteklabs.com
curl -I http://api.traderideas.deepteklabs.com

# Should return HTTP responses (even if 502/503, that's OK - means DNS is working)
```

### 8.5 Cloudflare Proxy (Orange Cloud)

**If your domain uses Cloudflare proxy (orange cloud icon):**

When DNS resolves to Cloudflare IPs (like `104.21.x.x` or `172.67.x.x`), your domain is behind Cloudflare proxy. This affects SSL certificate setup:

**Check if using Cloudflare proxy:**
```bash
dig traderideas.deepteklabs.com +short
# If returns Cloudflare IPs (104.x.x.x, 172.x.x.x), proxy is enabled
```

**Options for SSL with Cloudflare:**

1. **Use DNS Challenge (Recommended)** - Works with proxy enabled
2. **Temporarily Disable Proxy** - Get certs, then re-enable proxy
3. **Use Cloudflare SSL** - Let Cloudflare handle SSL (easiest)

See Section 9.2 for detailed instructions.

**⚠️ Only proceed to SSL setup once DNS records are working!**

---

## 9. SSL Certificates

### 9.1 Install Certbot

```bash
sudo apt install -y certbot python3-certbot-nginx
```

### 9.2 Obtain SSL Certificate

**⚠️ Prerequisites:**
- DNS A records must be configured and propagated (see Section 8)
- Domains must resolve to your server's IP (or Cloudflare if using proxy)
- Port 80 must be accessible from the internet (for HTTP challenge) OR DNS API access (for DNS challenge)

**⚠️ Important:** If your nginx config already references SSL certificates that don't exist, you need to fix this first.

**Check if using Cloudflare proxy:**
```bash
dig traderideas.deepteklabs.com +short
# If returns Cloudflare IPs (104.x.x.x, 172.x.x.x), you're using Cloudflare proxy
```

**If using Cloudflare Proxy, use Option D (DNS Challenge) or Option E (Cloudflare SSL)**

**Option A: Use Standalone Mode (Works if NOT using Cloudflare proxy)**

```bash
# Stop nginx temporarily
sudo systemctl stop nginx

# Get certificates using standalone mode
sudo certbot certonly --standalone -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com

# Start nginx again
sudo systemctl start nginx

# Now certbot can update the nginx config
sudo certbot --nginx -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com
```

**Option B: Temporarily Comment Out SSL in Nginx Config**

```bash
# Edit nginx config
sudo nano /etc/nginx/sites-available/institutional-trader

# Temporarily comment out SSL lines:
# - Comment out: listen 443 ssl http2;
# - Comment out: ssl_certificate and ssl_certificate_key lines
# - Change: listen 443 ssl http2; to: listen 80;

# Example temporary config (HTTP only):
# server {
#     listen 80;
#     server_name traderideas.deepteklabs.com;
#     # ssl_certificate /etc/letsencrypt/live/traderideas.deepteklabs.com/fullchain.pem;
#     # ssl_certificate_key /etc/letsencrypt/live/traderideas.deepteklabs.com/privkey.pem;
#     ...
# }

# Test and reload
sudo nginx -t
sudo systemctl reload nginx

# Now get certificates (certbot will automatically update the config)
sudo certbot --nginx -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com

# Certbot will automatically:
# - Get the certificates
# - Update your nginx config to use them
# - Add SSL redirects
```

**Option C: Use Certbot with Nginx Plugin (If config is clean and NOT using Cloudflare proxy)**

```bash
# Replace with your actual domain
sudo certbot --nginx -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com
```

Follow the prompts:
- Enter your email address
- Agree to terms
- Choose whether to redirect HTTP to HTTPS (recommended: Yes)

**Option D: DNS Challenge (Recommended for Cloudflare proxy)**

If your domain is behind Cloudflare proxy, use DNS challenge:

```bash
# Install Cloudflare plugin for certbot
sudo apt install -y python3-pip
sudo pip3 install certbot-dns-cloudflare

# Create Cloudflare API token directory
sudo mkdir -p /etc/letsencrypt
sudo chmod 700 /etc/letsencrypt

# Create Cloudflare credentials file
sudo nano /etc/letsencrypt/cloudflare.ini
```

Add your Cloudflare API token:
```ini
# Cloudflare API token (get from: Cloudflare Dashboard → My Profile → API Tokens → Create Token)
# Permissions needed: Zone:Zone:Read, Zone:DNS:Edit
dns_cloudflare_api_token = YOUR_CLOUDFLARE_API_TOKEN
```

```bash
# Secure the credentials file
sudo chmod 600 /etc/letsencrypt/cloudflare.ini

# Get certificates using DNS challenge
sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini \
  -d traderideas.deepteklabs.com \
  -d www.traderideas.deepteklabs.com \
  -d api.traderideas.deepteklabs.com

# Update nginx config manually (certbot won't auto-update with DNS challenge)
sudo nano /etc/nginx/sites-available/institutional-trader
# Update ssl_certificate paths to:
# ssl_certificate /etc/letsencrypt/live/traderideas.deepteklabs.com/fullchain.pem;
# ssl_certificate_key /etc/letsencrypt/live/traderideas.deepteklabs.com/privkey.pem;

sudo nginx -t
sudo systemctl reload nginx
```

**Option E: Use Cloudflare SSL (Easiest with Cloudflare proxy)**

If using Cloudflare proxy, you can let Cloudflare handle SSL:

1. **In Cloudflare Dashboard:**
   - Go to SSL/TLS → Overview
   - Set encryption mode to **"Full"** or **"Full (strict)"**
   - Cloudflare will automatically provide SSL certificates

2. **On your server, use self-signed or Cloudflare Origin Certificate:**
   ```bash
   # Generate self-signed certificate (Cloudflare will accept it)
   sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
     -keyout /etc/ssl/private/nginx-selfsigned.key \
     -out /etc/ssl/certs/nginx-selfsigned.crt
   
   # Update nginx config to use self-signed cert
   sudo nano /etc/nginx/sites-available/institutional-trader
   # Change to:
   # ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
   # ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
   
   sudo nginx -t
   sudo systemctl reload nginx
   ```

3. **Or get Cloudflare Origin Certificate:**
   - Cloudflare Dashboard → SSL/TLS → Origin Server → Create Certificate
   - Copy certificate and key
   - Save to `/etc/ssl/certs/cloudflare-origin.crt` and `/etc/ssl/private/cloudflare-origin.key`
   - Update nginx config to use these files

### 9.3 Auto-Renewal

Certbot sets up auto-renewal automatically. Test it:

```bash
sudo certbot renew --dry-run
```

---

## 10. Database Access for Friend

### Option A: Supabase (Recommended)

#### Method 1: Create New Supabase User (Best for Collaboration)

1. **Invite via Supabase Dashboard:**
   - Go to your Supabase project
   - Navigate to **Settings > Team**
   - Click **Invite Member**
   - Enter your friend's email
   - Choose role (Developer or Admin)
   - They'll receive an invitation email

2. **Share Connection Details:**
   - Your friend will need:
     - `SUPABASE_URL` (same as yours)
     - `SUPABASE_ANON_KEY` (same as yours - safe to share)
     - Their own account credentials

3. **For Backend Access:**
   - If they need service key access, you can:
     - Share `SUPABASE_SERVICE_KEY` (keep this secure!)
     - Or create a separate Supabase project for them
     - Or use Row Level Security (RLS) policies to limit access

#### Method 2: Create Database User (PostgreSQL User)

If using direct PostgreSQL connection:

```sql
-- Connect to Supabase via psql or Supabase SQL Editor
-- Create a new user
CREATE USER friend_user WITH PASSWORD 'secure_password_here';

-- Grant necessary permissions
GRANT CONNECT ON DATABASE postgres TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;

-- Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;
```

**Connection String for Friend:**
```
postgresql://friend_user:secure_password_here@db.your-project.supabase.co:5432/postgres
```

### Option B: Local PostgreSQL

If using local PostgreSQL:

```bash
# Connect to PostgreSQL
sudo -u postgres psql

# Create user for friend
CREATE USER friend_user WITH PASSWORD 'secure_password_here';

# Grant permissions
GRANT CONNECT ON DATABASE institutional_trader TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;

# Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;

\q
```

**Connection String for Friend:**
```
postgresql://friend_user:secure_password_here@your-server-ip:5432/institutional_trader
```

**Important:** If your friend needs to connect from outside your network:
1. Configure PostgreSQL to accept remote connections
2. Set up firewall rules
3. Consider using SSH tunnel for security

### Option C: SSH Tunnel (Most Secure)

Create an SSH tunnel for your friend:

```bash
# On your friend's machine
ssh -L 5432:localhost:5432 user@your-server-ip

# Then they can connect using:
# postgresql://friend_user:password@localhost:5432/institutional_trader
```

### Option D: Read-Only Access (If Friend Only Needs to Read Data)

```sql
-- Create read-only user
CREATE USER friend_readonly WITH PASSWORD 'secure_password_here';
GRANT CONNECT ON DATABASE institutional_trader TO friend_readonly;
GRANT USAGE ON SCHEMA public TO friend_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO friend_readonly;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO friend_readonly;
```

---

## 11. Monitoring & Maintenance

### 10.1 Health Checks

```bash
# Check backend health
curl https://api.yourdomain.com/health

# Check services status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx
```

### 10.2 Logs

```bash
# Backend logs
sudo journalctl -u institutional-trader-backend -n 100 -f

# Python service logs
sudo journalctl -u institutional-trader-python -n 100 -f

# Nginx access logs
sudo tail -f /var/log/nginx/access.log

# Nginx error logs
sudo tail -f /var/log/nginx/error.log
```

### 10.3 Update Application

```bash
# Pull latest changes (if using Git)
cd /opt/institutional_trader
git pull

# Update backend
cd backend
npm install --production
sudo systemctl restart institutional-trader-backend

# Update Python service
cd python_service
source venv/bin/activate
pip install -r requirements.txt
deactivate
sudo systemctl restart institutional-trader-python

# Update frontend
cd ../frontend
npm install
npm run build
sudo systemctl reload nginx
```

### 10.4 Backup Database

**For Supabase:**
- Use Supabase Dashboard > Database > Backups
- Or use `pg_dump` with connection string

**For Local PostgreSQL:**
```bash
# Create backup
sudo -u postgres pg_dump institutional_trader > backup_$(date +%Y%m%d).sql

# Restore backup
sudo -u postgres psql institutional_trader < backup_20240101.sql
```

---

## 12. Troubleshooting

### Git Pull Fails - DNS Resolution Error

**Error:** `fatal: unable to access 'https://github.com/...': Could not resolve host: github.com`

**This means your container/VM cannot resolve DNS names.**

**Quick Fix:**

```bash
# 1. Check current DNS configuration
cat /etc/resolv.conf

# 2. If empty or missing, add DNS servers
sudo nano /etc/resolv.conf
```

Add these lines:
```
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 1.1.1.1
```

**For LXC Containers (Persistent Fix):**

```bash
# Edit container DNS configuration
# On Proxmox host, edit container config:
nano /etc/pve/lxc/<container-id>.conf

# Add DNS servers:
nameserver: 8.8.8.8
nameserver: 8.8.4.4
```

Or via Proxmox Web UI:
- Go to your container → Options → DNS
- Set DNS servers: `8.8.8.8, 8.8.4.4`

**For Ubuntu VMs/Containers (Systemd-resolved):**

```bash
# Edit systemd-resolved config
sudo nano /etc/systemd/resolved.conf
```

Uncomment and set:
```
[Resolve]
DNS=8.8.8.8 8.8.4.4 1.1.1.1
FallbackDNS=1.1.1.1 8.8.8.8
```

**If using Tailscale (DNS server 100.100.100.100):**

If your `/etc/resolv.conf` shows Tailscale DNS (`100.100.100.100`) but it's timing out, add fallback DNS servers:

```bash
# Edit systemd-resolved config
sudo nano /etc/systemd/resolved.conf
```

Set:
```
[Resolve]
DNS=100.100.100.100 8.8.8.8 8.8.4.4 1.1.1.1
FallbackDNS=8.8.8.8 8.8.4.4 1.1.1.1
```

This keeps Tailscale DNS as primary (for Tailscale network access) but adds public DNS as fallback.

```bash
# Restart systemd-resolved
sudo systemctl restart systemd-resolved

# Test DNS
nslookup github.com
ping -c 2 github.com
```

**Alternative: Use IP Address (Temporary Workaround)**

If DNS still doesn't work, you can manually update git remote:

```bash
# Get GitHub IP
nslookup github.com 8.8.8.8

# Or use known GitHub IPs (may change)
# Update git remote to use IP (not recommended for long-term)
git remote set-url origin https://140.82.121.3/deepkoluguri/INSTITUTIONAL-FLOW-TRADING-PLATFORM.git
```

**Verify DNS is Working:**

```bash
# Test DNS resolution
nslookup github.com
dig github.com

# Test connectivity
ping -c 2 github.com
curl -I https://github.com
```

### Backend Won't Start

```bash
# Check logs
sudo journalctl -u institutional-trader-backend -n 50

# Check if port is in use
sudo netstat -tlnp | grep 3000

# Test database connection
cd /opt/institutional_trader/backend
node -e "import('./src/db.js').then(m => m.testConnection())"
```

### Python Service Won't Start

**Step 1: Check detailed error logs**
```bash
# View recent logs with full error messages
sudo journalctl -u institutional-trader-python -n 100 --no-pager

# Follow logs in real-time
sudo journalctl -u institutional-trader-python -f
```

**Step 2: Verify systemd service file has correct port**
```bash
# Check current service file
sudo cat /etc/systemd/system/institutional-trader-python.service

# If port is 8000, update it to 8010:
sudo nano /etc/systemd/system/institutional-trader-python.service
# Change: --port 8000 to --port 8010
# Then reload and restart:
sudo systemctl daemon-reload
sudo systemctl restart institutional-trader-python
```

**Step 3: Test manually to see actual error**
```bash
cd /opt/institutional_trader/backend/python_service
source venv/bin/activate

# Check if .env file exists and has correct DB config
cat .env

# Test startup manually
uvicorn main:app --host 0.0.0.0 --port 8010
```

**Step 4: Common issues and fixes**

**Database connection timeout:**
- The service now has a 10-second timeout and will start even if DB is temporarily unavailable
- Check database is reachable: `psql -h <DB_HOST> -U <DB_USER> -d institutional_trader`
- Verify `.env` file in `backend/python_service/` has correct database credentials

**Port already in use:**
```bash
# Check if port 8010 is in use
sudo netstat -tlnp | grep 8010
# Or
sudo lsof -i :8010

# Kill process if needed
sudo kill -9 <PID>
```

**Missing dependencies:**
```bash
cd /opt/institutional_trader/backend/python_service
source venv/bin/activate
pip install -r requirements.txt
```

**Step 5: Verify backend .env has correct Python service URL**
```bash
# Check backend .env
cat /opt/institutional_trader/backend/.env | grep PYTHON_SERVICE_URL

# Should be:
# PYTHON_SERVICE_URL=http://localhost:8010

# If not, update it:
nano /opt/institutional_trader/backend/.env
# Add or update: PYTHON_SERVICE_URL=http://localhost:8010
# Then restart backend:
sudo systemctl restart institutional-trader-backend
```

### Nginx Errors

```bash
# Test configuration
sudo nginx -t

# Check error logs
sudo tail -f /var/log/nginx/error.log

# Reload configuration
sudo systemctl reload nginx
```

### Database Connection Issues

```bash
# Test PostgreSQL connection (local)
psql -h localhost -U trader_user -d institutional_trader

# Test Supabase connection
psql "postgresql://postgres:password@db.project.supabase.co:5432/postgres"
```

### Firewall Issues

```bash
# Check firewall status
sudo ufw status

# Allow ports (if using UFW)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 22/tcp  # SSH
sudo ufw enable
```

### SSL Certificate Issues

```bash
# Check certificate status
sudo certbot certificates

# Renew certificate manually
sudo certbot renew

# Check certificate expiration
sudo certbot certificates | grep Expiry
```

---

## Quick Reference Commands

```bash
# Start all services
sudo systemctl start institutional-trader-backend
sudo systemctl start institutional-trader-python

# Stop all services
sudo systemctl stop institutional-trader-backend
sudo systemctl stop institutional-trader-python

# Restart all services
sudo systemctl restart institutional-trader-backend
sudo systemctl restart institutional-trader-python

# Check status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx

# View logs
sudo journalctl -u institutional-trader-* -f

# Test health
curl https://api.yourdomain.com/health
```

---

## Security Checklist

- [ ] All `.env` files have secure passwords
- [ ] `JWT_SECRET` is at least 32 characters and random
- [ ] SSL certificates installed and auto-renewal enabled
- [ ] Firewall configured (only necessary ports open)
- [ ] Database users have appropriate permissions (principle of least privilege)
- [ ] Regular backups configured
- [ ] System updates applied regularly
- [ ] SSH key authentication enabled (disable password auth)
- [ ] Rate limiting enabled in backend
- [ ] CORS origin set to production domain only

---

## Next Steps

1. ✅ Deploy application to Proxmox
2. ✅ Configure database access for friend
3. ✅ Set up monitoring/alerting (optional: UptimeRobot, Sentry)
4. ✅ Configure automated backups
5. ✅ Set up CI/CD pipeline (optional)

Your application should now be running on your Proxmox server! 🎉

