24 KiB

Raw Blame History

🚀 Proxmox Deployment Guide

Complete guide for deploying the Institutional Trader platform on a Proxmox server.

Prerequisites
Proxmox VM/Container Setup
System Dependencies
Application Setup
Database Configuration
Systemd Services
Nginx Configuration
SSL Certificates
Database Access for Friend
Monitoring & Maintenance
Troubleshooting

1. Prerequisites

Proxmox server with network access
Domain name pointing to your server's IP (for SSL)
SSH access to the Proxmox host
Basic knowledge of Linux command line

2. Proxmox VM/Container Setup

Option A: Ubuntu LXC Container (Recommended - Lightweight)

Create LXC Container:
- In Proxmox Web UI, go to your node
- Click "Create CT" (Container)
- Choose Ubuntu 22.04 or 24.04 template
- Configure:
  - Hostname: institutional-trader
  - Password: Set root password
  - CPU: 2-4 cores ✅ (You have 4 CPUs - perfect!)
  - Memory: 4-8 GB RAM ✅ (You have 31.75 GiB - excellent!)
  - Disk: 20-50 GB ✅ (You have 294.23 GiB - plenty of space!)
  - Network: Bridge with static IP (recommended) or DHCP
  - Unprivileged: Yes/No (see note below)

Note on Unprivileged Containers:

✅ Your configuration will work - Unprivileged containers are fine for this deployment
Services run on ports > 1024 (Node.js: 3000, Python: 8000) - no issues
Nginx can run as reverse proxy (may need to bind to ports > 1024 or use host network)
PostgreSQL can be installed and run normally
SSL certificates work fine with Certbot

Start Container:

# In Proxmox shell or via Web UI
pct start <container-id>

Access Container:

# From Proxmox host
pct enter <container-id>

# Or SSH if you configured it
ssh root@<container-ip>

Option B: Ubuntu VM (More Isolation)

Create VM:
- In Proxmox Web UI, go to your node
- Click "Create VM"
- Choose Ubuntu 22.04 or 24.04 ISO
- Configure:
  - CPU: 2-4 cores
  - Memory: 4-8 GB RAM
  - Disk: 50-100 GB
  - Network: Bridge adapter
- Install Ubuntu Server (minimal installation)
Access VM:
```
ssh <user>@<vm-ip>
```

3. System Dependencies

Important: Unprivileged Container Considerations

If your container is unprivileged (which yours is), there are a few adjustments:

Nginx Port Binding:
- Unprivileged containers cannot bind to ports < 1024
- Solution: Use Nginx on host or configure port forwarding
- Alternative: Run Nginx on port 8080/8443 and forward from host
Systemd:
- May need to enable systemd in container
- Check: systemctl status should work
File Permissions:
- Some operations may need different permissions
- Usually not an issue for this application

Your Configuration Analysis:

✅ 4 CPUs - Perfect (recommended 2-4)
✅ 31.75 GiB RAM - Excellent (recommended 4-8 GB)
✅ 294.23 GiB Disk - Plenty of space (recommended 20-50 GB)
⚠️ Unprivileged: Yes - Will work, but see Nginx note below

Enabling Systemd (if needed)

If systemd doesn't work in your container, you may need to enable it:

# Check if systemd is working
systemctl status

# If it fails, you may need to enable it in Proxmox
# Edit container config on Proxmox host:
pct set <container-id> -features nesting=1
pct set <container-id> -features keyctl=1

Run these commands on your Ubuntu container/VM:

# Update system
apt update && apt upgrade -y

# Install Node.js 20.x (LTS)
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt install -y nodejs

# Install Python 3.11+ and pip
apt install -y python3 python3-pip python3-venv

# Install PostgreSQL client (for remote database connection)
apt install -y postgresql-client

# Install Nginx
apt install -y nginx

# Install Git
apt install -y git

# Install build tools (for native modules)
apt install -y build-essential

# Install PM2 (process manager - optional but recommended)
npm install -g pm2

# Install Uvicorn (for Python service)
pip3 install uvicorn

# Verify installations
node --version  # Should be v20.x.x
python3 --version  # Should be 3.11+
nginx -v

4. Application Setup

4.1 Clone and Prepare Repository

# Create application directory
mkdir -p /opt/institutional_trader
cd /opt/institutional_trader

# Clone your repository (or upload files)
# Option 1: If using Git
git clone <your-repo-url> .

# Option 2: If uploading files manually
# Use SCP or Proxmox file browser to upload files

4.2 Backend Setup

cd /opt/institutional_trader/backend

# Install dependencies
npm install --production

# Create production .env file
cp env.production.example .env
nano .env

Backend .env configuration:

NODE_ENV=production
PORT=3000

# Supabase Configuration (if using Supabase)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_ANON_KEY=your_anon_key
SUPABASE_SERVICE_KEY=your_service_role_key

# OR Direct PostgreSQL Connection (if using local/remote PostgreSQL)
DATABASE_URL=postgresql://postgres:password@100.121.163.23:5432/institutional_trader

# CORS Configuration
CORS_ORIGIN=https://yourdomain.com

# Security
JWT_SECRET=your_super_secret_jwt_key_min_32_chars_long
RATE_LIMIT_WINDOW=15
RATE_LIMIT_MAX=100

# Python Service (if using)
PYTHON_SERVICE_URL=http://localhost:8000

4.3 Python Service Setup

cd /opt/institutional_trader/backend/python_service

# Create virtual environment
python3 -m venv venv

# Activate virtual environment
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Deactivate (we'll use systemd to manage this)
deactivate

4.4 Frontend Setup

cd /opt/institutional_trader/frontend

# Install dependencies
npm install

# Create production .env file
cp env.production.example .env.production
nano .env.production

Frontend .env.production configuration:

VITE_API_URL=https://api.yourdomain.com
VITE_WS_URL=wss://api.yourdomain.com
VITE_SUPABASE_URL=https://your-project.supabase.co
VITE_SUPABASE_ANON_KEY=your_anon_key

Build frontend:

npm run build

This creates a dist folder with production-ready files.

5. Database Configuration

Option A: Using Supabase (Cloud Database - Recommended)

Set up Supabase:
- Go to supabase.com
- Create a new project
- Get your connection details from Settings > API
- Run database schema scripts from backend/database/ in Supabase SQL Editor
Connection String:
- Use DATABASE_URL or SUPABASE_URL + keys in backend .env
- See Database Access for Friend section for sharing access

Option B: Remote PostgreSQL Database

Your database is hosted at 100.121.163.23:5432. Configure connection:

Update backend .env:

USE_LOCAL_DB=true
LOCAL_DB_HOST=100.121.163.23
LOCAL_DB_PORT=5432
LOCAL_DB_USER=postgres
LOCAL_DB_PASSWORD=your_postgres_password
LOCAL_DB_NAME=institutional_trader

Or use DATABASE_URL format:

DATABASE_URL=postgresql://postgres:your_password@100.121.163.23:5432/institutional_trader

Test connection:

# Test from container
psql -h 100.121.163.23 -p 5432 -U postgres -d institutional_trader

# Or test from Proxmox host
apt install -y postgresql-client
psql -h 100.121.163.23 -p 5432 -U postgres -d institutional_trader

Note: Make sure the remote PostgreSQL server allows connections from your Proxmox container IP. You may need to:

Configure pg_hba.conf on the database server to allow your container's IP
Configure firewall rules to allow port 5432 from your container
Ensure PostgreSQL is listening on the correct interface (not just localhost)

6. Systemd Services

6.1 Backend Service

Create systemd service for Node.js backend:

sudo nano /etc/systemd/system/institutional-trader-backend.service

Add:

[Unit]
Description=Institutional Trader Backend API
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend
Environment=NODE_ENV=production
EnvironmentFile=/opt/institutional_trader/backend/.env
ExecStart=/usr/bin/node src/server.js
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-backend
sudo systemctl start institutional-trader-backend
sudo systemctl status institutional-trader-backend

6.2 Python Service

Create systemd service for Python service:

sudo nano /etc/systemd/system/institutional-trader-python.service

Add:

[Unit]
Description=Institutional Trader Python Service
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend/python_service
Environment="PATH=/opt/institutional_trader/backend/python_service/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
ExecStart=/opt/institutional_trader/backend/python_service/venv/bin/uvicorn main:app --host 0.0.0.0 --port 8000
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-python
sudo systemctl start institutional-trader-python
sudo systemctl status institutional-trader-python

6.3 View Logs

# Backend logs
sudo journalctl -u institutional-trader-backend -f

# Python service logs
sudo journalctl -u institutional-trader-python -f

# All logs
sudo journalctl -u institutional-trader-* -f

7. Nginx Configuration

⚠️ Important: Unprivileged Container Nginx Setup

If your container is unprivileged, Nginx cannot bind to ports 80/443 directly. You have two options:

Option A: Run Nginx on Proxmox Host (Recommended)

Install Nginx on the Proxmox host
Configure it to proxy to your container's services
Container services run on ports 3000 (backend) and 8000 (Python)
See "Nginx on Host" section below

Option B: Use Port Forwarding

Run Nginx in container on ports 8080/8443
Forward ports from Proxmox host to container
Configure firewall rules

Option C: Enable Privileged Mode (Less Secure)

Convert container to privileged mode
Allows binding to ports 80/443
Less secure but simpler

7.1 Create Nginx Configuration

For Container (if using Option B):

sudo nano /etc/nginx/sites-available/institutional-trader

Copy and customize the configuration (replace yourdomain.com with your domain):

# Backend API Server
server {
    listen 443 ssl http2;
    server_name api.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    # SSL Configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # API Routes
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
    }

    # WebSocket Support
    location /ws {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # WebSocket specific timeouts
        proxy_read_timeout 86400;
    }

    # Health Check
    location /health {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }
}

# Frontend Server
server {
    listen 443 ssl http2;
    server_name yourdomain.com www.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    # SSL Configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    root /opt/institutional_trader/frontend/dist;
    index index.html;

    # Security Headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # Gzip Compression
    gzip on;
    gzip_vary on;
    gzip_min_length 1024;
    gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml+rss application/json;

    location / {
        try_files $uri $uri/ /index.html;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }

    # Static assets caching
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }
}

# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com api.yourdomain.com;
    return 301 https://$server_name$request_uri;
}

7.2 Enable Site

# Create symlink
sudo ln -s /etc/nginx/sites-available/institutional-trader /etc/nginx/sites-enabled/

# Test configuration
sudo nginx -t

# Reload Nginx
sudo systemctl reload nginx

7.3 Nginx on Proxmox Host (For Unprivileged Containers)

If your container is unprivileged, install Nginx on the Proxmox host:

On Proxmox Host:

# Install Nginx
apt install -y nginx

# Create configuration
nano /etc/nginx/sites-available/institutional-trader

Nginx Config (on Proxmox host, pointing to container):

# Backend API Server
server {
    listen 443 ssl http2;
    server_name api.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    location / {
        proxy_pass http://<container-ip>:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location /ws {
        proxy_pass http://<container-ip>:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_read_timeout 86400;
    }
}

# Frontend Server
server {
    listen 443 ssl http2;
    server_name yourdomain.com www.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;

    root /var/www/institutional-trader/dist;
    index index.html;

    location / {
        try_files $uri $uri/ /index.html;
    }
}

Copy frontend files to host:

# On container, create archive
cd /opt/institutional_trader/frontend
tar czf /tmp/frontend-dist.tar.gz dist/

# On Proxmox host, copy from container
pct pull <container-id> /tmp/frontend-dist.tar.gz /tmp/
tar xzf /tmp/frontend-dist.tar.gz -C /var/www/institutional-trader/

8. SSL Certificates

8.1 Install Certbot

sudo apt install -y certbot python3-certbot-nginx

8.2 Obtain SSL Certificate

# Replace with your actual domain
sudo certbot --nginx -d yourdomain.com -d www.yourdomain.com -d api.yourdomain.com

Follow the prompts:

Enter your email address
Agree to terms
Choose whether to redirect HTTP to HTTPS (recommended: Yes)

8.3 Auto-Renewal

Certbot sets up auto-renewal automatically. Test it:

sudo certbot renew --dry-run

9. Database Access for Friend

Option A: Supabase (Recommended)

Method 1: Create New Supabase User (Best for Collaboration)

Invite via Supabase Dashboard:
- Go to your Supabase project
- Navigate to Settings > Team
- Click Invite Member
- Enter your friend's email
- Choose role (Developer or Admin)
- They'll receive an invitation email
Share Connection Details:
- Your friend will need:
  - SUPABASE_URL (same as yours)
  - SUPABASE_ANON_KEY (same as yours - safe to share)
  - Their own account credentials
For Backend Access:
- If they need service key access, you can:
  - Share SUPABASE_SERVICE_KEY (keep this secure!)
  - Or create a separate Supabase project for them
  - Or use Row Level Security (RLS) policies to limit access

Method 2: Create Database User (PostgreSQL User)

If using direct PostgreSQL connection:

-- Connect to Supabase via psql or Supabase SQL Editor
-- Create a new user
CREATE USER friend_user WITH PASSWORD 'secure_password_here';

-- Grant necessary permissions
GRANT CONNECT ON DATABASE postgres TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;

-- Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;

Connection String for Friend:

postgresql://friend_user:secure_password_here@db.your-project.supabase.co:5432/postgres

Option B: Local PostgreSQL

If using local PostgreSQL:

# Connect to PostgreSQL
sudo -u postgres psql

# Create user for friend
CREATE USER friend_user WITH PASSWORD 'secure_password_here';

# Grant permissions
GRANT CONNECT ON DATABASE institutional_trader TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;

# Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;

\q

Connection String for Friend:

postgresql://friend_user:secure_password_here@100.121.163.23:5432/institutional_trader

Important: Make sure the PostgreSQL server at 100.121.163.23:

Allows remote connections from your friend's IP
Has pg_hba.conf configured to allow connections
Has firewall rules allowing port 5432
Consider using SSH tunnel for additional security

Option C: SSH Tunnel (Most Secure)

Create an SSH tunnel for your friend:

# On your friend's machine
ssh -L 5432:100.121.163.23:5432 user@your-proxmox-server-ip

# Then they can connect using:
# postgresql://friend_user:password@localhost:5432/institutional_trader

This creates a secure tunnel through your Proxmox server to the database.

Option D: Read-Only Access (If Friend Only Needs to Read Data)

-- Create read-only user
CREATE USER friend_readonly WITH PASSWORD 'secure_password_here';
GRANT CONNECT ON DATABASE institutional_trader TO friend_readonly;
GRANT USAGE ON SCHEMA public TO friend_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO friend_readonly;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO friend_readonly;

10. Monitoring & Maintenance

10.1 Health Checks

# Check backend health
curl https://api.yourdomain.com/health

# Check services status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx

10.2 Logs

# Backend logs
sudo journalctl -u institutional-trader-backend -n 100 -f

# Python service logs
sudo journalctl -u institutional-trader-python -n 100 -f

# Nginx access logs
sudo tail -f /var/log/nginx/access.log

# Nginx error logs
sudo tail -f /var/log/nginx/error.log

10.3 Update Application

# Pull latest changes (if using Git)
cd /opt/institutional_trader
git pull

# Update backend
cd backend
npm install --production
sudo systemctl restart institutional-trader-backend

# Update Python service
cd python_service
source venv/bin/activate
pip install -r requirements.txt
deactivate
sudo systemctl restart institutional-trader-python

# Update frontend
cd ../frontend
npm install
npm run build
sudo systemctl reload nginx

10.4 Backup Database

For Supabase:

Use Supabase Dashboard > Database > Backups
Or use pg_dump with connection string

For Remote PostgreSQL:

# Create backup
pg_dump -h 100.121.163.23 -p 5432 -U postgres -d institutional_trader > backup_$(date +%Y%m%d).sql

# Restore backup
psql -h 100.121.163.23 -p 5432 -U postgres -d institutional_trader < backup_20240101.sql

11. Troubleshooting

Backend Won't Start

# Check logs
sudo journalctl -u institutional-trader-backend -n 50

# Check if port is in use
sudo netstat -tlnp | grep 3000

# Test database connection
cd /opt/institutional_trader/backend
node -e "import('./src/db.js').then(m => m.testConnection())"

Python Service Won't Start

# Check logs
sudo journalctl -u institutional-trader-python -n 50

# Test manually
cd /opt/institutional_trader/backend/python_service
source venv/bin/activate
uvicorn main:app --host 0.0.0.0 --port 8000

Nginx Errors

# Test configuration
sudo nginx -t

# Check error logs
sudo tail -f /var/log/nginx/error.log

# Reload configuration
sudo systemctl reload nginx

Database Connection Issues

# Test PostgreSQL connection (remote)
psql -h 100.121.163.23 -p 5432 -U postgres -d institutional_trader

# Test Supabase connection
psql "postgresql://postgres:password@db.project.supabase.co:5432/postgres"

Firewall Issues

# Check firewall status
sudo ufw status

# Allow ports (if using UFW)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 22/tcp  # SSH
sudo ufw enable

SSL Certificate Issues

# Check certificate status
sudo certbot certificates

# Renew certificate manually
sudo certbot renew

# Check certificate expiration
sudo certbot certificates | grep Expiry

Quick Reference Commands

# Start all services
sudo systemctl start institutional-trader-backend
sudo systemctl start institutional-trader-python

# Stop all services
sudo systemctl stop institutional-trader-backend
sudo systemctl stop institutional-trader-python

# Restart all services
sudo systemctl restart institutional-trader-backend
sudo systemctl restart institutional-trader-python

# Check status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx

# View logs
sudo journalctl -u institutional-trader-* -f

# Test health
curl https://api.yourdomain.com/health

Security Checklist

All .env files have secure passwords
JWT_SECRET is at least 32 characters and random
SSL certificates installed and auto-renewal enabled
Firewall configured (only necessary ports open)
Database users have appropriate permissions (principle of least privilege)
Regular backups configured
System updates applied regularly
SSH key authentication enabled (disable password auth)
Rate limiting enabled in backend
CORS origin set to production domain only

Next Steps

✅ Deploy application to Proxmox
✅ Configure database access for friend
✅ Set up monitoring/alerting (optional: UptimeRobot, Sentry)
✅ Configure automated backups
✅ Set up CI/CD pipeline (optional)

Your application should now be running on your Proxmox server! 🎉

24 KiB Raw Blame History