deepkoluguri
/
institutional-trader
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
institutional-trader/README/PROXMOX_DEPLOYMENT.md

1345 lines
35 KiB
Markdown
Raw Blame History

# 🚀 Proxmox Deployment Guide
Complete guide for deploying the Institutional Trader platform on a Proxmox server.
## Table of Contents
1. [Prerequisites](#1-prerequisites)
2. [Proxmox VM/Container Setup](#2-proxmox-vmcontainer-setup)
3. [System Dependencies](#3-system-dependencies)
4. [Application Setup](#4-application-setup)
5. [Database Configuration](#5-database-configuration)
6. [Systemd Services](#6-systemd-services)
7. [Nginx Configuration](#7-nginx-configuration)
8. [DNS Configuration](#8-dns-configuration)
9. [SSL Certificates](#9-ssl-certificates)
10. [Database Access for Friend](#10-database-access-for-friend)
11. [Monitoring & Maintenance](#11-monitoring--maintenance)
12. [Troubleshooting](#12-troubleshooting)
---
## 1. Prerequisites
- Proxmox server with network access
- Domain name pointing to your server's IP (for SSL)
- SSH access to the Proxmox host
- Basic knowledge of Linux command line
---
## 2. Proxmox VM/Container Setup
### Option A: Ubuntu LXC Container (Recommended - Lightweight)
1. **Create LXC Container:**
- In Proxmox Web UI, go to your node
- Click "Create CT" (Container)
- Choose Ubuntu 22.04 or 24.04 template
- Configure:
- **Hostname**: `institutional-trader`
- **Password**: Set root password
- **CPU**: 2-4 cores
- **Memory**: 4-8 GB RAM
- **Disk**: 20-50 GB
- **Network**: Bridge with static IP (recommended) or DHCP
2. **Start Container:**
```bash
# In Proxmox shell or via Web UI
pct start <container-id>
```
3. **Access Container:**
```bash
# From Proxmox host
pct enter <container-id>
# Or SSH if you configured it
ssh root@<container-ip>
```
### Option B: Ubuntu VM (More Isolation)
1. **Create VM:**
- In Proxmox Web UI, go to your node
- Click "Create VM"
- Choose Ubuntu 22.04 or 24.04 ISO
- Configure:
- **CPU**: 2-4 cores
- **Memory**: 4-8 GB RAM
- **Disk**: 50-100 GB
- **Network**: Bridge adapter
- Install Ubuntu Server (minimal installation)
2. **Access VM:**
```bash
ssh <user>@<vm-ip>
```
---
## 3. System Dependencies
Run these commands on your Ubuntu container/VM:
```bash
# Update system
apt update && apt upgrade -y
# Install Node.js 20.x (LTS)
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt install -y nodejs
# Install Python 3.11+ and pip
apt install -y python3 python3-pip python3-venv
# Install PostgreSQL client (if using local DB)
apt install -y postgresql-client
# Install Nginx
apt install -y nginx
# Install Git
apt install -y git
# Install build tools (for native modules)
apt install -y build-essential
# Install PM2 (process manager - optional but recommended)
npm install -g pm2
# Install Uvicorn (for Python service)
pip3 install uvicorn
# Verify installations
node --version # Should be v20.x.x
python3 --version # Should be 3.11+
nginx -v
```
---
## 4. Application Setup
### 4.1 Clone and Prepare Repository
```bash
# Create application directory
mkdir -p /opt/institutional_trader
cd /opt/institutional_trader
# Clone your repository (or upload files)
# Option 1: If using Git
git clone <your-repo-url> .
# Option 2: If uploading files manually
# Use SCP or Proxmox file browser to upload files
```
### 4.2 Backend Setup
```bash
cd /opt/institutional_trader/backend
# Install dependencies
npm install --production
# Create production .env file
cp env.production.example .env
nano .env
```
**Backend `.env` configuration:**
```env
NODE_ENV=production
PORT=3000
# Supabase Configuration (if using Supabase)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_ANON_KEY=your_anon_key
SUPABASE_SERVICE_KEY=your_service_role_key
# OR Direct PostgreSQL Connection (if using local/remote PostgreSQL)
DATABASE_URL=postgresql://postgres:password@localhost:5432/institutional_trader
# CORS Configuration
CORS_ORIGIN=https://yourdomain.com
# Security
JWT_SECRET=your_super_secret_jwt_key_min_32_chars_long
RATE_LIMIT_WINDOW=15
RATE_LIMIT_MAX=100
# Python Service (if using)
PYTHON_SERVICE_URL=http://localhost:8010
```
### 4.3 Python Service Setup
```bash
cd /opt/institutional_trader/backend/python_service
# Create virtual environment
python3 -m venv venv
# Activate virtual environment
source venv/bin/activate
# Install dependencies
pip install -r requirements.txt
# Deactivate (we'll use systemd to manage this)
deactivate
```
### 4.4 Frontend Setup
```bash
cd /opt/institutional_trader/frontend
# Install dependencies
npm install
# Create production .env file
cp env.production.example .env.production
nano .env.production
```
**Frontend `.env.production` configuration:**
**⚠️ Critical:** The frontend must have a `.env.production` file with the correct API URL, otherwise it will try to connect to `localhost:3010` which won't work in production.
```env
VITE_API_URL=https://api.yourdomain.com
VITE_WS_URL=wss://api.yourdomain.com
VITE_SUPABASE_URL=https://your-project.supabase.co
VITE_SUPABASE_ANON_KEY=your_anon_key
```
**Build frontend:**
```bash
npm run build
```
This creates a `dist` folder with production-ready files.
---
## 5. Database Configuration
### Option A: Using Supabase (Cloud Database - Recommended)
1. **Set up Supabase:**
- Go to [supabase.com](https://supabase.com)
- Create a new project
- Get your connection details from Settings > API
- Run database schema scripts from `backend/database/` in Supabase SQL Editor
2. **Connection String:**
- Use `DATABASE_URL` or `SUPABASE_URL` + keys in backend `.env`
- See [Database Access for Friend](#9-database-access-for-friend) section for sharing access
### Option B: Local PostgreSQL (Self-Hosted)
```bash
# Install PostgreSQL
apt install -y postgresql postgresql-contrib
# Start PostgreSQL
systemctl start postgresql
systemctl enable postgresql
# Create database and user
sudo -u postgres psql
# In PostgreSQL prompt:
CREATE DATABASE institutional_trader;
CREATE USER trader_user WITH PASSWORD 'secure_password_here';
GRANT ALL PRIVILEGES ON DATABASE institutional_trader TO trader_user;
\q
# Import schema
cd /opt/institutional_trader/backend/database
sudo -u postgres psql institutional_trader < schema.sql
sudo -u postgres psql institutional_trader < missing_tables.sql
# Import other SQL files as needed
```
Update backend `.env`:
```env
USE_LOCAL_DB=true
LOCAL_DB_HOST=localhost
LOCAL_DB_PORT=5432
LOCAL_DB_USER=trader_user
LOCAL_DB_PASSWORD=secure_password_here
LOCAL_DB_NAME=institutional_trader
```
---
## 6. Systemd Services
### 6.1 Backend Service
Create systemd service for Node.js backend:
```bash
sudo nano /etc/systemd/system/institutional-trader-backend.service
```
Add:
```ini
[Unit]
Description=Institutional Trader Backend API
After=network.target postgresql.service
[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend
Environment=NODE_ENV=production
EnvironmentFile=/opt/institutional_trader/backend/.env
ExecStart=/usr/bin/node src/server.js
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
```
Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-backend
sudo systemctl start institutional-trader-backend
sudo systemctl status institutional-trader-backend
```
### 6.2 Python Service
Create systemd service for Python service:
```bash
sudo nano /etc/systemd/system/institutional-trader-python.service
```
Add:
```ini
[Unit]
Description=Institutional Trader Python Service
After=network.target postgresql.service
[Service]
Type=simple
User=root
WorkingDirectory=/opt/institutional_trader/backend/python_service
Environment="PATH=/opt/institutional_trader/backend/python_service/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
EnvironmentFile=/opt/institutional_trader/backend/python_service/.env
ExecStart=/opt/institutional_trader/backend/python_service/venv/bin/uvicorn main:app --host 0.0.0.0 --port 8010
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
```
**Note:** The `EnvironmentFile` directive is optional - the service will also load `.env` automatically via `load_dotenv()` in the code. However, adding it explicitly ensures environment variables are available even if the code path changes.
Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable institutional-trader-python
sudo systemctl start institutional-trader-python
sudo systemctl status institutional-trader-python
```
### 6.3 View Logs
```bash
# Backend logs
sudo journalctl -u institutional-trader-backend -f
# Python service logs
sudo journalctl -u institutional-trader-python -f
# All logs
sudo journalctl -u institutional-trader-* -f
```
---
## 7. Nginx Configuration
### 7.1 Create Nginx Configuration
```bash
sudo nano /etc/nginx/sites-available/institutional-trader
```
Copy and customize the configuration (replace `yourdomain.com` with your domain):
```nginx
# Backend API Server
server {
listen 443 ssl http2;
server_name api.yourdomain.com;
ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
# SSL Configuration
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
# Security Headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# API Routes
location / {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_cache_bypass $http_upgrade;
}
# WebSocket Support
location /ws {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# WebSocket specific timeouts
proxy_read_timeout 86400;
}
# Health Check
location /health {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Host $host;
add_header Cache-Control "no-cache, no-store, must-revalidate";
}
}
# Frontend Server
server {
listen 443 ssl http2;
server_name yourdomain.com www.yourdomain.com;
ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
# SSL Configuration
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
root /opt/institutional_trader/frontend/dist;
index index.html;
# Security Headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
# Gzip Compression
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml+rss application/json;
location / {
try_files $uri $uri/ /index.html;
add_header Cache-Control "no-cache, no-store, must-revalidate";
}
# Static assets caching
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
}
# Redirect HTTP to HTTPS
server {
listen 80;
server_name yourdomain.com www.yourdomain.com api.yourdomain.com;
return 301 https://$server_name$request_uri;
}
```
### 7.2 Multi-Container Setup (Nginx on Separate Container)
If you're running nginx on a separate container/VM that proxies to your application containers:
**Example configuration for proxying to another Proxmox container:**
```nginx
server {
listen 443 ssl http2;
server_name traderideas.deepteklabs.com;
ssl_certificate /etc/letsencrypt/live/traderideas.deepteklabs.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/traderideas.deepteklabs.com/privkey.pem;
# Security Headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Content-Type-Options "nosniff" always;
# Backend API Routes
location /api/ {
proxy_pass http://192.168.8.151:3000/api/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_buffering off;
}
# WebSocket Support (if using WebSockets)
location /ws {
proxy_pass http://192.168.8.151:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 86400; # 24 hours for long-lived connections
}
# Health Check
location /health {
proxy_pass http://192.168.8.151:3000/health;
access_log off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
add_header Cache-Control "no-cache, no-store, must-revalidate";
}
# Frontend (served from application container)
location / {
proxy_pass http://192.168.8.151:8080/;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_hide_header Cache-Control;
add_header Cache-Control "no-cache, no-store, must-revalidate" always;
}
# Static assets with caching
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot|otf|map)$ {
proxy_pass http://192.168.8.151:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
expires 1y;
add_header Cache-Control "public, immutable";
access_log off;
}
}
```
**Important notes for multi-container setup:**
- Replace `192.168.8.151` with your application container's IP
- Ensure containers can communicate (same network/bridge)
- Port 3000 = Backend API, Port 8080 = Frontend (adjust as needed)
- Test connectivity: `curl http://192.168.8.151:3000/health` from nginx container
### 7.3 Enable Site
```bash
# Create symlink
sudo ln -s /etc/nginx/sites-available/institutional-trader /etc/nginx/sites-enabled/
# Test configuration
sudo nginx -t
# Reload Nginx
sudo systemctl reload nginx
```
---
## 8. DNS Configuration
**⚠️ Important:** Before setting up SSL certificates, you must configure DNS records pointing to your server.
### 8.1 Find Your Server's Public IP Address
```bash
# On your server, find the public IP
curl ifconfig.me
# Or
hostname -I
# Or check in Proxmox Web UI: Datacenter → Your Node → Network
```
### 8.2 Configure DNS A Records
Go to your domain registrar's DNS management panel (where you registered `traderideas.deepteklabs.com`) and add these A records:
| Type | Name | Value | TTL |
|------|------|-------|-----|
| A | `@` (or blank) | `<your-server-ip>` | 3600 |
| A | `www` | `<your-server-ip>` | 3600 |
| A | `api` | `<your-server-ip>` | 3600 |
**Example for `traderideas.deepteklabs.com`:**
- `traderideas.deepteklabs.com` → `<your-server-ip>` (A record with name `@` or blank)
- `www.traderideas.deepteklabs.com` → `<your-server-ip>` (A record with name `www`)
- `api.traderideas.deepteklabs.com` → `<your-server-ip>` (A record with name `api`)
### 8.3 Verify DNS Propagation
Wait 5-15 minutes for DNS to propagate, then verify:
```bash
# Check if DNS records are resolving
dig traderideas.deepteklabs.com +short
dig www.traderideas.deepteklabs.com +short
dig api.traderideas.deepteklabs.com +short
# All should return your server's IP address
# Alternative: Use nslookup
nslookup traderideas.deepteklabs.com
nslookup www.traderideas.deepteklabs.com
nslookup api.traderideas.deepteklabs.com
# Or use online tools:
# - https://dnschecker.org
# - https://www.whatsmydns.net
```
**Common DNS Providers:**
- **Cloudflare**: Dashboard → DNS → Records → Add record
- **Namecheap**: Domain List → Manage → Advanced DNS → Add New Record
- **GoDaddy**: DNS Management → Add Record
- **Google Domains**: DNS → Custom Records → Add Record
### 8.4 Test Domain Accessibility
Once DNS is propagated, test that domains are reachable:
```bash
# From your server
curl -I http://traderideas.deepteklabs.com
curl -I http://www.traderideas.deepteklabs.com
curl -I http://api.traderideas.deepteklabs.com
# Should return HTTP responses (even if 502/503, that's OK - means DNS is working)
```
### 8.5 Cloudflare Proxy (Orange Cloud)
**If your domain uses Cloudflare proxy (orange cloud icon):**
When DNS resolves to Cloudflare IPs (like `104.21.x.x` or `172.67.x.x`), your domain is behind Cloudflare proxy. This affects SSL certificate setup:
**Check if using Cloudflare proxy:**
```bash
dig traderideas.deepteklabs.com +short
# If returns Cloudflare IPs (104.x.x.x, 172.x.x.x), proxy is enabled
```
**Options for SSL with Cloudflare:**
1. **Use DNS Challenge (Recommended)** - Works with proxy enabled
2. **Temporarily Disable Proxy** - Get certs, then re-enable proxy
3. **Use Cloudflare SSL** - Let Cloudflare handle SSL (easiest)
See Section 9.2 for detailed instructions.
**⚠️ Only proceed to SSL setup once DNS records are working!**
---
## 9. SSL Certificates
### 9.1 Install Certbot
```bash
sudo apt install -y certbot python3-certbot-nginx
```
### 9.2 Obtain SSL Certificate
**⚠️ Prerequisites:**
- DNS A records must be configured and propagated (see Section 8)
- Domains must resolve to your server's IP (or Cloudflare if using proxy)
- Port 80 must be accessible from the internet (for HTTP challenge) OR DNS API access (for DNS challenge)
**⚠️ Important:** If your nginx config already references SSL certificates that don't exist, you need to fix this first.
**Check if using Cloudflare proxy:**
```bash
dig traderideas.deepteklabs.com +short
# If returns Cloudflare IPs (104.x.x.x, 172.x.x.x), you're using Cloudflare proxy
```
**If using Cloudflare Proxy, use Option D (DNS Challenge) or Option E (Cloudflare SSL)**
**Option A: Use Standalone Mode (Works if NOT using Cloudflare proxy)**
```bash
# Stop nginx temporarily
sudo systemctl stop nginx
# Get certificates using standalone mode
sudo certbot certonly --standalone -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com
# Start nginx again
sudo systemctl start nginx
# Now certbot can update the nginx config
sudo certbot --nginx -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com
```
**Option B: Temporarily Comment Out SSL in Nginx Config**
```bash
# Edit nginx config
sudo nano /etc/nginx/sites-available/institutional-trader
# Temporarily comment out SSL lines:
# - Comment out: listen 443 ssl http2;
# - Comment out: ssl_certificate and ssl_certificate_key lines
# - Change: listen 443 ssl http2; to: listen 80;
# Example temporary config (HTTP only):
# server {
# listen 80;
# server_name traderideas.deepteklabs.com;
# # ssl_certificate /etc/letsencrypt/live/traderideas.deepteklabs.com/fullchain.pem;
# # ssl_certificate_key /etc/letsencrypt/live/traderideas.deepteklabs.com/privkey.pem;
# ...
# }
# Test and reload
sudo nginx -t
sudo systemctl reload nginx
# Now get certificates (certbot will automatically update the config)
sudo certbot --nginx -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com
# Certbot will automatically:
# - Get the certificates
# - Update your nginx config to use them
# - Add SSL redirects
```
**Option C: Use Certbot with Nginx Plugin (If config is clean and NOT using Cloudflare proxy)**
```bash
# Replace with your actual domain
sudo certbot --nginx -d traderideas.deepteklabs.com -d www.traderideas.deepteklabs.com -d api.traderideas.deepteklabs.com
```
Follow the prompts:
- Enter your email address
- Agree to terms
- Choose whether to redirect HTTP to HTTPS (recommended: Yes)
**Option D: DNS Challenge (Recommended for Cloudflare proxy)**
If your domain is behind Cloudflare proxy, use DNS challenge:
```bash
# Install Cloudflare plugin for certbot
sudo apt install -y python3-pip
sudo pip3 install certbot-dns-cloudflare
# Create Cloudflare API token directory
sudo mkdir -p /etc/letsencrypt
sudo chmod 700 /etc/letsencrypt
# Create Cloudflare credentials file
sudo nano /etc/letsencrypt/cloudflare.ini
```
Add your Cloudflare API token:
```ini
# Cloudflare API token (get from: Cloudflare Dashboard → My Profile → API Tokens → Create Token)
# Permissions needed: Zone:Zone:Read, Zone:DNS:Edit
dns_cloudflare_api_token = YOUR_CLOUDFLARE_API_TOKEN
```
```bash
# Secure the credentials file
sudo chmod 600 /etc/letsencrypt/cloudflare.ini
# Get certificates using DNS challenge
sudo certbot certonly \
--dns-cloudflare \
--dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini \
-d traderideas.deepteklabs.com \
-d www.traderideas.deepteklabs.com \
-d api.traderideas.deepteklabs.com
# Update nginx config manually (certbot won't auto-update with DNS challenge)
sudo nano /etc/nginx/sites-available/institutional-trader
# Update ssl_certificate paths to:
# ssl_certificate /etc/letsencrypt/live/traderideas.deepteklabs.com/fullchain.pem;
# ssl_certificate_key /etc/letsencrypt/live/traderideas.deepteklabs.com/privkey.pem;
sudo nginx -t
sudo systemctl reload nginx
```
**Option E: Use Cloudflare SSL (Easiest with Cloudflare proxy)**
If using Cloudflare proxy, you can let Cloudflare handle SSL:
1. **In Cloudflare Dashboard:**
- Go to SSL/TLS → Overview
- Set encryption mode to **"Full"** or **"Full (strict)"**
- Cloudflare will automatically provide SSL certificates
2. **On your server, use self-signed or Cloudflare Origin Certificate:**
```bash
# Generate self-signed certificate (Cloudflare will accept it)
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout /etc/ssl/private/nginx-selfsigned.key \
-out /etc/ssl/certs/nginx-selfsigned.crt
# Update nginx config to use self-signed cert
sudo nano /etc/nginx/sites-available/institutional-trader
# Change to:
# ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
# ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
sudo nginx -t
sudo systemctl reload nginx
```
3. **Or get Cloudflare Origin Certificate:**
- Cloudflare Dashboard → SSL/TLS → Origin Server → Create Certificate
- Copy certificate and key
- Save to `/etc/ssl/certs/cloudflare-origin.crt` and `/etc/ssl/private/cloudflare-origin.key`
- Update nginx config to use these files
### 9.3 Auto-Renewal
Certbot sets up auto-renewal automatically. Test it:
```bash
sudo certbot renew --dry-run
```
---
## 10. Database Access for Friend
### Option A: Supabase (Recommended)
#### Method 1: Create New Supabase User (Best for Collaboration)
1. **Invite via Supabase Dashboard:**
- Go to your Supabase project
- Navigate to **Settings > Team**
- Click **Invite Member**
- Enter your friend's email
- Choose role (Developer or Admin)
- They'll receive an invitation email
2. **Share Connection Details:**
- Your friend will need:
- `SUPABASE_URL` (same as yours)
- `SUPABASE_ANON_KEY` (same as yours - safe to share)
- Their own account credentials
3. **For Backend Access:**
- If they need service key access, you can:
- Share `SUPABASE_SERVICE_KEY` (keep this secure!)
- Or create a separate Supabase project for them
- Or use Row Level Security (RLS) policies to limit access
#### Method 2: Create Database User (PostgreSQL User)
If using direct PostgreSQL connection:
```sql
-- Connect to Supabase via psql or Supabase SQL Editor
-- Create a new user
CREATE USER friend_user WITH PASSWORD 'secure_password_here';
-- Grant necessary permissions
GRANT CONNECT ON DATABASE postgres TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;
-- Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;
```
**Connection String for Friend:**
```
postgresql://friend_user:secure_password_here@db.your-project.supabase.co:5432/postgres
```
### Option B: Local PostgreSQL
If using local PostgreSQL:
```bash
# Connect to PostgreSQL
sudo -u postgres psql
# Create user for friend
CREATE USER friend_user WITH PASSWORD 'secure_password_here';
# Grant permissions
GRANT CONNECT ON DATABASE institutional_trader TO friend_user;
GRANT USAGE ON SCHEMA public TO friend_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO friend_user;
GRANT SELECT, USAGE ON ALL SEQUENCES IN SCHEMA public TO friend_user;
# Grant permissions on future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO friend_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, USAGE ON SEQUENCES TO friend_user;
\q
```
**Connection String for Friend:**
```
postgresql://friend_user:secure_password_here@your-server-ip:5432/institutional_trader
```
**Important:** If your friend needs to connect from outside your network:
1. Configure PostgreSQL to accept remote connections
2. Set up firewall rules
3. Consider using SSH tunnel for security
### Option C: SSH Tunnel (Most Secure)
Create an SSH tunnel for your friend:
```bash
# On your friend's machine
ssh -L 5432:localhost:5432 user@your-server-ip
# Then they can connect using:
# postgresql://friend_user:password@localhost:5432/institutional_trader
```
### Option D: Read-Only Access (If Friend Only Needs to Read Data)
```sql
-- Create read-only user
CREATE USER friend_readonly WITH PASSWORD 'secure_password_here';
GRANT CONNECT ON DATABASE institutional_trader TO friend_readonly;
GRANT USAGE ON SCHEMA public TO friend_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO friend_readonly;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO friend_readonly;
```
---
## 11. Monitoring & Maintenance
### 10.1 Health Checks
```bash
# Check backend health
curl https://api.yourdomain.com/health
# Check services status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx
```
### 10.2 Logs
```bash
# Backend logs
sudo journalctl -u institutional-trader-backend -n 100 -f
# Python service logs
sudo journalctl -u institutional-trader-python -n 100 -f
# Nginx access logs
sudo tail -f /var/log/nginx/access.log
# Nginx error logs
sudo tail -f /var/log/nginx/error.log
```
### 10.3 Update Application
```bash
# Pull latest changes (if using Git)
cd /opt/institutional_trader
git pull
# Update backend
cd backend
npm install --production
sudo systemctl restart institutional-trader-backend
# Update Python service
cd python_service
source venv/bin/activate
pip install -r requirements.txt
deactivate
sudo systemctl restart institutional-trader-python
# Update frontend
cd ../frontend
npm install
npm run build
sudo systemctl reload nginx
```
### 10.4 Backup Database
**For Supabase:**
- Use Supabase Dashboard > Database > Backups
- Or use `pg_dump` with connection string
**For Local PostgreSQL:**
```bash
# Create backup
sudo -u postgres pg_dump institutional_trader > backup_$(date +%Y%m%d).sql
# Restore backup
sudo -u postgres psql institutional_trader < backup_20240101.sql
```
---
## 12. Troubleshooting
### Git Pull Fails - DNS Resolution Error
**Error:** `fatal: unable to access 'https://github.com/...': Could not resolve host: github.com`
**This means your container/VM cannot resolve DNS names.**
**Quick Fix:**
```bash
# 1. Check current DNS configuration
cat /etc/resolv.conf
# 2. If empty or missing, add DNS servers
sudo nano /etc/resolv.conf
```
Add these lines:
```
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 1.1.1.1
```
**For LXC Containers (Persistent Fix):**
```bash
# Edit container DNS configuration
# On Proxmox host, edit container config:
nano /etc/pve/lxc/<container-id>.conf
# Add DNS servers:
nameserver: 8.8.8.8
nameserver: 8.8.4.4
```
Or via Proxmox Web UI:
- Go to your container → Options → DNS
- Set DNS servers: `8.8.8.8, 8.8.4.4`
**For Ubuntu VMs/Containers (Systemd-resolved):**
```bash
# Edit systemd-resolved config
sudo nano /etc/systemd/resolved.conf
```
Uncomment and set:
```
[Resolve]
DNS=8.8.8.8 8.8.4.4 1.1.1.1
FallbackDNS=1.1.1.1 8.8.8.8
```
**If using Tailscale (DNS server 100.100.100.100):**
If your `/etc/resolv.conf` shows Tailscale DNS (`100.100.100.100`) but it's timing out, add fallback DNS servers:
```bash
# Edit systemd-resolved config
sudo nano /etc/systemd/resolved.conf
```
Set:
```
[Resolve]
DNS=100.100.100.100 8.8.8.8 8.8.4.4 1.1.1.1
FallbackDNS=8.8.8.8 8.8.4.4 1.1.1.1
```
This keeps Tailscale DNS as primary (for Tailscale network access) but adds public DNS as fallback.
```bash
# Restart systemd-resolved
sudo systemctl restart systemd-resolved
# Test DNS
nslookup github.com
ping -c 2 github.com
```
**Alternative: Use IP Address (Temporary Workaround)**
If DNS still doesn't work, you can manually update git remote:
```bash
# Get GitHub IP
nslookup github.com 8.8.8.8
# Or use known GitHub IPs (may change)
# Update git remote to use IP (not recommended for long-term)
git remote set-url origin https://140.82.121.3/deepkoluguri/INSTITUTIONAL-FLOW-TRADING-PLATFORM.git
```
**Verify DNS is Working:**
```bash
# Test DNS resolution
nslookup github.com
dig github.com
# Test connectivity
ping -c 2 github.com
curl -I https://github.com
```
### Backend Won't Start
```bash
# Check logs
sudo journalctl -u institutional-trader-backend -n 50
# Check if port is in use
sudo netstat -tlnp | grep 3000
# Test database connection
cd /opt/institutional_trader/backend
node -e "import('./src/db.js').then(m => m.testConnection())"
```
### Python Service Won't Start
**Step 1: Check detailed error logs**
```bash
# View recent logs with full error messages
sudo journalctl -u institutional-trader-python -n 100 --no-pager
# Follow logs in real-time
sudo journalctl -u institutional-trader-python -f
```
**Step 2: Verify systemd service file has correct port**
```bash
# Check current service file
sudo cat /etc/systemd/system/institutional-trader-python.service
# If port is 8000, update it to 8010:
sudo nano /etc/systemd/system/institutional-trader-python.service
# Change: --port 8000 to --port 8010
# Then reload and restart:
sudo systemctl daemon-reload
sudo systemctl restart institutional-trader-python
```
**Step 3: Test manually to see actual error**
```bash
cd /opt/institutional_trader/backend/python_service
source venv/bin/activate
# Check if .env file exists and has correct DB config
cat .env
# Test startup manually
uvicorn main:app --host 0.0.0.0 --port 8010
```
**Step 4: Common issues and fixes**
**Database connection timeout:**
- The service now has a 10-second timeout and will start even if DB is temporarily unavailable
- Check database is reachable: `psql -h <DB_HOST> -U <DB_USER> -d institutional_trader`
- Verify `.env` file in `backend/python_service/` has correct database credentials
**Port already in use:**
```bash
# Check if port 8010 is in use
sudo netstat -tlnp | grep 8010
# Or
sudo lsof -i :8010
# Kill process if needed
sudo kill -9 <PID>
```
**Missing dependencies:**
```bash
cd /opt/institutional_trader/backend/python_service
source venv/bin/activate
pip install -r requirements.txt
```
**Step 5: Verify backend .env has correct Python service URL**
```bash
# Check backend .env
cat /opt/institutional_trader/backend/.env | grep PYTHON_SERVICE_URL
# Should be:
# PYTHON_SERVICE_URL=http://localhost:8010
# If not, update it:
nano /opt/institutional_trader/backend/.env
# Add or update: PYTHON_SERVICE_URL=http://localhost:8010
# Then restart backend:
sudo systemctl restart institutional-trader-backend
```
### Nginx Errors
```bash
# Test configuration
sudo nginx -t
# Check error logs
sudo tail -f /var/log/nginx/error.log
# Reload configuration
sudo systemctl reload nginx
```
### Database Connection Issues
```bash
# Test PostgreSQL connection (local)
psql -h localhost -U trader_user -d institutional_trader
# Test Supabase connection
psql "postgresql://postgres:password@db.project.supabase.co:5432/postgres"
```
### Firewall Issues
```bash
# Check firewall status
sudo ufw status
# Allow ports (if using UFW)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 22/tcp # SSH
sudo ufw enable
```
### SSL Certificate Issues
```bash
# Check certificate status
sudo certbot certificates
# Renew certificate manually
sudo certbot renew
# Check certificate expiration
sudo certbot certificates | grep Expiry
```
---
## Quick Reference Commands
```bash
# Start all services
sudo systemctl start institutional-trader-backend
sudo systemctl start institutional-trader-python
# Stop all services
sudo systemctl stop institutional-trader-backend
sudo systemctl stop institutional-trader-python
# Restart all services
sudo systemctl restart institutional-trader-backend
sudo systemctl restart institutional-trader-python
# Check status
sudo systemctl status institutional-trader-backend
sudo systemctl status institutional-trader-python
sudo systemctl status nginx
# View logs
sudo journalctl -u institutional-trader-* -f
# Test health
curl https://api.yourdomain.com/health
```
---
## Security Checklist
- [ ] All `.env` files have secure passwords
- [ ] `JWT_SECRET` is at least 32 characters and random
- [ ] SSL certificates installed and auto-renewal enabled
- [ ] Firewall configured (only necessary ports open)
- [ ] Database users have appropriate permissions (principle of least privilege)
- [ ] Regular backups configured
- [ ] System updates applied regularly
- [ ] SSH key authentication enabled (disable password auth)
- [ ] Rate limiting enabled in backend
- [ ] CORS origin set to production domain only
---
## Next Steps
1. ✅ Deploy application to Proxmox
2. ✅ Configure database access for friend
3. ✅ Set up monitoring/alerting (optional: UptimeRobot, Sentry)
4. ✅ Configure automated backups
5. ✅ Set up CI/CD pipeline (optional)
Your application should now be running on your Proxmox server! 🎉
Reference in New Issue View Git Blame Copy Permalink