10 KiB
10 KiB
🏗️ LuckyChit Production Architecture
Current Setup
Your application runs across 2 LXC containers with Cloudflare in front:
Internet
│
▼
┌─────────────────────────────────────┐
│ Cloudflare CDN │
│ (SSL/TLS, DDoS Protection, CDN) │
│ Domain: chitfund.deepteklabs.com │
└──────────────┬──────────────────────┘
│ HTTPS/HTTP
▼
┌─────────────────────────────────────┐
│ LXC 1: Nginx Proxy │
│ - Reverse Proxy │
│ - Ports: 80, 443 │
│ - Can cache responses ⚠️ │
└──────────────┬──────────────────────┘
│ HTTP (internal)
▼
┌─────────────────────────────────────┐
│ LXC 2: Application Server │
│ IP: 192.168.8.148 │
│ ┌───────────────────────────────┐ │
│ │ PM2 Process Manager │ │
│ ├───────────────────────────────┤ │
│ │ Backend API │ │
│ │ - Port: 3000 │ │
│ │ - Node.js + Express │ │
│ │ - Process: luckychit-api │ │
│ ├───────────────────────────────┤ │
│ │ Frontend │ │
│ │ - Port: 8080 │ │
│ │ - Flutter Web (Static) │ │
│ │ - Process: luckychit-frontend │ │
│ └───────────────────────────────┘ │
│ ┌───────────────────────────────┐ │
│ │ PostgreSQL Database │ │
│ │ - Port: 5432 │ │
│ │ - Database: luckychit │ │
│ └───────────────────────────────┘ │
└─────────────────────────────────────┘
Data Flow
User Request Flow
Browser
→ Cloudflare (SSL/TLS, CDN)
→ Nginx Proxy (LXC 1)
→ PM2 Frontend (LXC 2:8080)
→ Browser renders Flutter app
API Request Flow
Browser
→ Cloudflare
→ Nginx Proxy (LXC 1)
→ PM2 Backend API (LXC 2:3000)
→ PostgreSQL Database
← Response
← JSON Response
← Proxied Response
← CDN Cached/Direct
← Response
Cache Layers (Your Problem!) 🎯
1. Browser Cache
- Location: User's browser
- Status: ✅ Fixed (added cache-control meta tags)
- Control:
index.htmlmeta tags
2. Cloudflare Cache
- Location: Cloudflare edge servers
- Status: ⚠️ Can cache
- Control: Cloudflare dashboard
- Fix: Purge cache in dashboard
3. Nginx Proxy Cache ⚠️ LIKELY CULPRIT!
- Location: LXC 1 (nginx proxy container)
- Status: ❌ Probably caching!
- Control:
/etc/nginx/sites-available/chitfund - Fix: See
NGINX_PROXY_CACHE_FIX.md
4. PM2 Static Server
- Location: LXC 2 (backend container)
- Status: ✅ Serves fresh files from
build/web - Control: Rebuild with
flutter build web
Port Mapping
| Service | LXC | Port | Accessible From | Purpose |
|---|---|---|---|---|
| Nginx | LXC 1 | 80 | Internet (via Cloudflare) | HTTP Proxy |
| Nginx | LXC 1 | 443 | Internet (via Cloudflare) | HTTPS Proxy |
| Backend API | LXC 2 | 3000 | LXC 1 (nginx) | REST API |
| Frontend | LXC 2 | 8080 | LXC 1 (nginx) | Flutter Web App |
| PostgreSQL | LXC 2 | 5432 | Localhost only | Database |
Network Configuration
LXC 1 (Nginx Proxy)
# Public facing
Public IP: <your-public-ip>
Ports: 80, 443 (open to internet)
# Internal
Internal IP: <nginx-lxc-ip>
Can reach: LXC 2 (192.168.8.148)
LXC 2 (Application)
# Internal only
IP: 192.168.8.148
Ports: 3000, 8080 (open to LXC 1)
Not directly accessible from internet
Nginx Proxy Configuration
Your nginx config on LXC 1 should look like:
# /etc/nginx/sites-available/chitfund
server {
listen 80;
server_name chitfund.deepteklabs.com;
# Frontend (Flutter)
location / {
proxy_pass http://192.168.8.148:8080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
# IMPORTANT: Disable caching!
proxy_no_cache 1;
proxy_cache_bypass 1;
add_header Cache-Control "no-cache, no-store, must-revalidate";
}
# Backend API
location /api {
proxy_pass http://192.168.8.148:3000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_no_cache 1;
}
}
Deployment Flow
When You Deploy Code:
# 1. Local Machine
git add .
git commit -m "Update"
git push origin prodnew
# 2. LXC 2 (Application Server)
ssh luckychit@192.168.8.148
cd /home/luckychit/apps/chitfund
./deploy-frontend-only.sh
# This rebuilds Flutter and restarts PM2
# 3. LXC 1 (Nginx Proxy)
ssh root@<nginx-lxc-ip>
./nginx-clear-cache.sh
# This clears nginx cache
# 4. Cloudflare
# Go to dashboard and purge cache (if needed)
# 5. Browser
# Hard refresh: Ctrl + Shift + R
Cache Clearing Procedure
Full Cache Clear (Use After Deployment):
# 1. Clear nginx cache (LXC 1)
ssh root@<nginx-lxc-ip>
sudo rm -rf /var/cache/nginx/*
sudo systemctl reload nginx
# 2. Clear Cloudflare cache
# Login to Cloudflare dashboard
# Caching → Purge Everything
# 3. Hard refresh browser
# Ctrl + Shift + R (Windows)
# Cmd + Shift + R (Mac)
Troubleshooting Guide
Problem: Changes Not Showing
Check each cache layer:
- Browser: Hard refresh (
Ctrl + Shift + R) - Cloudflare: Purge cache in dashboard
- Nginx: Clear cache on LXC 1
- PM2: Verify build is fresh on LXC 2
Problem: 502 Bad Gateway
Check connectivity:
# On LXC 1 (nginx), test backend:
curl http://192.168.8.148:3000/health
curl http://192.168.8.148:8080
# If these fail:
# - PM2 is down on LXC 2
# - Firewall blocking between LXCs
# - Network routing issue
Problem: Backend Not Responding
Check PM2 on LXC 2:
ssh luckychit@192.168.8.148
pm2 status
pm2 restart all
Security Considerations
Current Setup:
- ✅ Cloudflare provides DDoS protection
- ✅ SSL/TLS handled by Cloudflare
- ✅ Backend LXC not directly exposed to internet
- ✅ Only nginx proxy is public-facing
- ⚠️ No SSL between nginx and backend (OK for internal network)
Recommendations:
- ✅ Keep backend LXC internal-only
- ✅ Use firewall rules between LXCs
- ✅ Limit nginx proxy access to only necessary ports
- ⚠️ Consider fail2ban on nginx proxy
- ⚠️ Consider firewall rate limiting
Firewall Configuration
LXC 1 (Nginx Proxy)
# Allow HTTP/HTTPS from internet
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 22/tcp # SSH
sudo ufw enable
LXC 2 (Application)
# Allow only from nginx proxy
sudo ufw allow from <nginx-lxc-ip> to any port 3000
sudo ufw allow from <nginx-lxc-ip> to any port 8080
sudo ufw allow 22/tcp # SSH
sudo ufw enable
Monitoring & Logs
LXC 1 (Nginx)
# Access logs
sudo tail -f /var/log/nginx/access.log
# Error logs
sudo tail -f /var/log/nginx/error.log
# Check status
sudo systemctl status nginx
LXC 2 (Application)
# PM2 logs
pm2 logs
# PM2 status
pm2 status
# PM2 monitoring
pm2 monit
Backup Strategy
What to Backup:
-
Database (LXC 2)
pg_dump -U luckychit luckychit > backup.sql -
Nginx Config (LXC 1)
tar -czf nginx-config.tar.gz /etc/nginx/ -
Application Code (LXC 2)
cd /home/luckychit/apps/chitfund git status # Should be clean # Code is in git, no backup needed -
PM2 Config (LXC 2)
pm2 save # Saves to ~/.pm2/dump.pm2
Performance Optimization
Current Performance:
User → Cloudflare (CDN, <50ms globally)
→ Nginx Proxy (LXC 1, ~1ms)
→ PM2 App (LXC 2, ~10ms)
→ PostgreSQL (~5ms)
Total latency: ~16ms + network time
Recommended Optimizations:
-
Enable nginx caching for static assets only
- Cache images, CSS, JS (7 days)
- Don't cache HTML, JSON
-
Use Cloudflare caching selectively
- Cache static assets
- Bypass for API and HTML
-
Enable gzip compression in nginx
gzip on; gzip_types text/plain text/css application/json application/javascript; -
Use PM2 cluster mode (already fixed in ecosystem.config.js)
Quick Reference
SSH Access:
# Nginx Proxy (LXC 1)
ssh root@<nginx-lxc-ip>
# Application Server (LXC 2)
ssh luckychit@192.168.8.148
Clear All Caches:
# 1. Nginx (LXC 1)
sudo rm -rf /var/cache/nginx/* && sudo systemctl reload nginx
# 2. PM2 rebuild (LXC 2)
cd /home/luckychit/apps/chitfund && ./force-cache-bust.sh
# 3. Cloudflare
# Dashboard → Caching → Purge Everything
# 4. Browser
# Ctrl + Shift + R
Check Everything:
# Nginx (LXC 1)
systemctl status nginx
curl http://192.168.8.148:3000/health
# PM2 (LXC 2)
pm2 status
pm2 logs --lines 20
Summary
Your Issue: Changes not showing because of nginx proxy caching
Solution:
- Clear nginx cache on LXC 1
- Add
proxy_no_cacheto nginx config - Clear Cloudflare cache
- Hard refresh browser
See: NGINX_PROXY_CACHE_FIX.md for detailed nginx fix
Architecture documented! Your caching issue is in the nginx proxy layer. 🎯