453 lines
10 KiB
Markdown
453 lines
10 KiB
Markdown
# 🏗️ LuckyChit Production Architecture
|
|
|
|
## Current Setup
|
|
|
|
Your application runs across **2 LXC containers** with Cloudflare in front:
|
|
|
|
```
|
|
Internet
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────┐
|
|
│ Cloudflare CDN │
|
|
│ (SSL/TLS, DDoS Protection, CDN) │
|
|
│ Domain: chitfund.deepteklabs.com │
|
|
└──────────────┬──────────────────────┘
|
|
│ HTTPS/HTTP
|
|
▼
|
|
┌─────────────────────────────────────┐
|
|
│ LXC 1: Nginx Proxy │
|
|
│ - Reverse Proxy │
|
|
│ - Ports: 80, 443 │
|
|
│ - Can cache responses ⚠️ │
|
|
└──────────────┬──────────────────────┘
|
|
│ HTTP (internal)
|
|
▼
|
|
┌─────────────────────────────────────┐
|
|
│ LXC 2: Application Server │
|
|
│ IP: 192.168.8.148 │
|
|
│ ┌───────────────────────────────┐ │
|
|
│ │ PM2 Process Manager │ │
|
|
│ ├───────────────────────────────┤ │
|
|
│ │ Backend API │ │
|
|
│ │ - Port: 3000 │ │
|
|
│ │ - Node.js + Express │ │
|
|
│ │ - Process: luckychit-api │ │
|
|
│ ├───────────────────────────────┤ │
|
|
│ │ Frontend │ │
|
|
│ │ - Port: 8080 │ │
|
|
│ │ - Flutter Web (Static) │ │
|
|
│ │ - Process: luckychit-frontend │ │
|
|
│ └───────────────────────────────┘ │
|
|
│ ┌───────────────────────────────┐ │
|
|
│ │ PostgreSQL Database │ │
|
|
│ │ - Port: 5432 │ │
|
|
│ │ - Database: luckychit │ │
|
|
│ └───────────────────────────────┘ │
|
|
└─────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Data Flow
|
|
|
|
### User Request Flow
|
|
|
|
```
|
|
Browser
|
|
→ Cloudflare (SSL/TLS, CDN)
|
|
→ Nginx Proxy (LXC 1)
|
|
→ PM2 Frontend (LXC 2:8080)
|
|
→ Browser renders Flutter app
|
|
```
|
|
|
|
### API Request Flow
|
|
|
|
```
|
|
Browser
|
|
→ Cloudflare
|
|
→ Nginx Proxy (LXC 1)
|
|
→ PM2 Backend API (LXC 2:3000)
|
|
→ PostgreSQL Database
|
|
← Response
|
|
← JSON Response
|
|
← Proxied Response
|
|
← CDN Cached/Direct
|
|
← Response
|
|
```
|
|
|
|
---
|
|
|
|
## Cache Layers (Your Problem!) 🎯
|
|
|
|
### 1. Browser Cache
|
|
- **Location**: User's browser
|
|
- **Status**: ✅ Fixed (added cache-control meta tags)
|
|
- **Control**: `index.html` meta tags
|
|
|
|
### 2. Cloudflare Cache
|
|
- **Location**: Cloudflare edge servers
|
|
- **Status**: ⚠️ Can cache
|
|
- **Control**: Cloudflare dashboard
|
|
- **Fix**: Purge cache in dashboard
|
|
|
|
### 3. Nginx Proxy Cache ⚠️ **LIKELY CULPRIT!**
|
|
- **Location**: LXC 1 (nginx proxy container)
|
|
- **Status**: ❌ Probably caching!
|
|
- **Control**: `/etc/nginx/sites-available/chitfund`
|
|
- **Fix**: See `NGINX_PROXY_CACHE_FIX.md`
|
|
|
|
### 4. PM2 Static Server
|
|
- **Location**: LXC 2 (backend container)
|
|
- **Status**: ✅ Serves fresh files from `build/web`
|
|
- **Control**: Rebuild with `flutter build web`
|
|
|
|
---
|
|
|
|
## Port Mapping
|
|
|
|
| Service | LXC | Port | Accessible From | Purpose |
|
|
|---------|-----|------|-----------------|---------|
|
|
| **Nginx** | LXC 1 | 80 | Internet (via Cloudflare) | HTTP Proxy |
|
|
| **Nginx** | LXC 1 | 443 | Internet (via Cloudflare) | HTTPS Proxy |
|
|
| **Backend API** | LXC 2 | 3000 | LXC 1 (nginx) | REST API |
|
|
| **Frontend** | LXC 2 | 8080 | LXC 1 (nginx) | Flutter Web App |
|
|
| **PostgreSQL** | LXC 2 | 5432 | Localhost only | Database |
|
|
|
|
---
|
|
|
|
## Network Configuration
|
|
|
|
### LXC 1 (Nginx Proxy)
|
|
```bash
|
|
# Public facing
|
|
Public IP: <your-public-ip>
|
|
Ports: 80, 443 (open to internet)
|
|
|
|
# Internal
|
|
Internal IP: <nginx-lxc-ip>
|
|
Can reach: LXC 2 (192.168.8.148)
|
|
```
|
|
|
|
### LXC 2 (Application)
|
|
```bash
|
|
# Internal only
|
|
IP: 192.168.8.148
|
|
Ports: 3000, 8080 (open to LXC 1)
|
|
Not directly accessible from internet
|
|
```
|
|
|
|
---
|
|
|
|
## Nginx Proxy Configuration
|
|
|
|
Your nginx config on **LXC 1** should look like:
|
|
|
|
```nginx
|
|
# /etc/nginx/sites-available/chitfund
|
|
|
|
server {
|
|
listen 80;
|
|
server_name chitfund.deepteklabs.com;
|
|
|
|
# Frontend (Flutter)
|
|
location / {
|
|
proxy_pass http://192.168.8.148:8080;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection 'upgrade';
|
|
proxy_set_header Host $host;
|
|
|
|
# IMPORTANT: Disable caching!
|
|
proxy_no_cache 1;
|
|
proxy_cache_bypass 1;
|
|
add_header Cache-Control "no-cache, no-store, must-revalidate";
|
|
}
|
|
|
|
# Backend API
|
|
location /api {
|
|
proxy_pass http://192.168.8.148:3000;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Host $host;
|
|
proxy_no_cache 1;
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Deployment Flow
|
|
|
|
### When You Deploy Code:
|
|
|
|
```bash
|
|
# 1. Local Machine
|
|
git add .
|
|
git commit -m "Update"
|
|
git push origin prodnew
|
|
|
|
# 2. LXC 2 (Application Server)
|
|
ssh luckychit@192.168.8.148
|
|
cd /home/luckychit/apps/chitfund
|
|
./deploy-frontend-only.sh
|
|
# This rebuilds Flutter and restarts PM2
|
|
|
|
# 3. LXC 1 (Nginx Proxy)
|
|
ssh root@<nginx-lxc-ip>
|
|
./nginx-clear-cache.sh
|
|
# This clears nginx cache
|
|
|
|
# 4. Cloudflare
|
|
# Go to dashboard and purge cache (if needed)
|
|
|
|
# 5. Browser
|
|
# Hard refresh: Ctrl + Shift + R
|
|
```
|
|
|
|
---
|
|
|
|
## Cache Clearing Procedure
|
|
|
|
### Full Cache Clear (Use After Deployment):
|
|
|
|
```bash
|
|
# 1. Clear nginx cache (LXC 1)
|
|
ssh root@<nginx-lxc-ip>
|
|
sudo rm -rf /var/cache/nginx/*
|
|
sudo systemctl reload nginx
|
|
|
|
# 2. Clear Cloudflare cache
|
|
# Login to Cloudflare dashboard
|
|
# Caching → Purge Everything
|
|
|
|
# 3. Hard refresh browser
|
|
# Ctrl + Shift + R (Windows)
|
|
# Cmd + Shift + R (Mac)
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting Guide
|
|
|
|
### Problem: Changes Not Showing
|
|
|
|
**Check each cache layer:**
|
|
|
|
1. **Browser**: Hard refresh (`Ctrl + Shift + R`)
|
|
2. **Cloudflare**: Purge cache in dashboard
|
|
3. **Nginx**: Clear cache on LXC 1
|
|
4. **PM2**: Verify build is fresh on LXC 2
|
|
|
|
### Problem: 502 Bad Gateway
|
|
|
|
**Check connectivity:**
|
|
|
|
```bash
|
|
# On LXC 1 (nginx), test backend:
|
|
curl http://192.168.8.148:3000/health
|
|
curl http://192.168.8.148:8080
|
|
|
|
# If these fail:
|
|
# - PM2 is down on LXC 2
|
|
# - Firewall blocking between LXCs
|
|
# - Network routing issue
|
|
```
|
|
|
|
### Problem: Backend Not Responding
|
|
|
|
**Check PM2 on LXC 2:**
|
|
|
|
```bash
|
|
ssh luckychit@192.168.8.148
|
|
pm2 status
|
|
pm2 restart all
|
|
```
|
|
|
|
---
|
|
|
|
## Security Considerations
|
|
|
|
### Current Setup:
|
|
- ✅ Cloudflare provides DDoS protection
|
|
- ✅ SSL/TLS handled by Cloudflare
|
|
- ✅ Backend LXC not directly exposed to internet
|
|
- ✅ Only nginx proxy is public-facing
|
|
- ⚠️ No SSL between nginx and backend (OK for internal network)
|
|
|
|
### Recommendations:
|
|
- ✅ Keep backend LXC internal-only
|
|
- ✅ Use firewall rules between LXCs
|
|
- ✅ Limit nginx proxy access to only necessary ports
|
|
- ⚠️ Consider fail2ban on nginx proxy
|
|
- ⚠️ Consider firewall rate limiting
|
|
|
|
---
|
|
|
|
## Firewall Configuration
|
|
|
|
### LXC 1 (Nginx Proxy)
|
|
```bash
|
|
# Allow HTTP/HTTPS from internet
|
|
sudo ufw allow 80/tcp
|
|
sudo ufw allow 443/tcp
|
|
sudo ufw allow 22/tcp # SSH
|
|
sudo ufw enable
|
|
```
|
|
|
|
### LXC 2 (Application)
|
|
```bash
|
|
# Allow only from nginx proxy
|
|
sudo ufw allow from <nginx-lxc-ip> to any port 3000
|
|
sudo ufw allow from <nginx-lxc-ip> to any port 8080
|
|
sudo ufw allow 22/tcp # SSH
|
|
sudo ufw enable
|
|
```
|
|
|
|
---
|
|
|
|
## Monitoring & Logs
|
|
|
|
### LXC 1 (Nginx)
|
|
```bash
|
|
# Access logs
|
|
sudo tail -f /var/log/nginx/access.log
|
|
|
|
# Error logs
|
|
sudo tail -f /var/log/nginx/error.log
|
|
|
|
# Check status
|
|
sudo systemctl status nginx
|
|
```
|
|
|
|
### LXC 2 (Application)
|
|
```bash
|
|
# PM2 logs
|
|
pm2 logs
|
|
|
|
# PM2 status
|
|
pm2 status
|
|
|
|
# PM2 monitoring
|
|
pm2 monit
|
|
```
|
|
|
|
---
|
|
|
|
## Backup Strategy
|
|
|
|
### What to Backup:
|
|
|
|
1. **Database (LXC 2)**
|
|
```bash
|
|
pg_dump -U luckychit luckychit > backup.sql
|
|
```
|
|
|
|
2. **Nginx Config (LXC 1)**
|
|
```bash
|
|
tar -czf nginx-config.tar.gz /etc/nginx/
|
|
```
|
|
|
|
3. **Application Code (LXC 2)**
|
|
```bash
|
|
cd /home/luckychit/apps/chitfund
|
|
git status # Should be clean
|
|
# Code is in git, no backup needed
|
|
```
|
|
|
|
4. **PM2 Config (LXC 2)**
|
|
```bash
|
|
pm2 save
|
|
# Saves to ~/.pm2/dump.pm2
|
|
```
|
|
|
|
---
|
|
|
|
## Performance Optimization
|
|
|
|
### Current Performance:
|
|
|
|
```
|
|
User → Cloudflare (CDN, <50ms globally)
|
|
→ Nginx Proxy (LXC 1, ~1ms)
|
|
→ PM2 App (LXC 2, ~10ms)
|
|
→ PostgreSQL (~5ms)
|
|
```
|
|
|
|
**Total latency**: ~16ms + network time
|
|
|
|
### Recommended Optimizations:
|
|
|
|
1. **Enable nginx caching for static assets only**
|
|
- Cache images, CSS, JS (7 days)
|
|
- Don't cache HTML, JSON
|
|
|
|
2. **Use Cloudflare caching selectively**
|
|
- Cache static assets
|
|
- Bypass for API and HTML
|
|
|
|
3. **Enable gzip compression in nginx**
|
|
```nginx
|
|
gzip on;
|
|
gzip_types text/plain text/css application/json application/javascript;
|
|
```
|
|
|
|
4. **Use PM2 cluster mode** (already fixed in ecosystem.config.js)
|
|
|
|
---
|
|
|
|
## Quick Reference
|
|
|
|
### SSH Access:
|
|
```bash
|
|
# Nginx Proxy (LXC 1)
|
|
ssh root@<nginx-lxc-ip>
|
|
|
|
# Application Server (LXC 2)
|
|
ssh luckychit@192.168.8.148
|
|
```
|
|
|
|
### Clear All Caches:
|
|
```bash
|
|
# 1. Nginx (LXC 1)
|
|
sudo rm -rf /var/cache/nginx/* && sudo systemctl reload nginx
|
|
|
|
# 2. PM2 rebuild (LXC 2)
|
|
cd /home/luckychit/apps/chitfund && ./force-cache-bust.sh
|
|
|
|
# 3. Cloudflare
|
|
# Dashboard → Caching → Purge Everything
|
|
|
|
# 4. Browser
|
|
# Ctrl + Shift + R
|
|
```
|
|
|
|
### Check Everything:
|
|
```bash
|
|
# Nginx (LXC 1)
|
|
systemctl status nginx
|
|
curl http://192.168.8.148:3000/health
|
|
|
|
# PM2 (LXC 2)
|
|
pm2 status
|
|
pm2 logs --lines 20
|
|
```
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
**Your Issue**: Changes not showing because of **nginx proxy caching**
|
|
|
|
**Solution**:
|
|
1. Clear nginx cache on LXC 1
|
|
2. Add `proxy_no_cache` to nginx config
|
|
3. Clear Cloudflare cache
|
|
4. Hard refresh browser
|
|
|
|
**See**: `NGINX_PROXY_CACHE_FIX.md` for detailed nginx fix
|
|
|
|
---
|
|
|
|
**Architecture documented! Your caching issue is in the nginx proxy layer.** 🎯
|
|
|