institutional-trader/backend/DATABASE_QUOTA_FIX.md

141 lines
3.9 KiB
Markdown

# Fix: Database Size Quota Exceeded
## Problem
Your Supabase project has been restricted due to exceeding the database size quota:
```
Service for this project is restricted due to the following violations: exceed_db_size_quota
```
## Immediate Solutions
### Option 1: Contact Supabase Support (Recommended)
1. **Go to Supabase Support**: https://supabase.help
2. **Explain your situation** - You need access restored
3. **Request options**:
- Temporary quota increase while you clean up
- Upgrade to a paid plan
- Guidance on reducing database size
### Option 2: Upgrade Your Plan
1. Go to Supabase Dashboard → Settings → Billing
2. Upgrade to a paid plan with higher storage limits
3. This will immediately restore access
### Option 3: Reduce Database Size
If you have access to the dashboard, you can:
1. **Check current database size:**
- Go to Database → Database Size
- See which tables are using the most space
2. **Clean up old data:**
```sql
-- Example: Delete old options flow data (older than 90 days)
DELETE FROM "OptionsFlow_monthly"
WHERE "CreatedDate" < NOW() - INTERVAL '90 days';
-- Example: Delete old price data (keep only last 30 days)
DELETE FROM prices_intraday_1m
WHERE ts < NOW() - INTERVAL '30 days';
```
3. **Archive instead of delete:**
- Move old data to archive tables
- Or export to external storage (S3, etc.)
4. **Optimize tables:**
```sql
-- Vacuum to reclaim space
VACUUM FULL;
-- Reindex tables
REINDEX DATABASE postgres;
```
5. **Check for large indexes:**
```sql
-- Find large indexes
SELECT
schemaname,
tablename,
indexname,
pg_size_pretty(pg_relation_size(indexrelid)) AS index_size
FROM pg_stat_user_indexes
ORDER BY pg_relation_size(indexrelid) DESC
LIMIT 20;
```
## Temporary Workaround: Use Supabase Client
While waiting for quota resolution, you can try using the Supabase client with API keys (if the API is still accessible):
1. **Get your API keys:**
- Go to Supabase Dashboard Settings API
- Copy:
- **Project URL**: `https://mycttyevrycmkhlvwgfk.supabase.co`
- **service_role key**
2. **Update `backend/.env`:**
```env
# Remove DATABASE_URL (direct connection won't work)
# DATABASE_URL=...
# Use Supabase client
SUPABASE_URL=https://mycttyevrycmkhlvwgfk.supabase.co
SUPABASE_SERVICE_KEY=your_service_role_key_here
PORT=3010
NODE_ENV=development
CORS_ORIGIN=http://localhost:5173
```
**Note:** This may still not work if the entire project is restricted, but it's worth trying.
## Prevention: Monitor Database Size
Once access is restored, set up monitoring:
1. **Create a monitoring query:**
```sql
SELECT
pg_database.datname,
pg_size_pretty(pg_database_size(pg_database.datname)) AS size
FROM pg_database
ORDER BY pg_database_size(pg_database.datname) DESC;
```
2. **Set up alerts** in Supabase Dashboard for database size
3. **Implement data retention policies:**
- Automatically archive/delete old data
- Keep only necessary historical data
- Use partitioning for large tables
## Recommended Data Retention Strategy
For a trading system, consider:
- **Options Flow**: Keep last 90 days active, archive older
- **Price Data (1m)**: Keep last 30 days, aggregate to 5m/15m for older
- **Daily Data**: Keep indefinitely (small size)
- **Alerts**: Keep last 90 days, archive older
## Next Steps
1. **Immediate**: Contact Supabase support at https://supabase.help
2. **Short-term**: Clean up old data if you can access the database
3. **Long-term**: Upgrade plan or implement data retention policies
## Supabase Plan Limits
Check your current plan limits:
- **Free Plan**: 500 MB database size
- **Pro Plan**: 8 GB database size
- **Team Plan**: 8 GB database size (with more features)
Consider upgrading if you need more storage for historical trading data.