On May 28, I analyzed one of my websites' traffic over 20 minutes to figure out whether my current 2-core setup was holding up β or holding me back. This is what I found.
From access_20min.log.txt
, I counted:
~13,400 total requests
Many from bots: Googlebot, Bingbot, IAS
Many hits on SSR-heavy pages: /swimmer/
, /rankings/
, /records/
Thatβs roughly:
670 requests per minute
11 requests per second
Under ideal conditions:
~20β50 requests/sec (API or SSR)
Assumes good caching, minimal disk I/O, responsive DB
But SwimStandards also:
Uses Next.js SSR + Feathers.js API
Runs MongoDB aggregation per request
Handles bot load 24/7
Shows socket hang up errors and TIME_WAIT spikes
You're operating at the edge of stability for a 2-core server.
Red flags:
socket hang up
during peak periods
TIME_WAIT
sometimes exceeds 300
MongoDB slow logs show query latency
High I/O due to bot hits on uncached pages
You want margin for real traffic surges
You want better SSR/API responsiveness
You want fewer 502/socket errors without blocking bots
Move MongoDB to its own box (if not already)
Use pm2 scale feathers-dna 2
to add more backend workers
Enable Redis or file-based caching
Use Cloudflare or NGINX with caching to protect SSR routes
sudo awk -v d="$(date -d '20 minutes ago' '+%d/%b/%Y:%H:%M')" '$0 ~ d,0' /var/log/apache2/other_vhosts_access.log | awk '{print $2}' | sort | uniq -c | sort -nr | head -20
wc -l access_20min.log.txt
sudo netstat -anp | grep :443 | awk '{print $6}' | sort | uniq -c
sudo awk '{print $2}' /var/log/apache2/other_vhosts_access.log | sort | uniq -c | sort -nr | head -20
watch -n 1 "netstat -anp | grep :5052 | wc -l"
sudo netstat -anp | grep :443 | awk '{print $6}' | sort | uniq -c
free -m
uptime
sudo grep -i "slow query" /var/log/mongodb/mongod.log | jq -c 'select(.attr.durationMillis > 800)' > slow.log
If your site handles real-time rankings and dynamic pages like mine, a 2-core box can survive β but it wonβt thrive. The more traffic (bot or not), the harder it gets to avoid dropped connections, long delays, and random SSR failures.
For me, this test showed itβs time to move up β or keep scraping by with every tweak possible.