This is an interesting project. I am writing a tiny little script to scrape my developer personal website.
The website serves static files and is hosted on Vercel.
What are some things to think about?
- Vercel has a hard cap of 100 builds on the free plan.
- So what does that mean for scraping
- Could you use warm IPs?
- What about using warm IPs behind a proxy, like SOCKS5?
- How would you avoid detection with a proxy?
- Cloud security is pretty good, especially with a big VC backed company like Vercel. This would be a hard task to accomplish for someone unexperienced in this stuff but I feel confident that asking more experienced pen testers, etc, would result in at the least, a brainstorming session.