Melvin_PulpMiner
Jun 13, 2025A: Hey! Just to clarify—are you asking if individual websites might block access when using PulpMiner to extract data?
Not sure if that's the question being asked but I'm interested in the answer to that question. A lot of websites do block automated browsing and scrapers.
Yes. If your product is blocked on a website I want to use, I won't be able to use it.
Currently to avoid this blocking issues we are using rotating proxies in the scrapers we implemented.
Ok great, might you have the option in the future to use our own proxies as well? If lots of people are using the same scrapers and proxies then it might get blocked quicker. If I was putting something to use for something important I'd want to use my own proxies to keep control of that.
Hi, richard884.
Currently the PulpMiner hasn't implemented the Bring you own Proxy feature since we are handling it for you. But it is in the vision but hasn't taken up for any feature development work.