Q: Hello, will this software scrape an entire state?
If I put the state as my location? Or at least run the top 100-200 cities in that state?
Grant_LocalScraper
May 15, 2024A: Hello,
We are limited by the number of results that the website gives. Google Maps for example, you can't ask it for an entire state and get thousands of results. You need to work within the data limits that each site gives you and increase the number of searches you do. Let's take Google Places as an example, it has 300 results per search. If you believe that each city probably has less than 300 results, then you need to do your 100-200 searches and get 300 results (max) from each city. If you feel there should be more than 300 then you need to find ways to divide the city into smaller search areas. Like the 8 boroughs of new york or by zip code as the smallest area.
If you want the most data in one go then I recommend Yellow Pages USA, but scraping them will require some sort of proxies as outlined on the ReadMe page for the program.