SumoDudeQ: Hi, haven't tested this out yet, but couple main things.
Will the website we're scraping know that we're scraping it? (No it wouldn't be, for example facebook or twitter, just smaller more regular sites)
Lets say I want to collect products from a car dealership website. I dont need all 9,736 car listings. I just want the 100 red Chevy cars. Can it narrow down the search & only scrape specific categories?
Also let's say I scrape out all the red chevy car listings. What can I do with that data after? Can I shoot it over into my own website to populate my own car catalog page? (I'm new to this, guessing that can be done via webhooks or API or something?)
Can it scrape google maps, or yelp, manta business listings, etc, or are any of those considered "Premium"?
Wondering how credits work too. Would one credit = one car listing in this case?
Thanks
Ardy_BrowseAI
May 15, 2024A: Hi there,
- We open sites and interact with them like a regular user would. We also use rotating IP addresses and we do not run too many automations simultaneously on the same site to avoid putting too much pressure on their server. Typically, sites can only detect scraping when it's done as a logged in user and its speed or volume or IP change is suspicious.
- If the site offers a search functionality, the robot can be trained to use the search and narrow down the results before capturing them. It can do anything you can do on that site.
- The data can be passed on to other tools you use. We have native integrations with Zapier, Airtable, Google Sheets, Pabbly Connect, Make.com (soon), and more. We also have API and webhooks that you can use for custom integrations.
- It's designed to be able to scrape any website. More info here:
https://help.browse.ai/supported-sites
- That is right, each credit would get you one row of data, which could be one car listing in this case.
I don’t understand about the 1 credit one record either.
I have an event website with speakers. Name, company, position.
Is extracting the name one credit, the company another credit and the position a third credit?
Oh no! Each credit lets you extract a *row*, not a *cell*. So all of those (assuming they are in a single row of data) can be captured with a single credit. The exception would be extracting from Premium sites.
You can learn more about this here:
https://help.browse.ai/how-are-credits-calculated