Q: Crawl Pages (url) automatically for a whole site?
Is there a way for your app to Crawl Pages (url) automatically for a whole site? That way we don't have to add each page one one by one?
Schiocco
Edited Jan 22, 2025A: Yes! You can do it with one click. Automatic scheduled re-training is also possible via cron jobs
roadkillciscoPLUSOk I got it to crawl entire site by just putting in the main url and then clicking the "Extract URLs" but how do I set it up to "Automatic scheduled re-training"?
duarne3
Verified purchaser
Is there a limit to the number of pages it can crawl on a website?
If you need assistance, please feel free to contact us via chat at https://board.support/docs . You can login using your Support Board Cloud account. Thank you! Best regards.
Yes, but it is not related to the number of pages. The limit checks the number of characters in the text to train the chatbot.