Q: Index a complete website

I wan't the AI Chatbot have complete knowledge of the website it is based on.
I have added the domain (without /).
It seems hat i have to add each subpage by manually. Because that is a site with 300+ subpages, that isn't very comfortable.

Is there an automatic option to crawl all subpages?
As best solution also with a listing, what is indexed at which date and an option to remove specific sites if needed.

torsten3dSep 29, 2025
Share
Helpful?
Log in to join the conversation
Related questions
View product details