Q: Crawl Pages (url) automatically?

Is there a way for your app to Crawl Pages (url) automatically for a whole site? That way we don't have to add each page one one by one?

roadkillciscoPLUSJan 22, 2025
Founder Team
tinytalk

tinytalk

Jan 22, 2025

A: Hello,

Thank you for reaching out.
Yes, there is a toggle called "Crawl Website" next to the input where you add your website url. This will crawl all the linking pages as it finds. Alternatively you can also provide a sitemap.xml if you have one already, for instance https://example.com/sitemap.xml

If you've tried this and it didn't work as you expected, please reach out to us at support@tinytalk.ai and we'd be happy to look into it together.

All the best,
Oscar

Share
Helpful?
Log in to join the conversation