Q: 1) When scrapping the content of the webpage, does it get everything or just the main contents - I know most ...
scrappers would just get everything including useless sidebars, menu nav, etc.
2) Are you able to do a google search and get top 10 results and scrape?
Yaro_DocGPT_AI
May 15, 2024A: Hi!
1) About scraping
- There is functions for getting specific tags like h1,h2, title, description. (like WEB_H1....)
- Also there is function for getting whole content like: WEB_URL
2) About google search. i have planned this function soon, but it will use official API from Google that is paid (also needed to provide api key), their cost about 0.01$ per request.
I dont like use "black" workarounds for scraping google, they are very unstable.
Let me know if you are interested in this func, i will prioritize it in backlog
There are following functions for scraping:
WEB SCRAPPING:
=WEB_URL("https://docgpt.ai")
=WEB_TITLE("https://docgpt.ai")
=WEB_DESCRIPTION("https://docgpt.ai")
=WEB_H1("https://docgpt.ai")
=WEB_H2("https://docgpt.ai/")
=WEB_HEADINGS("https://docgpt.ai")
=WEB_PARAGRAPHS("https://docgpt.ai")
=WEB_URLS_DATA(A37:A39)