Q: Can I build website databases with your platform? ETL Capabilities
1. Can I compile a website database? - Create structured data within your application whilst adhering to a db schema (data type + constraints)? IE, Automation to loop through rows and gather multiple columns records (car models + URL ). Send API request per row to (web scraper) and map results to other columns in same row. How reliable are LLM structured outputs (hallucination %)?
2. Am I able to transform data (standardize records)?
3. Can I perform RAG with tables? IE, SQL like rag chains, where I request info that requires the LLM to perform an SQL query, and return answer based on results from SQL query.
4. What about using variables (in automations and tables) for LLM content creation? For example, 'Use the information from column 2 (format) to create a poem for column 1 (subject)'
5. Can I store scraped results your vector db?
Dung_Diaflow
Mar 23, 2025A: Hi JD-Jack,
Thank you for your detailed questions! Here’s how Diaflow can support your needs:
1, 2, 3: Building Website Databases, Transforming Data, and RAG
You can create a website database using a Diaflow Table (similar to Airtable) with an integrated AI flow. Simply use the Diaflow Table node in your flow to manage structured data. However, we currently don’t support looping through data rows. That said, this is a fantastic suggestion, and we’ll consider adding it in future sprints—stay tuned!
4: LLM Content Creation with Variables
Yes, this is absolutely possible! First, create a Diaflow Table with your data (e.g., subjects in Column 1). In your app flow, add a Diaflow Table node to pull data from Column 1. Then, insert an LLM model node to generate a poem based on that data, using Column 2 (format) as a guide. Format the output as needed, and finally, add another Diaflow Table node to save the generated poems back into your table.
5: Storing Data in Vector DB
You can store any scraped results in Diaflow Vector and retrieve them in flows as needed. This works great for training AI or referencing data across your projects.
Thank you for your thoughtful ideas—they really help us improve!
Best,
Diaflow team