Q: How Large Can A Humata Knowledgebase Be?
For example, if I purchase Tier 4 and upload 18,000 pages monthly, that adds up to a LOT of individual documents accumulating over time. At any point would the AI start to become limited as to how "deep" it searches into the database when responding to user queries?
Also, what is the largest an individual document can be? I have one that has over 4milion characters. Would this need to be broken up into smaller parts before uploading?
Cyrus_Humata
Dec 20, 2024A: For a single document, it can be 100MB or ~2,500 pages long. If it goes beyond that, what I recommend for your 4M character file is to create a dedicated folder for that and break it into smaller modular files, then you can press Ask All inside of that folder to query everything collectively. My general impression is that the overall LLM/AI architecture will improve and since we pioneered and productionized RAG the capabilities of search will only progress, perhaps significantly overtime.
DougLietzPLUSHello Cyrus... could you also please answer the first part of my question. If I have a knowledgebase containing hundreds of thousands of documents (mostly video transcripts in my case), will AI be able to analyze all of them at once, so that the user gets the best possible, most accurate "grounded" answer to their query?
Yes, we built Humata to be able to find the best relevant answer across very large knowledge bases. If you set it to "Grounded" (Grounded mode strictly cites your documentation for precise answers.) mode in Settings > Chat > Grounded, then you should be able to accomplish that.