Q: What is the maximum number of tokens that Triplo can process in a single submission?
I didn't find an answer to that question either in the Question section here or on your website. Though the 2M monthly token limit per device is clear, I want to know how much of that one can use in a single prompt.
Felipe_TriploAI
Nov 18, 2024A: Good question!
Short answer: it can process prompts as long as those accepted by the model you chose.
Long answer: Triplo AI does not interfere, limit, change, log any of your prompts. You'll be able to prompt as cleanly as possible the model you selected. The maximum length accepted by the model is one of it's features/limitations (not Triplo AI).
Maximum prompt size is usually described along the model name. GPT 4o-mini 128k means a prompt+results of a maximum of 128.000 tokens. Google Palm 1Mi, a maximum of prompt+results of 1 Million tokens.
To be fair, if you're using very large prompts, even considering context, you're probably doing something wrong and models will perform badly... Although they publicize these large "context" windows, the truth is that they don't correctly process long prompts and tends to lack performance on them.
Out of my curiosity, why do you need, what's the use case for large prompts?
Take care
Felipe
111039994008754896576Thanks for your response. To answer your question, I'm looking to transcribe and summarize longer YouTube videos, hence the token inquiry.
I struggled processing very long videos transcriptions... 2h or so. None of the models were able to process them in a single pass. I had to scrape and do it in steps. For the regular ones, piece of cake.
Take care
Felipe