Good for generic helper bot, but...
Let me start by saying I purchased Tier 3 which is the highest tier available at AppSumo
TLDR; this product has many flaws and is not ready for production. There are better customer support AI available on AppSumo that performs better than Fini.
The bad:
1. There is no way to prevent the bot from answering questions irrelevant to the documentation provided. To me this is the deal breaker because people can exploit and use the bot as generic GPT bot. Apparently, in order to do this we should purchase the custom tier (starts at $299/m) to get access to the inhouse built models.
2. They advertised that Tier 2 & 3 users are using GPT-4, but my Tier 3 account is apparently on GPT-3 (I asked the bot what LLM they are using and they replied with GPT-3)
3. There are a lot of bugs that needs to be ironed out, for example when we're using AskFini section to try out the bot, sometimes the instruction prompt are leaking when switching from one bot to another. I also cannot make the chrome extension work (setup the API and instruction, but no chat section is opening up after that)
The good: Can still be used as generic GPT bot with static instruction prompt.
Deepak_FiniAI
May 9, 2024Vincent-- thanks for your feedback.
1. For preventing answers irrelevant to the documentation - our standard models work with 95% of the cases. For 5% we have to deploy inhouse models. We are working hard to bring it to the appsumo tier in the next few weeks.
2. As you would read on Appsumo we use a multi-LLM layer to get responses to maintain the highest accuracy. The evaluation layer is the last layer where the answers go through and there we use a standard GPT3.5 model. The major key to get the best quality answers is the answer layer where we use GPT-4 for Tier2/ 3.
3. Not sure what you mean by instruction prompt leaking?