Q: OpenAI Batch API and Cached Message
Hi, been playing around with the tool and it has been great so far!
I'm wondering if it would be possible to add support for Batch API from OpenAI. It is a method for sending a group of requests to be processed and returned within a 24 hour timeframe in exchange for a 50% discount. I see this potentially being useful for those who want to use a more expensive and more capable model, are okay with waiting for a response, and would like to pay significantly less in API charges. API reference here: https://platform.openai.com/docs/guides/batch/overview
Second question: is there any way to turn off the cached message when you try to refresh a formula that has been set to false for refresh?
Wei_Sheng
Oct 30, 2024A: Hi thanks for your suggestions! I just took a look at the OpenAI Batch API, however, it doesn't seem possible that this can be done in Google Sheets because Google Sheets formulas only have a process time of up to 6 minutes, so it can't wait for the batch process of OpenAI to complete. However, we will look into this and see if there is a better way to achieve it.
For your second question, we will add an option to turn off the cache message. For now, you can leave a column for the messages, then choose to hide the column.
Wei