I already use other, general-purpose AIs (like Voila and ChatGPT). But I still glad to come across PenPilot, since – on paper – it seemed like a highly specialized, customizable writing assistant with a huge potential. So, I purchased the top tier immediately.

But when I started playing around with it, it became painfully clear that it still has a long way to go before it can become a useful tool in our arsenal. Here are some of the most frustrating issues I've come across:

1) Personas & instructions get ignored repeatedly – I've created a writing persona, gave detailed instructions regarding the style & format of the output, and provided an example. Despite all this, the output created by the AI felt boring, average – soulless, even – and NOTHING like what the persona I created might suggest. Same thing happened when I asked the AI to rewrite an article it created: it disregarded ALL my instructions, and just cranked out another version. Oh and my instructions regarding the approximate word count? There was a 35% difference!

2) The AI itself is slow – I know AI tools don't always respond to requests immediately, but I had to wait 10+ minutes for a rewrite... I mean, c'mon, it can't be THIS slow!

3) "Unable to process requests" – this error message (or something similar) has popped up multiple times throughout the ~90 minute testing session I had with the tool. It's possible that this was just a temporary glitch and I just chose a bad time to test PenPilot. But even if that is, in fact, the case, it was still frustrating.

4) Knowledge base feels unintuitive – in theory, a knowledge base sounds amazing! And it is, IF the AI can take full use of it. Unfortunately, that was not my experience with PenPilot. For one, once you create a category in the knowledge base, you can't delete it (or at least I couldn't figure out how to delete it). This means that if you accidentally create a duplicate category, you'll always be left guessing which one is the "real" one. Secondly, it seems the AI only uses ~80% of the knowledge base for the content it creates. I uploaded a single (!!!) 8-page pdf into my knowledge base, and the AI still left out key information from the article (despite me specifically telling it to include that information when I prompted it to rewrite the originally generated version).

5) The AI doesn't adjust & adapt: it doesn't seem to adapt to your style in real time. Now, this might be wishful thinking, but I'd like the AI to learn from each and every conversation I have with it; from each and every edit I make in the generated content; from each and every interaction – it needs to learn & adapt, otherwise it will always be stuck on the same – average – level. Currently, it doesn't seem like the AI is learning from ANY interaction. It just does the same thing over and over again with marginal changes – despite specific instructions, a detailed persona, and lots of manual editing.

6) It might be good in English, but it's pretty bad in other (smaller) languages. AI capabilities will always be behind in smaller languages – I know that. But when creating an article with PenPilot in another language, I was met with several weird sentence structures, grammatical errors, and even a few typos (!!!). Now, I get the weird sentence structures... I might even be understanding towards a few grammatical errors every now-and-then... But typos?! Seriously?

7) I don't get the SEO settings... It might just be me, but the current information under the SEO tab is too little to inform changes that need to be made to the article. Additionally, I though that once I gave it a primary keyword, the AI would automatically suggest modifications to achieve a better SEO-value. Clearly, I was wrong.

8) When I click "Rewrite content" (readability; plagarism; proofreading), it says the maximum character count for a prompt is 10,000 characters. Now, either don't offer the option to rewrite my article; of if you do, then make sure it works! Because now, I clicked on it only to discover that – apparently – I can't use this feature for long-form articles.

9) The tool has too much "noise". Look, not everyone will agree with me, but I was looking for a specialized writing assistant to help me craft better content, faster. Nothing else. I don't need image generation built in – I have other tools for that. I don't need a to-do list – I have other, specialized tools for that too. And if it's not going to help the AI create higher ranking content, then I don't need an SEO-checker either – again, I have far more capable, purpose-built tools for that too. I'd rather have PenPilot do one thing – and do it very, VERY well – than do multiple things at a mediocre level.

To summarize: the concept of PenPilot is great. And if they take my – and some other users' – feedback seriously, I believe it has the potential to turn into an irreplaceable writing assistant. But at the moment, it's just not there yet – far from it, actually.