Socrates Reviews

I'm a lawyer and I'm using it for local chat since the first hour and I can say it's really a good product. No problem to interact with my documents, even if the ask them in a different language thant their text. I really hope it's not going down. Jon, stay on it!

This tool has great potential, but it seems the project has stalled. Updates are delayed, and Jon does not respond to messages, even those sent by AppSumo customer service.

Save your money. I would argue that most AI tools are likely to fade away due to prohibitive ongoing costs. Instead, opt for those that support models like BYOK.

The project and idea hold great promise, but it seems Jon is allowing it to die.

I got a T5 because i find the idea really great. And I realy like how it works.
However with the standard model. it gives AI reasoning in every cell starting an ending with some code :(
<think> XXXXX endless AI reasoning XXXX <think>

I was so excited about this AI tool because I routinely need to read contracts and other PDF documents and my other AI tools don't do this. Further complicating it, many PDF's are not ready OCR. I gave it 2 jobs this week...

First was comparing two similar contracts to see what was different, then going carefully through the newest one to interpret archaic legal language and ensure I understood the terms of agreement. It was somewhat useful, but the response outputs are unnecessarily long and it doesn't respond well to requests to be concise. Further, there is no real effort put into making it have a personality. Instead of addressing me as "you" it calls me "they" as if it's responding to a 3rd party instead of me directly. Kind of tedious. I got some helpful information but it took a long time to distill it down to simple and actionable. Also because it is so verbose, it's hard to go back and find the useful parts of the chat.

Second was what I hoped would be a super quick and useful job. Search 2 PDF's from my son's high school teachers and make a combined list of school supplies. LONG answers from Socrates! The list is cluttered with [1][2] annotations and broken down into sections that I didn't ask it to make. I just wanted a quick list I could copy/paste into Notes before I hit the store. Example of the complexity it adds instead of simply doing the task:

"<think> Alright, let me try to figure this out. The user provided two class handouts, one for Math 1P and another for English. They want a combined shopping list based on these documents.
Looking at the Math 1P document, it mentions that only completed work done in pencil will be accepted. So pencils are a must. It also talks about using notebooks for warm-ups, notes, and a toolkit. This suggests that students need a notebook, probably a spiral one since that's common for note-taking.

The English class document requires a large, three-ring binder or a 200-page spiral notebook. It also specifies blue or black ballpoint pens. So from English, I need to add a binder or spiral notebook and pens.

Both classes mention bringing supplies every day and being prepared. The Math class also mentions extra paper, a pencil sharpener, highlighters, and a class folder. So I should include those as well.

I should organize these items into categories for clarity. That way, the user can easily see what's needed for each class or general use. Combining similar items, like notebooks and writing utensils, makes the list more efficient. I'll make sure to include the specific details from each document, like the type of pens and notebooks, to cover all bases."

When I asked, "what are those numbers in brackets?" I wanted to copy/paste a simple shopping list, not be forced to copy extraneous info.

"<think> Alright, so the user is asking about the numbers in brackets in my previous response. Let me try to figure this out.
In my answer, I used references like [1] and [2]. These numbers are citations from the context provided. Each context piece has a number, like [1], [2], and [3]. When I included the supplies in the shopping list, I added these numbers in brackets to show which context each item came from.

So, for example, when I mentioned "Pencils" for the Math class, I put [1] because that information was from the first context document. Similarly, the binder for English class was from the second context, so I used [2].

I think the user is asking for clarification on what these numbers mean. They might be confused about why there are numbers in brackets or what they refer to. My job is to explain that these are citations from the provided context documents, helping the user understand the sources of each item in the shopping list.

I should make sure to explain this clearly, so the user knows that each bracketed number corresponds to a specific document they provided. That way, they can refer back to the original context if they need more details about any item.

Those numbers in brackets (e.g., [1], [2]) are citations that refer to specific pieces of context provided earlier. Each number corresponds to a document or piece of information that supports the answer or statement being made. This helps in identifying the source of the information and ensures clarity."

Then I tried telling it to "be concise," but it responded with the same long reply again.

This is honestly absurd if the point is to save time by having the tool read docs and make the information useful. It's adding complexity instead of pulling out necessary info. Maybe good for compiling research data, but not so much for practical day to day application. I really need a tool to work with PDF's on the regular, but this one misses the mark for practical use cases.

I had high hopes for Socrates since I wanted to create many learning materials from textbooks for my students. I've been doing this with ChatGPT, Claude, Gemini etc. so far and will probably have to continue doing it with those... I have a prompt that should ask 3 interactive questions at the beginning. Every LLM understands this, but Socrates doesn't understand it at all. It seems to only understand 10% of the prompt (they are in German). I've contacted support and am waiting to see what the team says - If German isn't supported, I'll initiate a refund. 2 tacos, since I like the UI