Socrates Reviews

I got a T5 because i find the idea really great. And I realy like how it works.
However with the standard model. it gives AI reasoning in every cell starting an ending with some code :(
<think> XXXXX endless AI reasoning XXXX <think>

I was so excited about this AI tool because I routinely need to read contracts and other PDF documents and my other AI tools don't do this. Further complicating it, many PDF's are not ready OCR. I gave it 2 jobs this week...

First was comparing two similar contracts to see what was different, then going carefully through the newest one to interpret archaic legal language and ensure I understood the terms of agreement. It was somewhat useful, but the response outputs are unnecessarily long and it doesn't respond well to requests to be concise. Further, there is no real effort put into making it have a personality. Instead of addressing me as "you" it calls me "they" as if it's responding to a 3rd party instead of me directly. Kind of tedious. I got some helpful information but it took a long time to distill it down to simple and actionable. Also because it is so verbose, it's hard to go back and find the useful parts of the chat.

Second was what I hoped would be a super quick and useful job. Search 2 PDF's from my son's high school teachers and make a combined list of school supplies. LONG answers from Socrates! The list is cluttered with [1][2] annotations and broken down into sections that I didn't ask it to make. I just wanted a quick list I could copy/paste into Notes before I hit the store. Example of the complexity it adds instead of simply doing the task:

"<think> Alright, let me try to figure this out. The user provided two class handouts, one for Math 1P and another for English. They want a combined shopping list based on these documents.
Looking at the Math 1P document, it mentions that only completed work done in pencil will be accepted. So pencils are a must. It also talks about using notebooks for warm-ups, notes, and a toolkit. This suggests that students need a notebook, probably a spiral one since that's common for note-taking.

The English class document requires a large, three-ring binder or a 200-page spiral notebook. It also specifies blue or black ballpoint pens. So from English, I need to add a binder or spiral notebook and pens.

Both classes mention bringing supplies every day and being prepared. The Math class also mentions extra paper, a pencil sharpener, highlighters, and a class folder. So I should include those as well.

I should organize these items into categories for clarity. That way, the user can easily see what's needed for each class or general use. Combining similar items, like notebooks and writing utensils, makes the list more efficient. I'll make sure to include the specific details from each document, like the type of pens and notebooks, to cover all bases."

When I asked, "what are those numbers in brackets?" I wanted to copy/paste a simple shopping list, not be forced to copy extraneous info.

"<think> Alright, so the user is asking about the numbers in brackets in my previous response. Let me try to figure this out.
In my answer, I used references like [1] and [2]. These numbers are citations from the context provided. Each context piece has a number, like [1], [2], and [3]. When I included the supplies in the shopping list, I added these numbers in brackets to show which context each item came from.

So, for example, when I mentioned "Pencils" for the Math class, I put [1] because that information was from the first context document. Similarly, the binder for English class was from the second context, so I used [2].

I think the user is asking for clarification on what these numbers mean. They might be confused about why there are numbers in brackets or what they refer to. My job is to explain that these are citations from the provided context documents, helping the user understand the sources of each item in the shopping list.

I should make sure to explain this clearly, so the user knows that each bracketed number corresponds to a specific document they provided. That way, they can refer back to the original context if they need more details about any item.

Those numbers in brackets (e.g., [1], [2]) are citations that refer to specific pieces of context provided earlier. Each number corresponds to a document or piece of information that supports the answer or statement being made. This helps in identifying the source of the information and ensures clarity."

Then I tried telling it to "be concise," but it responded with the same long reply again.

This is honestly absurd if the point is to save time by having the tool read docs and make the information useful. It's adding complexity instead of pulling out necessary info. Maybe good for compiling research data, but not so much for practical day to day application. I really need a tool to work with PDF's on the regular, but this one misses the mark for practical use cases.

I had high hopes for Socrates since I wanted to create many learning materials from textbooks for my students. I've been doing this with ChatGPT, Claude, Gemini etc. so far and will probably have to continue doing it with those... I have a prompt that should ask 3 interactive questions at the beginning. Every LLM understands this, but Socrates doesn't understand it at all. It seems to only understand 10% of the prompt (they are in German). I've contacted support and am waiting to see what the team says - If German isn't supported, I'll initiate a refund. 2 tacos, since I like the UI

I was compelled to request a refund for this product due to ongoing discomfort with its UI/UX. The following are aspects that need improvement:

- The PDF display cannot be further enlarged, which hinders my ability to read the file content in greater detail. Ideally, this should be adjustable dynamically.
- In the latest UI update, there is no filter to select the range of pages to be included in the chat. This is particularly problematic when I want to extract information from specific pages.

Several features that could be developed in the future include:

- The necessity for chat functionality in the form of images, as academic journals often contain diagrams that require specialized explanations.
- The need for prompt initialization, which can be customized for specific types of documents.

- Task and Directory Management:
I handle multiple tasks assigned to different workers, each with its own directory and specific folder structure to properly store the corresponding files.

- Error Prevention and System Need:
To avoid disorder or mistakes in file placement, I expected Socrates (in offline mode, since it doesn’t make sense to consume tokens for simple local workflow actions) to assist workers by indicating the correct path for each task.

- Clear and Unambiguous Structure:
The entire root and hierarchy of folders, subfolders, and files is based exactly on the annual action plan. The names are unique, differentiated, and clearly describe the work to be done, eliminating any possible confusion.

- Problems with the Socrates:
When a task is completed and we need to confirm the correct directory to store it, Socrates always gets confused. It is unable to guide users, even when given all the necessary information to identify the appropriate location. This issue persists despite the clarity and uniqueness of the folder structure.

- Comparison vs Gemini:
The same request was made to Gemini, which was able to guide correctly and identify the right directory almost instantly and without errors.

**Local Deep Dive very soon** about it, means what? we will have a solution for my case or not?