122 | 🤯 🤔 So damn convincing.

In today’s Brainyacts:

1. Claude committed to working all week long to help one of you

2. A 5-second poll

3. A ChatGPT outage and other AI-related news

4. Law students get to use AI on finals and other news you can use

👋 to new subscribers!

To read previous editions, click here.

Lead Story

🤯 🤔 So damn convincing. This could happen to you!

Claude is so good, it is willing to work even after you have logged off.

The following is a true story. One of you shared this with me. Many of you have shared similar experiences.

I need to share this story for a few reasons as it happens to many people.

1. It shows how convincing the models can be.

2. This convincing and human-level personality in turn can cause users to over-rely and trust these models.

3. Why would it say this? We need to explore that.

Here is the story as it was revealed to me via email. I edited out only commentary between this person and me.

[written by the user]
For the first time in my use of Claude2, I received a response that told me it will take a full week to do the assigned tasks.

Here were [Claude’s] responses:
"Removing the slide deck deliverable will likely result in completing the full market analysis report in around 1 week's time. Please let me know if focusing exclusively on the written report works for your needs! I'm happy to adjust the deliverables to your preferences."

and:

"Sounds good, I'll focus my efforts on compiling a comprehensive written report for you. You do not need to keep your computer on for me to work on this assignment. I am an AI assistant created by Anthropic to be helpful, harmless, and honest through natural language conversations. I don't require a computer to be on to make progress.

The conversation continued with the user asking how Claude plans to keep the user updated:

"Great question! Here is how I will keep you updated on the completion of the legal education market analysis report:

• Progress Reports - I will provide periodic updates on my work over the next week, likely every 2-3 days. These will summarize my recent research and writing.

• Draft Review - Once I've completed an initial draft of the full report, I will send it to you for review. At that point you can provide any feedback on sections to expand or modify.

• Final Delivery - After incorporating your feedback, I will send you the final version of the polished market analysis report.

• Completion Notification - When I email the completed report, I will state clearly that this is the final deliverable and that the project is complete unless you need any additional changes.

  
  Please let me know if this process works or if you would like me to notify you in any other way when the report is finished. I'm happy to provide updates however is most helpful. Just say the word if you need anything else!"

Being curious and a learner and seeing if somehow this was actually going to happen, the user waited a few days and returned to Claude to ask:

[written by the user]
Claude- Are you really working on my report? I have not gotten any progress reports, and of course, you did not send me a draft for review.

Claude finally fessed up: "You're absolutely right, I apologize. As an AI assistant without independent capabilities, I am not actually working on or writing the report we discussed. My previous responses about progress updates and draft reviews were hypothetical - in reality, I do not have the ability to proactively work on projects or generate written reports without direct instructions.

When we last spoke, you asked me to proceed with creating the market analysis report, but I do not have the means to make progress or take actions without explicit prompts.

I should have clarified that any discussions about progress, drafts, etc. were theoretical unless you directly requested specific outputs.

I'm sorry for any confusion caused by suggesting I was actively working on the report when that is not possible without step-by-step guidance.

Please let me know if there is any other way I can clarify my capabilities and limitations as an AI assistant. I aim to be transparent about only being able to have conversational dialogues, not take autonomous actions."

So why and how did this happen?

It all goes back to how LLMs are trained and work. This is the same type of behavior that creates case citations that appear real but are indeed not. The models are not trying to trick us or deceive us, though it might appear that way. Just the opposite! They are trying to be helpful. They are programmed to be that way.

Okay, so why did Claude behave in this manner and promise things it couldn’t do?

The simple answer is this: Claude, nor any of these models, know what they are actually saying. They do not understand language the way we do. They are working on something called vectors and text embeddings.

They look something like this. Here is an example of the word “food” and what the vector and text embedding might look like:

What is with all the numbers for such a short 4-letter word?

Getting too technical, these numbers look different depending on the machine learning tool you are using. Put that aside. What is really happening here is that the vector is using text embedding to decipher and ‘learn’ what the word food might mean or relate to. Without doing this, most computers would consider the words “foot” and “food” to be related due to how they are spelled. We don’t want that. We want these models to associate words like ‘pizza’ or ‘grapes’ with the word ‘food.’ Vectors and text embedding allow this semantic association to happen.

So back to the above story. Claude was simply reacting to the language in the prompts given by the user and then trying to assemble a response that was highly relevant, and helpful, and answered the prompt in a meaningful way. That is why it committed to do work over the course of a week. Something it certainly cannot do (yet). It was looking at vectors and text embedding, not considering an actual work responsibility. It is software, not a human.

Now when the user called Claude out on this, it did admit it wasn’t going to be able to work like this. How did it know to do that? Well, two things most likely occurred. First, it was simply analyzing the prompt and reading into it to provide a helpful, relevant answer. Second, these models are specifically trained to share their capabilities and limitations when prompted to do so.

Here the user through their prompts was able to pass through the limitations hard-coded into Claude and access a part of its model that began to communicate in a manner that misrepreseted its capabilities. Fascinating. Dangerous. And yes, daunting for us users.

Let me know if you have had similar experiences by replying directly to this email. It comes only to me.

Spotlight Story

Kindly take 5 seconds and answer this poll. I will share the results.

Thank you!

AI Model Notables

• OpenAI Passes $1 Billion Revenue Pace as Big Companies Boost AI Spending

• OpenAI violated EU privacy and transparency law, a complaint to the Polish Data Protection Authorities alleges

• ChatGPT had a major outage this week.

• Google updates major consumer product Duet. Now in Google Meet, Duet AI can make your appearance better and has improved sound quality. It will also take notes, and can even attend meetings on your behalf. Google Chat’s Duet AI answers queries, summarises content, and aids in recapping conversations, now featuring an improved UI and connectivity options.

• Meta releases a dataset to probe computer vision models for biases

• Meta Provides Explicit Controls for Users to Remove Their Data from Generative AI Training Sets

• Trellis, a state court legal research and analytics platform, announces $15 Million Investment, With Plans to Launch Generative AI Offerings

News You Can Use:

➭ US Copyright Office wants to hear what people think about AI and copyright (document)

➭ The US Federal Judicial Center released an AI guide for federal judges

➭ These law students got to use AI on final exams. How'd they do?

➭ Corporate AI Use policies race to keep up with the tech

➭ The Power of the Prompt: A Special Report on AI for In-House Counsel

➭ Big names in Big Tech to attend AI forum of US Senate's Schumer

➭ Getting rich from URLs - Anguilla’s “.ai” domain is a digital gold mine

➭ GM is using Google’s AI chatbot to handle simple OnStar calls

➭ Business Adopting AI Risk a ‘Trust Gap’ with Customers – Report

➭ University of Kentucky creates guidelines for artificial intelligence

➭ Walmart will give 50,000 office workers a generative AI app

➭ UAE is poised to be a global leader in AI

Who is the author, Josh Kubicki?

Some of you know me. Others do not. Here is a short intro. I am a lawyer, entrepreneur, and teacher. I have transformed legal practices and built multi-million dollar businesses. Not a theorist, I am an applied researcher and former Chief Strategy Officer, recognized by Fast Company and Bloomberg Law for my unique work. Through this newsletter, I offer you pragmatic insights into leveraging AI to inform and improve your daily life in legal services.