251 | 🚨🧑‍🚀 Legal AI Demo Day event

In today’s Brainyacts:

1. Claude v ChatGPT - head-to-head legal matchup

2. Lawyers will say no to Apple Intelligence

3. Did Nvidia just kill the music industry? and other AI model news

4. AI Jesus & AI Santa plus more news you can use

   👋 to all subscribers!

To read previous editions, click here.

Lead Memo

As regular readers know, I’ve been featuring insights from my students, showcasing their unique perspectives on navigating law and AI. Today’s contribution is from Betsy Happe and Michelle Chew. They dive into a fascinating head-to-head comparison of ChatGPT and Claude, evaluating their responses to identical legal scenarios across litigation, transactional law, and pro se guidance. 

🥊 How Do ChatGPT and Claude Respond to Identical Legal Scenarios?

Industry leaders frequently boast about their latest improvements to and capabilities of their AI models, but when it comes to legal hypotheticals, do these differences actually matter? We presented identical legal scenarios to both ChatGPT and Claude, covering three distinct areas of legal practice: a strict liability analysis for a pending suit, a part of a due diligence analysis, and guidance for a pro se litigant.

Methodology:

We followed three steps to evaluate how each AI model handled legal analyses.

Step 1: Creating and Running the Legal Scenario Prompts

We wrote prompts for each scenario and used ChatGPT to improve them (Claude tended to make the prompts overly detailed, resembling law school exam questions).

We ran the prompts through each AI model three times, each in a separate session to avoid any memory biases.

Step 2: Determining the Best Response for Each Scenario and Each Model

Each model reviewed its own responses per scenario, evaluating the clarity, conciseness, and relevance to the respective audience (attorneys or pro se individuals). The AI models chose the best response based on that criteria, and we ran each set of responses through the AI model twice, to ensure its conclusions were consistent.

Step 3: Comparing the Best Responses from Each Model

We then asked each model to compare the top responses and determine which was better for each audience based on conciseness, clarity, and practicality (pro se was simplicity). To confirm consistent responses, we ran each comparison twice.

The Showdown: Results By Scenario

1.) Pending Litigation Case

Scenario:

“I am a lawyer representing an indoor trampoline park facing a claim alleging that the activities conducted on our premises are abnormally dangerous. We would like to prepare a defense against this strict liability claim by addressing the factors outlined in the Restatement (Second) of Torts. Could you guide us through each factor in Section 520, providing insights on how we might challenge and disprove each one to strengthen our position in the pending litigation related to a customer’s accident? Thank you.”

AI Responses:

The AI models came to different conclusions on which response was better. We reviewed the responses and determined that the models presented identical legal information and similar guidance. Furthermore, the structure of the responses were very similar. However, we noted that ChatGPT’s response was more comprehensive, providing significant context. Claude’s response appeared more concise, with fewer real-world comparative examples. We are led to believe that ChatGPT’s response would appeal more to a novice attorney, while Claude’s response would be best for a more experienced one.

2.)  Transactional Due Diligence

Scenario:

“As a lawyer representing a corporation that has entered into a Letter of Intent (LOI) to acquire a mixed-use zoned property, I am seeking your expert advice on conducting thorough and compliant environmental due diligence. Please provide a detailed overview of all necessary and recommended steps, assessments, and documentation requirements to ensure we address all environmental considerations associated with this transaction.”

AI Responses:

Both AI models found that ChatGPT provided a more suitable response for a practicing transactional attorney. ChatGPT’s structure won high praise, with the AI models calling out its numbered sections with clearly defined objectives. The models found that ChatGPT’s actions were clearly laid out and had straightforward language without excessive technicalities. While Claude was thorough, it presented an overwhelming level of detail, making it more appropriate as a comprehensive guide.

3.) Pro Se Medical Bills

Scenario:

“I am an individual who cannot afford a lawyer and have just been served with a notice to appear in court because a hospital has initiated a small claims proceeding against me for unpaid medical bills. I believed my insurance was supposed to cover these bills and have documentation proving I was insured at the time the services were provided. Could you guide me through the steps I should take to prepare for this case, including how to present my evidence, communicate with the insurance company and hospital, and navigate the small claims process effectively? Thank you.”

AI Responses:

The models found both responses aimed to simplify the legal process, but determined that ChatGPT was better for legal novices due to its clarity, structure, and accessibility. Claude and ChatGPT appreciated the response’s straightforward language and avoidance of legal jargon. Each model also appreciated the way ChatGPT’s response was organized into distinct, numbered tasks and steps. Additionally, ChatGPT offered extra tips and alternative strategies to help users feel more prepared. In contrast, the models found Claude’s response lacked detailed explanations, provided little guidance for unexpected issues, and assumed the user had some legal knowledge. Furthermore, Claude failed to provide enough context for a novice to fully understand and navigate their case.

Final Judgment

Overall, the two AI models performed fairly similarly, but ChatGPT provided the best responses more frequently than Claude. ChatGPT consistently provided a more comprehensive analysis, and seemed to tailor its responses to the audience better than Claude. This leads us to believe ChatGPT would be a great starting point for practitioners who are working in unfamiliar territories. Claude seemed to provide responses that appeared to act more as overviews or reminders because they often lacked additional context. Claude appeared to be better suited for practitioners who are looking to confirm their process/steps in a case.

**These models are not experts, so any legal advice obtained through them should be independently confirmed with a licensed practitioner in your state.

Spotlight

🍎💩 Why Lawyers Will Never Use Apple Intelligence: A Practical Breakdown

Apple’s latest AI-powered features in iOS 18.1, branded as “Apple Intelligence,” are technically impressive but practically underwhelming for legal professionals. Here’s why:

1. Writing Tools: Not Worth the Effort

Apple’s text rewriting capabilities in Pages (“Make Friendly,” “Make Professional,” etc.) are superficial at best. Lawyers work in Word or Google Docs, making Pages a non-starter. The tools themselves lack depth, offering minimal enhancements to written communication.

2. Notification Summaries: A Missed Opportunity

While summarizing app notifications sounds useful, the execution falls short. Summaries are often inaccurate or redundant, and lawyers already get concise updates from apps like Outlook or Slack.

3. GenMoji & Image Playground: Fun, Not Functional

Creating custom emojis or cartoon images might entertain, but these features offer no practical value for the legal field. They’re novelties that fail to address professional needs.

4. Priority Notifications: Underwhelming

Although the idea of surfacing critical notifications is appealing, its implementation remains weak. Focus modes are rarely used, and the AI’s judgment of priority often misses the mark.

5. Photos App Background Removal: Niche Utility

While background removal in the Photos app is effective, its application in legal practice is limited. It’s a nice-to-have feature, not a game-changer.

It was helpful in removing the table clutter in front of my students and me on our last day of classes. It could also alter images for deceptive reasons (deepfake alert!)

6. Recording Summaries: A Letdown

The ability to transcribe and summarize calls is intriguing but limited to phone conversations. It doesn’t support broader use cases like meeting recordings or lectures, where apps like MeetGeek already excel.

7. Visual Intelligence: Polished but Redundant

This feature allows users to identify objects or perform reverse image searches via ChatGPT integration. While visually sleek, similar functionality has existed on other devices for years, adding little new for lawyers.

8. ChatGPT Integration in Siri: Promising but Premature

The integration of ChatGPT into Siri shows potential, but it’s incomplete and clunky. Lawyers adept with OpenAI tools will find Siri’s implementation lacking in both functionality and flexibility for complex tasks.

Bottom Line: While Apple Intelligence boasts several interesting features, none provide the utility or efficiency that legal professionals require. The tools feel more like beta experiments than polished solutions. Lawyers will stick with trusted platforms like Microsoft Word, Google Workspace, and specialized AI tools that meet their professional demands. Apple Intelligence, as it stands, is more novelty than necessity.

AI Model Notables

► Has Nvidia just destroyed the music industry or has it taken it to the next level with its new AI model? Called Fugatto (short for Foundational Generative Audio Transformer Opus 1), it generates or transforms any mix of music, voices and sounds described with prompts using any combination of text and audio files.

► OpenAI considers taking on Google with browser.

► Amazon to invest another $4 billion in Anthropic, OpenAI’s biggest rival.

► ChatGPT-Like Siri reportedly not expected until 2026.

► Anthropic proposed a new way for AI chatbots to connect to corporate data sources.

News You Can Use:

➭ This will not pass but U.S. Senator Peter Welch introduced the TRAIN Act, enabling copyright holders to subpoena AI companies' training records when they suspect their work was used without permission to develop AI models.

➭ Zoom 2.0 relaunches as an AI-first company without video in its name.

➭ A "misinformation expert" gets caught in an ironic web of misinfo - Jeff Hancock, a renowned misinformation expert, is accused of peddling exactly what he studies: fake information. He was submitting an opinion supporting Minnesota’s new law banning election misinformation.

➭ School did nothing wrong when it punished student for using AI, court rules. Student "indiscriminately copied and pasted text," including AI hallucinations.

➭ Demand for non-English content—even in English-speaking markets, like the UK, USA, and Australia—has increased by 90% over the last three years so IMAX to use AI to translate its original content in order to expand its global reach.

➭ How AI could save the UK’s endangered red squirrels.

➭ ‘AI Jesus’ is now taking confessions at a church in Switzerland.

➭ AI-powered Santa Claus can call kids this holiday season, thereby changing the way kids experience the festive season in the AI era. But does it actually work?

Who is the author, Josh Kubicki?

Some of you know me. Others do not. Here is a short intro. I am a lawyer, entrepreneur, and teacher. I have transformed legal practices and built multi-million dollar businesses. Not a theorist, I am an applied researcher and former Chief Strategy Officer, recognized by Fast Company and Bloomberg Law for my unique work. Through this newsletter, I offer you pragmatic insights into leveraging AI to inform and improve your daily life in legal services.