• The Brainyacts
  • Posts
  • 183 | ✋🙅‍♂️ Wait, GPT-4 Didn't Pass the Bar?

183 | ✋🙅‍♂️ Wait, GPT-4 Didn't Pass the Bar?

Brainyacts #183

It’s Tuesday. You will miss this one today but there is another next Tuesday. Free Event: See 10 “Innovating with AI” students showcase the new AI products and processes they’ve built over the last two months.

Let’s dig in.

In today’s Brainyacts:

  1. Introducing Decentralized AI

  2. Doubting GPT-4’s Bar Score

  3. OpenAI Voice Cloning and other AI model news

  4. Beyoncé takes stand against AI and other AI-related content

👋 to new subscribers!

To read previous editions, click here.

Lead Memo

🧶 🪮 Facebook gave your private messages to Netflix. Netflix gave Facebook your viewing history

And this is why I want to introduce you to Decentalize AI. Our personal data is an asset to everyone but ourselves! For background on the Facebook/Netflix ploy, read here.

Explainer on Decentralized AI

Generative AI is increasingly woven into the fabric of our lives from Microsoft’s Copilot to Google’s use of Gemini in Gmail and Workspace. I get weary of BigTech controlling how I interact with this tech, what data is used to build these models, and what data of mine it uses. I imagine you might get a bit worried too. That is why I want to introduce the emerging movement of decentralized AI.

Understanding decentralized AI is crucial for the average consumer of generative AI tools. Unlike traditional AI, which is often controlled by large tech companies, decentralized AI champions a different vision—one where power and control are in the hands of individual users.

Why Decentralized AI Matters

Decentralized AI is about shifting control from big tech to individual users, enhancing data privacy, and ensuring broader access to advanced AI tools. This movement is driven by the desire to bypass potential conflicts over intellectual property, navigate through looming government regulations, and address the scarcity of high-end computing resources. 

Advocates of decentralized AI, like Emad Mostaque, who recently gave up his gig as CEO of Stability AI to throw himself into this movement, argue for a shift away from centralized entities like OpenAI and Google, advocating for AI that reflects local knowledge and culture and is governed by the people.

At the core of this movement is the use of blockchain technology, known for underpinning cryptocurrencies, to create AI systems that are tamper-proof, ethical, and transparent. This contrasts with the often opaque "black box" systems managed by industry giants. Blockchain's role extends to enabling decentralized networks that can offer more affordable access to computing power, thus democratizing the technology's benefits.

Impact on Consumers and the AI Landscape

For the average user, decentralized AI promises a future where personal data is not just a commodity for corporations, but a valuable asset controlled by the individual. Initiatives like Personal AI's subscription-based assistant, which uses blockchain to secure personal data, illustrate a shift towards empowering users to manage their digital presence and interactions with AI technologies.

Moreover, decentralized AI can foster innovation and customization in AI tools, leading to services that better reflect individual needs and preferences. For instance, platforms like Vana enable users to pool and monetize their data, providing a tangible example of how decentralized AI can benefit consumers directly. 

Challenges and Questions for the Future

Despite the optimism, decentralized AI is still in its infancy and faces significant challenges. The reliance on blockchain technology, while promising for data security and transparency, has not yet achieved widespread adoption beyond the realm of cryptocurrencies. Moreover, the track record of decentralized projects in other domains has been mixed, with many initiatives failing to gain long-term traction.

For you all, this emerging landscape raises several questions:

  1. Data Ownership: How will decentralized AI redefine our understanding of data ownership and privacy?

  2. Access and Inequality: Can decentralized AI truly level the playing field, providing equal access to AI tools and resources?

  3. Governance: What governance models will ensure that decentralized AI systems are ethical, transparent, and accountable?

  4. Technology Integration: How will decentralized AI integrate with existing AI systems and infrastructures?

Oh, and I know you have been all looking for a way to connect GenAI with Blockchain! 🤣

Spotlight

🙅‍♂️ New Research Refutes GPT-4 ‘s Bar Score

Research paper here.

Perhaps the most widely touted of GPT-4’s at-launch, zero-shot capabilities has been its reported 90th-percentile performance on the Uniform Bar Exam. This paper begins by investigating the methodological challenges in documenting and verifying the 90th-percentile claim, presenting four sets of findings that indicate that OpenAI’s estimates of GPT-4’s UBE percentile are overinflated.

  1. GPT-4's Performance on the Bar Exam: OpenAI claimed that GPT-4 scored in the top 10% of candidates on the bar exam, which is the test required for law practice in most U.S. states. This was surprising because it marked a significant improvement from its predecessor, GPT-3.5, and because GPT-4 didn't perform as well on other essay-based exams.

  2. Questions about the Claim: There are doubts about the 90th percentile claim. The bar exam includes essays, and GPT-4's performance in essay writing hasn't been as strong on other tests. Also, the organization that administers the bar exam doesn't release percentile rankings, so it's unclear how the 90th percentile was calculated.

  3. Lack of Documentation: OpenAI hasn't publicly provided detailed evidence to support the 90th percentile score on the bar exam, though it has provided such documentation for other tests.

  4. External Evaluation: The evaluation of GPT-4 on the bar exam was conducted by experts outside of OpenAI, focusing on average performance rather than percentile rankings. Their findings don't necessarily support the 90th percentile claim.

  5. Actual Percentile Estimate: Based on available data, such as unofficial percentile charts from Illinois, GPT-4’s score might actually be around the 68th percentile overall and 48th percentile for the essay section, which is significantly lower than the 90th percentile.

AI Model Notables

 OpenAI can clone voices with just 15 seconds of audio - amidst growing concerns over AI voice misuse, OpenAI is implementing measures like audio watermarking and monitoring to ensure responsible deployment

 OpenAI updates image generator DALL-E The DALL·E enabling you to edit images by selecting an area of the image to edit and describing your changes in chat or you can also provide a prompt with your desired edit in the conversation panel, without using the selection tool

 OpenAI opens up ChatGPT to users without needing to sign up

 OpenAI and Microsoft plan $100 Billion AI ‘Stargate’ (supercomputer)

 Apple says its latest AI model ReALM is even better than OpenAI’s GPT4

 Can Microsoft truly get Copilot to run locally on AI PCs?

► Here come the ads: The generative AI search engine Perplexity, backed by Jeff Bezos and positioning itself as a Google rival, is preparing to monetize by letting brands sponsor the "related questions" that account for 40% of queries on its platform

News You Can Use:

 But 34% of Americans haven’t even heard of ChatGPT 

The U.S. House has imposed a ban on the use of Microsoft's AI Copilot by congressional staff, citing cybersecurity concerns

The Federal Trade Commission's new impersonation rule comes into force today, arming the agency against AI-powered scams that mimic government agencies and businesses

AI Safety: UK and US sign landmark agreement

Conservative US judge says AI could strengthen 'originalist' movement

US, Japan to call for deeper cooperation in AI, semiconductors

 Beyoncé’s new album ‘Cowboy Carter’ is a statement against AI music

NY’s AI Chatbot for small businesses suggests they break laws, steal wages

The US plans to unveil new copyright laws for AI this year

Was this newsletter useful? Help me to improve!

With your feedback, I can improve the letter. Click on a link to vote:

Login or Subscribe to participate in polls.

Who is the author, Josh Kubicki?

Some of you know me. Others do not. Here is a short intro. I am a lawyer, entrepreneur, and teacher. I have transformed legal practices and built multi-million dollar businesses. Not a theorist, I am an applied researcher and former Chief Strategy Officer, recognized by Fast Company and Bloomberg Law for my unique work. Through this newsletter, I offer you pragmatic insights into leveraging AI to inform and improve your daily life in legal services.

DISCLAIMER: None of this is legal advice. This newsletter is strictly educational and is not legal advice or a solicitation to buy or sell any assets or to make any legal decisions. Please /be careful and do your own research.8