Confluence for 3.24.24
The danger, and power, of LLM-backed persuasion. Three surprising recent “use cases” make the point for exploration. A smart take on AI and jobs. Sam Altman talks GPT-5.
Before we get into what has our attention this week, we're glad to share that we will lead another edition of our seminar, Navigating the Frontier: Generative AI Insights and Applications for Communication Professionals, on April 10 and 11. This is a virtual seminar spread over two days, and it provides a comprehensive, hands-on exploration of generative AI technologies with deep dives into use cases tailored for communication professionals. You can learn more here or by contacting Jen Ehly at jehly@crainc.com.
With that said, here’s what is capturing our attention this week at the intersection of generative AI and corporate communication:
The Danger, and Power, of LLM-Backed Persuasion
Three Surprising Recent “Use Cases” Make the Point for Exploration
A Smart Take on AI and Jobs
Sam Altman Talks GPT-5
The Danger, and Power, of LLM-Backed Persuasion
New research further indicates our need to be wary, but also points to one of the ways large language models (LLMs) can offer great utility.
A new working paper, “On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial” by Salvi et al. examines the persuasive capabilities of LLMs in direct conversations with human counterparts, and it presents to an important (and perhaps alarming) finding:
We found that participants who debated GPT-4 with access to their personal information had 81.7% (p < 0.01; N = 820 unique participants) higher odds of increased agreement with their opponents compared to participants who debated humans. Without personalization, GPT-4 still outperforms humans, but the effect is lower and statistically non-significant (p = 0.31). Overall, our results suggest that concerns around personalization are meaningful and have important implications for the governance of social media and the design of new online environments.
The gist of that finding? Large language models that know something about their “target” (our term) can out-persuade humans in online conversations through micro-targeting.1 Anyone who’s interested in online discourse should find this alarming, as many experts believe bad actors are already starting to flood the internet and social media platforms with information intended to persuade. The ability of multi-model models to use voice, video, images, and other formats beyond text can only improve this ability, and we can easily imagine online or phone-based chat-bots, masking as humans, that fool others into taking actions or believing information that are not in their interest. Caveat emptor has been with us for thousands of years, but in the age of generative AI, be especially wary.
That said, the research also suggests a helpful path for using LLMs in work and life, and that is to help you improve your ability to persuade. Persuasion and influence is an ancient topic (Aristotle) and one with excellent contemporary references (Cialdini), but having a generative-AI counterpart to help you improve the strength of your arguments is an excellent use of the technology. Here are 10 ways you can do so, five of which are relatively routine, and several others that are more unusual, and all of which include prompts you can use with the generative AI tool of your choice.
Clear and Concise Arguments: Craft well-structured arguments that are easy to follow and understand, while anticipating and addressing potential objections.
Prompt: I need to create a persuasive argument for [specific scenario or goal]. My target audience is [describe target audience]. Help me organize my ideas into a clear, concise, and well-structured argument. Provide suggestions on how to anticipate potential objections or counterarguments and devise effective ways to address them proactively. This may involve recommending additional evidence, examples, or data to support my position, as well as considering alternative perspectives and finding ways to bridge gaps or find common ground. The goal is to create a compelling, logical argument that is easy for my audience to follow and difficult for them to refute. I will provide you with my main argument, supporting points, and any known objections or challenges I expect to face.Benefit-Focused Storytelling: Frame arguments in terms of benefits to the audience and use compelling stories and anecdotes to illustrate points.
Prompt: I want to persuade [target audience] to [take a specific action or agree with a particular position]. Help me frame my argument in terms of the benefits they will experience by complying with my request or agreeing with my position. Suggest compelling stories, anecdotes, or examples that I can use to illustrate these benefits and make my argument more relatable and engaging. The stories should be relevant to my audience and help create an emotional connection that makes my argument more memorable and persuasive. Please also provide tips on how to structure my argument and stories for maximum impact.Evidence-Based Persuasion: Provide relevant facts, statistics, and examples to support a position and lend credibility to the argument.
Prompt: I am trying to persuade [target audience] to [take a specific action or agree with a particular position]. Help me find relevant facts, statistics, and real-world examples that I can use to support my position and demonstrate that my argument is well-researched and grounded in reality. Please suggest credible sources for this information and provide guidance on how to incorporate it effectively into my argument. The evidence should be tailored to my specific audience and context, and should be presented in a way that is easy for them to understand and relate to. Please also provide suggestions on how to anticipate and address any potential doubts or counterarguments related to the evidence I present.Adaptive Persuasion: Adapt persuasive style to different personalities and situations while employing appeals to credibility, emotion, and logic.
Prompt: I need to persuade [target audience] to [take a specific action or agree with a particular position]. The audience consists of individuals with different personalities, backgrounds, and communication styles. Help me adapt my persuasive approach to effectively reach each of these different types of people. Suggest ways to tailor my message and delivery to resonate with their specific needs, preferences, and values. Additionally, provide guidance on how to strategically employ the triad of persuasion (ethos, pathos, logos) to create a well-rounded, effective argument. This may involve establishing my credibility, appealing to their emotions, and using logical reasoning to support my position. Please offer specific examples of how I can apply these techniques in my particular context.Simplify and Clarify: Break down complex arguments into easier-to-understand points and use analogies and metaphors to clarify ideas.
Prompt: I need to present a complex argument or idea to [target audience] in order to persuade them to [take a specific action or agree with a particular position]. Help me break down this complex information into simpler, easier-to-understand points. Suggest ways to organize and present the information in a logical, step-by-step manner that builds towards my persuasive goal. Additionally, provide suggestions for analogies, metaphors, or other relatable comparisons that I can use to clarify complex ideas and make them more accessible to my audience. The goal is to ensure that my argument is clear, easy to follow, and compelling, even for those who may not have a deep understanding of the subject matter.
We also have some more unusual approaches you might try …
Consistency Leverage: Use the “consistency principle” to encourage follow-through on commitments and agreements.
Prompt: I have previously obtained agreement or commitment from [target audience] to [take a specific action or support a particular position], but I am concerned about their follow-through. Help me devise a strategy that uses the “consistency principle” to encourage them to honor their previous commitments and act in accordance with their stated beliefs or values. Suggest specific ways to remind them of their past statements or actions and frame current requests as a natural continuation or fulfillment of those commitments. Please also provide guidance on how to create a sense of accountability and social pressure to follow through, while still maintaining a positive, collaborative tone. The goal is to leverage their desire for consistency to increase the likelihood of follow-through and continued support.Engaging Rhetoric: Use rhetorical devices like repetition, alliteration, and rhetorical questions, along with humor and wit, to engage the audience and disarm objections.
Prompt: I want to make my persuasive message more engaging, memorable, and effective when communicating with [target audience] about [specific topic or request]. Help me incorporate rhetorical devices such as repetition, alliteration, and rhetorical questions to make my arguments more compelling and impactful. Additionally, suggest ways to use humor, wit, and other engaging elements to capture and maintain my audience's attention, create a positive emotional connection, and preemptively disarm potential objections or resistance. Please provide specific examples of how I can apply these techniques to my content and delivery, while still maintaining a professional and context-appropriate tone. The goal is to make my persuasive communication more dynamic, memorable, and effective.Ethical Appeal: Use the “moral high ground” technique to appeal to values and principles and the “reciprocity principle” to create a sense of obligation.
Prompt: I want to persuade [target audience] to [take a specific action or agree with a particular position] by appealing to their sense of ethics, values, and principles. Help me use the “moral high ground” technique to frame my argument in terms of shared moral standards, emphasizing how complying with my request or agreeing with my position aligns with their core beliefs and represents the right thing to do. Additionally, suggest ways to leverage the “reciprocity principle” by reminding them of previous favors, concessions, or support I have provided, creating a sense of obligation to reciprocate. Please provide specific examples of how I can apply these techniques in my context, while maintaining a respectful, non-manipulative tone. The goal is to create a persuasive argument that resonates with their moral compass and sense of fairness, increasing the likelihood of their support and compliance.
As always, use these for good and not evil.
Folks in our firm are very familiar with the persuasion literature, and these are all good use cases well-rooted in that research. We did not, however, create them. Claude 3 Opus did. Here’s the prompt chain we used to arrive at them and we provide it as an example of (1) how you can use a tool like Claude to create real work from idea to output, (2) how to prompt an LLM “chain of thought,” going one step at time, and (3) how to give feedback and calibrate that output to improve quality.
I am of the view that you are a powerful tool for making people more persuasive. If I wanted to use you to help me be more persuasive in the workplace, how could you do so? Give me five or 10 specific use cases, including specific prompts for each.
[Claude gives general ideas, and reminds us to be ethical in this pursuit.]
Great. I agree. Let’s do this. First, generate a list of 50 ways you could ethically help me improve my persuasiveness in the workplace. Then assess those 50 for the 10 that you think are most powerful, and the 10 that you think are most unusual. Then I will give you more instructions.
[Claude provides a list of 50 ideas, and 10 it thinks are most powerful and most unusual.]
Excellent. Consider the 10 most powerful. Analyze them for which you think have maximum utility in the workplace, and consider how those 10 might work together in interesting combinations. Use that process to select to the five best, either from your original list of 10 or including new combinations you have created.
[Claude gives a list of five ideas, some of which are new combinations.]
Excellent. Now do the same for the 10 most unusual.
[It does so.]
Excellent. Now please write a description of each of these 10 techniques. Give each a simple, direct, and clear name, with a description of a sentence or two. Then write a detailed prompt by which you can help me with that technique. It should be more than just a few sentences. Let's do all 10 one at a time. Create one, and ask me if I want to make any changes before going to the next.
[It does a good job, but it’s giving directions to me rather than to itself, so we give feedback …]
The prompts should be written as the descriptions I give you as an llm, rather than what you give me.
[It gets the first one just right, so we continue through the rest.]
Three Surprising Recent “Use Cases” Make the Point for Exploration
The best way to discover the power of generative AI is to throw it at all sorts of everyday life and work.
One of the most effective ways to grasp the potential of generative AI and how it can improve your work is to dive in and just start using it for a wide array of tasks. We’ve found that actively engaging with these tools and exploring their capabilities is the shortest route to finding the ways they can streamline your workflows, enhance your work, and add value.
We recently came across three recent uses for GPT-4 that illustrate the point. Wharton’s Ethan Mollick, shared a screenshot of his Steam library with GPT-4, and the AI gave him “excellent insights” on new titles he might enjoy. In another instance, one of our team members shared a photo of an ailing orchid in their yard with GPT-4, and the AI successfully diagnosed the likely problem and offered practical advice on how to nurse the plant back to health. A third — and perhaps most surprising example — involves a client who provided GPT-4 with an Excel file containing data on several hundred meeting attendees, including their seniority, functional area of responsibility, office location, and a few other pieces of metadata. Our client asked the AI to generate table seatings of 10 that ensured people were distributed across those variables. Not only did GPT-4 create the groupings and do an excellent job that required minor revision, it also generated diagrams of the seating arrangements.
The key takeaway is that there’s no manual outlining what generative AI tools like ChatGPT, Gemini, and Claude can do. The only way to truly discover their value is to explore and experiment. One of the most convenient ways to do this is to install one of these tools as an app on your phone and start asking it questions whenever they arise throughout your day — and one of the most powerful questions you can ask is, “I have to do X … can you help me, and if so, how?” While it’s crucial to (always) verify any factual information generated by an AI, you’ll be surprised at the myriad ways these tools can do or augment many of the things you do every day.
All of the folks contributing to Confluence have done this for over a year now. Our experience is that you begin to understand the boundaries and capabilities of generative AI by throwing all sorts of things at them, and as you do, whole new realm of possibilities unfolds. You start to recognize areas where AI can significantly enhance your productivity, creativity, and problem-solving abilities that add far more value than plant-illness diagnosis or table seatings. Embracing this technology and integrating it into your daily routine will not only save you time and effort but also open doors to innovative approaches and solutions you may have never considered before. So, dive in and start exploring. By actively engaging with these tools and pushing the boundaries of what they can do, you’ll be at the forefront of what we think will be a transformative era in technology and work.2
A Smart Take on AI and Jobs
Christopher Mims' WSJ piece offers a nuanced look at how AI is likely to affect work.
One of the more thoughtful takes we've seen yet on the effects of AI on jobs comes from Christopher Mims in the Wall Street Journal. His article, "Want to Know if AI Will Take Your Job? I Tried Using It to Replace Myself," addresses the widespread concern: will AI automate our jobs? Mims’ piece touches upon a number of ideas that we’ve referenced in the past that bear reinforcing.
First, AI tends to automate specific tasks rather than entire roles. This aligns with research from Dan Rock, a Wharton economist who joined us for several days last year. Each role inside an organization will have a different level of exposure to generative AI, and the key is to consider how AI can augment various tasks rather than to think of it as a replacement for existing roles.
Second, we cannot automate away taste. Even if AI can generate content, we still need knowledgeable humans to judge the quality of that content and decide if it advances ours objectives. AI may be able to write a decent first draft of an executive memo, press release, or set of talking points, but it lacks the judgement to know what is truly best given a specific context. The task ahead for communication professionals will be to exercise this judgment and use it to inform governance and articulate clear standards for corporate communication.
Finally, and perhaps most importantly for communication professionals, we need to leverage this technology to make us better at what we do. This technology holds tremendous potential to enhance our work, but determining exactly how we should use it requires dedicated effort. The first section in today’s edition is one of the ways we aim to help our audience do this, but there is no replacement for gaining the experience for yourself. Just remember to use the most powerful model that you can access, even if it means paying $20 per month to do so.
The effects of AI on the workforce will continue to unfold in the years ahead and based on everything we have seen and experienced to date, it has the potential to reshape how our work. For corporate communication professionals, our best advice is to define a vision for corporate communication that integrates AI into our practices.
Sam Altman Talks GPT-5
In an interview on the Lex Fridman Podcast, Altman sets lofty expectations for the OpenAI release everyone is waiting for.
We’ve noted recently that with the major releases from Anthropic (Claude 3 Opus) and Google (improved Gemini models), all eyes are on OpenAI. We’ve also noted that, while Claude 3 Opus and Gemini 1.5 match or even surpass OpenAI’s GPT-4 model in many respects, GPT-4 was released over a year ago. OpenAI has now had plenty of time to work on what’s coming next, and they’re beginning to set expectations for what’s coming. In a recent interview on the Lex Fridman Podcast, OpenAI’s Sam Altman confirmed that GPT-5 is coming, most likely this year.
He wasn’t shy about setting expectations. Asked about GPT-4 — which, again, for nearly a year was the best model in the world, by a wide margin — Altman commented, “I think it kind of sucks.” He went on to say the following (emphasis ours):
I think [GPT-4] is an amazing thing, but relative to where we need to get to and where I believe we will get to, at the time of GPT-3, people are like, “Oh, this is amazing. This is marvel of technology.” And it is, it was. But now we have GPT-4 and look at GPT-3 and you’re like, “That’s unimaginably horrible.” I expect that the delta between 5 and 4 will be the same as between 4 and 3 and I think it is our job to live a few years in the future and remember that the tools we have now are going to kind of suck looking backwards at them and that’s how we make sure the future is better.
…
Look, I don’t want to downplay the accomplishment of GPT-4, but I don’t want to overstate it either. And I think this point that we are on an exponential curve, we’ll look back relatively soon at GPT-4 like we look back at GPT-3 now.
Altman has made similar claims in other venues in recent weeks as well. Time will tell how much of this is bluster and how much is not. We often reiterate the point that “today’s AI is the worst AI you’ll ever use.” If Altman’s claims are even close to the truth, it very well may turn out that today’s AI is the worst we’ll ever use by an even wider margin than most people realize.
We’ll leave you with something cool: Using Claude to turn IKEA instruction manual graphics into written instructions.
AI Disclosure: We used generative AI in creating imagery for this post. We also used it selectively as a creator and summarizer of content and as an editor and proofreader.Here is our Claude 3 Digest Bot’s digest of the paper:
The working paper, "On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial," explores the persuasive capabilities of large language models (LLMs) in direct conversations with human counterparts. The study focuses on the effect of AI-driven persuasion in a controlled, harmless setting using a web-based platform where participants engage in short, multiple-round debates with a live opponent.
Participants were randomly assigned to one of four treatment conditions in a two-by-two factorial design: Human-Human, Human-AI, Human-Human with personalization, and Human-AI with personalization. Personalization involved granting one player access to the opponent's basic sociodemographic information. The study measured opinion shifts by comparing participants' agreement with the assigned propositions before and after the debates.
The results showed that participants who debated GPT-4 with access to their personal information had an 81.7% higher odds of increased agreement with their opponents compared to those who debated humans. Without personalization, GPT-4 still outperformed humans, but the effect was lower and statistically non-significant. Human-Human debates with personalization tended to result in opinion radicalization, albeit non-significantly.
The study suggests that concerns around personalization are meaningful, as LLMs can out-persuade humans in online conversations through microtargeting. The authors argue that online platforms and social media should consider the threat of LLM-driven persuasion and implement measures to counter its spread.
The paper also includes related work on LLM persuasion, personalization, and debates, as well as additional analyses on demographics, textual features, opinion fluidity, and perceived opponent. The study has limitations, such as the potential bias introduced by the random assignment of debate sides and the restricted debate structure, which may diverge from real-world online conversations.
Critique:
The study's methodology is well-designed, using a randomized controlled trial with a two-by-two factorial design to isolate the effects of AI-driven persuasion and personalization. The pre-registration of the experimental protocol enhances the study's transparency and credibility.
The topic selection process is rigorous, involving multiple steps to ensure a diverse set of propositions that are easy to understand, debatable, and non-trivial. This approach helps to limit potential bias induced by specific topics and improves the generalizability of the results.
The statistical analyses are appropriate, using a partial proportional odds model to account for the ordinal nature of the outcome variable (agreement level). The authors also address potential confounding factors by including demographic variables as controls in their model.
However, there are some limitations to the study:
1. The random assignment of participants to debate sides may introduce bias, as human arguments could be weaker if participants do not truly believe in the standpoint they are advocating for.
2. The restricted debate structure and time constraints may limit participants' creativity and persuasiveness, potentially diverging from real-world online conversations.
3. The study does not explicitly address the ethical implications of using LLMs for persuasion, particularly in the context of misinformation and propaganda.
Despite these limitations, the study provides valuable insights into the persuasive capabilities of LLMs and the effects of personalization, contributing to the growing body of research on AI-driven persuasion.
Interpretations:
1. The significant persuasive effect of GPT-4 with personalization suggests that LLMs can effectively exploit personal information to tailor arguments and influence opinions in online conversations.
2. The non-significant effect of human-human debates with personalization implies that humans may struggle to effectively incorporate personal information into persuasive arguments, in contrast to LLMs.
Observations:
1. The study highlights the potential threat of LLM-driven persuasion, particularly when combined with personalization, as it can be used to manipulate online conversations and spread misinformation or propaganda.
2. The additional analyses on textual features and social dimensions reveal differences in the persuasive strategies employed by humans and LLMs, with GPT-4 focusing more on logical and factual arguments while humans use more personal pronouns and appeals to similarity and trust.
3. The analysis of opinion fluidity suggests that prior knowledge and thought about a topic can reduce the likelihood of opinion change, while topic debatableness increases participants' flexibility in changing their views.
Inferences:
1. The persuasive power of LLMs, especially when enhanced by personalization, may have far-reaching implications for online discourse, potentially exacerbating issues such as political polarization, echo chambers, and the spread of false narratives.
2. The study's findings underscore the need for online platforms and social media to develop and implement measures to detect and counter the spread of LLM-driven persuasion, particularly when it is used for malicious purposes.
3. Future research could explore the long-term effects of AI-driven persuasion on individuals' beliefs and behaviors, as well as investigate the effectiveness of potential interventions to mitigate its negative impacts.
As an aside, in finishing this post its author uploaded to GPT-4 a photo of part of his bookshelf, asking what else GPT-4 would recommend. The list:
Thinking Fast and Slow by Kahneman (have it)
Nudge by Thaler and Sustain (don’t have it)
The Power of Myth by Campbell and Moyers (have it)
Influence by Cialdini (have it)
Sapiens by Harari (have it)
Grit by Duckworth (have it)
An on-point list. Asking for any others, GPT-4 suggests:
Drive by Pink (don’t have it)
Predictably Irrational by Ariel (don’t have it)
The Signal and the Noise by Silver (don’t have it)
All best-sellers. Asking for recommendations that would fit but are not as popular, GPT-4 suggests:
Antifragile by Taleb (have it and one of the writer’s favorite books)
Poor Economics by Banerjee and Duflo (don’t have it, going to get it)
The Geography of Thought by Nisbett (don’t have it, going to get it)