Confluence for 11.10.2024

Another source of bias in LLMs. Microsoft Copilot is coming to individual 365 subscriptions. MIT study strengthens the argument for co-intelligence. Mistral AI is gaining momentum.

Midjourney prompt: A Georgia O'Keeffe painting of purple dahlias, with the word "Confluence" in the bottom left corner.

Welcome to Confluence. While we won’t post about it this week, we are increasingly impressed — and a bit put back — by the level of thoughtfulness and collaboration that Anthropic’s Claude Sonnet 3.5 “New” model is able to provide. This is the model most of us use in our day-to-day work. Until now we’ve been quick to say that large language models are prediction machines, and that they aren’t actually “thinking.” But … boy, Sonnet 3.5 New sure seems to be thinking. More to come on that next week. In the meantime, here’s what has our attention this week at the intersection of generative AI and corporate communication:

• Another Source of Bias in LLMs

• Microsoft Copilot Is Coming to Individual 365 Subscriptions

• MIT Study Strengthens the Argument for Co-Intelligence

• Mistral AI Is Gaining Momentum

Another Source of Bias in LLMs

New research shows how large language models (LLMs) reflect the values embedded within different languages.

We often write and speak about bias within generative AI tools, whether in chatbots like Claude or ChatGPT or in image generation tools like Midjourney. Now recent research highlights another dimension of bias in LLMs — one rooted in the language used to train and prompt the models themselves.

The research team found something striking, yet unsurprising: When prompting LLMs to provide information about specific political and historical figures in different languages (in this case, English versus Chinese), the outputs reflected markedly different moral assessments. A similar gap emerged when comparing responses from LLMs built and trained in different countries. When you consider how LLMs work, this bias makes sense — language necessarily reflects the norms and frames of the culture in which it exists, and models trained on massive bodies of text, regardless of language, reflect the patterns that exist within their training data. If there are differences, no matter how subtle, in the values English, Chinese, and other languages reflect, those differences will show up in how LLMs respond.

For corporate communicators and leaders working with these tools, this presents both challenges and opportunities. The bias effect is most pronounced when dealing with politically or culturally sensitive topics — areas where many organizations tend to already tread with care. But it also suggests the need for thoughtful practices when using LLMs for translating content across languages and cultures.

The most important piece of advice we can offer is to treat AI translations as drafts rather than final copy. While these tools excel at converting text between languages, they may subtly alter tone or connotation in ways that only native speakers will catch. We need to keep humans in the loop to make sure we can capture and correct potential issues and unwanted biases.

Big picture, our view on bias in LLMs remains the same. While it’s important to be aware of potential sources of bias, it shouldn’t stop you from leveraging the power of generative AI. These tools are exceptional at translating text across languages and, in most organizational contexts, the type of bias this research identifies is less likely to cause significant issues. The key is maintaining appropriate oversight — having native speakers review LLM translations to ensure quality and cultural appropriateness.

Microsoft Copilot Is Coming to Individual 365 Subscriptions

The rollout is only in select markets, but could it be a sign of what’s coming?

Earlier this year, Microsoft introduced Copilot Pro, a $20/month add-on for individual Microsoft subscribers that adds generative AI capabilities to Office applications like Word, PowerPoint, in Excel. This week, The Verge reports that Microsoft announced that in select markets — Australia, New Zealand, Malaysia, Singapore, Taiwan, and Thailand — Copilot will no longer be an add-on and instead will automatically come with every individual 365 subscription. It will come with a slight price increase — $4-$5 AUD per month in Australia, for example.

It’s a seemingly minor announcement but it may be a bigger hint of what’s coming. In January of this year — which, as it happens, was the same month Copilot Pro was announced — we wrote that “at some point [Copilot] will be part of [Microsoft 365’s] standard offering.” This week’s announcement is a small step in that direction and could foreshadow what’s to come for individual subscriptions in additional markets and enterprise 365 subscriptions.

If Microsoft does move in that direction, it will have significant implications for corporate communications teams. As we’ve written before, the more that software providers like Microsoft embed Copilot and other generative AI capabilities within the tools individual consumers use every day, the more the potential benefits and accompanying organizational challenges multiply. The utility for corporate communication is increasingly apparent, and we comment on various aspects of that utility nearly every week in Confluence. The challenges and second-order consequences are less obvious but just as important. As we wrote of Copilot in January:

So far most employees have not used these models to create content because they either have yet become facile with something like GPT-4, or their organizations don’t allow them to do so. But as Microsoft continues to integrate Copilot more deeply into its 365 and Office suite and make it more affordable (and we believe it will do so — in fact, we believe at some point it will be part of the standard offering), corporate communication professionals will increasingly take advantage of these tools in creating content. It won’t be long, though, before their internal clients (teams, leaders, and other employees) realize they can create much of this mundane content themselves. The role of content creation will shift from the professional, to the professional using the AI, to the internal client using the AI. This will have profound implications for governance, task and role definitions, skill development, and organizational design in our space. This is coming quickly. The time to start thinking about it is now.

MIT Study Strengthens the Argument for Co-Intelligence

A new depth to our understanding of how AI complements professional work.

A new study from MIT, “Human-AI Collaboration Increases Performance and Innovation Under Time Pressure” by Neil Thompson, Baek-Young Choi, and Yufei Wu, examines how AI affects professional work, looking in particular at its influence on materials science research. While the study focuses on a field different from communication, its findings provide empirical evidence for what we’ve long believed about effective human-AI collaboration.

The study tracked researchers using AI to automate routine ideation tasks — about 50% of the initial work — which gave them more time for evaluation and refinement. This mirrors the recommendation we’ve made before: generative AI can often handle time-consuming tasks like first drafts and data analysis, creating space for the strategic and nuanced work that drives value.

The data also reveal an interesting dynamic: the benefits of AI varied across users. Top scientists nearly doubled their output with AI assistance, while others saw only modest gains. The difference came down to their skill in evaluating and prioritizing AI suggestions. We expect this finding holds true for communication teams: those who build skills in assessing AI outputs will see stronger results.

The study also found that AI didn't simply accelerate existing approaches — it helped researchers explore new territory and tackle more ambitious work, suggesting that AI might do more than automate familiar processes. This might look like helping teams identify unexpected audience segments, create novel engagement strategies, and develop surprising messaging approaches.

The message here is straightforward, and a data-backed confirmation of what we’ve long believed: successful human-AI partnership won't come from resisting or embracing AI wholesale, but from building proficiency in managing it and treating it as a collaborator. The best way to build that skillset? Use generative AI every day, especially for tasks for which you believe it may not be able to add value. Try it, and see.

Mistral AI Is Gaining Momentum

Mistral is making a strong case for sophisticated design over sheer size.

In a market dominated by American tech giants and Chinese startups, a French company is quietly making moves in generative AI development. After 18 months of rapid growth and recent deals with major corporations, Mistral AI has evolved from an ambitious research project into a serious player in the LLM market.

Unlike other generative AI companies racing to build ever-larger models, Mistral has taken a different path. They’ve focused on architectural efficiency, proving that sophisticated design can match the performance of larger models. Their models shine in both the quality of writing and the ability to analyze long documents — both crucial features for business applications.

Mistral’s European origins have shaped their development in notable ways. Their architecture reflects European privacy standards, and their development process aligns with EU regulations. This European perspective has become increasingly relevant as organizations worldwide grapple with evolving AI governance.

The company’s focus on efficiency translates to practical advantages. Their models require less computational power than many alternatives, and they’re designed to be adapted for specific business needs. Major enterprises are taking notice, particularly in sectors where regulatory compliance and resource efficiency matter most. As generative AI matures into a core business technology, Mistral’s success suggests raw performance is no longer enough — efficiency and regulatory alignment have become equally important factors in how businesses choose and deploy AI solutions.

---

We’ll leave you with something cool: Thanks to AI, we might just see The Beatles take home a Grammy for Record of the Year in 2025.

---

AI Disclosure: We used generative AI in creating imagery for this post. We also used it selectively as a creator and summarizer of content and as an editor and proofreader.