Hi all,
Here’s the truth: your company’s most valuable knowledge likely isn’t written down in any manual. It’s not documented in spreadsheets or stored in databases. It exists in the intuitive understanding of your best employees. This “invisible” expertise has been impossible to capture and scale. But artificial intelligence is about to change that. I have invited a friend of Exponential View’s, Jeremy Kahn, to share his view on what this means for the future of business. Jeremy is Fortune’s AI editor and author of a new book “Mastering AI: A Survival Guide to Our Superpowered Future”.
Enjoy!
Overcoming Polanyi’s Paradox
By Jeremy Kahn
As companies try to figure out how best to derive value from AI, one of the most important things they’ll have to reckon with is Michael Polanyi. Polanyi was a mid-20th-century Hungarian-British polymath, known today for his contributions to philosophy and information theory. In particular, Polanyi wrote about the importance of what he called tacit knowledge—information that exists, but which is undocumented, and which, in fact, may be undocumentable because it consists of the judgment and intuition of professionals and experts, derived from years of experience.
Polanyi’s catchphrase explanation of tacit knowledge is:
We know more than we can tell.
In the early 2000s, MIT economist David Autor speculated that Polanyi’s idea of tacit knowledge helped explain why widespread digital technologies such as the computer and the internet had not resulted in a big increase in labor productivity. Autor argued that for many tasks, especially for knowledge workers, digital technology failed to capture the tacit knowledge needed to perform those tasks well.
Given their ubiquity, it is tempting to infer that there is no task to which computers are not suited. But that leap of logic is unfounded. Human tasks that have proved most amenable to computerization are those that follow explicit, codifiable procedures. […] Tasks that have proved most vexing to automate are those that demand flexibility, judgment and common sense—skills that we understand only tacitly—for example, developing a hypothesis or organizing a closet. In these tasks, computers are often less sophisticated than preschool-age children.
Autor called this Polanyi’s Paradox, a reference to both Polanyi and Solow’s Paradox—named for the late economist Robert Solow, who in 1987 quipped that “you can see the computer age everywhere but in the productivity statistics.”
But today, AI holds out the possibility of overcoming Polanyi’s Paradox. One of the strengths of AI—especially AI based on unsupervised learning, including today’s large language models—is its ability to find patterns in unstructured data. In other words, AI can potentially capture tacit knowledge. It can learn effective sales techniques not from a set of rules about how to sell, but from simply looking for patterns in transcripts of calls from top-performing sales reps. It could potentially learn the best legal argument to make in a particular case before a particular judge not from a handbook written by a top lawyer, but from analyzing the past arguments of winning lawyers who appeared in that judge’s court.
This is already happening in business, medicine, law, and academia. GongAI is a company that uses LLMs to analyze audio transcripts and chat logs. The software company Diligent used GongAI’s platform to analyze transcripts of its best sales reps and build an AI coach that boosted the success of its sales calls by 7.4% and helped its sales reps meet their quotas three weeks faster. Similarly, researchers at Stanford University and MIT examined the effects of an AI coaching tool developed by an unnamed Fortune 500 enterprise software company and deployed by a company that had a contact center in the Philippines. The tool was trained on the transcripts of dialogues that the company’s top-performing customer service agents had with customers. The tool was remarkably effective, boosting productivity—as measured by the number of issues successfully resolved per hour—by 14% overall. And the coaching helped the least experienced agents the most, boosting their productivity by 34%, as they were able to gain from the tacit knowledge of the most experienced and successful call handlers.

Researchers at the Florida State University, Tallahassee and the University of Buffalo used the feedback that expert surgeons provided in video-based assessments of surgeons in training to create an AI system that could analyze videos and provide coaching in real-time. And Instrumentl, a company known for its deep data on grants, is testing an AI grant writing tool that relies in part on extracting the tacit knowledge from past successful grant proposals.
The question for businesses that want to use AI most effectively then will be what are the sources of data that they have available that might contain not simply the explicit knowledge of the firm—but the tacit knowledge of its best employees, too?
And here’s where things get tricky.
By its very nature, a lot of tacit knowledge is never recorded. It is passed on orally, in conversations between staff or with customers, or perhaps never passed on at all. Today, it is routine to record certain contact center calls for “training and quality assurance” purposes, and these can, and have already, been used by some companies to create AI “coaches” that can help less experienced call handlers provide more effective responses to queries. But for multi-million dollar contracts, where sales cycles are measured over quarters, sales meetings might not be recorded. Nor are a lot of internal meetings where business strategy or, even project management, decisions are discussed. And yet it is precisely in these meetings where the data exists. With this insight captured, a business could train an AI copilot or coach to elevate the performance of less experienced staff. It could also help businesses to train potent AI agents that will take actions on behalf of the company, armed with the skill to make nuanced judgments derived from the expertise of the business’s most talented employees.
As
recently outlined in the Sunday newsletter, the faces of AI in enterprise are many but Copilots attuned to the tacit knowledge could be incredibly powerful…Copilots [are] AI that works alongside you in real-time, suggesting code as you type or refining your writing on the fly. They’re more like having a knowledgeable colleague looking over your shoulder than an intern working independently.
Keep reading with a 7-day free trial
Subscribe to Exponential View to keep reading this post and get 7 days of free access to the full post archives.