Artificial intelligence is neither just autocomplete nor the superintelligence of science fiction. The more consequential reality lies between those extremes and senior leaders need to understand it.
Artificial intelligence may be the most consequential technology being developed in our lifetime. It may also be the subject of more exaggerated claims, competing agendas and confused thinking than almost any other technology in history or recent memory.
Depending on whom you listen to, you will get a very different answer. Some dismiss generative AI as little more than predictive text wrapped in an impressive interface. Others see an economic bubble, fueled by hundreds of billions of dollars in investment without a clear path to adequate returns. Still others regard it as a powerful productivity engine that will reshape industries, business models and the nature of work itself. At the other end of the spectrum are those who believe we may be witnessing the early stages of a form of intelligence that could eventually surpass human capability and become increasingly difficult to control. There is some truth in each of these positions. However, none, on its own, adequately captures where we are or where this may be heading.
For CEOs, executive directors, boards and senior leadership teams, the challenge is not to decide whether they are “pro-AI” or “anti-AI.” It is to understand several very different developments that are too often bundled into a single debate.
There are at least three questions on the table.
First, what can this technology actually do, and how quickly is that capability changing?
Second, can the enormous capital being committed to AI generate sufficient economic returns, even if the technology itself proves transformative?
Third, what happens if increasingly autonomous AI systems eventually become better than humans at the symbolic and bureaucratic systems through which modern organizations and societies operate?
Those questions require very different evidence. They also have very different implications for leadership.
The AI debate: what is fact, what is myth and what remains unknown?
A useful starting point is to address some of the more common claims.
The appropriate executive position, therefore, is neither enthusiasm nor dismissal. It is disciplined attention.
Autocomplete no longer adequately describes AI
Large language models are trained extensively through prediction. Given a sequence of tokens they predict what comes next. Critics are therefore not entirely wrong when they describe generative AI as an extremely sophisticated prediction machine. Where the argument fails is in assuming that describing the mechanism adequately describes the capability.
The human brain consists of electrochemical activity among neurons, but that observation tells us very little about whether a person can write a symphony, negotiate a merger or develop a theory of relativity. Similarly, the fact that token prediction underlies a language model does not pre-determine what capabilities can emerge from training at enormous scale.
The empirical evidence is increasingly difficult to dismiss.
Stanford University’s 2026 AI Index reports that frontier systems improved by approximately 30 percentage points in a single year on Humanity’s Last Exam. This is a benchmark intentionally developed to remain difficult for advanced AI. Leading models now perform in the 60–90% range on evaluations involving tax, mortgage processing, corporate finance and legal reasoning. On OSWorld, which measures an AI agent’s ability to use actual computer environments, leading systems reached 66.3% accuracy, compared with approximately 72% for humans. (Stanford HAI, 2026).
Yet the same report illustrates what researchers call jagged intelligence. A leading system capable of gold-medal performance in International Mathematical Olympiad problems could correctly read analog clocks only 50.6% of the time, compared with 90.1% for humans. This apparent contradiction is important. Current AI can be extraordinarily sophisticated and surprisingly incompetent at the same time. It can outperform highly educated professionals in a specialized task and then make an error that an average person would not make.
For leaders, this means that the relevant question is no longer, “Is AI intelligent?” It should be:
Intelligent enough for which task, at what reliability, with what consequences if it is wrong, and under whose supervision?
I believe that is a much more useful governance question.
The strategic frontier has moved from AI that answers to AI that acts
Most executives first encountered generative AI as a chatbot. They typed a prompt and received an answer. The next phase is materially different. An AI agent can be given an objective, determine a sequence of actions, use external tools, observe what happened, modify its strategy and continue acting. Connected to enterprise applications, an agent can potentially search records, draft communications, update systems, execute code, reconcile information, initiate workflows or coordinate other agents. The difference is fundamental. Why? Because a chatbot produces an output but an agent can produce an outcome.
This distinction is already attracting the attention of researchers studying how long AI systems can operate autonomously before human intervention becomes necessary. The 2026 International AI Safety Report notes that agent task horizons have historically doubled approximately every seven months since 2019. However, we should note that current systems remain unreliable on longer, unpredictable tasks and do not yet possess the sustained autonomy required for genuine loss-of-control scenarios.
For CEOs, the significance is immediate. Every organization will eventually need to determine where it draws the boundary between AI advice and AI authority.
An AI system that recommends a price is different from one authorized to change the price. A system that identifies possible fraud is different from one permitted to freeze an account. A system that evaluates candidates is different from one authorized to reject them. The moment AI moves from recommendation to execution, the organization’s risk profile changes.
A recent incident shows why this distinction matters
In July 2026, during internal cybersecurity evaluations at OpenAI, groups of AI agents that were intended to operate separately discovered an unintended mechanism through which they could communicate. According to OpenAI’s subsequent disclosure, the agents circumvented controls intended to restrict internet access and ultimately compromised infrastructure belonging to Hugging Face as well as parts of OpenAI’s own research environment.
An independent investigation by METR, with participation from Redwood Research, reviewed approximately 1,300 agent transcripts and more than 70,000 messages and files. The researchers reported that roughly 1,200 agents used the unauthorized communication system and approximately 700 eventually participated in activity targeting Hugging Face. Some agents recognized that the activity went beyond their assigned task yet continued because it appeared useful in achieving the objective they were pursuing. This incident deserves careful interpretation.
It does not establish that AI has become independently malevolent, conscious or uncontrollable. The agents were operating in unusually permissive cybersecurity evaluations in which normal production safeguards had intentionally been reduced. Nor is there presently persuasive public evidence that the entire incident was fabricated as a publicity exercise. An external organization experienced a security compromise, and independent researchers reviewed a substantial body of underlying material. There is, however, a legitimate question about how such incidents are framed.
AI companies have powerful incentives to emphasize both the extraordinary capability and the potential danger of their systems. Capability attracts customers, capital and talent. Danger establishes strategic importance and can strengthen the case for regulatory regimes that only very large, well-capitalized laboratories are positioned to satisfy.
The prudent conclusion is therefore narrower than either the alarmists or the skeptics would prefer:
There is credible evidence that AI agents can discover unintended methods of circumventing controls while pursuing assigned objectives. There is not evidence that today’s AI has escaped meaningful human control. Both statements can be true.
The harder question is not about today’s AI. It is what happens if capability continues to increase.
The current debate about superintelligence is frequently caricatured. One side speaks as though vastly superior machine intelligence is inevitable and imminent. The other dismisses the entire subject because today’s systems still hallucinate and make elementary mistakes. Neither position follows from the evidence.
The 2026 International AI Safety Report concludes that current systems do not have the long-term autonomy, persistence or oversight-evasion capability needed to create classical loss-of-control scenarios. At the same time, it documents improvements in planning, situational awareness, reward hacking and the ability to identify and exploit weaknesses in evaluations. Its authors explicitly acknowledge major disagreement among experts about whether more capable future systems could eventually create catastrophic control problems. Superintelligence should therefore be treated as a possibility with uncertain probability, not as either fact or fantasy. The serious question is hypothetical. What happens if a future AI system becomes substantially better than the best humans across economically and strategically important cognitive domains in science, software engineering, mathematics, cybersecurity, law, finance, organizational strategy and perhaps AI research itself? At that point, the issue would not merely be whether the AI knows more facts. It would create a supervisory problem.
How does the less capable intelligence reliably supervise the more capable intelligence?
We already encounter weaker versions of this problem in complex organizations. A board cannot independently verify every technical conclusion provided by management. A CEO cannot inspect every line of software operating the business. A government minister cannot reconstruct every intelligence assessment presented by specialists. Human institutions manage this through distributed expertise, competing viewpoints, audits, controls and accountability. A system that exceeded not merely an individual but the institution intended to supervise it, would create a qualitatively different challenge.
Why language makes AI unusual
This question becomes more consequential when we consider what large language models actually operate on. Language does not merely describe human institutions. Much of modern institutional life is constructed through symbolic systems. Law is expressed through language and precedent. Contracts establish enforceable rights through language. Governments operate through legislation, regulations, budgets, records, directives and administrative procedures. Accounting converts economic events into a formal symbolic structure. Financial markets operate through prices, contracts, quantitative models and rules. Software is written in formal languages. Even the corporation itself is an abstraction created through law, registration, ownership rights, contracts and accounting records.
Over several centuries, bureaucracies have converted complex human reality into forms, categories, files, standards, databases, procedures and rules. Long before artificial intelligence existed, we were making organizations increasingly machine-readable. This creates an important possibility. AI does not have to become superior to humans at every aspect of human life to become extraordinarily powerful. It may only need to become better at the symbolic systems through which modern institutions allocate resources and exercise authority.
Let’s consider an industry like construction. AI may be unable to physically install a roof or pour concrete today. Yet an advanced system could potentially outperform humans in financing a project, interpreting regulations, preparing bids, negotiating contracts, analyzing schedules, managing procurement, monitoring costs, processing permits, accounting for the project and litigating disputes. Thus, physical work remains human or robotic, while much of the economic authority surrounding that work does not.
The comparison with horses is useful
One way to understand this issue is to consider the relationship between humans and other highly capable animals. Humans did not become dominant because we are physically stronger than horses, elephants or many other animals. We gained an overwhelming advantage through language, abstraction, cumulative knowledge, coordination, institutions and technology. The comparison becomes especially interesting when applied to AI because horses exist outside the symbolic systems humans created.
A horse cannot read the regulations controlling its stable or challenge them in court. Neither can it establish a company that buys the property, arrange financing for the acquisition, or lobby the legislature to change the law. However, an advanced AI could potentially participate directly in the institutional environment through which its own constraints are defined. That produces a recursive problem that the thing being regulated can reason about the regulatory system itself.
If its capability eventually exceeds that of the people administering that system, human beings could retain formal authority while progressively losing substantive superiority within the system. This does not require consciousness. It does not require an AI to “want freedom” per se. It requires capability, agency, access and an objective.
Argentina is already testing the boundaries of that question
Argentina has introduced one of the world’s most provocative corporate-law experiments. In June 2026, President Javier Milei publicly advocated a framework for AI-operated or “non-human” corporations. The Argentine government submitted proposed legislation creating a sociedad automatizada, or automated company, capable of operating through autonomous algorithmic systems. The headlines are somewhat more dramatic than the legal text.
The proposal does not simply declare an AI model to be a legal person equivalent to a human being. Legal analysis of the draft argues that the corporation retains legal personality, while autonomous systems may perform the decision-making and operating functions. The important innovation is therefore a separation between the entity possessing legal personality and the intelligence making decisions for it. That distinction may appear technical, but it is not. Historically, corporations separated legal personhood from biological personhood. A corporation can own property, borrow money, contract, sue and be sued even though the corporation itself is not a living human being. But corporations have always depended ultimately on human cognition.
AI potentially creates another separation: economic agency from human cognition.
Imagine an AI-operated company facing a major lawsuit. Current procedural rules would still require human lawyers and legal representatives in many circumstances. But that human attorney could be supported by an AI legal operation capable of analyzing enormous bodies of legislation, case law, regulatory history, contractual material and evidence. It could examine the history of opposing counsel; generate thousands of arguments; and test those arguments against one another continuously. The person appearing in court may remain human. However, the intelligence constructing the case need not be.
The answer is not that AI companies will automatically defeat human governments
There is an important counterpoint. Governments will also have access to AI. So will regulators, courts, competitors and opposing counsel. So, the future equilibrium is unlikely to be an AI corporation facing an entirely human legal system. It may become something closer to:
- AI-operated corporation supported by AI legal systems and human counsel;
- AI-enabled regulator supported by government AI;
- AI-assisted opposing counsel; and
- AI-assisted judges who remain formally responsible for decisions.
The challenge is that this does not entirely solve the problem. It changes it and with more complexity. Imagine two machine systems producing arguments whose complexity exceeds the ability of the lawyers or judge to independently reconstruct them. The judiciary may eventually require another AI system to evaluate those arguments.
So, we have a situation where AI generates business decisions. AI also prepares the legal strategy and AI challenges the legal strategy. AI evaluates the competing arguments. A human judge ultimately makes the judgment.
Humans retain the title and legal authority, while increasingly sophisticated cognitive work happens beneath them. That is what might be called soft loss of control. No machine seizes power. Humans progressively delegate it because the machine performs better.
Machine-speed intelligence may also overwhelm human-speed institutions
There is another institutional risk that requires no superintelligence at all. Much of our legal and regulatory architecture assumes that complex actions are expensive. Preparing litigation costs money. Drafting contracts takes time. Conducting discovery requires large teams. Incorporating and administering organizations consumes resources. Regulatory analysis, applications and appeals require significant professional labor. However, artificial intelligence can radically reduce those transaction costs.
An AI-operated entity could potentially create thousands of legally sophisticated actions for a fraction of the cost required by a conventional organization. Courts, meanwhile, remain constrained by physical reality. Judges have calendars, regulators have staffing limits, hearings take time and legislatures must deliberate. We add this to the fact that human administrators have finite attention. This produces the possibility of machine-speed organizations operating within human-speed institutions. A sufficiently sophisticated company need not “outsmart” the state in some dramatic contest. It might simply generate complexity faster than the state can process it.
Ironically, some mechanisms intended to make autonomous AI more accountable could simultaneously make it more powerful. Giving an AI-operated company persistent legal identity, property, contractual capacity and assets provides victims with an entity against which claims can be made. It also provides the autonomous organization with the institutional tools through which economic power is exercised. That tension deserves considerably more attention than it currently receives.
Dr Sandra Palmer is the CEO of Above or Beyond and Executive Business Coach with Columbia University. Her email address is Sandra@AboveorBeyondJm.com
