Karen Hao on AI, AGI, and the Price of Conviction

In the ever-evolving world of artificial intelligence few voices are as insightful and critical as Karen Hao. Her work delves deep into the ethical and societal implications of AI challenging the narratives often presented by tech evangelists. This post explores Hao’s perspectives on the empire of AI the fervent believers in artificial general intelligence AGI and the potential costs associated with their unwavering convictions.

Understanding the Empire of AI

• Karen Hao is a journalist with expertise in AI’s societal impact. She’s written for MIT Technology Review The Atlantic and others. Penguin Random House
• Her book Empire of AI: Dreams and Nightmares in Sam Altman’s OpenAI published May 20, 2025 examines the rise of OpenAI its internal culture its shifting mission and how it illustrates broader trends in the AI industry.

What Empire of AI Means in Hao’s Critique

Hao frequently uses empire as a metaphor and sometimes more than a metaphor to describe how AI companies especially OpenAI amass power resources and influence in ways that resemble historical empires. Some of the traits she identifies include:

1. Claiming resources not their own: especially data that belongs to or is produced by millions of people without clear consent.
2. Exploiting labor: particularly in lower-income countries for data annotation content moderation often under poor working conditions.
3. Monopolization of knowledge production: The best AI researchers increasingly concentrated in big private companies academic research being subsumed filtered or overshadowed by company-oriented goals.
4. Framing a civilizing mission or moral justification: much like imperial powers did the idea that the company’s growth or interventions are for the greater good progress or saving humanity or advancing scientific discovery.
5. Environmental and resource extraction concerns: Data centers energy and water usage environmental consequences in places like Chile etc.

Key Arguments & Warnings Hao Raises

• That the choices in how AI is developed are not inevitable but the result of specific ideological and economic decisions. They reflect certain value systems often prioritizing scale speed profit dominance over openness or justice.
• That democratic oversight accountability and transparency are lagging. The public tends to receive partial narratives marketing and hype rather than deep context about what trade-offs are being made.
• That there are hidden costs environmental impacts labor exploitation unequal distribution of gains. The places bearing the brunt are often outside of Silicon Valley Global South lower-income regions.
• That power is consolidating certain entities OpenAI and its peers are becoming more powerful than many governments in relevant dimensions compute data control narrative control. This raises questions about regulation public agency and who ultimately controls the future of AI.

Possible Implications

• A demand for more regulation and oversight to ensure AI companies are accountable not just economically but socially and environmentally.
• Growing public awareness and potentially more pushback around the ethics of data usage labor practices in AI environmental sustainability.
• A need for alternative models of AI development: ones that emphasize fairness shared governance perhaps smaller scale or more distributed power rather than imperial centralization.

• Data Dominance: Companies amass vast datasets consolidating power and potentially reinforcing existing biases.
• Algorithmic Control: Algorithms govern decisions in areas like finance healthcare and criminal justice raising concerns about transparency and accountability.
• Economic Disruption: Automation driven by AI can lead to job displacement and exacerbate economic inequality.

A significant portion of the AI community believes in the imminent arrival of AGI a hypothetical AI system with human-level cognitive abilities. These AGI evangelists often paint a utopian vision of the future where AI solves humanity’s most pressing problems.

Hao urges caution emphasizing the potential risks of pursuing AGI without carefully considering the ethical and societal implications. She challenges the assumption that technological progress inevitably leads to positive outcomes.

The Cost of Belief

Unwavering belief in the transformative power of AI can have significant consequences according to Hao. It can lead to:

• Overhyping AI Capabilities: Exaggerated claims about AI can create unrealistic expectations and divert resources from more practical solutions.
• Ignoring Ethical Concerns: A focus on technological advancement can overshadow important ethical considerations such as bias privacy and security.
• Centralization of Power: The pursuit of AGI can concentrate power in the hands of a few large tech companies potentially exacerbating existing inequalities.