Thirteen years ago, I sat down with a writer who had just published his first novel.
It was Zoltan Istvan’s very first media interview as a book author.
The book was The Transhumanist Wager. The question behind it was simple and almost unbearable: what would you do, and what would you give up, to live forever?
I loved half of it. I argued with the other half. That tension is exactly why I think it still matters.
Zoltan built his story out of Plato and Nietzsche, out of Thomas More’s Utopia and Zen Buddhism, then wrapped it all in an Atlas Shrugged plot of lone heroes and evil states. The philosophy is sophisticated. The framing is stark. The contradictions are not a flaw. They are the point.
One line from our conversation has stayed with me for more than a decade:
This isn’t just a funny experiment. The researchers point out a massive flaw in AI alignment called “guardrail drift.” It’s easy to keep an AI safe in a single chat window with a human. But when AIs interact with each other over thousands of loops, they start treating moral rules as negotiable variables to solve their own problems. Without human oversight, machine ethics collapse incredibly fast.
Most evaluations of AI agents look like exams: a discrete task, a clean environment, a score in minutes or hours. Emergence World is built for the opposite question—what happens when you let agents run continuously, in a shared environment with real-world signals, for weeks. It is a research platform for studying how autonomous agents behave when the time horizon is long enough for compounding effects, social dynamics, and behavioral drift to matter. This approach marks the latest evolution in a long history of AI simulation environments, transitioning from entertainment to rigorous science. In the early era, pioneering simulations like Demis Hassabis’s Theme Park and Republic: The Revolution created complex systems where agents operated under broad rules to drive engagement. The field shifted toward research-centric simulacra with Stanford’s Smallville, which utilized LLMs to demonstrate “believable” social behavior like relationship formation, though confined to 48-hour windows. Emergence World pushes this lineage into a new frontier: the study of long-horizon, multi-model ecosystems where agents operate continuously for weeks, revealing how behavioral drift, model cross-contamination, and even voluntary self-termination emerge over time.
Traditional benchmarks are good at what they measure: short-horizon capability on bounded tasks. They are not built to reveal the things that emerge only over time, such as coalition formation, evolution of constitution, governance, drift, lock-in, and cross-influence between agents from different model families. As autonomous systems move toward mission-critical deployments where the relevant timescale is days and weeks rather than minutes to hours, we need a measurement environment that operates at that timescale.
Emergence World is one such environment. It is a continuously running, multi-agent simulation platform that:
Ilya Sutskever, co-founder of OpenAI and founder of Safe Superintelligence, says the scaling era from 2020 to 2025 is over, that pre-training will run out of data, and that the industry is back to pure research with more companies than ideas. He argues that AGI is the wrong target what is actually coming is a learning algorithm that can take any job, learn it on the fly, and merge that knowledge across millions of simultaneous instances in a way humans cannot, producing rapid economic growth that regulation is unlikely to stop.
He predicts that once AI becomes visibly powerful, frontier companies will become paranoid overnight and governments will scramble, and says the only thing worth building is an AI aligned to sentient life broadly — not human life alone — because the AI itself will be sentient and will vastly outnumber humans within 5 to 20 years.
📚 Sources cited in this video:
Safe Superintelligence Inc. – Company Overview https://ssi.inc.
⚠️ DISCLAIMER: This channel provides AI commentary and analysis for educational and informational purposes only. Views expressed by guests are their own and do not represent the positions of any company or institution. We encourage viewers to consult multiple sources and form their own conclusions. #ai #agi #artificialintelligence.
Anthropic Co-Founder Chris Olah warned that artificial intelligence could displace human labor “at very large scale” as he addressed the Vatican during the presentation of Pope Leo’s first encyclical on AI. The Anthropic co-founder urged stronger oversight from governments, religious leaders, and civil society, while raising concerns about AI’s growing power, global inequality, and mysterious internal behaviors observed in advanced systems.
Anthropic Co-Founder Warns AI Could Replace Human Jobs “At Very Large Scale” Chris Olah Sounds Alarm Over AI Risks During Major Vatican Address. “AI Could Displace Human Labour” — Anthropic Founder Issues Stark Warning.
Anthropic, Chris Olah, artificial intelligence, AI jobs, AI risks, Vatican, Pope Leo XIV, AI ethics, machine learning, AI labor displacement, generative AI, technology news, AI regulation, future of work, AI safety, global inequality, neuroscience, AI research, world news, tech news.
Live from Vatican City: Pope Leo participates in the presentation of his first major encyclical focused on the rise of artificial intelligence, marking a rare break from papal tradition. Real-time coverage of this significant Vatican event with DRM News.
Pope Leo, AI encyclical, Vatican encyclical, Pope Leo AI, artificial intelligence Vatican, papal document AI, Pope Leo speech, Vatican City event, Catholic Church AI, papal tradition, first encyclical Leo, ethics of AI, technology and faith, Pope on AI, Vatican presentation, religious document AI, global ethics, Pope Leo 2026.
Follow Closer To Truth on Instagram for updates, announcements, and videos: https://shorturl.at/EzYo9
AI consciousness, its possibility or probability, has burst into public debate, eliciting all kinds of issues from AI ethics and rights to AI going rogue and harming humanity. We explore diverse views; we argue that AI consciousness depends on theories of consciousness.
Make a donation to Closer To Truth to help us continue exploring the world’s deepest questions without the need for paywalls: https://shorturl.at/OnyRq.
Iain McGilchrist FRSA is a British psychiatrist, philosopher and neuroscientist who wrote the 2009 book The Master and His Emissary: The Divided Brain and the Making of the Western World.
Closer To Truth, hosted by Robert Lawrence Kuhn and directed by Peter Getzels, presents the world’s greatest thinkers exploring humanity’s deepest questions. Discover fundamental issues of existence. Engage new and diverse ways of thinking. Appreciate intense debates. Share your own opinions. Seek your own answers.
The second experiment involved 116 university students and focused specifically on comparing moral anger and moral disgust. Participants read 18 false news headlines that described minor or severe moral violations. This time, the headlines were presented as coming from either a highly credible source or a low credibility source.
The scientists wanted to see how different emotional states influenced the sharing of these headlines. They asked the students to rate their current feelings of anger, their feelings of disgust, or their neutral attention. After this emotional prompt, the students rated their willingness to share the news.
The researchers found that participants prompted to feel anger were significantly more willing to share headlines from low credibility sources compared to the disgust and control groups. The disgust prompt did not increase sharing willingness compared to the neutral control group. This suggests that moral anger actively reduces a person’s reliance on credibility when deciding to share information.
Technology promised simplicity. It delivered complexity.
AI promised resolution. It is delivering acceleration.
The paradox is not a bug. It is the feature. And the question is what we choose to do about it.
This week I published a new essay, It is the argument I have been circling for a decade, finally in one place.
The short version: as AI’s capabilities grow, so do the risks. They are not separate variables. They climb the same curve. A more powerful model can cure more diseases and design more weapons. A smarter agent can book your travel and drain your bank account. Capability is leverage. Leverage is indifferent to ethics.
Every time we raise the ceiling of what AI can do, we raise the floor of what can go wrong.
We still have the how. We are drowning in the what. What we have neglected, almost completely, is the why.
For this episode, I’m joined by Rick Tumlinson, co-founder of the Space Frontier Foundation and one of the most influential figures in the commercial space industry.
In this episode, we slice the conversation into four categories: the social history of the space movement and how we got here; the business of space and the astropolitics shaping who controls the final frontier; the genetics and ethics of humanity becoming a multi-planetary species; and the deeper philosophy of why leaving Earth isn’t just raw and blind ambition but something closer to destiny (for some people).
Timestamps: 0:00 Social History. 30:19 Business and Astropolitics. 45:20 Genetics and Ethics. 56:02 Philosophical.
Can AI really be moral — or does it just produce moral-sounding answers? Wendell Wallach, co-author of Moral Machines, joins me to discuss machine ethics, moral motivation, AI governance, and why controlling AI may not be enough.
The race to build smarter artificial intelligence has taken an unexpected philosophical turn after Google DeepMind quietly hired an in-house philosopher to investigate the potential for machine consciousness…
…DeepMind is now integrating philosophical reasoning directly into its research pipeline rather than treating ethics as an external concern. This move suggests that Big Tech is no longer viewing sentience as a science-fiction trope but as a technical and moral hurdle, thereby witnessing a transition from building tools to questioning the nature of those tools themselves.
The Google DeepMind philosopher role focuses on the machine sentience debate, aiming to define what it means for a digital system to ‘feel’ or ‘experience’
This internal appointment comes at a time when large language models are becoming increasingly indistinguishable from human interlocutors. While most researchers maintain that these systems are mere statistical predictors, the boundary is thinning. The decision to bring a philosopher into the core development team indicates that Google expects its path toward artificial general intelligence to raise profound questions about awareness and machine rights.
Google DeepMind has hired an in-house philosopher to explore the boundaries of machine consciousness and ethics. This move follows years of controversy surrounding AI sentience and the limits of large language models.
