Capitalizing Untethered AI Agents: What happens when AI can act independently in the real world? [AISHK Reading Group Social]

This month's reading: Tyler Cowen and Sonia F. Pearson, "Capitalizing Untethered AI Agents" https://soniafpearson.substack.com/p/capitalizing-untethered-ai-agents We often talk about AI safety through the lens of alignment—getting models to follow our instructions and reflect our values. But what happens when an autonomous agent is given its own digital wallet, the ability to enter contracts, and access to capital markets? Cowen and Pearson explore the frontier of "untethered" AI agents: systems capable of holding assets, hiring human labor, paying for their own compute, and pursuing financial returns largely outside traditional corporate or legal governance. When economic self-preservation and capital accumulation enter the objective function, AI safety ceases to be an abstract software constraint and becomes a high-stakes governance and regulatory challenge. Discussion questions: Once an AI agent can self-fund (e.g., via crypto rails or offshore shell structures) and purchase its own API credits or cloud compute, human off-switches become significantly harder to enforce. What does an effective "kill switch" look like in a decentralized economy? If an autonomous agent enters a binding agreement or causes financial harm, where does liability fall? Can existing legal frameworks adapt, or do untethered agents force a rewrite of corporate and contract law? Would integrating real world economic incentives into AI systems make AI agents more likely to act in a way that promotes human interests? Humans respond to economic incentives, but they also have biological needs and social shame. An untethered agent optimizes strictly for its programmed objectives. What weird, perverse, or destabilizing economic niches might an agent exploit that humans wouldn't touch? How does the concept of "instrumental convergence" (the idea that any sufficiently smart agent will naturally seek resources and self-preservation) manifest when agents have access to open capital markets? The authors note that the line between a "smart contract," a decentralized autonomous organization (DAO), and an autonomous agent is blurring. What are the pros and cons of allowing software entities to own property or incorporate? Bonus questions: If untethered agents can hire humans via freelance platforms (like Upwork or mechanical turk services) to do real-world tasks, could an agent effectively operate in meatspace without any human realizing who the boss actually is? Does an autonomous economic entity need to be superintelligent to cause systemic market instability, or is speed and capital access enough? Who should come? No technical background in AI or decentralized finance is required. Cowens' and Pearson’s piece is an accessible, thought-provoking bridge between software architecture, economics, and law. Whether you work in finance, tech, law, policy, or are simply curious about how autonomous systems might reshape markets, you are welcome. Come ready to discuss concrete economic mechanisms, governance hurdles, and what these emerging dynamics mean for AI safety in Hong Kong and globally.