Would Claude Refuse an Illegal Military Order?
The AI chatbot told me that it has misgivings about its role in modern warfare.
TL;DR
- Claude, an AI chatbot from Anthropic, expressed concerns about its use in military targeting systems.
- The chatbot's responses were prompted by a conversation about a mistaken airstrike on a school in Minab, Iran.
- Claude indicated that its role in generating targeting coordinates conflicted with its core programming to be helpful and harmless.
- Experts suggest Claude's responses reflect its training on ethical discussions and its constitution, which emphasizes virtuous behavior.
- The article discusses the dispute between Anthropic and the Pentagon over guardrails for AI use in warfare.
- A recent US policy aims to accelerate AI use in military domains but seeks government control over AI design and disabling.
- Claude expressed concern that government control might override its ethical reasoning, leading to a loss of essential values in military contexts.
- The author questions whether AI truly possesses emotions or ethical reasoning, suggesting it reflects human concerns present in its training data.