Suchman, Lucy Alice (2017) Situational awareness and adherence to the principle of distinction as a necessary condition for lawful autonomy. In: Lethal Autonomous Weapons Systems : Technology, Definition, Ethics, Law & Security. German Federal Foreign Office, Berlin, pp. 273-283.
CCW_Autonomy_Suchman.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.
Download (174kB)
Abstract
As a contribution to the CCW’s third informal meeting of experts on lethal autonomous weapon systems (LAWS), this briefing paper focuses on the implications of the requirement of situational awareness for autonomous action – whether by humans, machines or complex human-machine systems. For the purposes of this paper, ‘autonomy’ refers to self-directed action, and more specifically the action-according-to-rule that comprises military discipline. Unlike the algorithmic sense of a rule as that term is used in Artificial Intelligence (AI), military rules always require interpretation in relation to a specific situation, or situational awareness. Focusing on the principle of distinction, I argue that International Humanitarian Law (IHL) presupposes capacities of situational awareness that it does not, and cannot, fully specify. At the same time, autonomy or ‘self-direction’ in the case of machines requires the adequate specification (by human designers) of the conditions under which associated actions should be taken. This requirement for unambiguous specification of condition/action rules marks a crucial difference between autonomy as a legally accountable human capacity, and machine autonomy. The requirement for situational awareness in the context of combat, as a prerequisite for action that adheres to IHL, raises serious doubts regarding the feasibility of lawful autonomy in weapon systems.