National Engineering Policy Centre identifies challenges for the development of autonomous systems
12/08/2020
Safety and ethics must be central to the design of autonomous systems if they are to be developed and deployed in a way that ensures their benefits are widely distributed and no one is disadvantaged, according to a briefing published by the National Engineering Policy Centre, led by the Royal Academy of Engineering (RAEng). The technology enabling autonomous systems is advancing at pace, from self-driving cars and nuclear maintenance robots to drug delivery and diagnostics, long-distance shipping and protection from cyber attacks.Autonomous systems are set to enable a productivity and export boost for the UK and could help to provide wide economic and social benefits, such as replacing dull, dirty and dangerous tasks. Engineering plays a critical role in ensuring that the solutions adopted are safe, unbiased and designed to deliver on currently unmet needs. The briefing identifies some of the key challenges and opportunities that autonomous systems present and how the engineering community can work to ensure responsible innovation. It sets out some of the common issues that can arise from allowing autonomous systems to make informed decisions in complex environments and removing people from the decision-making process.
The National Engineering Policy Centre consulted members of the public, engineers, policy makers and regulators about their expectations of autonomous systems and the role that engineers should play. The briefing identifies some key actions needed, from establishing an agile regulatory environment to understanding public opinion and ensuring professional responsibility.
The challenges identified are:
- Technical – new validation and verification methods are required alongside simulation and real-world trials to be able to ensure the safety and certainty of a system’s capabilities.
- Professional responsibility – the right standards and codes of practice are needed to drive culture change in light of new challenges posed by autonomous systems, and these must evolve as best practice emerges.
- Regulation – the question of where regulation is required to enforce standards should be carefully considered. A leading, agile and responsive UK regulatory system that can connect across the regulators of many sectors is required as autonomous systems become more widespread.
- Oversight – as autonomous systems are deployed in increasingly large and more complex environments there will be liability and authority issues. There needs to be governance in place to judge if the uncertainty and risk a system creates justifies the benefits it brings.
- Public acceptance – there should be greater public involvement before new autonomous systems are deployed, to build trust between individuals and the service providers.
- Ethics – there must be mechanisms in the design process that enable collective, reflective, transparent decision-making to resolve uncertainty, address the lack of human oversight and inform how autonomous systems are deployed.
The National Engineering Policy Centre will now explore and test these conclusions, focusing on specific sectors, including transport and healthcare. It will consider the unique ways in which autonomous systems are developing in each sector and the specific challenges to safe and ethical deployment and seek to identify emerging good practice.
Professor Nick Jennings CB FREng, Chair of the National Engineering Policy Centre’s Autonomous Systems Working Group, said: “While we have generally worked out how to automate processes safely, the complexity of decision-making that may be handed over to autonomous systems creates major new challenges. It has been fascinating to uncover the common issues across different sectors and it is clear that engineers, working in collaboration with other disciplines, have a vital role to play in ensuring that the autonomous systems we develop and deploy are safe and for the benefit of all.”
The full briefing is available here: www.raeng.org.uk/policy-new/policy-projects-and-issues/safety-and-ethics-of-autonomous-systems
