We need to talk about uncertainty

Happy New Year! Have you made any New Year’s resolutions? Or have you deferred writing your list due to the uncertainty of what might happen with COVID and the various restrictions? Back at the start of the first lockdown, I wrote about the challenge of dealing with uncertainty and since then we have had to deal with uncertainty after uncertainty. Once the first lockdown was lifted, we were able to better plan events and possible travel, but then along came another. This summer, the cycle began again and we felt a bit more confident as events took place and people ventured out on holidays. However, there was still an air of uncertainty, especially coming into winter. Sure enough, a new variant has appeared and new restrictions have been imposed. As I write this, there is continued uncertainty in the scientific community about what might happen and whether further restrictions could be imposed. We are back in the situation where, as I wrote previously, there are any number of scenarios that could come to fruition and the impact of trying to cope with this is more worry and stress.

Going beyond COVID, there are other significant uncertainties in the world, albeit with potentially less immediate concerns for our day-to-day decisions until we feel their impact: climate change; economic volatility; and political instability. We continuously face some level of uncertainty, which can upset our immediate plans. The most common year-round variable in the UK is the weather, notwithstanding the recent named storms, which caused havoc in some areas of the country. Will it be wet or dry? The weather forecast may address this qualitatively with such phrases as ‘occasional showers throughout the morning’. Weather apps take a more quantitative approach, giving a percentage chance of rain on an hourly basis. Unless the value is either very high or very low it doesn’t really help you any more than the qualitative statement above.

One way in which we often deal with uncertainty is to ignore it and carry on with the confidence that everything will be fine. To some extent this is what we do in non-destructive testing (NDT). We generally avoid talking about uncertainty. When and where we do address the topic, we tend to use alternative terms such as probability of detection (POD) or reliability. While everyone in NDT is aware that there are errors associated with detection, characterisation and sizing, we tend to keep it to ourselves. So, when non-NDT personnel read a report stating ‘no defects’ they are left to believe that there are definitely no defects at all. When they read that ‘the size of a defect is 5 mm’, we do not necessarily inform them of the caveats behind such a statement.

I had a look through a number of ISO/EN/BS standards for ultrasonic testing and was unable to find any reference to uncertainty. Recently, we had an external audit of mechanical testing against the requirements of BS EN ISO/IEC 17025:2017. This standard is titled: ‘General requirements for the competence of testing and calibration laboratories’. Some NDT laboratories adhere to it, while other vendors apply different quality management systems. However, this particular standard mentions uncertainty in the context of: the selection, verification and validation of methods; technical records; the evaluation of uncertainty; and the requirements of test reports. Two requirements for including uncertainty in test reports, which are particularly applicable to NDT, are where it is relevant to the validity or application of the test results (for example its use in asset management decisions) and where the uncertainty affects conformity to a specification limit (for example the application of acceptance levels).

During the above audit, the auditor brought to my attention some documents available on the National Association of Testing Authorities (NATA), Australia, website. There are two recent documents in particular that address uncertainty in NDT. Both are in regard to the application of the ISO/EC 17025 standard for infrastructure and asset integrity. The first document runs through the criteria in the standard, highlighting that NDT involves both qualitative and quantitative uncertainty elements, that the former can be of greater significance and also that NDT methods have a degree of subjectivity. But this does not mean that it can be ignored: ISO/EC 17025 requires an estimation based on theoretical principles or practical experience when a rigorous evaluation is not possible. The second NATA document provides guidance on calculating quantitative uncertainty for different NDT methods and hopes that the information can show that NDT methods are not straightforward. This document also references the Health and Safety Executive (HSE) guidance on ultrasonic sizing errors (www.hse.gov.uk/comah/sragtech/ndt4.pdf) produced subsequent to the Programme for the Assessment of NDT in Industry (PANI) project.

Uncertainty as a subject will only become more important with the application of new digital technologies and so it is incumbent on us all to make sure we are up to speed on the topic. In the meantime, I hope you have enough certainty to make one or two resolutions for 2022.

Please note that the views expressed in this column are the author’s own personal ramblings for the purpose of encouraging discussion within NDT News. They do not represent the views of Jacobs or BINDT.

Letters can be mailed to The Editor, NDT News, Midsummer House, Riverside Way, Bedford Road, Northampton NN1 5NX, UK. Fax: +44 (0)1604 438301; Email: ndtnews@bindt.org or email Bernard McGrath direct at bernard.mcgrath1@jacobs.com

Comments by members

This forum post has no comments, be the first to leave a comment.

<< Back