"6+" or "Above six"? A simple error, apparently unnoticed for years, may have compromised a key autism screening guideline from the U.K.'s National Institute for Health and Care Excellence.

NICE's clinical guidelines, which influence treatment around the world, include a 10-item scale called the Autism Spectrum Quotient. Physicians are meant to use it in conjunction with their professional judgment to screen patients who warrant further assessment for the condition. 

But NICE's autism guideline apparently adopted the wrong cutoff. Instead of advising physicians to screen patients who score six and above — the recommendation of the Cambridge scientists who first developed the tool — the guideline recommended them to screen those who scored "above six." In a simple, 10-item test, this wording could be interpreted to mean seven or more. 

"Initially, my thoughts were of surprise and shock," Punit Shah, a psychologist at the University of Bath and an author of the peer-reviewed correspondence that detailed the error, published March 18 in The Lancet, told The Academic Times. "In more depth, it's worrying. NICE is a highly respected body, and its guideline informs clinical practice around the world. The existence of such a major, major mistake — the fact that there wasn't a checking of this — is quite worrying, and it makes you think, 'Well, are there other mistakes?' There's a broader issue here of investigating what happened to make sure it doesn't happen again."

Gillian Leng, chief executive of NICE and a visiting professor at University College London, told The Academic Times, "NICE is currently reviewing our autism guidelines and during a consultation asked health professionals to what extent they rely on AQ-10 when making decisions to offer a full autism assessment."

"As per the current recommendation in our guideline, they told us they do not use the AQ-10 test in isolation but alongside their clinical judgement," Leng said. "NICE is currently looking at new evidence and will of course update our recommendations where necessary."

Shah thinks patients who only scored six may have been mistakenly diverted from care. "The screening value is of six and above, and it's a value that maximizes how sensitive the tool is, but also makes it as specific as it can possibly be," Shah explained. 

"In the original research when the tool was developed, it was quite clearly stated that the screening value should be six and above," he said. "But in the actual NICE guidelines, for whatever reason — I can only put it down to human error — a score above six triggers a clinical referral."

Simon Baron-Cohen, the Cambridge clinical psychologist who created the tool, says six was definitely the cutoff in his 2012 publication — "This is crystal clear." But he doesn't think "erroneous" is the right word for the language in the NICE guideline. "As far as we know, NICE has not recommended 7 as the cut-point," he said.

Carrie Allison, who coauthored the 2012 publication with Baron-Cohen, pointed out that the AQ-10 document on NICE's website does, in fact, include the correct "more than 6 out of 10" language. "I hope this can be resolved swiftly with NICE to remove any ambiguity that remains and that nobody has been adversely affected," she added.

"There was an error," said Catherine Lord, a psychology professor and autism researcher at UCLA, when reached for comment. "The statement that six was the cutoff was not clear and so some people have interpreted as above six versus others interpreted as at six or higher."

Tim Nicholls, head of policy at the U.K.'s National Autistic Society, said, "This is concerning and we have contacted NICE to ask for clarification. A diagnosis can be life-changing. It can explain why you've always felt different, help you get vital support and take control of your life."

"We use the AQ10 as part of our own research," Shah, of the University of Bath, explained. "We knew the cutoff to be people scoring six or above — but one of my Ph.D. students and a coauthor of the paper [Rachel A. Clutterbuck] found this discrepancy. When we looked at the literature more closely, then it became abundantly clear that people had used different cutoffs." 

Shah added that NICE's own evidentiary appendices showed that they had identified the correct cutoff before mistakenly using the wrong standard in their published guidelines.

In an article for The Conversation, Shah and his colleagues mentioned research from Harvard Medical School and other institutions that may have used the inaccurate standard. "People will have made mistakes using this tool," Shah said. "I'm surprised there hasn't been more of a discussion about it so far, and that might be because people are realizing where they made mistakes using it."

"This mentions a study where they used 7 or more out of 10 as an inclusion criterion," Baron-Cohen said. "Research studies are free to set their own inclusion and exclusion criteria and that allows independent groups to test if results replicate when the same criteria are used." 

Allison agreed, adding, "I would also expect researchers to go to the original research paper in order that they can understand the context in which the study was conducted, how the cut-points are determined and how the results are interpreted."

Shah says it is impossible to know how many people may have missed out on treatment. "We just can't really know about the extent to which the guidelines have informed this process, because the AQ tool isn't the only thing that clinicians use to make their decisions," he explained. "It's not as straightforward as saying, 'Well, this is the precise number.' The direction of the mistake means we can infer that people are more likely to have missed the diagnosis rather than been overly referred."

Baron-Cohen noted, "I would be more concerned if there was evidence of possible autistic people being turned away by their doctors for scoring 6 out of 10, as this would indicate some people being deprived of the opportunity for a full diagnostic assessment. I haven't heard of a report like this and I would hope that GPs/family physicians were using some discretion in making a referral based on symptoms, not just a score on a screening measure."

"The new publication in The Lancet is helpful in identifying an ambiguity and this can be usefully tightened up," he added. "We are not aware that this ambiguity has affected research, as researchers tend to cite the paper attached which is very clear."

According to Nicholls, of the National Austistic Society, "Screening tools are not definitive and professionals should not rely too heavily on any single tool. But they can be useful for giving an indication that you could be autistic, and whether you should pursue a diagnosis. So it's essential that any tool or test is accurate and consistent."

Baron-Cohen says he intends to write to NICE and recommend they reword the guidance to "equal to or more than 6 out of 10" so as to avoid any ambiguity.

"The language used in the guidelines perhaps could have been more explicit about including scores of 6 and we hope to communicate this to the NICE committee," said Bonnie Auyeung, a psychology researcher at the University of Edinburgh and a coauthor of the 2012 paper with Baron-Cohen.

"It's unfortunate, but hopefully, this will be rectified soon," Lord, of UCLA, said. "I don't think many people in the U.S. have even heard of NICE. What is good about NICE is that it's a centralized effort to identify standards, but it has its own politics and problems."

Shah, too, hopes that the changes will be made rapidly. Until then, he says, clinicians and the general public must be alerted to the error.

"With these large, semi-governmental bodies, often the timetable can slow down progress, whereas rapid publication and the news cycle can help to raise awareness in the meantime," Shah said. "It was important to publish this as quickly as possible."

The correspondence, "Erroneous NICE guidance on autism screening," published March 18 in The Lancet, was authored by Lucy H Waldren, Rachel A Clutterbuck, and Punit Shah, University of Bath.