New study highlights the problems that can arise when data published for one task are used to train algorithms for a different one —


Important advances in synthetic intelligence (AI) over the previous decade have relied upon intensive coaching of algorithms utilizing large, open-source databases. However when such datasets are used “off label” and utilized in unintended methods, the outcomes are topic to machine studying bias that compromises the integrity of the AI algorithm, in keeping with a brand new research by researchers on the College of California, Berkeley, and the College of Texas at Austin.

The findings, revealed this week within the Proceedings of the Nationwide Academy of Sciences, spotlight the issues that come up when knowledge revealed for one activity are used to coach algorithms for a distinct one.

The researchers observed this subject after they failed to duplicate the promising outcomes of a medical imaging research. “After a number of months of labor, we realized that the picture knowledge used within the paper had been preprocessed,” mentioned research principal investigator Michael Lustig, UC Berkeley professor {of electrical} engineering and laptop sciences. “We needed to lift consciousness of the issue so researchers may be extra cautious and publish outcomes which are extra sensible.”

The proliferation of free on-line databases through the years has helped help the event of AI algorithms in medical imaging. For magnetic resonance imaging (MRI), specifically, enhancements in algorithms can translate into sooner scanning. Acquiring an MR picture includes first buying uncooked measurements that code a illustration of the picture. Picture reconstruction algorithms then decode the measurements to provide the photographs that clinicians use for diagnostics.

Some datasets, such because the well-known ImageNet, embody thousands and thousands of photos. Datasets that embody medical photos can be utilized to coach AI algorithms used to decode the measurements obtained in a scan. Research lead writer Efrat Shimron, a postdoctoral researcher in Lustig’s lab, mentioned new and inexperienced AI researchers could also be unaware that the information in these medical databases are sometimes preprocessed, not uncooked.

As many digital photographers know, uncooked picture information comprise extra knowledge than their compressed counterparts, so coaching AI algorithms on databases of uncooked MRI measurements is essential. However such databases are scarce, so software program builders generally obtain databases with processed MR photos, synthesize seemingly uncooked measurements from them, after which use these to develop their picture reconstruction algorithms.

The researchers coined the time period “implicit knowledge crimes” to explain biased analysis outcomes that end result when algorithms are developed utilizing this defective methodology. “It is a straightforward mistake to make as a result of knowledge processing pipelines are utilized by the info curators earlier than the info is saved on-line, and these pipelines usually are not all the time described. So, it is not all the time clear which photos are processed, and that are uncooked,” mentioned Shimron. “That results in a problematic mix-and-match method when creating AI algorithms.”

Too good to be true

To show how this apply can result in efficiency bias, Shimron and her colleagues utilized three well-known MRI reconstruction algorithms to each uncooked and processed photos based mostly on the fastMRI dataset. When processed knowledge was used, the algorithms produced photos that had been as much as 48% higher — visibly clearer and sharper — than the photographs produced from uncooked knowledge.

“The issue is, these outcomes had been too good to be true,” mentioned Shimron.

Different co-authors on the research are Jonathan Tamir, assistant professor in electrical and laptop engineering on the College of Texas at Austin, and Ke Wang, UC Berkeley Ph.D. pupil in Lustig’s lab. The researchers did additional assessments to show the consequences of processed picture information on picture reconstruction algorithms.

Beginning with uncooked information, the researchers processed the photographs in managed steps utilizing two frequent data-processing pipelines that have an effect on many open-access MRI databases: use of economic scanner software program and knowledge storage with JPEG compression. They educated three picture reconstruction algorithms utilizing these datasets, after which they measured the accuracy of the reconstructed photos versus the extent of knowledge processing.

“Our outcomes confirmed that every one the algorithms behave equally: When applied to processed knowledge, they generate photos that appear to look good, however they seem totally different from the unique, non-processed photos,” mentioned Shimron. “The distinction is very correlated with the extent of knowledge processing.”

‘Overly optimistic’ outcomes

The researchers additionally investigated the potential threat of utilizing pre-trained algorithms in a scientific setup, taking the algorithms that had been pre-trained on processed knowledge and making use of them to real-world uncooked knowledge.

“The outcomes had been putting,” mentioned Shimron. “The algorithms that had been tailored to processed knowledge did poorly after they needed to deal with uncooked knowledge.”

The photographs could look wonderful, however they’re inaccurate, the research authors mentioned. “In some excessive circumstances, small, clinically essential particulars associated to pathology may very well be utterly lacking,” mentioned Shimron.

Whereas the algorithms would possibly report crisper photos and sooner picture acquisitions, the outcomes can’t be reproduced with scientific, or uncooked scanner, knowledge. These “overly optimistic” outcomes reveal the danger of translating biased algorithms into scientific apply, the researchers mentioned.

“Nobody can predict how these strategies will work in scientific apply, and this creates a barrier to scientific adoption,” mentioned Tamir, who earned his Ph.D. in electrical engineering and laptop sciences at UC Berkeley and was a former member of Lustig’s lab. “It additionally makes it troublesome to check varied competing strategies, as a result of some may be reporting efficiency on scientific knowledge, whereas others may be reporting efficiency on processed knowledge.”

Shimron mentioned that revealing such “knowledge crimes” is essential since each business and academia are quickly working to develop new AI strategies for medical imaging. She mentioned that knowledge curators might assist by offering a full description on their web site of the strategies used to course of the information of their dataset. Moreover, the research presents particular pointers to assist MRI researchers design future research with out introducing these machine studying biases.

Funding from the Nationwide Institute of Biomedical Imaging and Bioengineering and the Nationwide Science Basis Institute for Foundations of Machine Studying helped help this analysis.