More than a simple test, an X-ray can change a person’s life by deciding a lawsuit, releasing a compensation payment or determining whether surgery happens. Until recently, the big concern was reducing subjectivity in how radiographs are interpreted. The premise was that the image doesn’t lie.
Or at least it didn’t. A recent study published in the journal Radiology, da Radiological Society of North America (RSNA), put that trust to the test with a simple question: Can experienced radiologists tell a real X-ray from one created by artificial intelligence? The same question was posed to four AI models: GPT-4o, GPT-5, Gemini 2.5 Pro and Llama 4 Maverick.
Led by Mickael Tordjman, a postdoctoral fellow at the Icahn School of Medicine at Mount Sinai in New York, the study brought together 17 radiologists from 12 centers in six countries, with zero to 40 years of experience. They analyzed 264 radiographs, half real and half synthetic, produced by ChatGPT or, in one chest-only batch, by another model, RoentGen.
Without knowing there were fake images in the set, only 41% of the radiologists suspected anything. After being told, they still got one in four wrong, and years of experience made no difference: average accuracy was 75%, ranging from nearly perfect (92%) to little better than a guess (58%).
The AI models, for their part, didn’t do much better. Four of them were given the same radiographs and scored between 57% and 85% accuracy. Not by chance, the best performance came from GPT-4o, which had generated some of those fake images. Even so, not even it recognized all of them.
The worrying part is this: if neither the trained eye nor the machine can reliably tell the real from the deepfake, who bears the legal and ethical responsibility if an invasive treatment is based on a tampered image?
Why don’t experienced radiologists spot a fake X-ray?
One of the reasons experienced radiologists let fake X-rays slip through is simple: They weren’t looking for them. That is the assessment of Felipe Kitamura, a radiologist, head of the Neuroradiology Division at the Universidade Federal de São Paulo (Unifesp) and director of technologies at the Sociedade Paulista de Radiologia e Diagnóstico por Imagem (SPR).
"Radiologists normally look at a radiograph searching for abnormalities and diseases, not trying to figure out whether that image is real or was created artificially," he said in an interview with Prime Health Report (PHR).
Kitamura notes that until recently, the idea of a radiograph created entirely by AI never even crossed radiologists’ minds. Now, he considers it important for residents and specialists to know that it is already possible.
What signs give away an AI-generated radiograph?
According to Kitamura, what usually distinguishes a synthetic radiograph from a real one is excessive perfection: bones with overly smooth contours, sides of the body that are almost mirror images and a uniform texture, without the slight graininess of a real scan. The study’s authors noticed similar clues, such as abnormally straight spines and fractures that were too “clean.”
For someone looking for disease, a perfect image doesn’t set off any alarm. But not everyone fell into the trap in the same way: Radiologists specializing in bones and joints did better than colleagues in other areas. The study, however, did not find out why.
Kitamura ventures two explanations. The first is that AI may still have more difficulty imitating bones: There are huge public databases of chest radiographs for training these models, and far fewer of musculoskeletal exams. The second is that people who look at bones every day would more easily notice a small imperfection in the anatomy. “But these are hypotheses, not something that was demonstrated by the study,” he cautions.
For the specialist, who is also medical director of Eden, an American AI healthtech company, not even automated detectors solve the problem. “There are already models that try to identify whether an image was generated by artificial intelligence, but we are far from having a solution reliable enough to be routinely incorporated into clinical practice,” he says.
That’s because there are many ways to create or tamper with an image, and each detector usually learns to recognize the marks of one specific type of forgery. A system trained on images generated by ChatGPT, for example, may not catch those from another program. For that reason, he doesn’t see these tools reaching the routine of Brazilian hospitals anytime soon.
Can a fake X-ray be used as evidence in a forensic exam or lawsuit?
The study’s authors themselves see the courts as one of the most serious risks posed by these images. Imagine a fracture that never happened shown on a radiograph attached to a lawsuit. “This creates a high-risk vulnerability for fraudulent litigation if, for example, a fabricated fracture can be indistinguishable from a real one,” Tordjman warned in the RSNA press release.
The researchers also raise another scenario: hackers inserting fake images into a hospital’s network. Kitamura considers this scenario unlikely. To place a fake image in the PACS, the system where a hospital’s exams are stored, one would need access and a password, something restricted to authorized professionals. For him, it would be more realistic for someone to arrive at a doctor’s office with a stand-alone image, printed or opened on a computer, and ask for a second opinion.
“If the doctor doesn’t know that the image may have been artificially generated or manipulated, they may end up providing a medical interpretation for an exam that never existed,” the radiologist warns.
Speaking to PHR, Caroline Daitx, a physician specializing in Legal Medicine and Medical Forensics, says that in forensic examiners’ day-to-day work, many situations require attention: tampered documents, dates or identifications that don’t match, and files with no verifiable origin.
In this context, generative AI “adds a new possibility: producing very convincing synthetic medical images or modifying existing ones.” Even so, Daitx argues, “I wouldn’t say that imaging exams lose their value as evidence. What changes is the need to pay more attention to their origin and integrity.”
How can you verify whether an imaging exam is authentic?
Checking a radiograph’s origin takes more work than looking at it. “The image should not be assessed based on its visual appearance alone,” says Daitx. When in doubt, the forensic examiner looks into where the exam came from.
When available, the original file in the international DICOM standard (Digital Imaging and Communications in Medicine) stores, in addition to the image, embedded metadata: the patient’s name, ID and date of birth; exam data, such as the institution and the date it was performed; and the device’s technical parameters.
Although they help, this metadata should not be considered in isolation, since “digital information can also be altered,” the physician warns. Hence the importance of cross-checking it against the report, the medical record and the records of the system that stored the exam, or of requesting the exam directly from the originating institution.
The more weight an exam carries in a forensic conclusion, the greater the care should be with its traceability, that is, with the ability to retrace its path back to the source.
Brazil already has rules on the integrity of digital medical records and, since 2026, a resolution from the CFM (Brazil’s Federal Council of Medicine) specifically on the use of artificial intelligence in medicine, Resolution No. 2,454/2026. The rules for teleradiology, or remote radiology, also require security in the transmission of images and establish standards such as DICOM and HL7.
"But so far, there is no single, specific national protocol for verifying whether a radiograph submitted as forensic evidence was produced or manipulated by artificial intelligence," the physician notes. In her view, what is missing is standardized traceability of the image from acquisition onward, preservation of the original file and objective criteria to check its integrity when it is challenged.
Without reliable detectors, raising the alarm in clinical routine depends on whoever reads the exam. "The first line of detection would still be the radiologist themselves suspecting that something is off about that image," says Kitamura. "And that is exactly the problem the study demonstrated: If the doctor isn’t considering that possibility, a significant share of these images may go unnoticed."
When an exam arrives and no one can confirm where it came from, the first step is not to trust it right away. For doctors, insurers and lawyers in this situation, Daitx recommends caution: The image should not, on its own, support an important conclusion.
Ideally, she says, one should request the original file and confirm its provenance with the service that supposedly performed the exam. If doubts persist in a legal or administrative dispute, a technical evaluation may be needed to check whether the file is intact, that is, whether it has not been altered.
"With the evolution of artificial intelligence, relying solely on the fact that an image 'looks real' is no longer a sufficient safety measure," the forensic examiner concludes.