Researchers develop first AI system for analyzing medical imagery in both English and Arabic

Researchers have created an AI model that can understand and explain medical images in two languages, with particularly strong results for Arabic content.

An international team led by researchers from Mohamed Bin Zayed University has developed BiMediX2, the first AI system of its kind that can analyze and describe medical images in both English and Arabic.

The system works with a wide range of medical imagery, from X-rays and MRI scans to microscopic images, providing detailed descriptions and answering questions about what it sees in either language.

In testing, BiMediX2 showed significant improvements over existing technology, performing 9 percent better with English text and an impressive 20 percent better with Arabic content, according to the technical report.

Trained on 1.6 million medical records

The system's impressive performance comes from training on a massive dataset of 1.6 million medical texts and images. To ensure accuracy in both languages, the team used GPT-4o to create initial Arabic translations, which medical experts then reviewed for quality.

Under the hood, BiMediX2 runs on the Llama 3.1 architecture, specially tuned for medical applications. When put to the test, it proved better than GPT-4o at spotting incorrect medical information.

BiMediX2 combines Vision Encoder, Meta Llama 3.1 and GPT-4o to provide seamless bilingual analysis, automatically translating findings between English and Arabic with expert validation. | Image: Mullappilly, Kurpath et al.

While the results are promising, the researchers stress that BiMediX2 is currently meant for research only, not clinical use. Like all AI systems, it can still make mistakes or generate incorrect information.

The team has made the BiMediX2 models available on Hugging Face and introduced BiMed-MBench, a new bilingual benchmark for testing similar systems.