In Week #180 of the Doctor Penguin newsletter, the following papers caught our attention:
1. Multimodal Transformer. Most current multimodal medical AI approaches lack a unified method to integrate features from different sources.
Zhou et al. developed a transformer-based model called IRENE that directly learns holistic representations from multimodal inputs (images, unstructured and structured text) rather than learning or concatenating modality-specific features. IRENE uses intra-directional and bidirectional intermodal attention to explicitly capture the connections between different modalities. In evaluations, IRENE outperformed an image-only model and non-unified multimodal diagnosis models in pulmonary disease detection (by 12% and 9%, respectively) and in adverse clinical outcome prediction in patients with COVID-19 (by 29% and 7%, respectively). IRENE also outperformed junior physicians in diagnosing 8 pulmonary diseases and performed comparable to senior physicians in 6 pulmonary diseases. These results demonstrate that the token-level bidirectional multimodal attention in IRENE effectively utilizes limited multimodal medical data and leverages complementary semantic information.
Nature Biomedical Engineering
Read paper
2. Physics-Informed Neural Network. Physics-informed neural networks (PINNs) effectively solve complex engineering problems using limited data. They achieve this by incorporating the underlying physics laws, described by partial differential equations (PDEs), into the loss function. By doing so, the models learn to minimize the loss function that accounts for additional physical constraints.
Sel et al. developed PINN models for the modeling of physiological time series for cuffless blood pressure estimation. They defined PDEs that describe cardiovascular phenomena with gradual changes and incorporated them into the loss function. The PINN models retain strong correlations (systolic: 0.90, diastolic: 0.89) and low error rates (systolic: 1.3 ± 7.6 mmHg, diastolic: 0.6 ± 6.4 mmHg) while significantly reducing the required amount training data by a factor of 15, compared to other state-of-the-art time series regression models like random forest regressor and transformer.
npj Digital Medicine
Read paper
3. Deployment. Unlike innovations such as computed tomography and magnetic resonance imaging that spread from high-income countries to low-income settings, radiology AI suggests an opposite trajectory.
In this perspective, Jha and Topol point out that radiology AI has yet to be widely implemented in many high-income countries, while it has seen rapid adoption in certain low-income and middle-income settings. They explain that this is because AI in radiology is driven by commercial and profit-focused models in high-income countries, where vendors must prove that AI brings financial benefits or reduces healthcare costs. On the other hand, in some low-income and middle-income settings where public and philanthropic funding is involved, the adoption of AI is primarily based on medical needs rather than financial returns.
The Lancet
Read paper
4. Deployment. What factors influence healthcare professionals' acceptance of AI in the hospital setting?
Lambert et al. conducted an integrative review of 42 articles to explore barriers and facilitators in this regard. Among the reasons for limited acceptance, various factors were identified. These include personal fears about losing professional autonomy, lack of integration in clinical workflow and routines, overly sensitive settings for alarm systems, and loss of patient contact. Technical reservations, such as unintuitive user interfaces and limitations due to weak internet connections, also impede the comprehensive usage and acceptance of AI. Moreover, human factors, such as personality and experience, play a role in shaping the perception of AI systems. To promote acceptance and implementation of AI systems in clinical settings, it is crucial to integrate them into existing routines and workflows, while providing adequate training and education to healthcare professionals.
npj Digital Medicine
Read paper
5. Sustainability. There is noticeable upward trend in the size, complexity, memory requirements, and training costs of AI/ML models.
Jia et al. analyze this trend and emphasize the resource sustainability challenges in AI/ML for healthcare. They point out that even leading hardware's energy efficiency and computing power struggle to keep up with the increasingly complex deep models required for improved accuracy and broader usage. Limited storage capacity and slow data transmission pose critical challenges in healthcare AI. Additionally, the growing number of medical images needed for training surpasses the capacity of domain experts responsible for labeling and verification. To address these issues, the authors suggest using pre-trained models to reduce resource consumption, employing model compression techniques like pruning to find the most efficient networks, utilizing self-supervised learning to minimize labeling expertise, adopting federated learning to distribute tasks and balance workload among participants, and implementing decentralized storage systems to safeguard data privacy while mitigating sustainability challenges.
Nature Machine Intelligence
Read paper
-- Emma Chen, Pranav Rajpurkar & Eric Topol