New training approach brings extra precision to medical AI practices

Artificial intelligence models can often excel at detecting patterns in images, sometimes surpassing human abilities. However, there are situations where a radiologist using an AI model to assess X-rays for pneumonia diagnosis must determine when to rely on the model's guidance and when to disregard it.

MIT and the MIT-IBM Watson AI Lab have proposed a tailored onboarding process to address this challenge. They have devised a system that instructs users on when to collaborate effectively with an AI assistant.

In this scenario, the training method identifies instances in which the radiologist might mistakenly trust the AI's advice when it's incorrect. The system autonomously formulates guidelines for optimal AI collaboration and communicates them in natural language.

During the onboarding phase, the radiologist practices collaborating with artificial intelligence through training exercises based on these guidelines, receiving feedback on both her and the AI's performance.

The researchers observed that this onboarding approach resulted in approximately a 5 percent enhancement in accuracy when humans and AI collaborated on image prediction tasks. Conversely, merely informing the user when to trust the AI without proper training yielded inferior results.

Crucially, the researchers' system operates entirely autonomously, utilizing data from both humans and AI as they perform a specific task to construct the onboarding process. It also possesses the flexibility to adapt to various tasks, making it versatile and applicable in numerous scenarios where humans collaborate with AI models, such as social media content moderation, writing, and programming.

It's common for people to receive AI tools without any accompanying training to assist them in discerning when these tools will be beneficial. This stands in contrast to most other tools, which typically come with instructional tutorials. Researchers are addressing this gap, tackling the problem methodologically and behaviorally.

The researchers foresee that such onboarding processes will become essential components of training for medical professionals. For instance, doctors who rely on AI for treatment decisions may undergo training similar to the one proposed. This might necessitate a reevaluation of practices ranging from continuing medical education to the design of clinical trials.

How does the onboarding technique work?

Traditional onboarding methods for human-AI cooperation often rely on training materials created by human experts for specific applications, limiting their scalability. Some alternative approaches involve using explanations, where the AI communicates its confidence in each decision, although research suggests that explanations are seldom beneficial.

Considering that the capabilities of AI models are continually advancing, expanding the potential areas where humans can benefit from them, and users' perceptions of these models are constantly evolving, researchers require an adaptable training process.

To achieve this, their onboarding technique is autonomously developed using data. It is constructed from a dataset containing numerous instances of a task, such as identifying the presence of a traffic light in a blurry image.

The system's initial step involves gathering data on the human and AI's performance on this task. In this scenario, the human, with the AI's assistance, attempts to predict whether blurry images contain traffic lights.

The system maps these data points onto a latent space, representing data where similar points are closely grouped together. It utilizes an algorithm to identify regions within this space where the human collaborates incorrectly with the AI. These regions highlight instances where the human trusted the AI's prediction erroneously and, conversely, where the human's prediction was correct but mistrusted by the AI.

For instance, humans might mistakenly rely on AI's judgment when viewing images depicting a nighttime highway.

Once these regions are identified, a secondary algorithm employs a large language model to articulate each region as a rule using natural language. The algorithm continually refines these rules by identifying contrasting examples. For example, it might describe a region as "disregard AI in cases of nighttime highways."

These rules serve as the foundation for constructing training exercises. The onboarding system presents an example to the human, such as a blurry nighttime highway scene, alongside the AI's prediction, and inquires whether the image contains traffic lights. The user can respond with 'yes,' 'no,' or defer to the AI's prediction.

In cases where the human's response is incorrect, they are provided with the correct answer and performance metrics for both the human and AI in these task instances. This process is repeated for each identified region, and after the training regimen, the user revisits the exercises where they previously erred.

Ultimately, this approach equips the human with valuable insights into these regions, fostering the expectation of more accurate predictions in the future.

AI is getting more accurate every day

The effectiveness of onboarding in enhancing accuracy was put to the test by the researchers across two tasks: identifying traffic lights in blurry images and answering multiple-choice questions spanning various domains, such as biology, philosophy, and computer science.

Initially, users were presented with a card containing information about the AI model, its training, and a breakdown of its performance in broad categories. Users were divided into five groups: some received only the card, some underwent the researchers' onboarding process, some followed a baseline onboarding procedure, some underwent the researchers' onboarding process alongside recommendations about trusting the AI, and others received recommendations alone.

Interestingly, only the researchers' onboarding process, without accompanying recommendations, significantly enhanced users' accuracy. It improved their traffic light prediction task performance by approximately 5 percent without introducing any slowdown. However, the impact of onboarding was less pronounced in the question-answering task, likely due to ChatGPT, the AI model, providing explanations with each answer, which conveyed trustworthiness.

The researchers have introduced an innovative approach for identifying instances where AI can be trusted and, importantly, for conveying this information to users, fostering improved interactions between humans and AI.

Recommendations for organizations considering investing in medical AI:

• Develop tailored onboarding processes for users to optimize collaboration with AI systems.
• Prioritize user education about AI model training, capabilities, and performance.
• Implement feedback mechanisms and regular training exercises to enhance user proficiency.
• Ensure transparency in AI decision-making processes and communicate clearly with users.
• Continuously adapt AI systems to evolving capabilities and user perceptions.
• Consider ethical implications and incorporate ethical considerations into AI practices.
• Standardize onboarding practices and encourage user autonomy in decision-making.
• Evaluate the effectiveness of onboarding processes and adjust based on user feedback.
• Foster responsible AI development and prioritize methods for user safety and trust.