New MIT Method Makes AI More Explainable

Scientists from the Massachusetts Institute of Technology have presented a new method that helps artificial intelligence models explain their predictions more clearly and accurately. The development was created in the MIT CSAIL laboratory and will be presented at the International Conference on Learning Representations. The technology can increase trust in AI in medicine, image analysis and other critical areas.

Why AI explainability is becoming critical

Modern artificial intelligence systems are often called a “black box,” xrust states. They are able to give accurate forecasts, but users do not always understand why the model came to exactly this result .

This is especially important in areas such as:

• medical diagnostics;
• analysis of medical images;
• scientific research;
• security systems.

For example, if the algorithm detects melanoma in a skin image , the clinician must understand what features in the image led to this conclusion. Without this, trust in the algorithm remains limited.

More information about the research laboratory can be found on the official MIT CSAIL website:
https://www.csail.mit.edu

class=»notranslate»>__GTAG4__ Method of “bottleneck of concepts”

One of the ways to make artificial intelligence clearer — use the so-called Concept Bottleneck Model (CBM) .

Its operating principle is simple:

1. First, the model identifies human-understandable features in the image.
2. Then uses these features for the final prediction.

For example, a bird recognition system can take into account:

class=»notranslate»>__GTAG23__ yellow legs;
• blue wings;
• beak shape;
• plumage color.

After analyzing these features, the algorithm determines the type of bird.

The intermediate stage — the “bottleneck” — allows the user to see the logic of solving the model.

The main problem of existing systems

Until now, most of these models worked with concepts that were predefined by people or language models .

But this approach has serious drawbacks:

• some concepts may be irrelevant;
• the description of features may be too general;
• the model can use hidden information that is not reflected in the explanations.

This problem is known as information leak.

As a result, AI sometimes relies on features that the user is not even aware of.

A new approach: extracting concepts from the model itself

Researchers have proposed an alternative method: extract concepts directly from the trained model.

The idea is that the neural network has already studied a huge amount of data and formed its own internal features for making decisions.

To obtain them, scientists used several technologies:

• sparse autoencoder;
• multimodal large language model (LLM);
• concept recognition module.

The process looks like this:

1. The autoencoder highlights the most important features within the model.
2. LLM translates them into clear text descriptions .
3. The system then automatically tags the images with these concepts.
4. After this, the model is trained to make predictions only based on the identified concepts .

As a result, the explanations become much closer to the real process of the neural network.

Limiting concepts for transparency

class=»notranslate»>__GTAG15__ To make the explanation even clearer, the researchers introduced an additional limitation.

For each forecast, the model can use no more than five concepts.

This gives several advantages at once:

• the model selects only the most important signs;
• explanations become brief;
• the likelihood of hidden factors is reduced.

Results of technology testing

The new method was tested on several computer vision tasks.

Among them:

• recognition of bird species;
• diagnosis of skin diseases using medical images.

According to the results of experiments, the system:

• showed higher accuracy than previous explainable AI models;
• gave more understandable explanations of the forecasts ;
• formed concepts that better correspond to real data.

Lead author of the study Antonio De Santis , graduate student at the Politecnico di Milano, explains the purpose of the development:

“We want to learn how to read the minds of computer vision models. Using more advanced concepts improves forecast accuracy and makes AI systems more accountable.”

Opinion of independent experts

The work was highly appreciated by experts in the field of data science.

Professor at the University of Würzburg Andreas Hoto noted:

class=»notranslate»>__GTAG11__ “This approach opens the way to explanations that better match how the model actually works, and creates a bridge between interpretive AI, symbolic AI, and knowledge graphs.”

More information about the future conference where the research will be presented:
https://iclr.cc

What does this mean for the future of artificial intelligence

The development of explainable AI could change approach to using algorithms in sensitive areas.

This is especially important for:

• medical diagnostic systems;
• autonomous technologies;
• scientific research;
• government and financial decisions.

The clearer the algorithms explain their findings, the easier it is for specialists to check and monitor their work .

Conclusions

The problem of explainability of artificial intelligence has been actively discussed in recent years. As the complexity of neural networks increases, more and more models become “black boxes” whose internal processes are difficult to interpret.

Research in the field of Explainable AI (XAI) is aimed at making algorithms more transparent and increasing user trust. MIT's work was one of the steps in the development of technologies that help understand exactly how AI makes decisions .

Xrust New MIT Method Makes AI More Explainable