Human-Centric AI: A Road Map to Human-AI Collaboration

AI systems perform tasks that normally would require human intelligence, and can accomplish them faster by rapidly consuming and analysing large amounts of complex data. That’s the theory. In practice, many instances of applied AI fail spectacularly. In some cases, we don’t even notice!

AI is typically a black box. In many cases we don’t understand why AI systems make decisions or predictions. This enigma becomes more pronounced the more complex AI decision-making becomes. Without this understanding, we won’t know when AI gives us a wrong answer. To harness AI’s full potential requires human-AI collaboration, which combines the best of both worlds: the profound analysis and pattern-recognition capabilities of AI with empathetic human decision-making.

To accomplish this, NEC Laboratories Europe is developing explainable human-centric AI that will allow these systems to explain themselves to human users, empowering us to make better and more informed decisions.

The biomedical researcher’s first question is: “What happens if someone takes the drug paliperidone and the chemical element calcium at the same time?” A conventional AI system might provide a correct prediction, saying, “It will cause pain,” but would stop there. A noncollaborative AI system can’t answer the researcher’s next question, “Why do you think it will cause pain?”

A collaborative AI system would then explain that each substance activates the particular proteins, LPA and MMP. When the researcher asks about the relationship between those proteins, the AI system would also explain that LPA further increases the activation of MMP, which is already activated by calcium. Thus, paliperidone indirectly exacerbates the upregulation of MMP already caused by calcium, which leads to increased pain.

Collaborative AI systems can increase user productivity when they interact naturally with humans and explain their predictions in a way that helps users accomplish their tasks.

When developers and users understand what the machine learning model learns, they can determine the value of its output, for example, with regards to bias. Developers can also audit explainable AI systems to assess the degree of legal compliance needed and understand why decisions and predictions were made. This helps foster trust amongst AI users and increases the adoption of AI for high-risk decision-making.

Currently, most research in explainable AI is algorithmic-centred: researchers explore how AI can be explained from a mathematical point of view. This approach does not consider what kind of explanations humans need – something that explainable human-centric AI accomplishes.

The need for explainable AI to ensure compliance, transparency and ethical use becomes all the more evident when we look at the pending Artificial Intelligence Act (AI Act) proposed by the European Union.1 Once in place, all AI systems deployed in the EU will be required to adhere to its regulatory and legal framework. Explainable AI’s ability to understand what AI systems have learnt will provide developers with a significant cost and time advantage in ensuring compliance. The draft EU AI Act currently specifies four risk categories of AI systems.

The lowest category is “Minimal Risk,” appropriate for spam filters, video games and in cases when the user needs to know that AI is being used and be able to opt out. “Limited Risk” refers to decisions or predictions made by AI systems that may pose some risk to users and therefore require transparency. AI chatbots that provide automated, low-risk customer service information is one example. “High Risk” includes domains that affect human rights or areas where AI will impact critical decision-making such as in areas of public services, law enforcement or medicine. Examples include large language models providing decisions or predictions for high-risk scenarios and autonomous vehicle systems.

For conformity assessments, explainable AI techniques can be applied to tell us what the AI system has learnt. After deployment, continuous market surveillance is called for. The highest risk category defines situations of unacceptable risk, such as threats to safety, livelihoods or human rights. In these areas, the use of AI will not be allowed.²

The AI Act also mandates that AI systems cede control to developers and users when desired or needed, which will make human-centric AI a key requirement for most AI systems moving forward. This is especially relevant when considering the rapid advancement in AI. While the EU is at the forefront of pending AI regulation, non-EU countries are following suit. In the United States, Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence was issued by President Biden on October 30, 2023.

To help us choose the most suitable explainable AI technology for a particular use case we need to understand the key explainable AI concepts.

AI models use two types of decision-making paradigms: transparent and opaque. A transparent model lets us follow how the model arrived at a decision, for example, a decision tree, where-as opaque AI models gives us black-box AI.

The figure below shows two decision trees used to help us decide if someone will be granted credit. The first tree has only two rules and is easily understood. We see that a person receives credit if they fulfil two conditions: 1. earn more than fifty thousand Euro a year and 2. have not defaulted on credit.

As a result, users may not trust their decisions or predictions. To resolve this, researchers are developing explainable methods for opaque models that improve their technical components. However, if human-AI collaboration requirements are ignored then humans may still not understand these methods.

When using an explainable method for an opaque model, we need to decide whether a post hoc or built-in explanation method should be used. Post hoc methods are added after a model has been trained and can explain the decision-making of any AI system. Well-known examples include Lime and Shap.3,4 Built-in explanation methods are included during the models development; Examples include influence functions and prototype-based learning.5,6

To choose the right method we must also consider the model’s use case. To help with this, we use the definitions of faithfulness and plausibility, which are important concepts when considering the success of an explanation method. When designing AI systems, faithfulness and plausibility account for the trade-off needed between how complex or plausible an explanation is and how well we understand the model’s reasoning process (how faithful it is). For high-risk scenarios, a method should have a high degree of both.

Faithfulness indicates how accurate an AI system’s explanation of its reasoning process is. If someone asks, “Why is the sky blue?” the system may answer, “Because someone painted it blue.” While a valid explanation, it is neither true nor faithful. Explanation methods will vary in how faithful they are. A high amount of faithfulness is needed when the model’s decision influences a high-risk scenario.

Plausibility assesses how clear the explanation is to a human, in other words, how human-centric it is. This is important because an explanation achieves its goal only if it helps a human user. As with faithfulness, a decision or prediction can vary depending on how plausible it is.

It is difficult for an explanation method to be both highly faithful and strongly plausible. A trade-off between the two is usually needed.

Building on our understanding of what it means to have faithful and plausible explanations for AI models, let’s examine how these explanations apply to post hoc and built-in explanation methods.

Post hoc methods have lower faithfulness than built-in methods, and their plausibility can only be slightly influenced. Built-in methods are designed to restrict an AI’s reasoning. The AI developer determines how plausible these methods are. However, using built-in methods often requires replacing the AI system with a new one. When this can’t be done using built-in methods are not possible.

Gradient Rollback is a highly faithful AI method that modifies how neural networks are trained. Developed by NEC Laboratories Europe, the method explains the prediction of a neural network by highlighting which training examples made the prediction. Compared to other explanation methods, Gradient Rollback offers more context and provides a user interface allowing users to ask for additional information.

NEC Laboratories Europe is extending the scope of prototype-based learning by incorporating human-centric AI concepts to ensure that explanations from prototype-based models are more plausible while achieving the highest amount of faithfulness. With that accomplished, prototype-based learning will be able to provide different types of explanations, such as training examples and counterfactuals. Once mature, applications of human-centric AI can be further extended by designing more customized concepts.

Prototype-based learning is designed to be explainable, and its explanations can be trusted. Experts can inspect whether a decision or prediction can be trusted based on the explanations given. Because of the different components that are included in machine learning, different explanations can be provided, resulting in a human-centric solution that is highly plausible.

The rapid acceptance of complex AI for general use, like large language models (LLMs), marks a milestone in its adoption. LLMs have had great success when applied to low-risk tasks but are not yet suited to medium-risk and high-risk scenarios. In the short time that complex AI has been in the public domain, it has been unintentionally and deliberately misused, and the consequent risks are increasing. Human-centric AI will have a great affect in helping guide its rapid development and responsible use.