Matrix

Matrix structure 

Per February 2023, a new organisational structure was introduced, to obtain a scalable organisation accommodating the growth of the HI Centre, while still addressing the CARE objectives described above. The new Matrix structure of the HI centre is an evolution of the original four CARE Research Lines into three perspectives: (i) four core scientific HI Challenges (problem space); (ii) which are crosscut by Special Interest Groups (SIG, solution space) that focus on specific AI techniques, and (iii) four Case Studies to anchor our research in concrete societal situations (evaluation space). Over time the different clusters, interest groups and case studies may change, depending on the developments in the field.

Matrix 2024
Challenges  

Collaboration & Synergy (CoSy)Bernd DudzikDaniel Balliet
In this challenge, we aim to provide a platform for research efforts directed at understanding, designing, developing, and evaluating the capacity of Hybrid Intelligent systems to facilitate collaboration with synergistic outcomes between different actors that complement each other. We take an inclusive view on the topic, where relevant interactions can take a wide variety of forms, e.g., comprising any number and combination of human-agent(s), human-human, or agent(s)-agent(s) as actors, and by including a broad spectrum of communication types, ranging from asynchronous unimodal communication all the way to fully synchronous multimodal exchanges. Importantly, we champion an interdisciplinary perspective, bridging fundamental and applied technological research with knowledge from the behavioural sciences related to human cooperation and teamwork.

More information on the CoSy challenge can be found here.

Dialogue  Catharine Oertel, Piek Vossen
The Dialogue cluster addresses the research objectives of collaborative HI and of explainable HI. HI scientists are connected which work on various aspects related to dialogues, such as long-term interaction, text versus dialog, human-agent interaction and human-robot interaction, and knowledge representation for dialogs. 

More information on this challenge can be found here.

Trust & Trustworthiness (T&T) – Myrthe Tielman, Roel Dobbe
In this challenge, we aim to inform the creation of trustworthy HI, and build and calibrate trust in human-AI systems. Our mission makes room for work at three key levels. Firstly, trustworthiness of technical HI advances: We aim to develop technical components of HI systems that are verifiably properties that promote trustworthiness, including (but not limited to) transparency, explainability, safety, and fairness. Secondly, trustworthiness of human-AI interactions and interfaces and the relation to the calibration of appropriate trust: We aim to understand how we can build systems which can actively understand how they are trusted, and which can interact with users in such a way that trust is calibrated to avoid situations of under and overtrust. Thirdly, trustworthiness of the broader sociotechnical system in which HI operates: We aim to understand the broader organizational and institutional factors that enable trustworthiness of HI technologies and interactions, and the calibration of trust at the use level, to inform the development and evolution of AI governance mechanisms.

More information on this challenge can be found here.

HI Methodologies and Metrics (M&M) Davide Dell’Anna, Tiffany Matej 
Our mission is to establish Hybrid Intelligence as a methodology, where “methodology” refers to “the rationale for the research approach, and the lens through which the analysis occurs.” Establishing the methodology includes, among other things, developing design patterns, shared ontologies, quality models (which can be further refined into measures and metrics) for HI systems. In other words, this challenge focuses on meta-level constructs, i,e., constructs that are not specific to a specific HI system or project, but they can be applied to a variety of HI systems. Examples of M&M projects are the HI Quality model (Davide Dell’Anna), the HI Terminology (Ilaria Tiddi), the HI Hackaton (Jasper van der Waa), the Generative HI paper (Catholijn Jonker, Max van Duijn).

More on the M&M Challenge can be found here


Special interest groups (SIG)
 

SIG Reinforcement Learning (RL) Shihan Wang, Frans Oliehoek
Applying machine learning, especially reinforcement learning techniques in the context of Hybrid Intelligence can be difficult: e.g. there are no big data sets about people’s behaviors, or how they cooperate in a team, and due to good
reasons (privacy concerns, etc.) it is not likely that they will arise any time soon. This SIG is dedicated to reinforcement learning and open to other relevant machine learning techniques for adaptive HI systems. It connects all members of the HI project that are interested in such learning techniques, either as object of study or as tool.

SIG NLP4HI Davide Grossi, Enrico Liscio, Lucia Donatelli
Natural Language Processing (NLP) is rapidly becoming the foundation of various HI systems, impacting fields such as democracy, healthcare, and law. As NLP technologies advance, understanding how humans and AI can process and utilize language to communicate, cooperate, and achieve goals is crucial. This SIG is committed to progressing both foundational and applied research at the intersection of NLP and HI. Our discussion will focus on the evolution of human-AI collaboration through the lens of NLP, asking how to develop real-world HI systems that understand, generate, and reason with language effectively, as well as how to engage with current state-of-the-art methods such as LLMs practically and ethically. Special attention will be given to how HI-NLP methods can enhance human deliberation, argumentation, and collective decision-making.

SIG Ethics and System Design Roel Dobbe
This challenge focuses on the normative dimensions of hybrid intelligence systems, related to research, development and its potential use in practice. In recent years, the ethics of AI systems has become a richly pursued topic. Doing justice to the social, ethical and legal/political aspects of hybrid intelligence requires taking a broader lens that situates a particular application in its broader context, including various stakeholders and their values and needs, as well as the growing number of institutional governance requirements (such as the AI Act or algorithm registers). In this SIG, we can address any needs to discuss or address such aspects. We invite colleagues from ethics and philosophy to help us give words to particular challenges. And we bring in tools and methods from sociotechnical system theory and design to help us enrich our own basis for analysing and designing the problems and solutions related to hybrid intelligence. As such, the SIG allows for learning and building common awareness of the normative dimensions, and it may inform how we do research and design hybrid intelligence in a more responsible and situated fashion.

SIG Theory of Mind (ToM) – Max van Duijn, Rineke Verbrugge, Ramira van der Meulen
Theory of Mind (ToM) is the ability to attribute mental states to others, in order to make sense of their behaviour and perspectives on the world. It is one of the building blocks of an intelligent system: one needs to be able to consider others’ perspectives to know how they will respond to one’s actions, which is beneficial in nearly any interaction, including cooperation and competition. In the literature, aspects of ToM are also referred to as, for example, ‘perspective taking’, ‘user/mental models’, ‘mind-reading’ and ‘intention recognition’. Within HI, we mostly concern ourselves with using ToM to benefit the collaboration and communication between robots, agents, and humans. ToM is, as such, mostly aligned with the Collaborative HI line, but it has strong ties to the Adaptive and Explainable HI lines as well. Within the SIG ToM we organise 2-3 meetings per semester, in which we read and discuss papers and share updates about relevant conferences, publications, etc. We also keep an “inventory” of what individual projects do with ToM and we reach out to HI researchers for ToM-related collaborations.


Case studies
 

Case study Health – Diabetes – Maaike de Boer, Myrthe Tielman, Chenxu Hao
Diabetes is a chronic lifestyle related disease. A change of lifestyle requires intensive, personalised support involving both the patient and their social environment. A hybrid intelligent coach should help diabetes patients to adopt a healthier lifestyle while at the same time lowering the workload of healthcare professionals. A key challenge is the creation of multi-party dialogues between HI system, patient and healthcare professional that create long-term engagement. Explainability is crucial in this domain.

Case study Virtual Museum Guide – Shenghui Wang, Victor de Boer
The use case is about transforming museum learning through an intelligent virtual guide that fosters inclusive, immersive, and interactive cultural exploration. By leveraging multimodal interactions along with the acquisition, representation, and presentation of polyvocal knowledge, we aim to guide and accompany diverse audiences, making cultural heritage accessible, engaging and meaningful. Through the power of Hybrid Intelligence, we aim to create immersive, interactive digital environments that connect individuals to the stories and knowledge of varied cultures, ensuring that everyone, regardless of background, can experience, learn from, and contribute to the richness of global heritage. 

More information can be found here.

Case study Robotic Surgery – Chirag Raman, Dalibor Vasilic, Catholijn Jonker
The aim of the use case is to create a human-agent coupled system capable of safely and effectively performing tasks in (micro)surgical environments. Specifically, the agent here refers to an exoscope on a robotic arm. Effective collaboration requires both the surgical team and the robotic agent to complement each others’ skills in a dynamic environment. For instance, the robotic microscope has to learn how best to align its position, orientation, and zoom with the activities of the members of the surgical team. This requires a mutual understanding of each upcoming surgical procedure, which is acquired during prior practice sessions, as well as dynamic adaptation during the surgery itself. The case study necessitates an inherently interdisciplinary approach, spanning topics including shared planning and reasoning, mental models and theory of mind,  multimodal nonverbal behavior dynamics, situated communication interfaces, continual and lifelong learning, and collaborative ontology determination.

More information on the case study can be found here.

Case study Scientific Assistant – Pradeep Murukannaiah, Ilaria Tiddi, Niklas Höpner
The aim of the scientific assistant is to support one or more steps in the scientific method cycle: formulating research questions, analysing the literature, formulating a hypothesis, designing an experiment, analysing data, and drawing conclusions. This will require a combination of symbolic and subsymbolic AI techniques, ranging from domain ontologies to deep learning, as well as theory of mind and shared planning to support the collaboration. 

More information on the case study can be found here.

Case study Law – Cor Steging, Stephanie Tan
Hybrid Law aims to apply the HI principles to the legal domain to strive towards societal goals such as access to justice. Most people do not know they are in a legal predicament until it is too late, and societal barriers can prevent them from getting the legal help they need. Creating legal HI systems can improve user literacy and help ordinary citizens in legal scenarios. HI systems can also assist legal personnel by helping them tackle an ever-expanding backlog of legal documents, which currently causes severe delays in justice. These Hybrid Law systems should make the right decisions for the right reasons, and be able to explain their reasoning for justice to occur. Furthermore, they require a human-centered design where the system adapts to the user for an effective and interactive collaboration.

More information on the case study can be found here.