Abstract: This dissertation explores how Large Language Models (LLMs) can effectively and responsibly contribute to complex decision-making processes. By combining AI and human intelligence, Hybrid Intelligence (HI) emerges, allowing the strengths of both humans and machines to be utilized. The work focuses on applying HI in deliberations, a venue where citizens collectively discuss complex societal issues. The goal of these deliberations is not to make decisions but to understand different perspectives and encourage a diversity of opinions. While these gatherings often take place in person, online deliberations offer the opportunity to involve more people and increase diversity. At the same time, this presents a challenge: the vast amount of generated data makes insightful analysis difficult. Although AI can assist with automatic analyses, it struggles to identify viewpoints that are expressed infrequently. When humans and AI work together, on the other hand, we can uncover new insights and better highlight nuances in opinions. This shifts the focus from only obtaining high-level attitudes (for or against) to being able to capture the underlying arguments and values of the opinion holders. In HI, collaboration between citizens and AI is a process that is both transparent and reliable, which supports a democratic debate.

Publications:

• van der Meer M, Facilitating Opinion Diversity through Hybrid NLP Approaches. In: Cao, Yang (Trista); Papadimitriou, Isabel; Ovalle, Anaelia; Zampieri, Marcos; Ferraro, Francis; Swayamdipta, Swabha (Ed.): Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, NAACL 2024, Mexico City, Mexico, June 18, 2024, pp. 272–284, Association for Computational Linguistics, 2024.
• van der Meer M; Vossen P; Jonker C M; Murukannaiah P K, Value-Sensitive Disagreement Analysis for Online Deliberation. In: Lorig, Fabian; Tucker, Jason; Lindström, Adam Dahlgren; Dignum, Frank; Murukannaiah, Pradeep K.; Theodorou, Andreas; Yolum, Pinar (Ed.): HHAI 2024: Hybrid Human AI Systems for the Social Good – Proceedings of the Third International Conference on Hybrid Human-Artificial Intelligence, Malmö, Sweden, 10-14 June 2024, pp. 481–484, IOS Press, 2024.
• van der Meer M; Vossen P; Jonker C M; Murukannaiah P K, An Empirical Analysis of Diversity in Argument Summarization. In: Graham, Yvette; Purver, Matthew (Ed.): Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 – Volume 1: Long Papers, St. Julian’s, Malta, March 17-22, 2024, pp. 2028–2045, Association for Computational Linguistics, 2024.
• van der Meer M; Falk N; Murukannaiah P K; Liscio E, Annotator-Centric Active Learning for Subjective NLP Tasks. In: Al-Onaizan, Yaser; Bansal, Mohit; Chen, Yun-Nung (Ed.): Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pp. 18537–18555, Association for Computational Linguistics, 2024.
• van der Meer M; Liscio E; Jonker C M; Plaat A; Vossen P; Murukannaiah P K, A Hybrid Intelligence Method for Argument Mining. In: J. Artif. Intell. Res., vol. 80, pp. 1187–1222, 2024.
• van der Meer M; Vossen P; Jonker C M; Murukannaiah P K, Do Differences in Values Influence Disagreements in Online Discussions? In: Bouamor, Houda; Pino, Juan; Bali, Kalika (Ed.): Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pp. 15986–16008, Association for Computational Linguistics, 2023.
• van der Meer M; Liscio E; Jonker C M; Plaat A; Vossen P; Murukannaiah P K, HyEnA: A Hybrid Method for Extracting Arguments from Opinions. In: HHAI2022: Augmenting Human Intellect, pp. 17–31, IOS Press, Amsterdam, the Netherlands, 2022.
• Shortall R; Itten A; van der Meer M; Murukannaiah P; Jonker C, Reason against the machine? Future directions for mass online deliberation. In: Frontiers in Political Science, vol. 4, 2022, ISSN: 2673-3145.
• Liscio E; van der Meer M; Siebert L C; Jonker C M; Murukannaiah P K, What values should an agent align with? In: Autonomous Agents and Multi-Agent Systems, vol. 36, no. 23, pp. 32, 2022.
• Kiseleva J; Li Z; Aliannejadi M; Mohanty S; ter Hoeve M; Burtsev M; Skrynnik A; Zholus A; Panov A; Srinet K; Szlam A; Sun Y; Hofmann K; Awadallah A; Abdrazakov L; Churin I; Manggala P; Naszadi K; van der Meer M; Kim T, Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021.
• van der Meer M; Reuver M; Khurana U; Krause L; Santamaría S B, Will It Blend? Mixing Training Paradigms & Prompting for Argument Quality Prediction 2022.
• Liscio E; van der Meer M; Jonker C M; Murukannaiah P K, A Collaborative Platform for Identifying Context-Specific Values. In: Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems, pp. 1773–1775, IFAAMAS, Online, 2021.
• Liscio E; van der Meer M; Siebert L C; Jonker C M; Mouter N; Murukannaiah P K, Axies: Identifying and Evaluating Context-Specific Values. In: Proc. of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021), pp. 799–808, IFAAMAS, Online, 2021.