Module 5: Virtualized Unbiasing Framework

The first section of the application is the scrollytelling part, where the user can be informed about bias and AI in general. Scrollytelling, also known as scroll-driven storytelling or scroll-based  storytelling, is a web design technique that involves using the scrolling action of a webpage to  reveal content in a narrative or visually engaging way. Instead of presenting information all at  once on a single page, scrollytelling unfolds content gradually as the user scrolls down the page. 

By strategically organizing and presenting data visually, this process enhances comprehension  and facilitates rapid understanding. To introduce the concepts of bias, AI, and bias in AI, we developed an animation-based scrollytelling experience (see Figure 1). 

The scrollytelling application consists of stages that explain what is AI, what is bias, what is bias in AI, and how it affects model outputs.

Figure 1: Scrollytelling example

LESSON 5. 2: STAGES OF AI TRAININGS

• A short description that summarizes the problem.
• The solution that either solved or mitigated the problem.
• Some reference material for further research on the topic. 

Real-life examples of bias in policies serve as digestible illustrations that underscore the critical  importance of understanding and addressing systemic inequalities. For each step, a short  description is provided along with ways that bias can occur. The intended use for this section is  to provide policy makers, stakeholders and ML engineers with the information needed in order  to prevent the occurrence of bias in workflows. 

Consider standardized testing in education, for instance, where biases can disproportionately  disadvantage certain demographic groups. Such policies can perpetuate societal disparities and  hinder equal opportunities. The importance of recognizing these biases lies in the potential to  empower a general audience. When individuals comprehend the tangible impacts of biased  policies, they are better equipped to advocate for change, engage in informed discussions, and  challenge discriminatory practices. By offering relatable examples, we empower the general  audience to navigate and contribute to conversations about fairness, justice, and equitable policy  reform in their communities and beyond. Real life examples are showcased in D4.3 demonstrator here.

Example from practice (AI4GOV project): OECD documents chatbot

To raise awareness of the importance of ethics in AI, and the importance of bias prevention approaches, AI4Gov consortium partner used the OECD papers as one of the data sources, consisting of various national AI policies and strategies.

OECD has a collection of various national AI policies and strategies. They have an online repository with over 800 AI policy initiatives from 69 countries, territories and the EU. We have developed tools for web scarping these documents and converting them to Markdown format; those documents have been ingested into another platform and enriched with the automatic classification.

AI4Gov is also developing "Policy-Oriented Analytics and AI Algorithms" in the context of Task: “Improve Citizen Engagement and Trust utilising NLP”. The aim is to develop several NLP algorithms in order to analyse large volumes of text data and also assist the respective AI experts. This particular component consists of the following sub-components:

• Question Answering Service: this service will provide the necessary tool for allowing the AI experts, developers, and policy makers to perform queries on the OECD papers regarding, among others, raising awareness among them of AI solutions. 
• Multilingual Bias Classification: this tool will support the multilingual identification and classification of bias in the OECD papers, providing all stakeholders information and enhanced knowledge of the types of bias that governments and public authorities take into consideration in their AI policies. It should be noted that, at this point of the project, a standalone implementation of this tool is available and its integration with the Policy-Oriented Analytics and AI Algorithms component will follow as the project progresses.   

The architecture of the Policy-Oriented Analytics and AI Algorithms is depicted in the following Figure. For the shake of completeness, a simplified version of the architecture, showcasing the internal workflow of this component is also provided below.

LESSON 5.4: CATALOG OF BIAS DETECTION AND MITIGATION STRATEGIES

The Catalogue builds upon training steps, providing tools and mitigation techniques for each of  the steps of training AI models. The central idea is not to create another text heavy framework, but  to provide a visual summary of existing bias detection and mitigation strategies in an  approachable and easy-to-grasp format. 

For the Bias Detector Catalogue, we have executed an extensive literature overview for bias  mitigation techniques, that are collected in our Gitlab repo.They serve as the input for the interactive visual  synthesis on the AI4gov platform.

Watch the Demo of the Bias Detector Catalog showcasing the examples of tools that can be utilized to detect or mitigate bias.