Three Questions about Scaled Deliberative Engagement

This is the third and final article of a series appearing here on how artificial intelligence can scale public deliberation processes to include large numbers of participants, and why this is important for future governance.

The first two articles in this series explain how we can use AI’s capacity to manage large volumes of qualitative data to scale deliberative processes, thereby transforming public engagement. This final article steps back to consider three basic questions that readers may be asking about our overall approach:

• Will politicians buy into it?

• Can we trust the technology?

• Does scaling engagement challenge representative government?

Can We Secure Political Buy-In?

Article 1 notes that securing political buy-in is a challenge for deliberation. Because citizen juries and mini-publics involve so few people, politicians worry that the larger community may not support the results. Scaling deliberation, we’ve argued, solves this problem by mobilizing a more people behind the results. However, these forums raise a further question: How can they be sure that participants won’t arrive at solutions that the politicians find unacceptable? Politicians want some assurance about the outcomes.

A scaled deliberative engagement can resolve this issue by limiting the scope of each dialogue in advance. In effect, process managers work with the political sponsors to set clear boundaries around the deliberations and the kinds of solutions that may be proposed. Participants are then free to explore and propose different solutions within these boundaries, but they cannot reach beyond them. This reassures the sponsoring politician: If they are comfortable with the scope of the dialogue, they should be comfortable with the results, as the following example shows.

Imagine a city government that wants to improve public transportation using this type of process. Council members:

• The government would work with engagement planners to set boundaries for the deliberation.

• The facilitator would then inform prospective participants of the boundaries, explaining the sorts of things that are in-scope and out-of-scope. Participants would have to agree to respect the boundaries to join the dialogue.

• Participants would concentrate on finding appropriate solutions, and the facilitator would ensure the discussion remains within the boundaries.

The council thus could be confident that the recommendations would align with the government’s capacity and expectations. Defining the dialogue’s boundaries thus helps secure political buy-in.

Can We Trust AI?

A second question concerns the public’s willingness to trust AI to gather, consolidate, and manage the large volumes of data that large-scale engagements can generate. This issue is especially important for public servants. As the Government of Australia recently stated, “government has an elevated level of responsibility for its use of AI,” so officials should be prepared “to explain, justify and take ownership of advice and decisions” based on AI. Our comments on trust here focus on two key issues: biases and hallucinations.

Bias occurs in AI when specific patterns contribute to its analysis, even though there may be little trace of them in the dialogue. Take ideological preferences: an AI’s analysis of a dialogue may reflect progressive ideas on social policy even though participants in the dialogue had little or nothing to say about them. Typically, such biases were embedded in the AI’s training data and then were instilled in its memory during training. As a result, the AI now uses them to help it summarize, organize, or analyze participants’ views.

So-called “hallucinations” pose a different risk. AI sometimes manufactures information about a situation, then presents it as a fact. For example, when reporting on a dialogue among participants, the model might claim that a particular view was widely endorsed, when it was barely mentioned.

Biases and hallucinations raise concerns over AI’s accuracy. If we want people to trust the model, its capacity to organize and analyze the data must be reliable and verifiable. When it comes to tasks like summaries, sentiment analysis, and topic modeling, advanced AI models are not perfect, but their level of reliability is high and improving quickly. Where there is still doubt, there are methods for testing the accuracy of the results.

Redundancy is one such method. By using a multiplicity of processes to analyze data, we can cross check the findings from different methods. Suppose the AI produces a summary containing the key points from a dialogue. Suppose it also does some topic modeling to identify and categorize the underlying themes or topics in the dialogue. These two methods are different, but they are close enough in nature that we can compare their results to see if they align. If significant discrepancies are found, there may be a problem.

Providing clear, auditable links to the original source material can enhance confidence in the results, as it allows for direct verification of the information presented.

Sentiment analysis can also be checked and verified. For example, if the analysis uncovers surprising patterns, such as an unexpected surge in anger over a particular issue, the data can be reviewed for bias or inaccuracies.

Participant review provides yet another way to verify accuracy. For example, after each public dialogue session, a summary report is circulated, which participants review to verify their comments.

In sum, this approach builds a range of safeguards into the process to ensure reliability, accuracy, transparency, and accountability, so that participants and decision-makers have confidence in the approach.

Does Technology-Enabled Deliberation at Scale Challenge Representative Government?

Some experts see public deliberation as an important step toward direct democracy. This is not how we see scaled deliberative engagements. In our view, this approach is closely aligned with representative democracy – and specifically, with cabinet government. Technology-enabled deliberation at scale is a new way for governments to manage complex and potentially controversial issues by engaging large cross-sections of the public in disciplined, highly inclusive dialogues. This helps build awareness, understanding, and a sense of public ownership of the findings that emerge, but there is no transfer of government decision-making authority to the participants. It is the minister who sets the boundaries around the process and who makes the final decision on whether to act on the recommendations. Participants can advise the government, but they cannot direct it. The process is not binding. We believe this kind of engagement strengthens and reinvigorates the policy process, but not by weakening the cabinet. On the contrary, scaled deliberation leads to a more responsive and resilient model of representative government.

This article was written by Don Lenihan PhD, Public Engagement Expert, President and CEO of Middle Ground Policy Research, Damian Carmichael former public servant with the Australian Government and Engagement Lead at Converlens and Tom Workman Director Converlens