AI, Democracy and the Questions Nobody Is Asking

Every time I join a conversation about artificial intelligence and democracy, I notice the same pattern: we either celebrate the technological revolution as inevitable and neutral, or we lose ourselves in science fiction scenarios about superintelligences spinning out of control. I understand why it happens. The spectacular always wins attention.

At Xquenda, we work alongside human rights organizations in the global south that are navigating — with limited resources and a great deal of uncertainty — their relationship with these systems. That daily work has convinced me of something: we are exaggerating some risks while ignoring the ones already causing harm. Four in particular.

The risk that concerns me most is not malicious AI. It is convinced AI. A system trained primarily on data produced in the global north, in English, from institutional positions of power, does not need bad intentions to cause harm. It only needs to not know what it does not know. And that is its most dangerous characteristic: it responds with fluency exactly where it should hesitate most. Indigenous knowledge is not in its training data. Southern epistemologies are not either. The experiences of communities that have historically not produced digitized text simply do not exist for the model. And the model does not know this. When that system is integrated into public policy decisions — who receives a social benefit, what medical treatment is recommended, which profile represents a risk — its blind spot becomes state policy. That is not a bug to be fixed in the next version. It is a political decision about whose knowledge counts.

The second risk is the speed of institutional delegation. Public institutions across Latin America are adopting AI tools at a pace that far outstrips their capacity to audit, understand, or regulate them. Not out of bad faith, but because of pressure to modernize, limited budgets for due diligence, and an industry very skilled at presenting its products as neutral solutions to deeply political problems. Organized civil society arrives late to those tables, when it arrives at all. And when it does, the important decisions have already been made.

The third risk is the silent concentration of epistemic power. A handful of companies, located in a handful of cities — San Francisco, New York, Berlin, Tokyo, Dubai — are defining what a correct answer looks like, what counts as problematic content, what futures are imaginable. What gets decided in those offices is not just technology: it is the framework within which millions of people in the global south will receive healthcare, education, and justice. This is not only a risk to privacy or employment. It is a risk to the diversity of ways of knowing and inhabiting the world. A democracy that hands over its common sense to systems designed thousands of kilometers away, without the participation of affected communities, is not modernizing. It is surrendering something it will not easily recover.

The fourth risk is the question no one is asking seriously enough: where should AI not be used? Public debate almost always assumes that the central question is how to use technology well, not whether it should be used in certain contexts at all. But that second question is urgent, and it is not being answered with the depth it deserves — not in medicine, not in law, not in education, not in social work, not in any discipline that operates with concrete human lives. There are decisions that should not be delegated to any algorithm, not because the technology fails, but because the act of delegating destroys something essential. Restorative justice, support for survivors of violence, community conflict resolution, asylum claim evaluation, grief accompaniment, risk assessment in gender-based violence cases: these are processes that require human presence, genuine listening, embodied accountability. Automating them does not make them more efficient. It hollows them out. And that hollowing has consequences that never appear in any performance metric, but do appear in the lives of people left on the other side of a decision that nobody signed.

What we need is not to stop technology or romanticize the past. It is to insist that the communities most affected by these decisions stand at the center of their design, auditing, and governance. Not as users. As authorities.

We keep talking about robots while algorithms are already deciding who receives medical care, who gets a loan, who is considered a risk. Not in the future. Right now. And the voices with the most to say about those decisions still have no seat at the table where they are made.

That is the conversation that is not yet happening with the urgency it deserves. And the time we spend on science fiction, nobody gives back.

Have you seen AI applied in your field in ways that missed something fundamental about the context? Who do you think is currently missing from the conversation about where these systems should and should not be used? I'd genuinely like to know what you're seeing from where you stand.