The power and the danger of generative AI

Language is the primary infrastructure of human knowledge—the shared medium in which we build concepts, frame explanations, test claims, and preserve collective memory. Social theory, science, and historical explanation do not merely use language; they depend on it to turn experience into arguments that can be criticized, revised, and improved. If texts can never correspond meaningfully to reality, then theory itself collapses—leaving only impressions, slogans, and power. The real issue is not whether language is a proxy, but what disciplines make it reliable: evidence, citation, debate, methodological transparency, and the ability to trace claims back to sources.

Ludwig Wittgenstein, an Austro-British philosopher who worked primarily in logic, in his later philosophy helps sharpen what is at stake. In Philosophical Investigations, he pushes back against the idea that words carry meaning by pointing to private mental objects or fixed essences. Instead, he emphasizes that meaning is anchored in public practice: “the meaning of a word is its use in the language.”

That deceptively simple remark relocates “meaning” from hidden interiors to the visible ways people coordinate, correct, teach, and apply expressions in real life. The same book adds the famous line: “to imagine a language means to imagine a form of life.”

Language is not a free-floating code; it is interwoven with human activities—asking, promising, measuring, accusing, explaining, proving—each governed by norms that are learned and enforced within a community.

This is why Wittgenstein’s discussion of rule-following matters so much. His reflections raise a destabilizing puzzle: no rule, by itself, mechanically forces one particular application, because any future action can be redescribed as “following” the rule. What stabilizes correct use is not a private interpretation but the shared practices that distinguish “going on correctly” from merely thinking one is doing so. In other words, the normativity of language—its right-and-wrong dimension—depends on a social environment where errors can be pointed out, justifications demanded, and standards maintained.

That Wittgensteinian picture dovetails with a powerful theme in twentieth-century epistemology: knowledge is not simply a psychological event but a status that requires justification. Sellars argued that calling something “knowledge” is not merely describing an inner episode; it is placing it within a network of reasons, entitlements, and responsibilities. Brandom extends this idea by treating assertions as moves in a game of giving and asking for reasons: to assert is to undertake commitments that others can challenge and to which one can be held accountable. On this view, language is not just how we report what we think; it is how we make claims answerable—how we convert mere talk into something that can count as knowledge.

Science, at its best, operationalizes these philosophical insights by institutionalizing criticism. Popper’s falsificationism—whatever its limits—captures a core feature of scientific seriousness: claims earn standing by exposing themselves to tests that could refute them, rather than insulating themselves with unfalsifiable elasticity. Longino adds a crucial social-epistemic dimension: objectivity is not guaranteed by individual purity or value-free detachment, but by robust critical interaction—especially the presence of alternative perspectives capable of identifying background assumptions, evidential gaps, and blind spots. Merton’s classic description of “organized skepticism” similarly stresses that reliability emerges where communities build norms and incentives for scrutiny rather than deference. Habermas, from a different angle, argues that everyday communication implicitly raises “validity claims” (truth, rightness, sincerity) that can in principle be contested—again highlighting that language becomes knowledge-like when it is structured for challenge and justification.

Once language is understood as infrastructure sustained by rule-governed social practices, the implications for generative AI come into focus. A language model can produce grammatically fluent, context-sensitive text—often astonishingly so—by learning statistical regularities from vast datasets. But fluency is not yet participation in a form of life. The system does not live the practices in which words are taught, corrected, and morally or epistemically policed. It does not fear being wrong, does not care about being challenged, and does not carry commitments in the way speakers do when they assert, promise, accuse, or testify. In Wittgenstein’s terms, it can generate moves that look like moves in our language-games, while lacking the human embedding—needs, purposes, accountability structures—through which those moves normally acquire their full normative weight.

This helps explain both the power and the danger of generative AI. The power is that models can approximate many surface competencies associated with language-games: summarizing, paraphrasing, drafting, answering questions, offering explanations. The danger is that humans are strongly disposed to attribute understanding and intention to coherent text. The “stochastic parrots” critique puts this sharply: because people naturally read meaning into linguistic form, systems that generate plausible language can be mistaken for systems that reliably track truth or responsibly reason—creating real risks of over-trust, bias amplification, and confident error. In a Wittgensteinian frame, the risk is a category mistake: treating a tool that produces linguistic appearances as if it were a participant in the social-normative practices that make language a vehicle for knowledge.

The remedy is not to declare language a mask and retreat into cynicism. That move would undercut theory itself. A Wittgenstein-informed response is more practical: focus on the conditions under which language becomes dependable. For human inquiry, reliability comes from disciplined practices—citation, transparency, methods, criticism, replicability, and the ability to trace claims back to sources. For AI-mediated inquiry, the same principle applies with added urgency. If a model produces an answer, the key question is not “Does it sound right?” but “Can I trace the basis of this claim, distinguish quotation/report from inference, and subject it to criticism?” This is how we keep AI outputs within the space of reasons rather than letting them float as rhetoric.

This is also why the most important design and usage choices are governance choices. When AI systems are deployed without strong provenance norms, they encourage a degraded language-game: one where eloquence substitutes for warrant. When they are embedded in workflows that demand evidence links, show uncertainty, invite adversarial review, and reward correction, they can support knowledge rather than impersonate it. In Wittgenstein’s terms, we are deciding what game we are playing with these tools: a game of quick rhetorical completion, or a game of inquiry where claims remain answerable to reasons and evidence.

Generative AI therefore intensifies an old lesson. Language is not merely a veil; it is the medium through which humans construct and contest understanding. But language does not become knowledge by sounding intelligent. It becomes knowledge when it is governed by norms—when reasons can be demanded, sources traced, methods disclosed, and claims revised under pressure. Wittgenstein’s emphasis on use, rule-following, and forms of life clarifies why fluent text can be epistemically empty: it can be detached from the practices that give words their disciplined grip on the world. The task is not abstinence from language technology, nor surrender to its charm, but conscious integration of these systems into the social machinery that makes language reliable in the first place.

The Way Forward

Use AI to accelerate inquiry, not replace it: generate outlines, questions, alternative framings, summaries, counterarguments, and research pathways—but require that any substantive claim be tied to evidence you can inspect. Make “show me where this comes from” the default follow-up, and treat uncited confidence as a warning sign.

Build workflows that keep AI inside the space of reasons: insist on traceability (links, quotations, data references), method transparency (what assumptions were made, what steps were taken), and explicit uncertainty (“I’m not sure,” ranges, competing interpretations). When the model cannot provide warrants, it should be used for exploration, not decision.

Adopt an adversarial reading posture: ask what would count against the answer, what alternative hypotheses exist, and what the strongest objections are. Use the tool to surface errors and blind spots—its own and yours—rather than to deliver rhetorical closure.

Match the tool to the task: reserve high-stakes uses (policy, law, medicine, finance) for settings with human review, domain expertise, and documented sources. In those contexts, the model’s role is best confined to drafting, summarizing, and organizing—never final judgment.

Finally, cultivate the right civic and organizational habits: teach people that fluency is not warrant, and that “sounding right” is not the same as “being answerable.” The goal is not to ban the tool or worship it, but to discipline it—so that AI-generated language is continuously pulled back into the practices that make language reliable: evidence, citation, debate, methodological clarity, and revision under critique.

Used this way, generative AI becomes less a producer of persuasive text and more a scaffold for thinking—helping us move faster without leaving behind the norms that make understanding possible.