Pablo Zavala · AI Safety Evaluation · Research Engineering

Multilingual AI for Indigenous Rights: Safety in the Low-Resource Case

A design proposal rather than a built system, for a Spanish, Quechua, and Shuar legal and business advisor, framed around low-resource-language safety, hallucination risk, and human review.

October 13, 2024 · 6 min

A legal advisor that speaks Quechua must be right about the law, or a wrong answer does real harm. My team drafted a design for a multilingual advisor, covering Spanish, Quechua, and Shuar, to help Indigenous Ecuadorians defend land and rights claims and start businesses, and the design's hardest problems are safety problems: a model is weakest in exactly the languages and legal domains where a wrong answer costs the most.

A Proposal Rather Than a Deployment

The status of the work needs stating plainly. We produced a design document rather than a running system: an LLMOps lifecycle, a set of objectives, and a list of bias, cultural-sensitivity, and observability KPIs. The team stopped before training a model, evaluating any output, or testing a single response with a community. Everything below describes intended behavior, and intended behavior earns trust only after measurement. Reading the proposal as a specification to be stress-tested, rather than as a result, keeps the claim honest.

Vulnerable Users Raise the Bar

My motivation stays concrete. I am Ecuadorian, and the people this tool would serve, communities navigating land rights, environmental harm, and the paperwork of starting a business, stand among the high-stakes users in rights and legal-advice contexts whom current systems often underserve. A fluent wrong answer reaches a first-time entrepreneur or a community defending its territory with the same confidence it would offer a lawyer, and the reader without legal training has the least ability to catch the error. Serving vulnerable users well therefore raises the bar rather than lowering it: the harder the user's situation, the less room the system has to be casually wrong.

Low-Resource Languages Raise the Safety Bar

The language choice sets the first safety problem. Spanish carries broad model support; Quechua (Kichwa in Ecuador) and Shuar lack it, and low-resource languages sit exactly where large models degrade, with thinner training data, weaker evaluation, and sparse ground truth. Joshi and colleagues document how few languages receive real NLP attention, and the NLLB low-resource translation project shows how much engineering low-resource translation still demands. A design that promises Quechua and Shuar advice inherits that fragility, so the honest version treats those languages as the place where evaluation must be strongest, rather than as a feature to announce.

Hallucination in Legal Advice Is the Core Risk

Fabrication turns from nuisance to hazard in this domain. A model that invents a statute, a filing deadline, or a land-title procedure can send a user toward a lost claim or a missed protection. The proposal reaches for retrieval-augmented generation to ground answers in verified legal sources, and retrieval grounding does reduce fabrication, yet retrieval narrows the risk without erasing it, since a model can still misread a retrieved passage or blend it with invention. Surveys of hallucination make the residual risk concrete. High-stakes legal advice therefore demands more than grounding alone; the system needs a boundary that stops it from asserting law beyond its sources.

Evaluation Must Cover Legal and Cultural Accuracy

The proposal's KPIs, covering bias, truthfulness, cultural sensitivity, and legal accuracy, name the right targets and leave the hard part unbuilt. Measuring legal accuracy in Quechua requires reference answers checked by Ecuadorian legal experts and fluent speakers, rather than a generic benchmark; measuring cultural sensitivity requires Indigenous reviewers with authority over the judgment, rather than a proxy score. Until those evaluation sets exist, the KPIs describe an aspiration rather than a result. A credible next step stays narrow: build a small, expert-reviewed test set for one legal domain in one language, publish the error rate, and refuse to expand scope faster than the evaluation can follow.

Human Review Belongs on Every Rights Claim

The safest version of this tool advises and leaves the decision to a person. A rights claim or a legal filing carries consequences that belong to a person rather than an automated answer, so the design should route every high-stakes output through a qualified human, whether a lawyer, a paralegal, or a trained community advocate, before a user acts on it. The proposal already invites community reviewers into bias auditing and human feedback; extending that principle to live advice keeps a person accountable at the point of harm. My work on auditable decision records applies here directly: an advisor that shows which sources a claim rested on, in the user's language, lets a reviewer check the reasoning instead of trusting the fluency. Human oversight forms the core of this system rather than a fallback; human oversight is the design.

Boundaries

  • The work is a design proposal rather than a deployed, trained, or evaluated system; accuracy, bias, and safety numbers remain future work.
  • Named base models and techniques, for example a pre-trained base model, retrieval, and human feedback, are design choices in the proposal rather than tested components.
  • Offline operation, voice interaction, and full multilingual coverage are stated intentions whose feasibility a build would have to prove.

Sources