---
id: ch12-known-limits
type: spine
title: "What testers and pilots found AI getting wrong"
readingTime: 3
standalone: true
core: false
teaser: "A wrong programme name, a daily usage cap, advice that ignored allergens: the project's own record of AI limits, and the fixes it names."
parent: null
diagram: null
recallQ: "What problems did PoliRural+ testers and pilots find with the project's AI tools, and which safeguards do the sources name?"
recallA: "Advisor testers met wrong or outdated facts (a Horizon 2020 reference), daily usage caps and generic advice; first-time JackDaw users felt lost and wanted more data-backed answers; pilots reported weak multilingual and place-name handling and heavy manual data curation. The named safeguards are human judgement on anything forward-looking, visible sources and limits, onboarding and in-tool help, and feedback channels."
highlights:
  - "February 2025: 8 structured walkthroughs of the Advisor found useful structure but wrong and outdated facts"
  - "Testers' own conclusion: for the future of a RAP, we need to think for ourselves"
  - "JackDaw first-timers felt lost; D2.3 recommends a 60-second tutorial and in-tool help"
  - "D4.2: user-facing outputs must carry sources, validity information and limits"
status: accepted
concept: ai-known-limits
conceptTitle: "Known limits of AI tools, and the safeguards"
parents: polirural-advisor|jackdaw
lang: en
lens: generic
depth: standard
formalism: none
visuality: balanced
lengthBand: deep
genre: explainer
state: edited
carriers: prose|table
---

The project tested its own AI tools and wrote down what went wrong. In February 2025, partners and pilot users ran **8 structured cognitive walkthroughs** of the [PoliRural+ Advisor](#c/polirural-advisor) (D2.3 Annex B). The Beta platform report (D4.2), the policy brief and the open-call report (D6.3) add more.

## The findings

| What went wrong | Where it was found | Safeguard the source names |
|---|---|---|
| The Advisor gave wrong information, naming the EU Horizon 2020 programme; a later answer was corrected | Advisor walkthroughs, D2.3 Annex B | Check facts; building context over the conversation improved accuracy |
| Answers were "a snapshot in the past", which makes forward-looking RAP advice "tricky" | Same | Human intervention for future planning |
| A daily usage limit on the free version cut sessions short | Same | Expanded access for key users, batch queries |
| AI suggestions were too generic, ignoring local needs and allergen limits | Päijät-Häme story, D2.3 §2.8 | A deliberate multi-step feedback process |
| Standard tools were not flexible enough for hands-on indicator work | Vidzeme story, D2.3 §2.8 | Custom GPT-based tools with region-specific instructions in the local language |
| First-time JackDaw users felt lost; they expected more data-backed answers | D2.3 evaluation of [JackDaw](#c/jackdaw) | 60-second tutorial, tooltips, a "Help Me Get Started" button; links to live RAP datasets |
| Multilingual and place-name handling, and map performance on rural connections | Pilot testing, D4.2 §6.4 | Strengthen both; show sources, data validity and limits in every output |
| Feeding rural, multilingual data to AI took months of manual work | Policy Brief No. 1 | Fund the "human layer": a data curation line, digital stewards |
| Applicants lacked clear maturity (TRL) expectations for JackDaw; feedback needed faster bug fixes | D6.3 §4.1 | Clearer upfront definitions; agile bug-resolution channels |

## What the testers concluded

The Advisor scored **1.0** for difficulty of access on a 1–4 scale (no problems) and **2.9** for clarity and usefulness of the dialogue, rated "useful". Testers valued its structure and its hints on funding instruments, including ones that fit Community-Led Local Development. One wrote the line that sums up the chapter: the Advisor gives ideas, but advice on the future of the RAP "can be tricky. **We need to think for ourselves**." Another noted that it "lacks detailed information that needs to be acquired during the project instead of asking a chatbot."

## Three safeguards that recur

1. **Human review** of anything that becomes a plan, a funding decision or a claim about the future (D2.3 Annex B; D4.2 §10.2).
2. **Visible sources and limits**: D4.2 §6.4 asks that user-facing outputs carry clear source references, data validity information and limitations relevant to planning decisions.
3. **Onboarding and feedback**: in-tool help for first-time users, and a widget asking "How helpful was this answer?" on a 1–5 scale with an optional comment (D2.3).

D4.2 frames such issues as evidence for the final development roadmap, not as failures of the Beta. For the evidence trail that makes errors visible, see [from data to decisions](#c/data-to-decisions).

**Sources:**
- D2.3 Rural-urban data analysis, Annex B Structured Evaluation of the PoliRuralPlus Services (§2 Advisor feedback, Tables 50–51); §2.8 Actual Pilot Stories (Stories 1–2); JackDaw recommendations §3–5 (Onboarding and In-Tool Help; Data and Service Integration; Pilot and Iterate) — [PDF](https://www.poliruralplus.eu/docs/382/D2.3_Rural_Urban_data_analysis_v1.0.pdf)
- D4.2 PoliRuralPlus platform Beta, §6.4 Issues, limitations and corrective actions; §10.2 — [PDF](https://www.poliruralplus.eu/docs/421/D4.2_PoliRuralPlus_Platform_Beta.pdf)
- D7.3 Policy Brief No. 1, Annex 10 (Execution challenges; Strategic recommendations 1 and 5) — [PDF](https://www.poliruralplus.eu/docs/349/D7.3_Policy_Brief_No.1.pdf)
- D6.3 Comprehensive Report on Funded Activities and Impact Assessment, §4.1.1 Application process; §4.1.3 Grant management and reporting — [PDF](https://www.poliruralplus.eu/docs/437/D6.3_Comprehensive_report.pdf)