General-purpose AI tools are powerful but not always compatible with highly specialized contexts.

Survey Solutions, the World Bank open-source platform for digital data collection, is used by Governments, multilaterals and data collection agencies in 175 countries worldwide.

One challenge with this platform is that when programming questionnaires, it uses a constrained dialect of C# with unique architectural rules. Developers building questionnaires on it need specialized guidance: cannot use certain classes, cannot declare variables freely, and the logic flows differently than standard C#. Yet when we asked Claude, Gemini, and GPT for help, they defaulted to mainstream C# advice – theoretically sound but practically useless.

Building a Specialized Chat-Bot

By building a chat-bot with a specialized narrow prompt explaining it how the Survey Solutions environment and interface differs from mainstream C# we started to see improved results immediately. The prompt was developed through a mixture of the official documentation and instruction from an experienced Survey Solutions programmer to catch all of those small quirks of programming. The chat-bot was launched in a beta phase with users providing feedback on examples where the advice was still not right, and the prompt was iterated to improve performance.

But the real breakthrough came from enabling better inputs. Two capabilities were added to make the system more user friendly while providing better inputs for the model to work with:

  • PDF Upload with Large Context Windows – Developers can now upload their exported questionnaire design directly. The system performs quality control, flagging logical inconsistencies, syntax errors, and architectural issues before they become headaches in the field.
  • Vision Capability – A screenshot tells a thousand words. Users upload their designer view, and the AI sees exactly what they are building – the branching logic, the variable dependencies, the nested rosters. This visual context dramatically improves the advice the system can offer.
  • Additionally, an educational component was added to the prompt, the chat-bot gives a brief description of how and why the proposed solution will work. The aim here was to not only give a one-off answer but to simultaneously build capacity and knowledge.

Key Takeaways

  • When users ask AI for help and get wrong answers (like generic code that breaks their survey) they get frustrated. If an AI wastes their time, they will quickly stop using it.
  • General AI simply doesn’t work for highly specific platforms. Teaching the AI the exact rules and quirks of a narrow system like Survey Solutions is the difference between advice that just sounds good and advice that actually works.
  • An AI is only as good as the information you give it. Instead of forcing users to write long, complicated explanations of their problem, letting them just upload a PDF or a screenshot gives the AI exactly what it needs to see. Make it easy for users to share their work, and the AI will give much better help in return.

Open for Exploration

The Survey Solutions AI Assistant is live and free to use: https://surveysolutions.impactengines.ai/

This is the kind of work Impact Engines pursues, applying our deep, specialist knowledge to tailor AI solutions. We believe AI is an incredibly powerful tool which we seek to harness for social good by optimizing processes, prompts and inputs.

Impact Engines builds responsible, practical AI for the not-for-profit and development sector. Learn more about our work at https://impactengines.ai