Translating user testing insights into impact
Testing & Evaluation
May 2024
International Institute of Tropical Agriculture

In order to design and target new dietary improvement programs and policies, IITA, Viamo and the government of Rwanda have partnered to generate a crowd-sourced tool to improve the availability, access, frequency and quality of data on diets. The group co-developed an unstructured supplementary service data (USSD) based system to collect diet quality data in Rwanda, adapting the global standard Digital Quality Questionnare (DQQ). To validate this modality of the DQQ deployment via USSD against face to face interviews (the standard data collection for dietary data), a validation study was run in July and August 2023 to assess how the modality of DQQ collection affects, if at all, the responses of participants.

Addressing a critical gap in global nutrition monitoring, the Global Diet Quality Project (DQQ) was launched in 2016 by the Global Alliance for Improved Nutrition (GAIN), in collaboration with Harvard School of Public Health and other partners. The initiative sought to create a standardized, low-cost method for collecting comparable dietary data across countries—a challenge that had long hindered efforts to track and improve diet quality worldwide.
Over the past decade, the project team developed and validated a robust, replicable methodology, resulting in the Diet Quality Questionnaire (DQQ). This innovative tool is being carefully adapted for each country through an inclusive process involving over 1,000 key informants in 140 countries, ensuring alignment with global guidance from the FAO, WHO, and UNICEF. In Rwanda, the team tried to make use of the ubiquously present technology of USSD to scale the survey distribution - but how did this change peoples' interaction with the survey questions?

Read More

Mobile phones are transforming how we collect data — faster, cheaper, and far more scalable than traditional in-person surveys.

Problem Statement

How might we improve the user experience of a mobile phone survey focused on diet quality?

Solution

Learning from users - a pop up usability lab

Over two weeks in Rwanda, we took a hands-on approach - part of the time based at the IITA office in Kigali, and part of the time right inside respondents' homes. Our team blended local expertise and global design thinking: a human-centered design specialist, a local scientist deeply familiar with the tool, and four trained enumerators. We tracked usability progress live through the platform provider, VIAMO, carefully monitoring data quality and drop-off rates before and after each design improvement. Research questions 1. Can respondents easily complete the questionnaire via USSD on their mobile phones? 2. Do survey respondents like to answer questions about their diet using their phones?

Approach & Methodology

Over two weeks in Rwanda, we took a hands-on approach — part of the time based at the IITA office in Kigali, and part of the time right inside respondents' homes.
Our team blended local expertise and global design thinking: a human-centered design (HCD) specialist, a local scientist deeply familiar with the tool, and four trained enumerators. We tracked usability progress live through our service partner, VIAMO, carefully monitoring data quality and drop-off rates before and after each design improvement.

Key Results

1. The overall start-completion rate rose from 58% to 70%.

2. The start-completion rate for women - with overal lower digital skills - rose from 56.5% to 70%

3. Data quality (measured in low deviant responses) increased significantly.

The lab testing revealed challenges in dealing with a scientific survey on USSD - even though frequently used - that made people drop the survey early, or even type in wrong answers

The lab testing revealed several pain points that risked undermining both data quality and participant engagement. The most pressing issues included

  • Frequent system errors and timeouts, which disrupted the survey experience and left users frustrated. Without the ability to correct or prevent errors within the USSD interface, respondents often struggled to navigate the process smoothly.
  • Comprehension proved another hurdle: some questions were unclear, and the small screens of basic phones made text difficult to read.
  • The lack of in-context help generally built in USSD systems left users without guidance when they needed it most.
  • Impact

    Recognizing these challenges as opportunities for improvement, the project team implemented a series of targeted enhancements.

  • Performance between the survey platform and telecommunications providers was optimized to minimize system errors and reduce interruptions.
  • Instructions were clarified and streamlined, with explicit instructions - such as when to respond “yes” - added to guide users and reduce data entry mistakes.
  • The survey structure was also revised to separate framing questions from response options, eliminating unnecessary repetition and making the process more intuitive
  • These changes not only addressed the technical and usability barriers but also ensured that the DQQ could deliver reliable, high-quality data while providing a smoother experience for participants.

    What our project partners are saying

    “This was such a cool, detailed piece of work! With the help of these rich insights we managed to improve our surveying tool, and increased completion rates for the diet surveying we were supporting our colleagues at the CGIAR with!”

    Anna Fraenzel, Viamo

    Is this service for you?

    Our usability testing is a complete, quick assessment covering the identification of your questions and doubts, testing design, participant screening, test implementation and recommendations for change. We help you see your product or service through the eyes of your users. It’s ideal for:

    • Web and mobile platforms
    • Digital tools and services
    • Even analog service experiences, with adaptation

    We customize every project to your needs. Whether you’re planning a redesign or just want an expert outside perspective - we’ll help you scope the right approach.

    “Usability testing allowa your the team to step into the users’ shoes and see the innovation from their perspective - small adjustments to service design can have a big impact on the user experience.” — Charlotte, Research & Strategy Lead | Glashaus Innovation

    Book A Call

    Check more projects

    VIEW

    Capturing hidden impacts with human-centered evaluation

    Mapping change together: Participatory evaluation captures hidden impacts
    VIEW

    In the breeder's shoes

    Persona development for digital phenotyping tools - Are we building the solution right for intended users?

    Let’s talk

    How your innovations cAN

    Thrive.