A structured approach, from the question to the use of findings
An evaluation or study is only useful if its findings are reliable and understood by those who must decide. Our approach follows six steps, framed by principles that apply to every assignment.
The six steps at a glance
- Step 1ScopingReview of the terms of reference, theory of change, evaluation questions and matrix based on the OECD-DAC criteria.
- Step 2DesignMixed methods, documented sampling plan, data collection tools pre-tested and translated into local languages.
- Step 3Data collectionDigital collection with KoboToolbox or ODK, trained local teams, remote methods where access is restricted.
- Step 4Quality controlReal-time supervision, back-checks, consistency checks and data quality assessment (DQA).
- Step 5AnalysisTriangulation of sources, disaggregation by sex, age, disability and location, clear separation of findings and interpretation.
- Step 6ReportingReadable reports, validation workshops and prioritised recommendations geared towards learning.
The six steps in detail
- 01
Scoping
Understanding the request before choosing methods.
- in-depth review of the terms of reference and programme documents;
- scoping interviews with the client and implementing teams;
- review or reconstruction of the theory of change;
- formulation of evaluation questions based on the OECD-DAC criteria and development of the evaluation matrix.
- 02
Design
Choosing methods that answer the questions within the limits of the context.
- combination of quantitative and qualitative methods (mixed methods);
- documented sampling plan: population, size, strata, selection, weights;
- tools programmed with entry checks, pre-tested and translated;
- risk analysis and security plan for each area.
- 03
Data collection
Collecting data in good conditions, for respondents and for quality.
- digital collection with KoboToolbox or ODK, with regular data uploads;
- trained enumerators, recruited wherever possible in the areas concerned;
- informed consent, interviews in safe and confidential settings;
- remote collection (telephone, key informants) when physical access is impossible.
- 04
Quality control
Detecting and correcting errors during data collection, not afterwards.
- daily data checks: interview duration, outliers, inconsistencies;
- back-checks: a share of respondents is re-contacted to verify answers;
- field supervision and daily team debriefs;
- data quality assessment (DQA) when we use partners' data.
- 05
Analysis
Establishing robust findings and stating how far they can be generalised.
- statistical analysis with reproducible scripts, applying weights where relevant;
- thematic analysis of qualitative data using a coding framework;
- triangulation of sources and methods;
- disaggregation by sex, age, disability and location, with limitations stated explicitly.
- 06
Reporting
Making findings useful to those who must decide.
- validation workshops with stakeholders;
- structured reports separating findings, interpretation and recommendations;
- prioritised recommendations stating who should act, on what and by when;
- feedback to communities where possible and safe.
Our methodological standards
These standards apply to every assignment. They are spelled out in the inception report, so that the client knows exactly how data will be produced and analysed.
OECD-DAC criteria and theory of change
The six criteria (relevance, coherence, effectiveness, efficiency, impact, sustainability) structure the evaluation questions; the theory of change links activities to expected results and allows the programme's assumptions to be tested.
Mixed methods and triangulation
Numbers tell us how much; interviews explain why. We cross-check sources (beneficiaries, partners, authorities, documents) and methods to verify every important finding.
Rigorous sampling
A survey's representativeness depends on its sampling design. We document every choice so that an external reader can judge the scope of the results.
Data quality control
Automatic checks in forms, daily reviews, back-checks and DQA: quality is built at every step.
Digital data collection
KoboToolbox and ODK reduce entry errors, allow real-time monitoring of data collection and work without a permanent connection.
Data and respondent protection
Informed consent, anonymisation, restricted access to databases, device encryption and deletion of identifying data once it is no longer needed.
Conflict sensitivity and Do No Harm
We analyse how data collection may interact with local tensions: choice of enumerators, locations, questions and timing.
Gender and inclusion
Female enumerators for interviews with women where needed, participation of people with disabilities and marginalised groups, disaggregated data.
Remote data collection in hard-to-reach areas
Telephone interviews, key informant networks, resident enumerators and image-based verification when security prevents site visits.
Planning an evaluation, a study or a monitoring system?
Send us your terms of reference or describe what you need. We will come back to you with an initial methodological reading and the terms of our involvement.