Dunja Dreesens

50 The survey was followed by an online RAND-modified Delphi. This consensus method was chosen with the aim to reach agreement with regard to different opinions, fluctuations in use and number of tools over time, and political involvement as the Dutch Ministry of Health, Welfare, and Sports was involved (via the National Health Care institute) in the development and monitoring of knowledge tools. The ministry wanted to have some influence regarding the quality of the tools developed, monitor efforts by the parties involved and see to an improved synchronized effort in developing these tools. Defining these tools therefore became of interest to the ministry as well. By using consensus methods, one can determine to which extent the participants agree with the issue posed and to which extent participants agree with each other. Choosing the RAND-modified Delphi also prevented more vocal people to dominate (153). The Delphi process consisted of two rounds. In both rounds, the participants were asked to score the tool types on a Likert scale according to the importance of the tool type (0-8; not important at all – very important) and their agreement with the definition provided (0-8; very much disagree – very much agree). In addition, they had the possibility to suggest amendments to the definitions provided for each tool and to add tools, preferably including a description. In the first round, the participants scored without seeing any scores of the other participants. In the second round, the scores and comments were visible to all participants, who were also able to discuss these with each other. The sequence of the tools was randomly determined for the first Delphi round. The different tool types were printed on paper and a colleague, who was not involved in the study, blindly selected the papers from a non-transparent bag. In the second round, we reversed the sequence of the tools and provided adapted definitions based on the first-round comments. The scoring and comments were analysed by the first author and checked by the project group before these were shared with the participants. Data analysis Consensus on relevance/importance of the tool type and agreement with the definition was defined as: the median of the scores ≥ 6 AND 70% of the participants’ scores were in the Likert score cluster 6-8. Consensus on irrelevance/unimportance of the tool type and disagreement with its definition was defined as the median of the scores ≤ 2 AND 70% of the participants’ responses are in the Likert score cluster 0-2. Tool types were discussed if: 30% of the responses were in group 0-2 AND 30% of the responses were in group 6-8. All other scores indicated ambiguity, and results scored by fewer than nine participants were considered invalid (see Appendix A). Concerning the suggestions for the definitions made by the participants, the first author compiled the suggestions, determined whether to adapt the definition, and discussed the findings with the project group. The same approach was followed for added tool types suggested by the participants. To conclude the study, a linguist aided the first author in editing and finalising the definitions of the tools for which the experts had assessed the definitions. Chapter 3