Data collection for my Delphi study was completed as planned on February 8th. (As if on cue, Clark was born the next morning!)
I received 12 completed surveys in response to the final consensus check. Thankfully, this was the minimum number I set out to gather – so the level of attrition was acceptable, especially through rounds 2 and 3 and the final consensus check. Twenty-four experts initially agreed to participate in the study. Of these, only 15 people actually completed round one. Thirteen completed round 2, and twelve completed both round 3 and the final consensus check. So, following round 1, the attrition rate was only 20%. Those who left the study were not among the significant dissenting opinions.
In the final consensus check, consensus was defined in the following way:
For the purposes of this study, consensus is defined as the state in which the results are “at least acceptable to every member [of the expert panel], if not exactly as they would have wished.” (Reid, 1988, as cited in Williams & Webb, 1994, p. 182).
Participants were then asked to rate their level of consensus with each of six summaries on the following scale (Participants were also invited to leave additional comments after rating their level of consensus with each summary):
- 5. Complete Consensus – I am in agreement with everything stated in this summary. The results are acceptable to me, if not exactly as I would have wished.
- 4. High Level of Consensus – I agree with most of what is stated in this summary, and I disagree in only minor or insignificant ways. The results are acceptable to me.
- 3. Moderate Level of Consensus – I agree with much of what is stated in this summary, but I also disagree in some ways. The results are acceptable to me.
- 2. Low Level of Consensus – I agree with some of what is stated in this summary, but I also disagree in some major or significant ways. However, the results are still acceptable to me.
- 1. No consensus – I disagree with most or all of what is stated in this summary. The results are not acceptable to me.
These ratings were used to find a level of consensus between the participants. Though many participants selected “5. Complete Consensus” in response to individual items, no items received that rating from all participants, so it would be inaccurate to report that there was complete consensus on any of the summarized themes. However, the participants ratings were averaged and the following scale was used to determine the level of consensus among the participants:
- 5.0 Complete Consensus
- 4.50-4.99 Very High Level of Consensus
- 3.50-4.49 High Level of Consensus
- 2.50-3.49 Moderate Level of Consensus
- 1.50-2.49 Low Level of Consensus
- 0.00-1.49 No Consensus
At this point in the process I have several new tasks ahead of me.
- Final Data Analysis: I will code and complete analysis of the Final Consensus Check comments. These will be used to modify the summaries used in the final consensus check for their final appearance in my report.
- Confirmability Measures: I will work with two of my colleagues, both of whom have recently completed in a doctoral dissertation focused on educational technology. They will aid me in a peer debriefing, in serving as devil’s advocates, and (in one case) in serving as an external auditor of my Delphi methodology.
- I will complete a literature realignment to account for the months between completion of my proposal and the beginning of this dissertation draft.
Then I can move on to rewrite chapters 1 through 3 to reflect the actual implementation of this study – and to compose chapters 4 and 5 to report my results and discuss their implications. Though I had hoped to have a draft of my dissertation by March 1st, I don’t think that is possible at this point. However, I still hope to have a draft completed sometime in March. I think it’s time to take on no new work until this is done… especially with a new baby in the house.