In the methods chapter of my proposal, one procedure I stated I would follow during the data collection and analysis phases of my study was to keep a research log. Because I am not revealing any sensitive data or sharing results that might skew the study, I have decided to share my experiences here as well. (Entries will appear in an edited form in order not to influence the study if participants happen to read this blog.)
UPDATE: It’s now 2008-01-02 and with the conclusion of data collection I am now adding back in a paragraph that does discuss specific results below. It begins with the word “Specifically”.
The first round of the study was originally scheduled to conclude tomorrow. However, I have only collected five responses. Out of more than sixty invitations to participate there are now fourteen confirmed participants, with a potential for two to three more. The good news is that their levels of expertise are very much what I had hoped for, which will add to the credibility of the study, though of course their identities will remain anonymous. However, the minimum number of responses I called for in my proposal was twelve, so I plan to send out reminders today and extend the deadline to Friday the 21st at least. It is a difficult time of year to conduct surveys. I knew this would be the case and I know I will need to be flexible in order to finish in time to graduate this May.
Nevertheless, once I received my first three responses I began to organize and prepare data for analysis. Also, I began early data analysis, using Tams Analyzer for OS X to create an initial coding scheme from the first three responses. Already the categories (and thus potential questions) I may include in the second round of the Delphi have already grown beyond my original six. I’m sure I will need to synthesize and condense the results to allow for a manageable and productive second round.
Specifically, there has been a focus on active learning, depth of learning, and differentiated learning, all of which may fall under my category of constructivist learning, as problem solving might, too. There has been some focus on hard fun, as well as the expected discussion of motivation and engagement. The importance (and inherent educational value) of gameplay has also been mentioned. There has been little mention so far of social benefits, other than some discussion of the natural marriage of games and the ZPD. One category discarded during the proposal stage was 21st Century Skills, but those issues are making an appearance in participant answers, particularly risk taking. Role playing has also reappeared in participant answers as well. Note that this early analysis has focused on question 1, which focuses on the potential benefits of MMORPGs in education. I have not yet begun analysis on question 2, which focuses on the potential drawbacks of MMORPGs in education.
This morning, I will be sending an email to the participants thanking those who have completed the first round and prompting others to complete the survey. A few participants who joined later will be receiving their round 1 questions this morning. And finally, a few others I expect might still want to join will receive an invitation or prompt for response.
I also plan to add two most recent responses to my Tams Analyzer project and add their content to my coding scheme for question 1. I will also begin reading and analyzing responses to question 2.
In addition, as I review my methods chapter I am looking ahead to identifying a colleague familiar with the subject matter to serve as a devil’s advocate to the results, and to identifying a colleague familiar with the method to serve as an external auditor.