Using New Technologies to Improve Qualitative Research Methods 使用新的技术来提高定性研究方法

《Using New Technologies to Improve Qualitative Research Methods 使用新的技术来提高定性研究方法》由会员分享，可在线阅读，更多相关《Using New Technologies to Improve Qualitative Research Methods 使用新的技术来提高定性研究方法（6页珍藏版）》请在装配图网上搜索。

1、Using New Technologies to Improve Qualitative Research Methods for Teacher Education and Evaluation of PerformanceJonathan GratchDepartment of Learning TechnologiesUniversity of North TexasScott J. WarrenDepartment of Learning TechnologiesUniversity of North TexasJian ZhangDepartment of Mathematics

2、and Computer Science Texas Womans UniversityAbstract: Qualitative methods have long been criticized for their time consuming process and narrow focus. The use of visual ethnography in research is as old as film itself. Critical CinEthnography combines of the visual element of cine visual ethnography

3、 and the rigorous research requirements of Critical Ethnography to produce a new method for educational research. In this paper we propose using a new technological advancement from a commercially available product to reduce the time necessary to move from transcription to analysis in the context of

4、 improving teaching by allowing administrators and experts to review a body of an instructors work rather than simple snapshots provided by walkthroughs and the yearly “dog and pony show currently required of teachers. IntroductionQuantitative and qualitative research methods have both been criticiz

5、ed over the course of the last four decades. In both instances, attacks have been leveraged from a theoretical perspective in terms of either method portraying concepts of truth and knowledge too narrowly (Hollis, 1994). In the case of quantitative methods, the low generalizability of findings due t

6、o limited sample sizes, lack of replicability, and the relative, rapidly changing social natures of the subjects of the research have challenged researchers to seek improvements to methods. Applying qualitative measures instead of quantitative has helped to address the limits of the laser-focus of m

7、uch quantitative research and its reliance on confirming or rejecting hypotheses while failing to capture how and why a result came to pass. However, qualitative methods have also come under fire because of the limited lens of singular researchers engaged in anthropological and observational methods

8、 that allow for high subjectivity or are criticized as being too time consuming or narrowly focused (Denzin & Lincoln, 2003).According to Creswell (2007), many researchers seek alternative theoretical approaches and discontinue traditional concepts. This has led to new methods of evaluating training

9、 and teaching. This change in approaches comes mainly from ethnography, but also includes writers in grounded theory, narrative research, and phenomenology. The focus of qualitative research should be on the phenomenon being explored rather than solely on the reader, the researcher, or the participa

10、nts being studied. There are some in the research community who believe the purpose of qualitative research should be to advance a social justice agenda; however, Creswell suggests that not all qualitative projects absolutely must have this agenda as the central characteristic. Other researchers “ex

11、pect educational research to produce generalizable, unambiguous, and immediately applicable solutions to complex educational problems (Freeman, Marrais, Preissle, Roulstson, & Pierre, 2007, p. 30). Finding research methods that can represent the multiple layers of human experience is challenging and

12、 sometimes limiting. Some argue that a qualitative method is not just a matter of opinion nor is good qualitative study simply a matter of meeting a checklist of criteria.Unlike many quantitative, positivist methods, qualitative research methods can be labor intensive. This makes the benefits of the

13、 context and depth of participant experience it provides difficult to justify for many researchers. Transcription of interviews and video data sets, such as captured in situ footage, is time consuming. Automated coding tools in ATLAS.ti and NVIVO require highly detailed transcripts, making the entir

14、e process of qualitative research challenging. The primary goal of this paper is to describe our methods for overcoming these problems and to report the outcomes from a pilot attempt to implement these technical methods for studies in the classroom environment towards a goal of helping instructors i

15、mprove their teaching.Participants and settingInterviews were recorded over a three-day period in a North Texas high school. Fifteen students, and eight faculty members participated in the study. Each group had a predesigned interview list. Students were selected randomly based upon an available poo

16、l. Teacher participation was based upon availability. Research methodsAll teacher participants were observed in the classroom. Interviews were conducted employing scripted and unscripted questions resulting from direct observations. These were used to gain further information on a particular subject

17、 or to delve deeper into interviewee views. All interviews require transcription and intensive coding, categorization, and building of themes.Ethnographic filmEthnographic film used for research, which is as old as the cinema, is another way to record cultural artifacts and subjects of interest in a

18、 research study. Rouch (2003) and de Heusch were two early ethnographic filmmakers that help to record and reveal cultural patterns. An early distinction that was useful in discussing this field was between ethnographic footage and ethnographic films. Footage was the raw material that came out of th

19、e camera with no sets of expectations attached to it, while films were structured works made for presentation to an audience (McDougall, 1978). Films first using cultural and educational studies appeared in the 1930s. As soon as technological development made it practical to do so, video recording b

20、egan to be used (Givvin, Hiebert, Jacobs, Hollingsworth, & Gallimore, 2005).Ethnographic film is used to join the art and skills of the filmmaker with a qualified intellect and insights of the ethnographer (Heider, 2006). Ethnographically shallow films are a come-in-shooting-and-get-out-fast approac

21、h to film making. This does not give a detailed description and analysis of human behavior based on a long-term study. Many documentary films devote much time to the portrayal of an individual person or event, which is an essential feature in ethnography, but it falls short of putting the data into

22、a cultural context.Critical ethnography emerged in the mid-1990s after the basic ethnographic methods were reformulated as an alternative to conventional and postmodern qualitative research. Some of the standards of ethnographic practice, such as grounded theory, the use of analytical concepts such

23、as validity, generalizability and so on were either rejected or reformulated and used in critical ethnographys (re)construction (Jordan & Yeomans, 1995). Two problems found in the 1990s with critical ethnography were: practitioners came from the ivory towers of academia and had trouble relating to t

24、hose they studied; further, their research findings were insufficiently protecting those at the research site. Critical ethnography is not in opposition to conventional ethnography; instead, it is about the relationships among knowledge, society, and political action. The critical theory perspective

25、 focuses on issues relating to ethics and performance (Madison, 2005). “Critical theorists, constructivists, and participatory/cooperative inquirers take their primary field of interest to be precisely that subjective and intersubjective social knowledge and the active construction and co-creation o

26、f such knowledge by human agents that is produced by human consciousness (Denzin & Lincoln, 2003, pg. 271). Thus, critical ethnography does not focus on testing of a hypothesis or outcomes, but on the discovery, interpretation, and application of local knowledge to how it can be used or put into pra

27、ctice. “Critical theorists tend to locate the foundations of truth in specific historical, economic, racial, and social infrastructures of oppression, injustice, and marginalization. Knowers (sic) are not portrayed as separate from (sic) some objective reality, but may be cast as unaware actors in s

28、uch historical realities (“false consciousness) or aware of historical forms of oppression, but unable or unwilling, because of conflicts, to act on those historical forms to alter specific conditions in this historical moment (“divided consciousness). (Denzin & Lincoln, pg. 273)Therefore, studies t

29、hat invoke a variety of data collection strategies and sources interviews, narratives, memos, newspaper accounts, library, and historical records, casual conversations, group discussions, photographs, epidemiologic statistics about usage patterns, etc. is a good use of scientific research that can u

30、ncover socio-culture knowledge about an unfamiliar or little-known group (Averill, 2006).Critical CineEthnographyThese methods, which we call Critical Cine-Ethnography, blend Carspeckens (1996) and seek the establishment of validity through a stringent five-step process. These data collection method

31、s also incorporate Rouchs (2003) approach to cine-ethnography. Specifically, we sought to incorporate what Denzin and Lincoln proposed is the major goal for social research:“Thus, we have two arguments proceeding simultaneously. The first, borrowed from positivism, argues for a kind of rigor in the

32、application of method, whereas the second argues for both a community consent and a form of rigor-defensible reasoning, plausible alongside some other reality that is known to author and reader-in ascribing salience to one interpretation over another and for framing and bounding an interpretive stud

33、y itself. (Denzin & Lincoln, 2003, p. 275)We have further established a series of goals and authenticity criteria for qualitative research methods that our own seek to address (p. 278). These are: Trustworthiness Rigorousness Fairness Ontological authenticity Educative authenticity Catalytic authent

34、icity Tactical authenticityIn order to reach these goals and meet these criteria, we have particularly sought to: Give participants and researchers voice, whether unified or disparate. Allow researcher reflexivity in filmed reflections on choice of research problem and participants as well as how we

35、 develop and change as researchers in the field (p. 283) Allow for multiple representations (film, textual, dramaturgical, audio, other) and allow for first and third person accounts. Allow viewer and researcher/filmmaker interpretations not one or the other Reveal researcher bias through self repor

36、t and discourse Include multiple observers and analysts who bring perspectives from multiple genders, races, etc. as part of analysis as well as participants to seek partial validity from findings/images Include narration only from validated themes while showing and letting speakers tell and confirm

37、 outcomes Leverage stringent validity construction process borrowed and adapted from Carspecken (1996).Interviews are necessary at several different times during the research process in a Critical CinEthnography. It is important to gather some information directly from participants prior to engaging

38、 in direct observation, if only to identify those you are observing as you build your thick record. Additional interviews as the need arises should be conducted throughout the data collection period as learning and teaching behaviors of interest come up in the class. Further, once the direct observa

39、tion is done, interviews should be conducted both immediately Interviewing, especially when done for film, is a challenging skill that takes years to gain competence in for gathering relevant data. It requires knowledge of ones participant, their values, and is sensitive to what they are comfortable

40、 with saying on film. While it may be wonderful as a researcher to have a participant on the record stating every systemic dysfunction they know of in their local school system, getting a participant to name these when their supervisors, peers, and students are likely to review everything they say.

41、A researcher must be sensitive in their questioning as to what it is acceptable to ask a teacher, student, or administrator, because the very nature of the question can create problems for participants and interviewer.Data analysis methodsIn order to conduct data analysis with a Critical CinEthnogra

42、phy, we adopted and modified Carspeckens methods outlined in his text Critical Ethnography in Educational Research (1996). This is a five-step process that involves first capturing data and researcher reflections in a thick record, conducting reconstructive analysis of the data, conducting follow-up

43、 interviews to validate findings from the reconstructive analysis, describing the relationship of the findings and participant speech acts with respect to other systems, and finally, explaining these findings with respect to cultural, power, political, and other systems. Each step in the process is

44、one of building from low inference observations up to a high level of inference that is grounded in low and medium inferences that have been confirmed through a stringent validation process. We provide an overview of this process here and break it further into a ten-step process as Step Two consists

45、 of a large number of equally important sub-stepsFilmingInterviews were recorded on three cameras, two of which permitted recording directly to flash memory, the third camera recorded to HDV cassette tapes. Both the interviewer and the subject were wired with a wireless microphone for accurate sound

46、 recording, the audio was then input into the cameras for recording onto the cameras default media. In all over 900 minutes of audio and video were recorded per camera over the three day period. Transcribing and analyzing all of the accumulated footage was proving to be a daunting and time consuming

47、 process.Technical analysis methodsIn order to lessen the process of transcribing all recorded footage we turned to the commercially available Adobe Master Suite CS6 ( adobe ). Adobe added a speech analysis feature in CS4 (2021) and significantly refined the application and added features such as fa

48、cial recognition and better speaker attribution. The speech analysis component of CS6 was originally implemented for projects moving from script stage into production to provide a comparison between the intended dialogue and the recorded content. The speech analysis component has the capability to c

49、ompare the spoken word against a reference script, and doing so increases the reliability of producing an accurate transcript. The speech analysis component is found in multiple applications within the Adobe Master Suite, including Premiere and Audition. For the purposes of this study we ran speech

50、analysis on video with audio under Premiere CS6, a nonlinear editing software tool. Adobe Premiere was selected as our editor of choice for both its inclusion of the speech analysis component and its ability to simultaneously handle multiple video formats within a single sequence or project, thus sa

51、ving us hundreds of hours by removing the need to transcode all the video footage to a an intermediate format. Removing the requirement to shift all of the different video formats to a single format to work within the editor produced the added bonus of reducing the amount of disk storage space neces

52、sary for the project. Speech analysis One of the most time consuming tasks in qualitative research is transcription of interviews and classroom video footage. The speech analysis tools in Adobe Premier allowed was expected to transcribe interviews at between a 60 to 80% accuracy rate, depending on t

53、he speaker. This substantially reduced the time it took to transcribe a 45-minute interview that would ordinarily take three to five hours to complete. With multiple interviews in a single study is onerous and discourages researchers from engaging in qualitative research. An added benefit is that th

54、e speech analysis tool embedded metadata within the transcript for later use. Premieres speech analysis retains the record of spoken word in relations to frame rate, audio sample, duration and provides a probability of accuracy of a spoken word at that given time/frame/sound stamp. The transcript ca

55、n be used as a reference guide within the editor and used to identify the specific data with context, which is viewable and searchable by other applications. Method OutcomesThe outcome of our research on the implementation of this technological tool was that it substantially improved both the accura

56、cy of our transcript and reduced the overall time it took to conduct analysis on the sample interviews. One of the deficiencies of the Adobe Premiere speech analysis tool is that multiple speakers, regardless of gender, often generate increased errors than a single speaker. This required more time s

57、pent correcting the transcript than was expected. One way we propose to overcome this problem is to separately employ microphones to capture each speaker, which will allow for separate automated coding, and should improve accuracy.Hand coding of the same speakers prior to employing the speech analys

58、is tool, the initial 48-minute interview took 4.5 hours to complete. In comparison, coding using the speech analysis tool took less 1 hour to transcribe and an additional hour to review and correct, for a total of 2 hours, a substantial improvement over traditional transcription. This allowed us to

59、produce a transcript and employ the automated coding tool within Atlas.ti, a commercially available, qualitative research product for initial completion of analysis in hours, reducing a months or years-long process to about a month for eight interviews.One major weakness of the tool is for environme

60、nts with many speakers not directing their voices to the microphone. In this case, the accuracy of the transcript drops to less than 30%. For one classroom filmed, the accuracy was less than 10%, which we hypothesize was because of the large amount of background noise that muffled student and teache

61、r voices.Application for Teacher EducationTwo likely avenues for employing Critical CinEthnographic methods are in the areas of 1.) improving the teaching of existing or future classroom instructors and 2.) to study the systems in which instructors teach in order to understand the political, social,

62、 economic, pedagogical, and other critical impacts on the act of teaching. The act of both capturing teaching as well as those the visual and auditory artifacts produced by educational institutions should allow better contextualization of how teaching happens, what leads to positive changes, and wha

63、t might restrict it. We provide two examples; the first is one in which our research seeks to improve practice through the observation of teaching. The second is an instance in which we sought to better understand the larger systems and their affordances within which instructors seek to teach.In the

64、 case of the first, one university instructor spent a semester capturing their own teaching as well as daily reflections on both the educational system in which they worked, their pedagogical practices, their lifeworld experiences, and artifacts that were the result of his teaching practice. In orde

65、r to reduce the analysis time later, the participant was asked to provide self-codes and develop categories for this reflection as he wrote them in order to better capture the experience. Analysts were then able to review the teaching that the instructor did, opening it for dramaturgical critique, while holding an understanding of the personal and professional experiences of that instructor which would impact practice. This allowed for specific recommendations to be made with an eye towards understanding what could be