Journal of Distance Education/Revue de l‘enseignement à distance (2001)

ISSN: 0830-0445

Assessing Social Presence In Asynchronous Text-based Computer Conferencing

Liam Rourke

Terry Anderson

D. Randy Garrison

Walter Archer

Liam Rourke is a doctoral student specializing in instructional technology in the University of Alberta’s Department of Educational Psychology. His e-mail address is lrourke@ualberta.ca.

Terry Anderson is a professor in the Faculty of Extension and Associate Director of the University of Alberta’s Academic Technologies for Learning (ATL) unit. Terry has 15 years of experience related to distance education as a teacher, researcher, and administrator, including employment as Director of Contact North in Northern Ontario. His e-mail address is terry.anderson@ualberta.ca.

D. Randy Garrison is a professor and Dean of the Faculty of Extension at the University of Alberta. His areas of research relate to the teaching and learning transaction in the context of adult, distance, and higher education. He has published extensively in both of these areas. His e-mail address is randy.garrison@ualberta.ca.

Walter Archer is a professor and Assistant Dean, Research, in the Faculty of Extension at the University of Alberta. He is also Acting Director of the new distance-delivered Master of Arts in Communications and Technology. He has 12 years experience with CMC and other modes of distance delivery. His e-mail address is walter.archer@ualberta.ca.

Abstract

Instructional media such as computer conferencing engender high levels of student-student and student-teacher interaction; therefore, they can support models of teaching and learning that are highly interactive and consonant with the communicative ideals of university education. This potential and the ubiquity of computer conferencing in higher education prompted three of the authors of the this article to develop a community of inquiry model that synthesizes pedagogical principles with the inherent instructional and access benefits of computer conferencing (Garrison, Anderson, & Archer, 2000). This article explicates one element of the model, social presence. Social presence is defined as the ability of learners to project themselves socially and affectively into a community of inquiry. A template for assessing social presence in computer conferencing is presented through content analysis of conferencing transcripts. To facilitate explication of the scheme and subsequent replication of this study, selections of coded transcripts are included, along with interrater reliability figures. The article concludes with a discussion of the implications and benefits of assessing social presence for instructors, conference moderators, and researchers.

Abstract

Les médias éducatifs, tel le forum électronique, sont susceptibles d‘engendrer un niveau élevé d‘interaction entre les étudiants et entre le tuteur et chaque étudiant; par conséquent, ils peuvent soutenir des modèles d‘enseignement et d‘apprentissage qui mettent en valeur l‘interaction et qui sont compatibles avec les idéaux communicationnels d‘une éducation universitaire. Ce potentiel, ainsi que l‘ubiquité du forum asynchrone dans l‘éducation supérieure, ont mené trois des auteurs de cet article à développer un modèle de communauté de recherche qui résume les principes pédagogiques applicables au forum électronique en incluant les bénéfices que l‘enseignement peut en retirer et les avantages relatifs à l‘accessibilité (Garrison, Anderson, & Archer, 2000). Cet article explique un élément du modèle : la présence sociale. On y définit la présence sociale comme la capacité des apprenants de se projeter sur le plan social et émotionnel dans une communauté de recherche. On y décrit un gabarit utilisé pour évaluer la présence sociale lors d‘une analyse de contenu de transcriptions de forums électroniques. Afin d‘aider la compréhension des modalités d‘évaluation et de faciliter la reproduction subséquente de cette étude, un échantillonnage de transcriptions codées ainsi que les barèmes de fiabilité inter-évaluateurs sont fournis. L‘article se termine par une discussion des conséquences et des avantages de l‘évaluation de la présence sociale du point de vue des formateurs, des animateurs de conférence et des chercheurs.

Computer-mediated conferencing (CMC) is unique among distance education media because of its ability to support high levels of responsive, intelligent interaction between and among faculty and students while simultaneously providing high levels of freedom of time and place to engage in this interactivity. These characteristics are making computer conferencing the dominant choice for distance learning in many institutions, especially when higher-order thinking goals are at issue. At the University of Alberta alone, over 400 courses include some form of computer conferencing (Susan Stein, ATL listserv, October 23, 1999). Therefore, it is important (a) to develop research methods that explore the nature of teaching and learning in these environments; (b) to apply these tools in authentic contexts; and (c) to use the results to develop instructional models that use this technology effectively. This article addresses each of these areas. It begins with a review of Garrison, Anderson, and Archer’s (2000) community of inquiry model, which was specifically designed to guide the use of computer conferencing to support critical thinking in higher education. The review focuses on one element of the model: social presence. A review of the development of the construct social presence is presented, followed by the introduction of a template for the assessment of social presence. This template is based on the content analysis of transcripts of computer conferencing. Selections of coded transcripts, a thorough description of the coding protocol, and a report of interrater reliability are followed by a discussion of the implications and benefits of assessing social presence for instructors, moderators, and researchers.

The Community of Inquiry Model

Garrison et al. (2000) present a model (see Figure 1) of the teaching and learning transaction that capitalizes on the ease and abundance of interaction that is possible with media such as computer conferencing. In this model, deep and meaningful learning, ostensibly the central goal of higher education, takes place in a community of inquiry composed of instructors and learners as the key participants in the educational process. The model assumes that in this community, learning occurs through the interaction of three core components: cognitive presence, teaching presence, and social presence.

Figure 1: Elements of an educational experience.

The first element in the model is the development of cognitive presence, which Garrison et al. (2000) define as “the extent to which the participants in any particular configuration of a community of inquiry are able to construct meaning through sustained communication.” The second element is teaching presence, which includes designing and managing learning sequences, providing subject matter expertise, and facilitating active learning. The third element is social presence, defined as the ability of learners to project themselves socially and emotionally in a community of inquiry. The function of this element is to support the cognitive and affective objectives of learning. Social presence supports cognitive objectives through its ability to instigate, sustain, and support critical thinking in a community of learners. It supports affective objectives by making the group interactions appealing, engaging, and thus intrinsically rewarding, leading to an increase in academic, social, and institutional integration and resulting in increased persistence and course completion (Tinto, 1987).

Social Presence

The genealogy of the construct social presence can be traced back to Mehrabian’s (1969) concept of immediacy, which he defined as “those communication behaviors that enhance closeness to and nonverbal interaction with another” (p. 203). His research suggested that nonverbal cues such as facial expressions, body movements, and eye contact increase the sensory stimulation of interlocutors. This in turn would lead to more intense, more affective, more immediate interactions.

Mehrabian’s (1969) work was followed up by communication theorists who studied a variety of media including facsimile machines, voice mail, and audio-teleconferencing in organizational settings. Short, Williams, and Christie (1976) postulated that the inability of these media to transmit nonverbal cues would, as Mehrabian had shown, have a negative effect on interpersonal communication. It was Short et al. who introduced and defined the term social presence as “the salience of the other in a mediated communication and the consequent salience of their interpersonal interactions” (p. 65).

Sproull and Keisler (1986) regarded the implications of Mehrabian’s (1969) work from a different perspective. They argued that the critical difference between face-to-face communication and mediated communication was the absence of social context cues. Their research indicated that the lack of cues to define the nature of a social situation led to uninhibited communication such as hostile and intense language (i.e., flaming), greater self-absorption, and a resistance to defer speaking turns to higher-status participants.

Daft and Lengel (1986) presented an articulated version of Short et al.’s (1976) and Sproull and Keisler’s (1986) theories. They agreed that the lack of nonverbal information would result in terse, pragmatic interchanges. However, they argued that in some situations this can be beneficial:

When messages are very simple or unequivocal, a lean medium such as CMC is sufficient for effective communication. Moreover, a lean medium is more efficient, because shadow functions and coordinated interaction efforts are unnecessary. [However], for receivers to understand ... information that is ambiguous, emphatic, or emotional, a richer medium should be used. (p. 57)

This literature suggests that CMC does not have the capacity to support social and affective interaction. However, recent reviews of the social presence literature question the extent to which this literature is generalizable to all communications media and to all applications of these media. Walther (1994) challenges these “filtered-cues” arguments and characterizes CMC as, in some cases, “hyper-personal” rather than the predicted impersonal (p. 9, italics added). Walther cites several studies in which “experienced CMC users rated text-based media, including e-mail and computer conferencing, as `as rich or richer’ than telephone conversations and face-to-face conversations” (p. 18).

The capacity of CMC to support highly affective interpersonal interactions is supported by studies that focus on its use in educational settings. Angeli, Bonk, and Hara (1998) conducted a content analysis of a course conducted entirely through CMC. They found that 27% of the total message content consisted of expressions of feeling, self-introductions, jokes, compliments, greetings, and closures. McDonald (1998) studied the development of group dynamics in educational computer conference settings and found that expressions of openness and solidarity were significant elements, rising from 18% and 40% of the total respectively when the conference began to 36% and 54% at its conclusion. Weiss and Morrison (1998) were skeptical about the capacity of computer conferencing to support a graduate seminar. One of the potential weaknesses, they ventured, was that “it would result in dry dialogue devoid of emotion” (p. 446). However, their analysis revealed 54 instances of humor and some episodes of hurt feelings. Kanuka and Anderson (1998) analyzed a professional development conference from a constructivist perspective. After a preliminary analysis, the authors added an extra category to their coding instrument to capture the overwhelming amount of social interchange that was occurring, an amount that was significantly higher than any of the other content they were measuring. Gunawardena (1994) assessed students’ subjective evaluations of a computer conference. On a five-point scale where 1 indicated a positive rating, the average rating for “sociable” was 2.23. Results such as these ameliorate the generalizations of filtered-cues theories and indicate that computer conferencing can support both the cognitive and affective dimensions of higher education.

Filtered-cues theorists such as Short et al. (1976), Sproull and Keisler (1986), and Daft and Lengel (1986) applied Mehrabian’s (1969) concept of immediacy to all applications of communication media. In the following section Mehrabian’s concept of immediacy is regarded in an educational context.

Teacher Immediacy

Andersen (1979) looked at the role of immediacy in postsecondary education and proposed the following definition of teacher immediacy: “Teacher immediacy is conceptualized as those nonverbal behaviors that reduce physical and/or psychological distance between teachers and students” (p. 544). She found that engaging in eye contact with students, adopting a relaxed body posture, using gestures, and smiling improved students affect toward the practices promoted in the course, the subject matter of the course, and the course instructor.

Gorham (1988) expanded the definition of teacher immediacy behaviors to include oral behaviors such as talking about experiences that have occurred outside class, using humor, addressing students by name, and praising students’ work or comments. Her results suggest that these types of behavior also contributed significantly to students’ affective learning.

Sanders and Wiseman (1990) extended this relationship to include behavioral and cognitive learning. They operationally defined cognitive learning as how much students thought they had learned in a course. They defined behavioral learning as the likelihood that students would actually attempt to use the behaviors, practices, or theories studied in the course. Positive correlations between both nonverbal and verbal teacher immediacy behaviors and student affective, behavioral, and cognitive learning were significant.

The designation of this line of research as teacher immediacy implies an instructor-centered perspective of the teaching-learning relationship where the teacher plays a central and authoritative role in the classroom. According to this perspective, the creation of a warm, open, and trusting environment is regarded primarily as the responsibility of the teacher. In the community of inquiry model, teachers and learners participate in a learning transaction that is more readily identified with constructivist rather than instructivist orientations. Therefore, in the community of inquiry model, social presence is regarded as a function of both learners and teachers.

Furthermore, it should be noted that teacher immediacy research has concentrated on the investigation of nonverbal and verbal behaviors in the face-to-face classroom. Current applications of computer conferencing are mainly text-based and largely asynchronous; therefore, to extend these concepts to educational uses of CMC, these behaviors will need to be reconsidered in their textual and asynchronous forms.

Content Analysis of Social Presence

Several writers offer broad declarations about the pedagogical benefits of computer conferencing; however, few of these generalizations are supported by empirical data. Part of the problem is the absence of effective means of studying this unique educational transaction (Hillman, 1999). One investigative tool that has shown promise is content analysis (Henri, 1989; Hillman, 1999; Howell-Richardson, & Melar, 1996; Kuehn, 1993; Mason, 1989; Kanuka & Anderson, 1998). Riffe, Lacy, and Fico (1998) define quantitative content analysis as:

the systematic and replicable examination of symbols of communication, which have been assigned numeric values according to valid measurement rules using statistical methods, in order to describe communication, draw inferences about its meaning, or infer from the communication to its context, both of production and consumption. (p. 22)

Content analysis is familiar to educational and communications researchers through the work of Flanders (1970), Sinclair and Coulthard (1975) and Bales (1950), among others. Flanders and Sinclair and Coulthard used the technique to study patterns of verbal interactions between teachers and students in the classroom, whereas Bales developed a tool for studying group interaction processes.

One of the problems experienced by researchers using these observational tools is the labor required to record accurately and completely and transcribe real-time, face-to-face interactions. Computer conferencing researchers have been drawn to the technique because conferencing software, which automatically and faithfully records all the online interactions in a machine-readable format, overcomes these problems.

An important step in content analysis is the development of categories and indicators that researchers can then use to analyze the transcripts. We arrived at the categories for social presence that we used through an iterative process. First, behavioral indices were derived from the three categories of social presence articulated by Garrison et al. (2000, i.e., emotional expression, open communication, and group cohesion). Second, indicators of social interaction that had been derived from the media capacity, teacher presence, and group interaction literature were applied deductively to the analysis. Third, additional indices were deduced from careful readings of the transcripts and then added to the coding scheme. This process culminated in the formation of three broad categories of communicative responses that contribute to social presence. These categories correspond directly to Garrison et al.’s original categories. However, they were relabeled to reflect better the nature of the emergent indicators that define them in this study. Open communication is now referred to as interactive responses, referring to indices of threaded interchanges combined with messages of a socially appreciative nature. Emotional presence has been renamed as affective responses and group cohesion as cohesive responses. Considering the exploratory nature of this study, assignment of indicators to categories is tentative. Further work using factor analysis is underway to confirm the existence of these three separate, although not necessarily orthogonal, categories of social presence. Each category is further described below and the indicators in a CMC environment are identified in italics.

Affective Responses

The expression of emotion, feelings, and mood is a defining characteristic of social presence as described by Garrison et al. (2000). The adjectives attributed to both social presence and teacher immediacy, for example, closeness, warmth, affiliation, attraction, openness, all point to affective interaction. Filtered-cues theorists argue that the capacity to express this type of socioemotional communication is reduced when body language, facial expressions, and vocal intonations are eliminated in text-based interaction. Affect is expressed in computer conferencing in a number of ways, including the use of emoticons (Falman, 1981), humor, and self-disclosure.

Kuehn (1993) noted that text-based, asynchronous interlocutors employ unconventional symbolic representations such as emoticons to facilitate expressiveness in the medium. Gunawardena and Zittle (1997) found that conference participants “enhanced their socioemotional experience by using emoticons to express missing nonverbal cues in written form” (p. 8).

Teacher immediacy literature has identified the use of humor as a contributory factor to immediacy and subsequently to learning (Christenson & Menzel, 1998; Christophel, 1990; Gorham, 1988; Gorham & Zakahi, 1990; Sanders & Wiseman, 1990). Gorham and Christophel (1990) note that humor is like an invitation to start a conversation; it aims at reducing social distance, and it conveys goodwill. Research by Eggins and Slade (1997) reinforces the importance of humor as an indicator of social presence. They found humor to be “a pervasive characteristic of [casual conversation], in contrast to its infrequent occurrence in formal, pragmatic interactions” (p. 155). They also postulate a connection between humor and critical discourse: “The construction of group cohesion frequently involves using conversational strategies such as humorous banter, teasing, and joking. These strategies allow differences between group members to be presented not as serious challenges to the consensus and similarity of the group” (p. 189).

The psychological explanation of social attraction and bonding between individuals includes self-disclosure. Cutler (1995) explains that “the more one discloses personal information, the more others will reciprocate, and the more individuals know about each other the more likely they are to establish trust, seek support, and thus find satisfaction” (p. 17). Shamp (1991) applied these notions to computer-mediated communication and built on Turkle’s (1997) observation that people have a tendency to attribute human characteristics to computers (anthropomorphism). Shamp suggested that people have an inverse tendency to attribute characteristics of computers to humans (mechanomorphism). Shamp discusses the negative implications of this tendency that relate directly to the facilitation of a community of inquiry. He notes that although CMC augments the number of people with whom an individual can interact, it does not necessarily augment the degree of exposure to the multifaceted nature of adult participants. For Shamp “the lack of perceived diversity in communication partners that [CMC] fosters has the potential to turn CMC into a closed system which allows little new and different information about the world to enter” (p. 158). The negative implications for the construction of knowledge are apparent. In regard to social presence, Shamp notes, “mechanomorphism could lead to computer communication that is not fulfilling or successful” (p. 158). He recommends the exchange of personal information to reduce feelings of social isolation and thus contribute to the formation of individualized impressions of interlocutors.

Teacher immediacy literature has provided an empirical justification for extending Shamp’s (1991) conclusions to educational applications of computer conferencing. Christenson and Menzel (1998), Gorham (1988), Gorham and Christophel, (1990), Gorham and Zakahi (1990), and Sanders and Wiseman (1990) found positive correlations between use of personal examples, personal anecdotes, and self-disclosure, and affective, cognitive, and behavioral measures of learning.

Interactive Responses

Short et al. (1976) identify “evidence that the other is attending” as a critical feature in the promotion of socially meaningful interaction. Eggins and Slade (1997) add that responses and rejoinders serve several beneficial purposes in conversation. They build and sustain relationships, express a willingness to maintain and prolong contact, and tacitly indicate interpersonal support, encouragement, and acceptance of the initiator. Using the “reply” feature to post messages, quoting directly from the conference transcript, and referring explicitly to the content of others’ messages are all types of interactive response in CMC.

Walberg (1984) conducted a meta-analysis of 3,000 studies that examined the effects of educational interventions designed to improve academic achievement. Reinforcement was at the top of his list and had an effect size of 1.17. Teacher immediacy research supports these findings, with studies by Christenson and Menzel (1998), Gorham (1988), and Gorham and Zakahi (1990) each finding that responses such as “praises students work, actions, or comments” contributed to teacher immediacy and subsequently to affective, behavioral, and cognitive learning. Sanders and Wiseman (1990) studied immediacy indicators individually and found a significant correlation (r=0.55) between “praises students work” and the three measures of learning. The importance of reinforcement to collaboration is supported by sociological theory. Social interaction theorists such as Mead and Cooley contend that the human needs for affiliation and self-esteem are on par with basic physiological needs (Stark, 1996). They point out that these needs can be satisfied only through interaction with others. From this perspective, reinforcement is the object that fuels the development and maintenance of interpersonal interaction. Complimenting and acknowledging, and expressing appreciation are ways of communicating reinforcement in a text-based medium.

Cohesive Responses

This category is exemplified by activities that build and sustain a sense of group commitment. It is defined in our analysis by three indicators: phatics and salutations, vocatives, and addressing the group as “we,” “our,” or “us.”

Phatics are defined as communication “used to share feelings or to establish a mood of sociability rather than to communicate information or ideas” (GuruNet, 1999). In this definition phatic communication is almost synonymous with social presence. Bussman (1998) suggests that phatics serve to confirm ties of union and include communicative acts such as formal inquiries about one’s health, remarks about the weather, or comments about trivial matters.

Vocatives, that is, addressing participants by name, are also an important expression of cohesion. Teacher immediacy literature has discovered an empirical connection between addressing students by name and cognitive, affective, and behavioral learning (Christenson & Menzel, 1998; Gorham, 1988; Gorham & Zakahi, 1990; Sanders & Wiseman, 1990). Seeking to explain this connection, Kelley and Gorham (1988) found support for a relationship between vocatives and immediacy of recall. Eggins and Slade (1997) support the use of vocatives to facilitate social presence, noting “the use of redundant vocatives would tend to indicate an attempt by the addresser to establish a closer relationship with the addressee” (p. 145).

A variation of the vocative effect occurs at the group level when participants refer to the group with inclusive pronouns such as we, our, us, or group. Mehrabian’s (1969) suggestion that the use of these pronouns connotes feelings of closeness and association has received support in the teacher immediacy literature (Christenson & Menzel, 1998; Gorham, 1988; Gorham & Zakahi, 1990; Sanders & Wiseman, 1990).

Coding

The relative presence of these 12 indicators reveals the level of social presence in an online community of inquiry. Low frequencies indicate that the social environment is cold and impersonal. Participants are using the conference in a purely pragmatic manner for terse exchanges of information, perhaps because they are being evaluated for quantitative participation. High scores indicate that the environment is warm and collegial. Participants feel a sense of affiliation with each other and a sense of solidarity with the group. This environment of approachability and closeness encourages the students to regard the conference and their interactions as intrinsically valuable and educationally profitable. This in turn supports students in the otherwise risky act of posting their tentative ideas and also in offering critiques of others’ hypotheses. As Eggins and Slade (1997) note, disagreement and critical evaluation are more characteristic of those who share strong bonds, rather than of new or transient acquaintances.

We established the construct social presence, its three categories—affective responses, interactive responses, and cohesive responses—and the 12 indicators (see Table 1). Our next step was to identify these indicators in the computer conferencing transcripts. Two perennial stumbling blocks to finding such indicators have been the unit of analysis and interrater reliability (Rourke, Anderson, Archer, & Garrison, 2000).

Unit of Analysis

Krippendorf (1980) describes the unit of analysis as a discrete element of text that is observed, recorded, and thereafter considered data. Many units have been experimented with, as noted in educational CMC literature; however, none has been sufficiently reliable, valid, and efficient to achieve preeminence. Syntactical units such as the sentence or the paragraph allow for consistent identification, but they are artificial and arbitrary designations that abide by logic that is usually external to the logic of the indicators of interest. An alternative is the “thematic unit,” which Budd, Thorp, and Donohue (1967) define as “a single thought unit or idea unit that conveys a single item of information extracted from a segment of content” (p. 34). Thematic units, such as Henri’s (1991) and McDonald’s (1998) “meaning unit” reflect the logic of the indicators; however, they resist reliable and consistent identification (Howell-Richardson & Mellar, 1996; Rourke et al., 2000). The most appropriate unit would combine the flexibility of the thematic unit, which allows coders to capture a unit in its natural form, with the reliable identification attributes of a syntactical unit. Evidence from our trials indicates that we have identified such a unit of analysis. Using the coding protocol presented below, we have attained percent agreement interrater reliability figures ranging from 0.91 on first coding to 0.95 on second application to a new set of transcripts (see Tables 1 and 2).

Interrater Reliability

Some authors criticize the percent agreement estimates of interrater reliability on the grounds that they do not account for chance agreement among coders (Capozzoli, McSweeney, & Sinha, 1999; Riffe et al., 1998). Instead, they prefer Cohen’s kappa, which is a chance-corrected measure. However, kappa presents a formulaic problem for coders using a thematic unit of analysis. The calculation of Cohen’s kappa requires a priori knowledge of the number of coding decisions to be made by the coders. For example, if coders used the sentence as the unit of analysis, they could count the number of sentences in a transcript, and this would constitute the number of decisions that coders would have to make. For reasons alluded to above and described in detail in Rourke et al. (2000), syntactical units are not appropriate for the analysis of computer conference transcripts. Briefly, indicators such as complimenting do not organize themselves into neat syntactical packages; furthermore, communication in CMC often reflects a telegraphic or conversational syntax style. Thus often there are no easily identifiable sentences or paragraphs. Also, chance agreement as calculated with kappa is negligible with a protocol consisting of 12 indicators. Therefore, we have elected to report the coefficient of reliability figure recommended by Holsti (1969): 2m/n 1+n 2, where m is the number of coding decisions on which the two coders agree, and n 1 and n 2 refer to the number of coding decisions made by raters 1 and 2 respectively.

Table 1: Model and Template for Assessment of Social Presence

Table 2: Sample of Coded Text

Illustrations of Social Presence

In the following section we present a selection of text, followed by a table illustrating how segments of the text were coded. The purpose of this section is to illustrate the concept of social presence to show how it is manifested in computer conferencing transcripts, and to facilitate replication of the protocol by subsequent researchers. To accomplish these objectives efficiently and to avoid ethical concerns, we have fabricated the following message, rather than presenting an actual message.

Message

Article No. 432: [Branch from no. 430]

posted by Sally on Mon, Oct. 12, 1999, 12:06

Subject: re: Week 6 questions

Hi Guys:

Just got home from a very hectic day, but I want to respond to some of the postings before I fall asleep at the dinner table ;-) Joe asked: “Do you have experience with either one of the models (be it extensive or limited)?” Where I work, we tried the author-editor model but since 1988 we have moved to the course team approach. I have to agree with Gerry’s very perceptive comment about the cost, time and other demands of this approach. What really frustrates me is that our textbook fails to mention any of these types of things. Does anyone else feel the same?

Well, that’s all for now. Guess I’ll have a little dinner and see what’s on the tube.

Cheers Sally

Methodology

In order to pilot test the efficacy and reliability of the proposed template for assessing social presence, selected transcripts from two graduate-level courses were examined. Selection 1 was taken from a graduate-level conference in workplace learning. This 13-week course was delivered at a distance and supported primarily by computer conferencing. The FirstClass(r) conferencing system was used. The conference was divided into weeklong, self-contained discussions that focused on one or two issues. The accumulated postings from the fifth week were selected for analysis. Fourteen people participated in this discussion, including the instructor, two student moderators selected from the group, and 11 other students. The discussion was led by the student moderators, whose functions included stimulating discussion, adding pedagogical comment, and summarizing discussion. The instructor passively monitored the discussion, becoming active only to close the discussion by summarizing the students’ messages with positive reinforcement and expert advice. A total of 90 messages were posted during the conference week containing 24,132 words.

The context for Selection 2 was similar to that of Selection 1: a 13-week, graduate-level course delivered at a distance, supported primarily by computer conferencing. The WEBCT(r) conferencing system was used. The content of this course was the theory and practice of distance education. This conference also was divided into weeklong, self-contained discussions. The accumulated postings from the sixth week of the conference were selected for analysis. Seventeen people participated in this discussion, including the instructor, two student moderators selected from the group, and 14 other students. The nature of the discussion and the role of the student moderators were equivalent to those described for Selection 1. The instructor was actively involved in the discussion, alternatively playing the role of participant and instructor. A total of 44 messages were posted during the conference week containing 6,260 words.

The selections from each course were compiled in the conferencing system and then imported into the qualitative analysis program AtlasTi(c). Messages were imported in a threaded format that often paralleled the chronological order in which the messages were posted. Codes were entered into the qualitative analysis package, and three researchers working together coded the messages, engaging in a constant dialogue and setting and checking rules and procedures throughout the coding process. Once a sound protocol had been established, two coders working independently followed this protocol in coding the two conference selections. On completion of the coding, interrater reliability was calculated using Holsti’s (1969) calculation for percent agreement.

Results

Transcript A contained 2.5 times as many instances of social presence than transcript B (n=362, 145 respectively). However, transcript A contained approximately twice as many messages and four times as many words as transcript B. These divergent numbers produced by comparable numbers of students (n=14, 17 respectively) during equivalent time frames (one week of discussion) illustrates that the raw number of instances of social presence, or the number of instances per message, are both skewed by differences in the number of words per message or per conference. Thus to compare two selections, we sum the raw number of instances and then divide by the total number of words. This allows for a more meaningful comparison of transcripts and should also facilitate comparisons across studies. We have labeled the result of this calculation “social presence density” based partly on the work of Mason (1991), who drew similar conclusions. Depending on the number of words in a transcript, this ratio often yields values that are extremely small (e.g., numbers in the 10-3 or 10-4 range) and thus hard to interpret. Therefore, as a final step in reporting our data, we have multiplied the social presence density figure by 1,000, which yields a unit of incidents per 1,000 words. In contrast to the raw number of instances of social presence, the aggregate social presence density for transcripts A and B was 22.83 and 33.54 respectively.

Aggregate interrater reliability for transcript A was 0.95; for transcript B it was 0.91. As expected, interrater reliability for individual indicators varied according to the manifest versus latent nature of the indicator. For example, manifest indicators such as continuing a thread and addressing participants by name, which are easily recognizable by coders, had reliability coefficients of 1.0, whereas latent variables, which require a degree of subjective interpretation by coders, had lower reliability. Humor, for example, yielded the lowest interrater reliability figure of 0.25. Riffe et al. (1998) characterize interrater reliability figures of between 0.80 to 0.90 as generally acceptable when applied to categories and indicators that have been used extensively. They add, “Research that is breaking new ground with concepts that are rich in analytical value may go forward with reliability levels that are somewhat below that range” (p. 131).

Discussion

The main purpose of this research was to develop and test the efficacy of a tool for analyzing the social presence component of educational computer conferences. Two transcripts were selected to meet these objectives; therefore, it is tempting to consider a comparative analysis of the data. However, fundamental research design criteria for comparative studies were not included in the design of this study. The value of this study is in the explication of the method rather than intensive analysis of these two illustrative conferences. Nonetheless, it is not inappropriate to discuss the types of analysis that are made possible with this instrument.

The aggregate social presence density rating for transcript B was considerably higher than that for transcript A. This confirmed the intuitive impressions that we formed while reading the transcripts of the sociability and educational effectiveness of the two conferences. This suggests that the template is able to expose and quantify important differences in social presence.

At the level of indicators, transcript B had higher social presence density ratings for all but three of the 12 indicators: complimenting, phatics, and salutations, and quoting directly from the transcript. Discourse analysts Eggins and Slade (1997) offer a possible explanation for this inconsistency. They use as an example an old married couple to illustrate the point that among people who share strong interpersonal bonds, superficial and formal social expressions (e.g., salutations, compliments) are less frequent and less necessary than among people who share transient or weak interpersonal bonds. Comments made by computer conferencing students offer some support for this interpretation. Results from Angeli et al.’s (1998) study of a computer conference support this hypothesis. They found that the incidence of purely social interaction, which they defined as statements that were not focused on the content of the course, declined significantly as the conference progressed and the students came to know one another better. For example, the students that Fabro and Garrison (1998) surveyed reported, “The cohort was generally conditioned in many ways to be polite and disagreement was taken as a personal affront. This group appeared to be quite timid and began to just agree with each other rather than challenge each other’s ideas” (p. 48).

Our system applies equal weighting to each of the 12 indicators of social presence. It is likely that further research will reveal that each of the indicators defines social presence differentially. For example, it is likely that the indicators of interactive responses related to interaction (using the reply feature to post messages, quoting from the transcript, referring explicitly to others’ messages, and asking other participants questions) should be viewed on a continuum ranging from weak to strong levels of interaction. The first two indicators listed above are realized through software features. Most conferencing systems provide users with the option of posting messages by replying to an existing message, and to include the contents of the message to which they are replying. Thus the presence of replies and quoted messages may be a superficial artifact of conferencing communication rather than a defining indicator of social presence. The remaining indicators represent a more labor-intensive, and thus a more conscious and willful, effort on the part of the student to interact with others. According to this hypothesis, referring to other students by name and referring explicitly to the contents of another’s message are better indicators of interaction than having another student’s name or message appear automatically due to software features. We are currently undertaking a study to determine the relative influence and importance of each of the indicators on social presence.

Although we postulate that fairly high levels of social presence are necessary to support the development of deep and meaningful learning, we expect that there is an optimal level above which too much social presence may be detrimental to learning. Discourse in a community of inquiry is not equivalent to social interaction over the garden fence or the bar at a neighborhood pub. Our exploratory study does not indicate if the levels of social presence that we have measured are sufficient, optimal, or even so large as to be detrimental to learning.

Although aggregate interrater reliability was high, it was not high for all indicators. Coding of two indicators that were postulated to have an important influence on social presence—expression of emotions and use of humor—did not achieve acceptable levels of reliability. Potter and Levine-Donnerstein (1999) suggest that interrater reliability for some types of categories can be improved through the refinement of categories and their definitions, coder training, and practice. However, these interventions do not work for what they call “latent projective” categories (e.g., use of humor). Other authors of content analysis studies (compare Newman, Webb, & Cochrane, 1995) have raters code transcripts independently at first and then meet to reconcile their disagreements. We have discouraged communication between coders during this stage of the coding process because, as Krippendorf (1980) argues, “communication invariably influences coding toward higher agreement and this lack of independence is likely to make data appear more reliable than they are” (p. 132). Yet to code accurately and reliably categories such as use of humor, it may be necessary for coders to discuss their decisions. We must also note, however, that the frequency of these two communicative responses was infrequent. For example, the social presence density ratings for use of humor and expression of emotions in transcript A were 0.24 and 0.46 respectively. Therefore, subsequent researchers may decide to exclude them from the analysis because, in short, they may be more trouble than they are worth.

Conclusion

We believe that the social presence density calculation provides an important quantitative description of computer conferencing environments. Social presence density calculation allows for the formulation and testing of hypotheses in which social presence is used as a dependent or independent variable. For example, exploratory studies could be undertaken in which educational interventions such as collaborative learning, guest speakers, or varying levels of reward for participation are manipulated and the effect on social presence measured. Alternatively, varying levels of social presence could be measured in terms of their effect on variables such as student satisfaction, achievement, and retention. The social presence density construct also allows for the exploration of the hypothesized relationship between social presence, cognitive presence, and teaching presence, and other indicators of participation, attitude, and learning in the community of inquiry.

Further study is needed, especially using instruments that triangulate participant perception of social presence and its value, and the relationship between social presence and objective measures of learning outcomes. However, we believe the tool presented here allows wider groups of practitioners and researchers to assess critically the level of social presence in their classes. Our methods are time-consuming, and reliability levels for latent projective variables such as humor will never attain extremely high or stable values. Further developments using a large corpus of exemplar illustrations of social presence indicators may some day allow for machine analysis of transcripts using systems such as latent semantic analysis (see http://lsa.colorado.edu). In the immediate future we invite others to further develop and verify the indicators and the results that we present. In our research group, further work is ongoing that extends this methodology to the remaining components of the community of inquiry model—cognitive presence and teaching presence.

References

Andersen, J. (1979). Teacher immediacy as a predictor of teaching effectiveness. In D. Nimmo (Ed.), Communication Yearbook 3 (pp. 543-559). New Brunswick, NJ: Transaction Books.

Angeli, C., Bonk, C., & Hara, N. (1998). Content analysis of online discussion in an applied educational psychology course [On-line]. Available:

http://crlt.indiana.edu/publications/crlt98-2.pdf

Bales, R. (1950). Interaction process analysis: A method for the study of small groups. Cambridge, MA: Addison-Wesley.

Budd, R., & Donohue, L. (1967). Content analysis of communication. New York: Macmillan.

Bussmann, H. (1998). Phatic communion. In G. Trauth, K. Kazzazi, & K. Kazzazi (Eds.), Routledge dictionary of language and linguistics (p. 358). London: Routledge.

Capozzoli, M., McSweeney, L., & Sinha, D. (1999). Beyond kappa: A review of interrater agreement measures. Canadian Journal of Statistics, 27(1), 3-23.

Christenson, L., & Menzel, K. (1998). The linear relationship between student reports of teacher immediacy behaviors and perceptions of state motivation, and of cognitive, affective and behavioral learning. Communication Education, 47, 82-90.

Christophel, D. (1990). The relationship among teacher immediacy behaviors, student motivation, and learning. Communication Education, 39(4), 323-340.

Cutler, R. (1995). Distributed presence and community in Cyberspace. Interpersonal Computing and Technology: An electronic Journal for the 21st Century, 3(2), 12-32.

Daft, R., & Lengel, R. (1986). Organizational information requirements, media richness and structural design. Management Science, 32(5), 554-571.

Eggins, S., & Slade, D. (1997). Analyzing casual conversation. Washington, DC: Cassell.

Fabro, K., & Garrison, D.R. (1988). Computer conferencing and higher-order learning. Indian Journal of Open Learning, 7(1), 41-53.

Falman, S. (1981). The unofficial Smiley FAQ [On-line]. Available: http://www.newbie.net/JumpStations/SmileyFAQ/index.html

Flanders, N. (1970). Analyzing teacher behavior. Reading, MA: Addison-Wesley.

Garrison, R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. Unpublished manuscript.

Gorham, J. (1988). The relationship between verbal teacher immediacy behaviors and student learning. Communication Education, 37, 40-53.

Gorham, J., & Christophel, D. (1990). The relationship of teachers’ use of humor in the classroom to immediacy and student learning. Communication Education, 39, 46-61.

Gorham, J., & Zakahi, W. (1990). A comparison of teacher and student perceptions of immediacy and learning: Monitoring process and product. Communication Education, 39, 355-367.

Gunawardena, C., & Zittle, F. (1997). Social presence as a predictor of satisfaction within a computer mediated conferencing environment. American Journal of Distance Education, 11(3), 8-26.

Henri, F. (1991). Computer conferencing and content analysis. In A. Kaye (Ed.), Collaborative learning through computer conferencing: The Najadeen papers (pp. 117-136). London: Springer-Verlag.

Hillman, D. (1999). A new method for analyzing patterns of interaction. American Journal of Distance Education, 13(2), 37-47.

Holsti, O. (1969). Content analysis for the social sciences and humanities. Don Mills: ON: Addison-Wesley.

Howell-Richardson, C., & Mellar, H. (1996). A methodology for the analysis of patterns of participation within computer mediated communication courses. Instructional Science, 24, 47-69.

Kanuka, H., & Anderson, T. (1998). On-line social interchange, discord and knowledge construction. Journal of Distance Education, 13(1), 57-74.

Kelly, D., & Gorham, J. (1988). Effects of immediacy on recall of information. Communication Education, 37, 198-207.

Krippendorf, K. (1980). Content analysis: An introduction to its methodology. London: Sage.

Kuehn, T. (1993). Communication innovation on a BBS: A content analysis. Interpersonal Computing and Technology: An Electronic Journal for the 21st Century, 1(2) [On-line]. Available: http://jan.ucc.nau.edu/~ipct/1993/n2/kuehn.txt

Mason, R. (1991). Analysing computer conferencing interactions. Computers in Adult Education and Training, 2, 161-173.

McDonald, J. (1998). Interpersonal group dynamics and development in computer conferencing. American Journal of Distance Education, 12(1), 7-25.

Mehrabian, A. (1969). Some referents and measures of nonverbal behavior. Behavior Research Methods and Instrumentation, 1(6), 205-207.

Newman, D., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Interpersonal Computer and Technology: An Electronic Journal for the 21st Century [On-line]. Available: http://www.helsinki.fi/science/optek/1995/n2/newman.txt

Potter, J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27(3), 258-284.

Riffe, D., Lacy, S., & Fico, F. (1998). Analyzing media messages: Using quantitative content analysis in research. Mawah, NJ: Erlbaum.

Rourke, L., Anderson, T., Garrison, R., & Archer, W. (2000). A review of methodological issues in analyzing computer conferencing transcripts. Unpublished manuscript.

Sanders, J., & Wiseman, R. (1990). The effects of verbal and nonverbal teacher immediacy on perceived cognitive, affective, and behavioral learning in the multicultural classroom. Communication Education, 39, 341-353.

Shamp, S. (1991). Mechanomorphism in perception of computer communication partners. Computers in Human Behavior, 7(3), 147-61.

Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. Toronto, ON: Wiley.

Sinclair, J., & Coulthard, M. (1975). Towards an analysis of discourse. London: Oxford University Press.

Sproull, L., & Keisler, S. (1986). Reducing social context cues: Electronic mail in organizational communication. Management Science, 32, 1492-1513.

Stark, R. (1996). Sociology (5th ed.). California: Wadsworth.

Tinto, V. (1987). Leaving college: Rethinking the causes and cures of college attrition. Chicago, IL: University of Chicago Press.

Turkle, S. (1997). Life on the screen: Identity in the age of the Internet. New York: Simon & Schuster.

Walberg, H. (1984). Improving the productivity of America’s schools. Educational Leadership, 41(8), 19-27.

Walther, J. (1994). Interpersonal effects in computer mediated interaction. Communication Research, 21(4), 460-487.

Weiss, R., & Morrison, G. (1998). Evaluation of a graduate seminar conducted by listserv. Proceedings of Selected Research and Development Presentations at the National Convention of the Association for Educational Communication and Technology (AECT).