Report on the Portfolio Assessment
of the Writing Across the Curriculum Program
in the College of Arts & Sciences
Teresa M. Redd, Director
November 26, 2001
TABLE OF CONTENTS
Executive Summary .
In June 2001, I organized the first portfolio assessment of the Writing Across the Curriculum (WAC) Program in the College of Arts & Sciences. The purpose of this assessment was (1) to determine what type of writing instruction was taking place and (2) to measure WAC students writing progress and proficiency. The assessment consisted of two types of evaluation, both focusing on the student "portfolio"a folder presenting examples of a students writing: First, I reviewed 178 portfolios consisting of all of the writing that the students had submitted during the Fall 2000 or Spring 2001 semester. Then, on June 4 and 5, WAC and English teachers rated 140 portfolios, each containing a representative set of papers that a WAC student had produced.
The assessment revealed the following:
Writing Instruction. Although the genres of writing vary from course to course, WAC teachers are assigning a great deal of writing, including frequent "prewriting" assignments but only occasional revisions. Most WAC teachers are also supplying sufficient feedback on formal papers, but a few need to respond more consistently to grammatical problems.
Writing Progress and Proficiency. Nearly three quarters of the students progressed in one or more areas of writing (i.e., content, arrangement, or style). However, most students achieved only an average level of writing proficiency.
In light of these findings, I recommend six measures to strengthen the WAC program, some of which I have already initiated this semester:
In June 2001, I organized the first portfolio assessment of the Writing Across the Curriculum (WAC) Program in the College of Arts & Sciences. The purpose of this assessment was (1) to determine what type of writing instruction was taking place and (2) to measure WAC students writing progress and proficiency. The assessment consisted of two types of evaluation, both focusing on the student "portfolio"a folder presenting examples of a students writing: First, I reviewed 179 portfolios consisting of all of the writing (e.g., journals, outlines, papers, essay exams) that the students had submitted during the Fall 2000 or Spring 2001 semester. Then, on June 4 and 5, WAC and English teachers rated 140 portfolios, each containing a representative set of papers that a WAC student had produced.
In 1991, under the leadership of the English Department, the College of Arts & Sciences launched an ambitious writing-across-the-curriculum program (WAC) at Howard University. The goals of the WAC program are twofold: to help students learn to write and write to learn. In other words, the writing-intensive courses that are the centerpiece of the WAC program help students master the text conventions of a particular discipline, while reinforcing skills learned in Freshman English. At the same time, the courses empower students to use writing assignments to read and think critically about the subject matter of the discipline.
At Howard the WAC program is a voluntary, collaborative venture guided by college-wide standards. Each year teachers volunteer to participate in a series of workshops: "Designing Assignments," "Revising Syllabi," and "Handling the Paper Load." During the workshops, the teachers collaborate on the revision of their syllabi to meet the Guidelines for Teachers of Writing-Intensive Courses in the Disciplines (see Appendix A). These guidelines call for frequent prewriting (i.e., written planning), writing, and rewriting of assignments that promote "writing to learn" as well as "learning to write." To enable teachers to respond to the assignments, the guidelines also limit the class size to 20 students. Once a syllabus meets the guidelines, the College's WAC Committee approves it and assigns the teacher's section a 700-level number and a WRTG suffix (e.g., BIOL 701-01 EVOLUTION-WRTG). At that point, the section fulfills the third writing requirement in the College of Arts & Sciences.
Because of the support of the English Department, the Dean's office, and teachers throughout the University, the WAC program has grown dramatically. As of May 2001, 92 teachers in the College of Arts & Sciences had completed the workshops. These teachers represent nearly every department in the College. Since August 1994, 11 teachers from Allied Health, 1 from Continuing Education, and 2 from Engineering have also joined their ranks. Consequently, the number of writing-intensive courses has soared. While Allied Health offered its first 19 writing-intensive courses between Fall 1994 and Spring 2001, Arts & Sciences scheduled 243 from Spring 1993 through Spring 2001. In addition, some departments, such as Math, History, Afro-American Studies, Physical Education, and Physics, have started requiring their majors to take a writing-intensive course within their field.
Rationale for a Portfolio Assessment
Since the WAC program seeks to improve student learning and writing in the disciplines, any assessment of WAC must take place in a disciplinary context. For instance, unlike the Freshman English program, the WAC program must measure the extent to which writing improves students learning of disciplinary content. The cumulative student and teacher evaluations (1993-2000) and a "paired classes" study of non-WAC vs. WAC versions of the same course suggest that students are using writing as a tool for learning (see Appendix B). However, we must devise more efficient ways to measure "learning from writing."
In the meantime, we can assess students ability to write within a discipline. We already know from the 1998 Write to Succeed pilot that a well-staffed Writing Center with close teacher-tutor communication dramatically improved our WAC students writing skills (see Appendix B). However, that study was small and dependent upon a one-semester grant. Although the English Department cannot afford to hire enough tutors to extend that study, we still need to know how well WAC students are writing in spite of the shortage of tutors. At the very least, an assessment can provide a baseline for comparison once we expand the Writing Centers staff and services.
Thus, a year ago, I began to plan a large-scale assessment of students progress and proficiency in handling the disciplinary and general conventions of writing. I decided not to organize a pretest-posttest essay assessment like the one underway in the Freshman English Program. The 1994 WAC pretest-posttest study had already revealed that the WAC assessment could not focus on pretest-posttest ratings of one type of essay because WAC teacherseven in the same divisionfocus on different genres of writing and different disciplinary text conventions (see Appendix B). We needed to assess the genre of writing that a particular teacher taught (e.g., abstracts, lab reports, case studies, or critiques). Also, according to recent research, to gain an accurate picture of what students could do, we needed to assess several writing samples produced by each student at different times of the semester (CCCC). In short, we needed to conduct a "portfolio assessment"an evaluation of a body of writing produced over an extended period. Many institutions across the U.S. consider portfolio assessment the most valid and reliable form of writing assessment, especially if each portfolio is rated by two or more teachers at the same time and in the same place (CCCC). With this thought in mind, in October 2000, I solicited $2,682 from the Dean and Provost to pay raters to assess portfolios of WAC students writing (see the budget in Appendix C).
Collection and Preparation of Portfolios
During the Fall 2000 and Spring 2001 semesters, I requested portfolios from 28 of the 36 scheduled classes. (I excluded three classes that were canceled due to low enrollment and five "duplicate" classes, i.e., classes that were the same as other classes offered during AY 2000-2001.) I received portfolios from only 179 students, or half, of the 358 students enrolled in the 28 classes. However, the collected portfolios represent nearly four fifths of the 28 classes. Table 1 lists the 22 classes by course and division.
Number of Portfolios by Division and Class
|HUM 1||Ancient Thought||Fall||9|
|HUM 2||Greek Drama||Fall||8|
|HUM 3||Ancient Law||Spring||7|
|HUM 4||Spartacus & Slave Revolts||Spring||7|
|HUM 5||Individual & Society||Spring||8|
|SOC SCI 1||Pre-Practicum||Spring||3a|
|SOC SCI 2||Comparative Black Literature||Spring||4|
|SOC SCI 3||African World||Spring||11a|
|SOC SCI 4||History of Economic Thought||Spring||5|
|SOC SCI 5||African American History||Spring||15|
|SOC SCI 6||Political Analysis||Spring||7|
|SOC SCI 7||Race & Public Policy||Spring||8a|
|SOC SCI 8||Sociological Theory||Fall||14|
|NAT SCI 1||Evolution||Spring||12|
|NAT SCI 2||Animal Parasitology||Spring||4|
|NAT SCI 3||History & Philosophy of Science||Fall||9|
|NAT SCI 4||Planetary Science||Spring||7|
|NAT SCI 5||Introduction to Analysis I||Fall||5a|
|NAT SCI 6||History & Philosophy of Phys Ed||Spring||3a|
|NAT SCI 7||Human Sexuality||Fall||19|
|NAT SCI 8||Addictive Behavior||Spring||8|
|NAT SCI 9||Abnormal Psychology||Fall||6|
aevaluated but not rated because of incompleteness or lateness
Having collected the student work, I put as many of the papers in chronological order as I could. After reviewing everything in the folders and recording my observations, I assembled a new set of portfolios for the raters, knowing that the raters would have time to read only a representative set of papers. For these portfolios, I selected only the first drafts that had been marked or graded by the teacher; thus, I excluded all rough drafts and revisions. At this point, I also excluded several incomplete folders that contained only one paper, regardless of the number of drafts. If there were many papers left in a folder, I selected papers from the beginning, middle, and end of the term, staying within the same genre of writing whenever possible. Thus, different portfolios featured different genres (e.g., book reviews vs. abstracts). However, I did not include journal entries unless they were formal pieces of writing and no other genre of formal writing was available. In all, I prepared 149 portfolios, from 17 classes, for rating (see the entries without an asterisk in Table 1).
Selection of Raters
Assessment expert Edward White(1992) has questioned whether English teachers alone can adequately assess portfolios "written in a discourse community we may know nothing about . . . written for contexts we do not share" (p. 72). Therefore, because WAC, unlike Freshman English, promotes the mastery of discipline-specific content and conventions, I recruited WAC teachers as well as English teachers to judge the portfolios and to develop the criteria for scoring. The five WAC teachers represented the Humanities, Social Science, and Natural Science divisions. (I did not recruit any teachers from Fine Arts since there were no Fine Arts portfolios.) On the other hand, the five English teachers were all writing faculty. Three had earned the rank of Master Teacher, and four had taught English courses that, like WAC courses, fulfilled the third writing requirement. I paired the raters so that each portfolio was read by a WAC teacher and an English teacher (see Table 2). Whenever possible, I assigned portfolios to WAC teachers who had expertise in the field represented by the portfolios. When the WAC and English partners could not resolve a dispute about a score, I designated myself as the third rater since I shared both WAC and English concerns.
The Five Pairs of WAC and English Raters
|Brown, Lee (Philosophy)||Noone, Patricia|
|Poser, Yvonne (Modern Languages)||Fowler, Cynthia|
|Williams, Daniel (Math)||Ellison, Kitty|
|Calloway, Denise (Physical Education)||Harrell, Wade|
|Ammons, Lila (Afro-American Studies)||Abdullah, Diayyah|
Developing Criteria. The raters evaluated the new portfolios over a period of two days, from 9 a.m. to 5 p.m. Monday, June 4 was devoted to developing evaluation criteria, then reading and scoring. I opened the session with introductory remarks about the purpose and nature of the assessment. Then, once I had distributed sample scoring guides from the English Department and Educational Testing Service, everyone discussed the following question: "How can we measure undergraduates writing progress and proficiency in a discipline-specific context?" Through animated discussion, the raters produced the criteria shown in Table 3 and a five-point scale (A = 5, B =4, C = 3, D = 2, F = 1) for grading the content, arrangement, and style of the students writing. By summing the scores for content, arrangement, and style, the raters arrived at a composite portfolio score of 3 to 15 points.
WAC and English Raters Criteria for an "A"
|Fulfillment of the rhetorical task|
|Familiarity with the subject matter|
|Analysis or synthesis (if expected)|
|Appropriate evidence (documented, if necessary)|
|Appropriate graphics (if required)|
|Other aspects of presentation|
|Correctness (Standard English grammar, spelling, and mechanics)|
|Artful use of language|
|Facility with the language of the discipline|
|Smooth integration of sources according to the conventions of the discipline|
Group Norming. Following the discussion, everyone participated in a "norming session" designed to clarify scoring criteria and resolve disagreements. Using the five-point scale and a score report form (see Appendix D), the group rated a sample portfolio from an economics class. Since the raters were engaged in a portfolio assessment, they graded the whole body of work in the portfolio without marking or scoring individual papers. In other words, they developed a general impression of the collection of papers in order to build a profile of the writer. After reading the sample economics portfolio, all raters agreed that the student had progressed during the term, but their composite scores for proficiency ranged from 8 to 13.5. However, the ensuing discussion built greater consensus.
Scoring and Norming in Pairs. Having developed a consensus, the raters read and scored portfolios from 1 p.m. to 5 p.m., stopping approximately every 50 minutes to take a break and to discuss in pairs the portfolios they had just rated in common. During these one-on-one conferences, the raters changed the content, arrangement, and/or style scores of 40% of the portfolios to resolve disagreements. However, in fewer than 10 cases, the raters asked me for my opinion. As a result of their conferences, the raters also decided not to rate 29 portfolios for progress because I could not ascertain the chronological order of the papers or because the portfolio contained only two papers.
During the session, the raters also asked me questions about the curricular context for certain papers. In most cases, I could answer these questions based on my examination of the syllabi, assignment sheets (if available), and other student submissions.
Second Group Norming. Tuesday, June 5 began with the reading of a sample portfolio from a classics course, followed by another group norming session. Once again the initial composite scores varied (from 8 to 11), but subsequent discussion brought the scores in line.
Final Scoring and Norming in Pairs. The rest of Tuesday was devoted to the evaluation of portfolios, punctuated by breaks and conferences roughly every 50 minutes. With extra help from the fastest raters, the group evaluated 140 portfolios. Nine portfolios were not rated for progress or proficiency, apparently because the raters overlooked the folders or ran out of time.
Progress Ratings. To determine whether students had progressed, I recorded the number of students who had earned a "Yes," "No," or "Unrateable" from both raters. However, whenever the raters disagreed, I served as the final arbiter. After rereading each controversial portfolio, l changed 9 of the 111 progress ratings to "Yes" and 9 to "No."
Proficiency Scores. To compute a portfolio score for proficiency, I averaged the two raters composite scores (i.e., the combined scores for content, arrangement, and style). Since a composite score could range from 3 to 15, if the raters composite scores were not within two points of one another, I added my score before averaging. I followed this procedure because Johnson et al. (2001) found that it produced the most reliable ratings:
When two raters assign nonexact scores, it appears that combining the scores of raters with the score of an expert results in a higher level of rater reliability than either (a) combining the original two scores, (b) substituting the score of an expert for the original two scores, or (c) combining the score of an expert with the closest rating from either rater. (p. 245)
After adjusting 12 of the 140 pairs of composite scores, I computed means for each student and class. However, I did not compare the means for classes since the number of rated portfolios in five classes was too small (< 5) for meaningful statistical tests. Instead, from the 140 scores, I computed the mean for all WAC students, regardless of class.
Observations Regarding Writing Instruction
As I mentioned above, prior to the scoring session, I reviewed all of the student submissions, including journals, outlines, rough drafts, revisions, and essay exams. My examination of 179 portfolios revealed the following:
Ratings for Progress and Proficiency
Writing Progress. The raters evaluated 111 portfolios for writing progress. As Table 4 reveals, the raters felt that nearly three quarters (72%) of the students had progressed in one or more areas of writing (i.e., content, arrangement, or style).
Number of Students Who Progressed as Writers
|Division Class||n||Not Rateda||Improved||Not Improved|
aMost of these portfolios were not rated because they could not be placed in chronological order or there were only two papers, which had been written at about the same time.
bWhile these portfolios did not reveal measurable improvement, they earned the highest scores.
cSince these papers were anonymous, the raters could not measure individual progress.
Writing Proficiency. The raters scored 140 portfolios for writing proficiency. The paired composite scores suggest that most WAC students are average writers in their disciplines since the mean of all 140 scores equals 10.15 (SD = 2.27) on a scale of 3 to 15, with scores ranging from 3.30 to 15.00. On the five-point scale, 10.15 would be equivalent to 3.38, or a "C." As Table 5 reveals, 10 and 11 are the most common means for a class, and the class means range from 7.71 to 13.40. However, it would be misleading to compare these means since the number of rated portfolios in some classes was quite small.
Table 5 Mean Composite Scores for Writing Proficiency
Mean Composite Scores for Writing Proficiency
|Division Class||n||Not Rateda||Mean||(SD)|
aApparently, the raters overlooked these folders or ran out of time.
bAlthough these papers were anonymous, they were organized by assignment so that the raters could compute a score for the class as a whole. Without these scores, the overall mean remains virtually unchanged (M = 10.13, SD = 2.38).
The following conclusions emerged from my review of student work and from the raters scores:
Writing Instruction. WAC teachers are providing substantive writing instruction: Although the genres of writing vary from course to course, WAC teachers are assigning a great deal of writing, including frequent prewriting but only occasional rewriting. Most WAC teachers are also supplying sufficient feedback on formal papers, but a few need to respond more consistently to grammatical problems.
Writing Progress and Proficiency. While taking a WAC class, most students progressed in one or more areas of writing (i.e., content, arrangement, or style). However, by the end of the term, most WAC students had achieved only an average level of writing proficiency in the discipline.
In light of these findings, I recommend six measures to strengthen the WAC program, some of which I have already initiated this semester (see the timetable for implementation in Appendix E):
On the one hand, WAC students who excelled in Freshman English may be struggling because in their WAC classes they encounter more complex writing tasks, tasks that require the analysis and synthesis of texts and data. Last year, under the leadership of Director Evora Jones, the Freshman English faculty began to address this issue by insisting that 002 and 003 essays cite evidence from texts, not just the students personal experiences. Last year we also revised the departmental final exams for both courses so that they too require writing that demands close reading and documentation.
On the other hand, grade inflation may also account for the persistence of so many errors beyond 003. Are some 003 writers slipping through the cracks into WAC classes because our grading has been inconsistent? The Freshman English faculty already follow a common grading scale, which is published in the 002 and 003 syllabi. However, last year the Freshman English faculty took additional steps to maintain consistently high standards by turning the departmental final for 002 and 003 into an "exit exam" graded by two or more English teachers: Students who received an unsatisfactory grade (less than a "C") from two teachers could no longer pass the course. Papers that initially received a "pass" and a "fail" were read by a third teacher.
Since we recently instituted all of these changes, it is too early to assess the impact on the WAC program: After all, last years Freshman English students are just beginning to enroll in WAC classes this semester. We will have to wait a few years for the next WAC assessment to gauge the success of our Freshman English reforms.
Guidelines for Teachers of Writing-Intensive Courses in the Disciplines
WAC Committee of the College of Arts & Sciences rev. 5/15/96
Writing Across the Curriculum (WAC) is an interdepartmental program that promotes writing to learn and learning to write in every discipline. Research has shown that student writing in the disciplines, including mathematics and physics, fosters better thinking and active learning. Because writing and learning are intrinsically linked, the WAC program aims to expand the role of writing in courses in each discipline. In addition, it seeks to reinforce writing skills learned in Freshman English while helping students master the text conventions of their chosen discipline. To achieve these goals, nearly half of U.S. colleges and universities have established WAC programs.
DESCRIPTION OF COURSES
Writing-intensive courses in the disciplines are the core of the WAC program at Howard University. They are designed to show students that writing well is essential to success in whatever field of study or career they pursue. In the College of Arts & Sciences, these courses fulfill the third writing requirement (following Freshman English 002 and 003). To enable teachers to respond to the frequent writing assignments, each course is limited to 20 students.
In the Directory of Classes and on transcripts, writing-intensive courses in the disciplines are designated by a 700-level number and a -WRTG suffix. If a course consists of multiple sections, only the 700-level sections will be writing-intensive.
1. No Beginners: Students who have not fulfilled the Freshman English requirement should not enroll in a writing-intensive section of a course, even if other sections of the course are open to freshmen.
2. No Repeaters: Students should not enroll in a writing-intensive section of a course they have already taken.
3.No Change in the Fulfillment of Other Requirements: A writing-intensive section will satisfy the same requirements (e.g., divisional requirements) as other sections of the course.
4. No Additional Credit: To fulfill the third writing requirement, a course must carry at least 3 credits. However, when a 3-credit course fulfills the third writing requirement, it is still worth only 3 credits.
5. No Old Section Number: Normally, when a section becomes writing-intensive, a 700-level number replaces the original number.
All students must have fulfilled the Freshman English requirement (Freshman English 002 and 003). To fulfill the requirement in the College of Arts & Sciences, students must not only pass 002 and 003, but earn a "C" or better in Freshman English 003. Otherwise, they must earn a "C" or better in 004 (the follow-up course for weak writers).
Writing assignments should play many roles. Students will use writing as follows:
1. to ensure that they read carefully
2. to make sense of their lessons
3. to think critically about the subject matter (especially through analysis and synthesis)
4. to organize their thoughts and present them in a comprehensible format
5. to master the text conventions of the discipline (e.g., formats, documentation styles, assumptions, acceptable evidence)
6. to reinforce the skills learned in Freshman English 002 or 003 (including editing skills)
1. Students will spend some class time discussing the writing required for the course.
2. Students will analyze and discuss examples of excellent writing in the discipline.
3. Students will use prewriting strategies (e.g., brainstorming, categorizing, freewriting, diagramming, outlining, and journal-keeping).
4. Students will complete several short writing assignments designed to improve their thinking and writing. These assignments can take many forms. For instance, in addition to formal research papers, lab reports, or essays, students may write rough drafts, journal entries, summaries, research proposals, annotated bibliographies, reviews of literature, book reviews, progress memoranda, and abstracts.
5. Students will be required (through grading incentives) to rewrite some papers. These rewrites should be substantive revisions, not merely the correction of grammatical and spelling errors. To revise, students should occasionally have access to "peer reviews" from their classmates as well as comments from the teacher.
6. The WAC Committee must approve the syllabi for writing-intensive courses.
Teachers should refer students to a Freshman English handbook (e.g., Harbrace, Allyn & Bacon, Little/Brown). In addition, teachers may order writing books specific to their discipline (e.g., the Harper Collins series on writing in biology, art, history, and other fields).
EVALUATION OF WRITING
Teachers should evaluate papers not only for an understanding of crucial concepts, but also for the clarity with which those concepts are expressed. In other words, teachers should consider the efficacy of expression (including such matters as unity, coherence, organization, grammar, and spelling). However, teachers need not correct the papers as an English teacher would.
Since students should be writing for various purposes in various stages and in various forms, teachers should vary their responses to writing. Certainly, first and final drafts do not have to be evaluated the same way. In fact, journal entries and some first drafts do not have to be marked or graded at all, just CREDITED. (Moreover, given checklists or guidelines, students can evaluate their classmates or their own first drafts.) Even final drafts can receive a variety of responses:
HOLISTIC: Evaluate on the basis of a general impression of the content, arrangement, and style of a text.
FOCUSED: Focus on the 2 or 3 most serious problems in the paper (e.g., inaccuracies, illogical organization, or the five-point errors on the Freshman English syllabi).
LIMITED: Assess the content and organization, while taking into account all English errors before a designated cut-off (e.g., the first 300 words).
RETURNED: Return an unmarked paper because it is so deficient that the student must confer with the teacher or a tutor and rewrite the paper (subject to a late penalty).
TRADITIONAL: Assign points for specific characteristics of the text and mark all errors (see the Freshman English syllabi).
Theory and practice indicate that faculty workshops contribute to the success of writing-intensive courses. Therefore, faculty members wishing to teach such courses will attend a workshop organized by the WAC Committee. The WAC Faculty Workshop will be a hands-on, collaborative venture--an occasion for colleagues to help each other develop syllabi for writing-intensive courses. Interested faculty members will attend 3 sessions (9 a.m. - 12 noon) in mid-May and submit a revised syllabus to the WAC Committee by May 31. When the WAC Committee approves the syllabus, the faculty member will receive a WAC Certificate to include in his or her merit-tenure-and-promotion file.
Only Certified WAC Teachers may teach 700-level courses that fulfill the third writing requirement. Therefore, the following policies apply:
Course Reassignment: If a WAC teacher cannot teach a 700-level course as scheduled, the course must be reassigned to another WAC teacher or canceled.
Team-Teaching: If a 700-level course is team-taught, at least one of the faculty members assigned to the course must be a certified WAC teacher. If this faculty member is the only team member to complete the workshop, s/he is responsible for assigning and responding to the students writing. If other faculty members or graduate students share that responsibility, they must also attend the workshop. However, the WAC Committee strongly recommends that all members of the teaching team participate in the workshop, regardless of their roles.
Previous WAC Experience: The WAC Committee will consider petitions from faculty members who wish to be excused from the workshop because of prior experience in a WAC program. A teacher with the requisite experience may conduct a 700-level course after the committee approves the teachers syllabus.
The WAC Faculty Workshop will provide resources for developing and evaluating lessons
and assignments that will not only help students learn to write but help them "write
Designing "Writing to Learn" and "Learning to Write" Activities
Designing "Writing to Learn" and "Learning to Write" Activities
Collaborating to Revise Syllabi: At the second meeting, veteran WAC teachers will share their syllabi and answer questions. In addition, WAC Committee members and workshop participants will collaborate on the revision of syllabi.
Handling the Paper Load: At the third meeting, participants will discuss and practice ways to respond to student writing. This session will offer techniques and resources that help teachers handle the paper load more effectively and more easily. Afterward, the participants will finish revising their syllabi and submit them to the WAC Committee.
RESOURCES (updated 10/26/00)
Teachers and students in the WAC program have access to many resources to help them solve writing problems:
WAC Committee: Dr. Teresa M. Redd, Director (6-6770), Dr. Ann Kelly (6-7757), Dr. Lee Brown (6-7358), and Dr. Annette Davis (6-6985).
Word-Processing Workshops: Academic Computing Services and the Center for Academic Reinforcement
Tutorial Assistance: English Departments Writing Center (Locke 100 and HEC 1024) and Center for Academic Reinforcement (Academic Support Building B, 1st floor lab).
Computer-Assisted Instruction: English Departments Writing Center (Locke 100)
WAC Website: www.english.howard.edu/wac
Grading Criteria: Freshman English 002 and 003 syllabi
References: Freshman English handbooks (Bedford Guide, Harbrace, Allyn & Bacon, Little/Brown); Barbara Walvoords Helping Students Write Well: A Guide for Teachers in All Disciplines (New York: Modern Language Association, 1986).
Acknowledgments: This document updates "Guidelines for Teachers of Writing-Intensive Courses in the Disciplines" (T. Redd & W. Schreck), which incorporated material from "Specifications for Courses Fulfilling the Writing Requirement Above 003" (WAC Ad Hoc Committee) and from "Working Paper on Writing in All Disciplines in the College of Liberal Arts" (A. Kelly).
A Brief History of WAC Assessment at Howard University
Method: From Spring 1993 to Spring 2000, students responded to questionnaires measuring their attitudes toward the writing component of their WAC classes. Students coded answers to nine questions about reading, thinking, organizing, and writing, using a Likert-type attitude scale. They also wrote answers to two questions about the strengths and weaknesses of the WAC component.
Results: The combined data show that most students feel that WAC courses are worthwhile, especially as a tool for learning the subject matter of the disciplines. On a scale of 1 to 4, students gave the highest ratings to WACs effects on their reading and thinking.
Method: From Spring 1993 to Spring 2000, teachers responded to questionnaires assessing their attitudes toward the writing component of their WAC classes. Teachers coded answers to the same nine questions that students were asked. However, they wrote responses to six questions about teaching and learning in their classes.
Results: Teachers generally gave their courses Likert ratings that were similar to the averages of the students responses. Their written responses (and, in some cases, attached writing samples) confirmed that they were using WAC techniques for prewriting, rewriting, and discussing assignments. Suggestions for improvement varied.
Method: The WAC Director led and taped discussions with the students and teachers in five WAC classes (biology, math, political science, history, and modern languages) in Spring 1994. Students were asked questions about WAC courses vs. Freshman English, WAC courses vs. other courses in the discipline, writing to learn, and WACs effects on their reading, thinking, and writing.
Results: All of the classes felt that writing had enhanced their learning of the subject matter. Many of the students claimed that the WAC course had dramatically improved their thinking, mainly because the requirement to write about texts had improved their reading. Fewer students claimed to be better writers as a result of the WAC courses: The most positive remarks about writing improvement came from the social sciences classes.
Paired Classes Study
Method: After the Fall 1993 semester, the WAC Director and a WAC Committee member from the English Department took advantage of the rare opportunity to compare the final exam grades in WAC and non-WAC sections taught by the same teacher. These included two sections taught by a history professor and two sections taught by a classics professor. The WAC team also compared the term papers in the WAC and non-WAC history sections. The WAC team scored the papers, using blind and holistic rating procedures. Essays were rated on a scale of 1-5 for idea development, support, organization, diction/sentence variety, and mechanics.
Results: Both WAC classes earned more points on the final exam than their non-WAC counterparts did. Moreover, both the history teacher and the WAC team gave the WAC term papers higher grades than they gave the non-WAC term papers. Because of the small sample sizes, the team could not determine whether the differences were statistically significant, but pre-existing ability differences did not account for the WAC advantage.
Method: In Fall 1994, teachers in five WAC classes (history, biology, physical education, modern languages, philosophy) assigned a WAC diagnostic essay ("What do you already know about the subject matter of this course?) and WAC final essay ("What do you know about the subject matter of this course?"). Because of the number of time references embedded in the WAC final essays, the essays could not be scored blindly, as planned Nevertheless, the WAC Director scored the essays holistically for development, organization, and language on a scale of 1-4.
Results: The study revealed that most students knew how to organize and develop a personal expository essay when they enrolled in the WAC classes. What the study did not reveal was how well they could organize and develop the type of disciplinary writing taught in their particular WAC class. Nor did the study indicate how well students could command the style of the discipline. As for general editing skills, there was no consistent improvement.
Write to Succeed Study
Method: The WAC Director analyzed data from the Fall 1998 pilot of the Write to Succeed Program. Teachers in the pilot gave a paper an Incomplete grade (e.g., "IC") if it contained a noteworthy writing problem and referred the student to the Writing Center, where a tutor helped the student edit the paper. Once the student had earned a Success Report from the tutor for successful editing, the teacher raised the paper grade one letter grade. To implement the pilot, the WAC Director received money from the Fund for Academic Excellence to hire and train six additional tutors. At the end of the project, she collected data from a chemistry class and a theatre class as well as Writing Center records.
Results: An analysis of the data revealed that the program (1) ensured that every student referred to the Writing Center earned a Success Report and/or a higher grade, (2) phased out "I" referrals for students who had earned Success Reportsand any others who had received "I" grades previously, and (3) decreased by 80% or more the percentage of a class receiving "I" for a writing assignment.
APPENDIX C: Budget
Note: Since I used the English Departments facilities (the Locke 100 Writing Center), the catering service and raters wages were the only expenses.
|Food (for 12)|
Monday, June 4
@$7.50 per person + $20 delivery
New York Deli Buffet
@$10.25 per person + $20 delivery
Wednesday, June 5
@$7.50 per person + $20 delivery
American Deli Buffet
@$8.25 per person + $20 delivery
|Wages (for 11)|
|11 raters||@$100 per day for 2 days||2,200.00|
APPENDIX D: Score Report Form
WAC Portfolio Assessment
for the ____________________Division
Raters Name _______________________
Raters Department __________________
Student Progress Proficiency
(initials) Yes or No? C A S Total
Note: If you wish to change a score after talking with your partner, do NOT cross out the original score. Just write the new score after the old one as follows: 5/6. If your final score totals are not within two points of one another, please show me the portfolio.
APPENDIX E: Timetable for Implementing Recommendations
Work with faculty to encourage rewriting and to improve feedback on grammar.
Seek funding for WAC-English rating sessions for professional development.
Seek funding for 20 additional tutors for the Writing Center.
Hire and train 20 additional tutors for the Writing Center.
Organize WAC-English rating sessions for professional development.
Continue to work with faculty and tutors as described above.
Continue to work with faculty and tutors as described above.
Continue to work with faculty and tutors as described above.
Collect portfolios for another assessment.
APPENDIX F: WAC Tips and Proofreading Sheet
TIPS FOR TEACHING WRITING ACROSS THE CURRICULUM
What if youre not teaching a WAC class this term? Well, you can still send students to tutors in the Center for Academic Reinforcement. In addition, you can assign certain students self-grading online exercises and ask them to bring you printouts of their scores for homework credit. The best site is Darlings Grammar (www.ccc.commnet.edu/grammar ).
On the other hand, if youre not teaching a WAC course, feel free to substitute a revision grade for an earlier grade. Or consider giving an "I" grade. For instance, if the content of a paper merits a "B" but the paper is disorganized or filled with errors, give the paper an "IC." Then, if the student submits a successful revision within a week, you can raise the grade (e.g., from an "IC" to a "B").
Remember the five-minute in-class writing assignment? At the start of a class, you announce that, during the last five minutes of class, students must write a response to a question about the lesson. Not only does this strategy encourage students to pay more attention in class, but it also gives you immediate feedback on your teaching. At first, you might wonder how to respond to all of that in-class writing. Have no fear. You can respond effectively without marking it. Simply give the students classwork credit (e.g., one point). If they misunderstood a concept, clear up the confusion during the next class period or conference. You can also return the Proofreading Evaluation Sheet and/or Writing Center Referral Form with papers that are riddled with errors.
Also, remember that the double-entry log used for outcomes assessment is also one version of a WAC journal: Students divide the page vertically and write a math problem, lecture notes, or a journal entry on one side of the page. Then on the opposite half, they comment on the reasoning or significance of what they have written. For more ideas, see the "Write to Learn" tips on the WAC web site (www.english.howard.edu/wac/sample.aspx), or for more student examples, see your "Write to Learn" handout from your WAC workshop. (If you need another copy, e-mail me and Ill send you a replacement by campus mail.)
PROOFREADING EVALUATION SHEET
(to supplement my comments
on the content and organization of your paper)
While reading your paper, I noticed a pattern of confusing, stigmatizing, or distracting errors. I have refrained from marking all of those errors so that you can practice detecting as well as correcting them. Using a grammar handbook, please look up the topics I have circled below and then correct your errors, UNDERLINING YOUR CHANGES. If you correct your errors by ___________, you may earn additional points.
SENTENCES (e.g., run-ons, comma splices, fragments, awkward construction)
VERBS (e.g., incorrect form, lack of agreement, wrong tense)
NOUNS (e.g., missing or incorrect plurals)
PRONOUNS (e.g., wrong form, lack of agreement, vague reference)
SPELLING AND CAPITALIZATION
WORD CHOICE (e.g., incorrect word form, awkward transition, wrong word)
PUNCTUATION (e.g., apostrophes, commas, semicolons, quotation marks)
TYPOS AND OMISSIONS
CITATION FORMAT (e.g., missing or incorrect form of MLA, APA, CBE, CMS)
Conference on College Composition & Communication (1998). Writing assessment: A position statement [Online]. Available: http://www.ncte.org/positions/assessment.html
Harris, M. (1992). The writing center and tutoring in WAC programs. In S. McLeod & M. Soven (Eds.), Writing across the curriculum: A guide to developing programs (pp. 154-174). Newbury Park, CA: Sage.
Johnson, R., Penny, J., & Gordon, B. (2001). Score resolution and the interrater reliability of holistic scores in rating essays. Written Communication, 18, 229-249.
National Network of Writing Across the Curriculum Programs (2001). Available: http://www.gmu.edu/departments/wac/wacnet.htm
White, E. (1992). Assigning, responding, evaluating (2nd ed). New York: St. Martins Press.