Qualitative Methods in Writing and Rhetorical Studies

Following up on conversations with two grad students this week, I’d like to detail my current thinking on the use of, and adaptations of, qualitative research methods in Writing and Rhetorical Studies. Just about everything published about qualitative methods expects the researcher to be a social scientist. While there are some fabulous publications in WRS that are fully invested in social science qualitative methods, most of us in the field aren’t trained in those methods. Right now I am advising a PhD student in Syracuse’s Curriculum and Teaching program. She’s taking three qualitative research methods courses—this semester. I don’t know of any WRS program that even offers such a possibility, much less requires it. Most training in WRS graduate programs is about the field itself—critical analysis of the conversations going on in it—without much depth about methods for contributing to those conversations. Indeed, if the WRS graduate program is within an English department, the students are often required to take courses in literature or literary theory—which can contribute to a naturalization of literary research methods as the foundation of research in Writing and Rhetorical Studies.

I’ve long been convinced that the narrative basis of much WRS scholarship is important and persuasive within the field, but carries little authority outside it. This matters when one is trying to effect system-wide change. That would be me, wanting to redirect the pedagogical energy that is currently being aimed excessively at the prevention of plagiarism. I’m a lot more convinced of the value of teaching students, in depth and throughout the curriculum, how to engage with and write from texts in an ethical and meaningful way. Such instruction, I suspect, can actually reduce students’ motivation to plagiarize deliberately, and it can also give them the writing skills, textual experiences, and mentoring they need for handling source texts ethically.

That has taken me into the realm of qualitative research, a realm in which I am a novice. It is also a realm in which most of the journals in Writing and Rhetorical Studies seldom publish.

So. Qualitative methods are little taught in most WRS graduate programs, and they are little published in WRS journals. It’s been my observation, moreover, that journals like Writing and Pedagogy, Written Communication, and the Journal of Writing Research are reluctant to publish articles that adapt but do not fully subscribe to qualitative social science methods.

So what’s a graduate student (or any scholar, for that matter) in Writing and Rhetorical Studies to do? It might seem silly to acquire expertise in methods that are so little used in one’s field. I believe, however, that WRS is increasingly interested in data-driven research. It’s my hope that over the decades to come, research methods will emerge in WRS that are distinctive to it—neither a wholesale adoption of the methods of literary criticism valued in literature departments, nor a migration of WRS to a social science discipline. Maybe it’s just confirmation bias on my part, seeing evidence everywhere of the thing I believe. Yet I believe it, all the same. I believe it’s important for PhD students in WRS to prepare themselves for a field more diverse than many of the English departments in which those PhD students received their MAs.

I don’t believe this requires a complete retooling; just a diversification. I have two pieces of advice for those who wish to write about data in genres other than personal narrative and hermeneutic interpretation.

My most basic advice is to code. No journal I know of resists publishing research whose data have been systematically analyzed. Coding is not a secret society; it can be learned through direct instruction, through mentoring, or through autodidacticism. Learning to code is an ongoing process; I know a lot more about it now than I did when I began coding, and I’ll know a great deal more a few years from now. I’d recommend starting with Saldaña, now in second edition. If this gives you a clear sense of how you might proceed, go with it. If, however, you’re still not certain, go to Charmaz, just out in a new edition. That will give you step-by-step procedures. And if you’re thinking your results should be quantified, Geisler has what you need. There are additional sources listed here.

My second piece of advice is to treat your data—whether observational, textual, archival, or interview—as data. Read up on whatever data-collection method you’re using. In addition to the well-grounded but ludicrously expensive Sage Handbook of Interview Research (get it through your library!), there are many other articles and books on interview-based research. Remember that your objective here is not to become a social scientist, yet to have the tools for thinking critically and strategically about the design and interpretation of interviews. The same is true for textual analysis. You can consult this bibliography for starters. I particularly recommend Titscher et al, even though it’s not a new title. There’s also a bibliography of sources on archival research.

If you’re feeling apprehensive (as I was at the beginning), be of good cheer. This is going to take a couple weeks of serious work. Not a lifetime. Not a new degree program. And the results will allow you to reach wider audiences and to participate in and influence a diversifying field of scholarly inquiry. And then? Then you just keep learning.

Writing in a new genre: Originality, and pain

I’ve started working in what is a new genre for me, creative non-fiction. I have a friend who teaches memoir and colleagues who teach creative non-fiction, but it’s a genre that until now I’ve associated with only as a reader. Writing in this genre is unsettling—mostly in a good way. For at least this morning, I feel that it’s therefore worth blogging about. What’s it like to be an experienced writer and teacher of writing who’s suddenly undertaking not just a new project but a new and very different genre of writing?  Exploring answers to that question will undoubtedly be useful for me. The possibility that it might be useful to others means that I write about it here in my blog rather than in my journal.

Writers talk a lot about the pain of writing. That’s not what this entry is about. It’s about the painful memories dredged up by the writing I’m doing right now.

A year ago a very dear friend told me that there was a book I was meant to write and that I needed to get to work on. Over lunch in a Syracuse restaurant, he described the book. I was nonplussed, and riveted. He was right!

I won’t even discurse on the fact that a friend told me what I should be writing, and that I hadn’t already seen it myself. After a lifetime of writing and a career of teaching writing, I take for granted that this is how very good writing often happens: One person sees what another person should be writing, and the chase begins. My students sometimes think I’m a magician when I’m the one telling them what they should be writing. I’ve given up trying to talk them down from that fantasy; if they want to think I’m a Muse, fine. Whatever makes ya happy. This particular variety of musedom, though, is not magical at all; it’s just a matter of paying attention to, and caring about, writing and writers. Not just writers but readers get insights, and it’s great when these can go back to the writers and produce fresh new ideas and directions.

After my friend made this suggestion to me, I mulled it over for awhile, but I just couldn’t see how the thing could work. I agreed that it was there, but it was behind a veil. Then last spring I was on a redeye where I had enough miles to upgrade to first class. The guy beside me was a corporate executive. He was an interesting guy and told me about his business. He asked me what I wrote, and I explained. Then he said “What are you working on now?” And out it came. In the middle of the night, somewhere between San Diego and New York, I wrote the TOC for that book as I spoke it aloud to him. I had the presence of mind then to pop open the iPad, log into Evernote, and write down what I’d just said. The book was a reality.

Other realities intervened, and as the months passed I just couldn’t find the space for working on that book. Other obligations pressed upon me.

And then two weeks ago I recognized that I was feeling long-term stress and frustration over life events that hadn’t been going my way. I needed an antidote, to prevent getting entrenched in negative thinking. So I decided it was time to get going on this book, and I pledged to put in an hour every morning.

Except for a several-days spell when pneumonia (gack) had me sleeping around the clock, that’s worked very well. In two weeks I’ve drafted 21 pages, and that’s pretty satisfying. I often tell grad students that Virginia Woolf counted 500 words as a good day’s writing. Using her standard—and I generally do—things are going well.

This morning, though, I found that this genre can be a very painful one. I’m not writing a cathartic memoir; this book is not a form of therapy, and its purpose is not to elicit sympathy or readers’ sense of identification. This book is a very deliberate argument about the nature of—or possibility of—originality. It does, nevertheless, incorporate autobiographical elements, and therein lies the pain. As I worked this morning, I dredged up some long-buried and painful memories. Unexpected, really disturbing. Stuff that was important to me, that I couldn’t deal with emotionally, and that I had for decades locked behind the wall of memory, inaccessible. Everybody’s heard the assertion that can produce unexpected insights and trigger old memories. I’m not sure I’ve ever had that experience myself, until now.

Onto the page it goes. In this early drafting, I’m not holding anything back. There are things in Chapter 4 that I know can’t stay, things that would injure people I love if I published them. But I’m committing them to writing in the early drafts, planning later to revise down the material that might hurt people I care about.

What I hadn’t expected is that the drafting would hurt me. I don’t mean to say I feel injured in any sustained way. The sudden, unexpected stab of suppressed painful memory, though, is something that never happened to me as I wrote textbooks and scholarly work. If it happens again as I work on this book, I guess I won’t be so surprised by it. But I sure was when it happened for the first time this morning.

Peer review—one version

In semesters past, I’ve done a variety of peer review techniques: small groups, one-on-one conversation, written feedback. With students consistently preferring the written feedback, about 3 semesters ago I began using only that.

I’ve also found that if students are working on separate projects but with common sources, the feedback is better. Today, for example, my FYC students have brought in drafts of their analysis of a single source. But the class is divided into three groups, each analyzing a different source. However, we’ve all read, worked with, and are familiar with all three sources. So today the folks who wrote analyses of Steven Johnson’s article are reading the analyses of students who were assigned Jamais Cascio’s piece; those who wrote analyses of Cascio are reading the analyses of students who were assigned Nicolas Carr; and so forth. They are not peer reviewing papers that responded to their own prompt, and that removes all student concerns about someone else pilfering their good ideas. This makes peer review purely magical.

And here’s the direction sheet I gave them:

Peer response to analysis drafts

  1. Don’t write on the paper you are reviewing

2. Open an email; address it to the writer; copy it to me (REHOWARD@SYR.EDU)

3. Take up the role of coach

4. Analyze the draft you are reading for these rhetorical concerns we’ve been tracking:
(a) How does this draft construct its audience, and how might it do so better?
(b) How effectively does word choice convey its message?
(c) As you read the draft, what do you see as its thesis? Copy it into your email. What advice do you have for revising that thesis?
(d) As you read the draft, what do you see as its evidence? List it in your email. What advice do you have for improving that evidence?
(e) As you read the draft, what do you see as its counterevidence? List it in your email. What advice do you have for improving that counterevidence?
(f) Where do you see the writer making logos-based rhetorical appeals? How effective are they? What suggestions for revision do you have?
(g) Where do you see the writer making pathos-based rhetorical appeals? How appropriate are they? What suggestions for revision do you have?
(h) As you read this draft, how authoritative does the writer sound—what sort of ethos does s/he project? What suggestions for revision do you have?
(i) What additional comments or advice do you have for the writer?

5. Send the email. Put the draft you’ve been reading on the bottom of the stack, and take another classmate’s paper off the top of the stack.

 

 

Course description, “Mixed Methods Research in Composition and Rhetoric”

CCR 635
Mixed Methods Research in Composition and Rhetoric
Rebecca Moore Howard
Syracuse University
Spring 2014

The catalog description for this course says it “[s]urveys research methods and methodologies. Focuses on reading research rhetorically, crafting researchable questions, and designing research studies.” In this outing of CCR 635, we’ll begin with that survey and then move  quickly to a project-based course in which we will address a curricular issue from the SU undergraduate writing program . As we examine that issue, we’ll develop research questions and the tools needed to answer them, choosing from ethnographic, archival, hermeneutic, and data-driven research methods. In pursuit of answers to our research questions, each student will read about, implement, and teach the rest of the class about a single research method, and everyone will learn how to code research data. The class will conduct pilot research for the projects we’ve designed, to test and hone those methods. At the conclusion of the semester, we will collaborate on a report of our work to the Director of Undergraduate Studies. We will be attentive, too, to possibilities for bringing our research to closure and publishing the results. The final project for each student will be to draft part or all of a hypothetical or real dissertation prospectus, focusing on the methods statement.

peeking into Rogerian rhetoric

Working on a project, I realized that I didn’t actually know much about Rogerian rhetoric, even though I endorse its principles. So I did a day’s worth of reading. Here’s the result:

Rogerian rhetoric was not developed by Carl Rogers. He was an influential psychologist in the 1950s and 1960s who developed a nuanced methodology (not method) for therapy sessions, involving empathy and active listening. Compositionists of that period were very much attuned to psychological approaches to teaching writing. The first to take up Rogers’ ideas was Young, Becker, and Pike (1970). Maxine Hairston then took up YB&P’s ideas and published two influential articles in the late 70s. Rogerian rhetoric had arrived, and it persisted in scholarship through the early 90s. Phyllis Lassner describes its appeal: In the formulations of Hairston and of Young, Becker, and Pike, “Rogerian argument would . . . provide a heuristic by which students come to understand the assumptions and biases on which their positions are based, as well as the positions, attitudes, and values of those who differ from them” (221).

During the last decade of its scholarly career, Rogerian rhetoric had its critics who who challenged its political assumptions. Lassner names Berlin as one of them, but a look into her cited text, the 1988 “Rhetoric and Ideology in the Writing Class,” reveals that Berlin mentions Rogers once—to associate him (interestingly, he himself and not his rhetorical posse) with expressionistic rhetoric (Berlin 484).] Invoking Berlin and Michael Awkward, Lassner works from a cultural studies POV in which one cannot be empathetic without taking race and gender into account; otherwise one is complicit in “cultural hegemony” (Zappen’s words) and manipulation. In Ede’s view, Lassner says, empathy cannot replace “judgment and guidance”; unlike therapy, the writer cannot avoid leading the reader to agreement. Ede points out, too, that even as Young, Becker, and Pike profess a non-agonistic rhetoric, they refer to the reader as “the opponent” (Lassner 222).

Rogerian rhetoric persists today, it would seem, primarily in textbooks, where it is a staple though not a universal. This day’s worth of research inclines me to continue endorsing it, but I want to avoid the two common errors of textbook rhetoric about Rogerian rhetoric: the direct attribution of it to Carl Rogers, and the complete erasure of argument against it.
 
I’ve posted a bibliography of my work here; it has the full citations for the works I reference here. I’ll welcome suggested additions to it, or revisions to what I say above.

Arguing against Turnitin

This week a colleague wrote to ask for strategies in arguing against the use of Turnitin. Although my answer encourages a labor-intensive approach, I’ll post and expand on it here, in case others might find it useful.
I find that the issue of Turnitin adherence derives from the instructor’s a priori model of students. As is evident in so much discourse in places like the Chronicle and IHE, many of our colleagues are entrenched in an agonistic stance toward students in the aggregate: students are lazy, illiterate, anti-intellectual cheaters who must prove their worth to the instructor. Turnitin and its automated assessment of student writing is a tool for that proof that instructors believe is labor-free for them. It’s really hard to argue them down from something so convenient that so neatly reifies their image of students. For these instructors, pointing out how poorly Turnitin performs, how limited its  algorithms are, how much it taints the instructor-student relationship, the extent to which it diverts instructor attention from the more important tasks of teaching students to write from sources, and how much it infringes on students’ right in copy–well, for these instructors, rational arguments against Turnitin fall on fallow ground.
That’s one of the great uses of the Citation Project: no instructors aren’t fascinated by its findings; none of them say the findings don’t matter; and all of them readily grasp that to teach students better requires mentoring, not machines. Recently I presented the research to a faculty group and pointed out that using Turnitin answers none of the problems revealed by the research, and it keeps both instructors and students focused on the avoidance of plagiarism rather than on the larger and more compelling questions of responsible and authentic engagement with sources. As I noted these problems with Turnitin, my audience began shifting in their seats and looking at each other: it was clear that they were Turnitin users who were suddenly uncomfortable with that fact. Will it change their practice? I won’t know, but I’ll hope so.
If you want to point people to the CP research, the links from this page might be helpful. Basically, I think the best anti-Turnitin argument is this indirect one. But it’s labor intensive!

Students’ Researched Writing: Reconfiguring the Teaching Agenda

—fragments from my keynote address at the Coastal Plains conference, University of Houston, April 5, 2013. This won’t read as a linear text, because a bunch of the presentation was delivered extemporaneously, and with visuals. But here are the bits that were written out prior to the presentation:

Introduction

I’m going to present some research today that is likely to matter to anyone in the room who teaches composition—which, I believe, is practically everyone in the room.

Around this research presentation I want to make two inflammatory statements that I hope you will ponder as you consider the research and think about it later. These two inflammatory statements describe issues that I am wrestling with right now, as a teacher and scholar. If you happen to subscribe to my Twitter feed, you already know what these statements are, as I was chatting about them online this morning.

For the rest of you, I’ll begin with inflammatory statement #1: We—all of us in this room—are victims of a lethal combination of presentism, classism, and technology fear.

By way of explanation, I’d like to ask you to pause a moment here and think about yourself as an academic researcher when you were in your last years of high school and early years of college. How did you DO research? What did you think its purpose was? Please just think about this for a moment. I can tell you about my own process:

****

My own history as a young academic researcher pushes against the presentism that teachers suffer from, a presentism that disinclines us to connect our students’  behaviors to our own. My own history inclines me to think that we should also resist the widespread discourse of student-hating (a form of classism) that is so prevalent in places like the Chronicle of Higher Education. It inclines me to think that if the internet is damaging literacy (here we have the technology fear I mentioned), that literacy was none too healthy to begin with. I was an A student in English, loved it, but it never occurred to me that anyone expected me to READ those books I was quoting in my research papers. Research papers were a quote-mining game, a stylistic exercise.

With that in mind, I’ll show you the research that I’ve been a part of. As you listen, please work actively to resist the urge to demonize the students whose researched writing we studied; to rail against the internet for its deadly effects on literacy; and to fear that civilization is coming to an end. Resist these urges.

Notes accompanying slides

  1. We see this paragraph, with its carefully cited weaving together of three different sources, and we think, “This is good writing for a first-year college student!” And it is—especially when we note that the student is using brackets and ellipses skillfully and correctly to alter quotations.
  2. When the paragraph is shaded to reveal how much of the material comes from sources, it’s a bit less impressive. There’s actually very little here that isn’t from a source; rather than using the sources to answer a research question and develop an argument, the student is taking up the role of medium, calling the sources to life and ordering selections from them in her text.And the student’s methods of source use are copying and patchwriting. While copying is a legitimate means of source use—all of us, no doubt, quote when we write from sources—it doesn’t actually demonstrate the writer’s relationship to or understanding of the source. Copying indicates only that the writer is able to transcribe. And patchwriting (here indicated in yellow) is what happens when a writer tries to paraphrase but sticks too closely to the language or syntax of the source. Patchwriting happens, we hypothesize, when the writer doesn’t understand the source. That may be because of unsophisticated reading techniques, because of unfamiliarity with the topic, or simply because the writer was working too quickly.It’s worth noting here too that all the citations in this paragraph are to sentences in the source. Did the student read—or understand—the entire source? We don’t know.Finally, note that the quotations all function in place of the student’s voice. They are cited in parenthetical references at the ends of sentences, but there’s no signal phrase to introduce them. Rather, they function as an alternative, and more authoritative, voice for the student. They are “dropped quotations,” with no signal phrase or other material to introduce the source and establish a conversation with it. Instead, the source speaks for the student writer.
  3. Z04 (the pornography paper you just saw a bit of) works with multiple sources within a single paragraph. Although we don’t yet have statistics to back this claim, it is our observation that the mode of these 174 papers is to devote a single paragraph to a single source. Like the pornography paper, this selection from Z05 veers from that typical pattern, instead weaving two sources together in the paragraph.
  4. Like the pornography paper, too, this one is depending on dropped quotations that speak instead of the student writer. Notice, though, that here, unlike the selection from the pornography paper, there’s plenty of student voice in this paragraph, not just a stitching together of source material.
  5. Whereas the pornography and Statute of Anne papers work with multiple sources in a single paragraph, this one follows the more typical pattern of partitioning sources into separate paragraphs. Look at the citations here: the student is apparently determined not to be “nailed” for plagiarism and is thus putting a parenthetical reference at the end of almost every sentence! Another thing to observe here is the use of signal phrases to reference the authors of the source. Despite the incessant parenthetical references, these signal phrases work to make the passage more conversational than what we’ve seen in the pornography and Statute of Anne papers, even though they do no more than name the authors. Why is the student choosing to work with these sources? Why should readers find them authoritative? The paper doesn’t answer such questions. Are we suggesting that instructors (at least those having their students work in MLA style) focus some instruction on signal phrases? Well, yes. —But not as a mechanical technique; rather, as a way of incorporating analysis, evaluation, and synthesis into the use of sources.
  6. In this paragraph the writer is citing two different pages in the source. As the pie chart “Page in Source” shows, this is itself an accomplishment; approximately half of the 1,911 citations in the study are to the first page of the source.
  7. The paragraph begins with summary. Given that under 7% of the 1,911 citations in this study were for summary from the source, this is very good news, indeed, because it demonstrates that the student has engaged the text at something larger than the sentence level.
  8. What follows the summary is patchwriting; what’s shaded in yellow here is very close to the words or syntax of the source.
  9. From patchwriting the student moves to paraphrase, here shaded in green. This phase of the Citation Project research was entirely textual; we don’t know who this student is, and we can’t ask her what was going on as she composed this paragraph. We speculate, though, that she was unaware of any shifts in how she was interacting with her source. To her it was probably entirely “summary” or “paraphrase.” To us researchers, though, it is both varied and significant, showing how easily a student who is trying to summarize or paraphrase can slide into patchwriting, a type of source use that is still classified as academic dishonesty at some institutions. Here we can plainly see that academic integrity is not the issue; reading and writing skills are.Do note, too, that the summary (shaded in pink) at the beginning of this paragraph is of the material that is patchwritten (yellow) and paraphrased (green) in the remainder of the paragraph. We don’t yet know how many of the 83 passages (out of 1,911) that we have coded as summary are actually summarizing material that the student has also incorporated through paraphrase, patchwriting, or copying, as is the case here. But we have certainly seen other cases of it.
  10. Like the writer of the body modification paper, this student seems determined to show that he is not plagiarizing. The source being used is from the scholarly journal Personal Relationships, and it reports the results of grant-sponsored research. The student is using parenthetical references to note the source, but does not include page numbers in those references. Citation Project researchers, studying the student’s paper beside the source text, discover that the paragraph shown here, as well as the paragraph that precedes it and the one that follows, are from page 2 of the Sbarra and Emery article. There are no other references to Sbarra and Emery in the paper.
  11. Here’s the passage marked up according to its source uses. There’s so much to say here, it’s hard to know where to begin. At the most simplistic level, it’s worth noting that this presents another example of extraordinarily heavy source use—particularly when we look at the source.
  12. Here’s the passage from the Sbarra and Emery source, marked up to show where the student drew information. For three consecutive paragraphs, the student is working not only from a single page of the source, but a single paragraph. Here the source is marked up for how it’s being used in the second of the three consecutive Sbarra and Emery paragraphs in the student’s paper. The blue, remember, is for copying; the yellow, for patchwriting; and the green, for paraphrase. And the purple? We’ve used that here as a highlight: something very interesting and encouraging is going on in the purple-shaded text. Let’s look at that in detail.
  13. This side-by-side comparison of bits from the source, and the student’s use of those bits, discourages us from being dismissive of what these 174 first-year composition students are doing in their researched writing. Frankly, we don’t see a lot of THINKING about the sources in the papers we’ve studied. Careful examination and reflection, however, does turn up these subtle indicators that students who may be pretty unsophisticated in their deployment of research sources (they are, after all, first-year college students with lots to learn) may nevertheless be thinking seriously and creatively about those sources. Here the student begins by doing a pretty good paraphrase of the source (the green material). Then the student offers his own example of how that might work; what’s in purple on the right is the student’s illustration and interpretation of what’s in purple on the left.

A concluding bit:

I’ll conclude with my second inflammatory statement. As context, I want to postulate that many if not most of the people in this room are focused on English studies with a literary bent. That was the nature of my own graduate studies at West Virginia University. And as I began my early teaching and scholarship, it was the hermeneutic bent of English studies that I brought to the classroom and my writing. I never questioned the universal efficacy of my careful reading of student texts. I made big generalizations based on those readings. For one of those generalizations, I became famous as a scholar. It was the early 1990s, and I said—basing my statement on my careful reading and interpretation of 9 student papers in a class I was teaching—that students patchwrite when they are working in unfamiliar academic discourse. However, the Citation Project’s quantitative analysis of papers produced by 174 students at 16 colleges nationwide suggests that they patch write ALL THE TIME, regardless of how difficult the source is. (I don’t have a pie chart for it yet, but we have done that analysis.) In the Q&A I’ll be glad to talk about how we interpret that finding, but before that, I want to give you my second inflammatory statement: Don’t place too much credence on your own careful reading of your own students’ writing. What you gather from your hermeneutic reading of your students’ texts is undoubtedly valid FOR THOSE STUDENTS. But you are only one reader, however skilled; you bring your own expectations, culture, and biases to your reading; and you should thus not overgeneralize about what you find. Instead, treat it as a form of pilot research. Your hermeneutic reading of your students’ texts raises issues for investigation, but until you are doing that systematically, until others are doing it with you, and until you have tested the replicability of your methods, what you have is a hypothesis, not a finding. I’ve learned that the hard way myself!

This is not to reject hermeneutic interpretation; far from it. This research began with interpretation, and after the data analysis stage, we turn to interpretation again. And here are some of our interpretive recommendations for our fellow teachers.

. . . followed by the list of pedagogical principles that I offer at every opportunity, including this one:

1.Less is more.

2.Work from shared texts.

3.Work from complex texts

4.Focus on intensive engagement with texts.

5.Teach methods for analyzing the rhetorical moves of a source.

6.Teach methods for interpreting and evaluating sources, especially Web pages.

7.Teach both shallow and deep reading—and the rhetorical occasions for both.

8.Teach paraphrase, summary, and methods for integrating them into written arguments and analyses.

9.Focus on rhetoric rather than mechanics of source use.

10.Less is more.

Why This Humanist Codes: A Genealogy of the Citation Project

Here’s my paper from the 2013 CCCC:

 

Rebecca Moore Howard

Conference on College Composition and Communication

Las Vegas, NV, 14 Mar. 2013.

 

 

Why This Humanist Codes

 

Several years ago, an applied linguist in Sweden emailed to introduce herself and tell me that she had published an article that tested my hypothesis about patchwriting. “Tested my hypothesis about patchwriting?” I was very much taken aback, even a little indignant. I didn’t have a hypothesis; I had findings. I had paid attention to what my students were doing in my classes, I had described it carefully; I had read exhaustively in the extensive postmodern theory of authorship and the scanty scholarship of student plagiarism, and I had written several articles and a book that had been very warmly received in my field.

That linguist, Diane Pecorari, had kindly attached a copy of her article,[1] in which she says, “Howard’s patchwriting model offers an intuitive appeal and, potentially, a powerful explanation for the phenomenon of non-prototypical plagiarism. However, too little empirical evidence for patchwriting exists” (320). Intuitive appeal? Too little empirical evidence? Surely the woman was mad. Surely she was failing to perceive the power of my analysis.

(As you can readily perceive, I was not trained in data-based methods. My PhD was in English, and although I had written a dissertation in what I would now call Writing Studies, all of my required coursework had been in literature and linguistics. In my department in the early 1980s, there were no courses offered in Writing Studies or in research methodology of any kind. Close reading was all one needed to know, and one learned it in every single class.[2])

 

I was still mulling over Pecorari’s assertions when I attended the 2006 conference on Writing Program Administration, at which Chris Anson delivered a keynote that stirred many of us in the audience. Anson said (and here I am quoting from the 2008 published version of his talk)[3] that our field, and the specific subfield of writing program administration—needs a reinvigorated research agenda. Anson explained that most arguments advanced by WPAs were based on belief—”whether theorized or not, whether argued from logic or anecdote, experience or conviction” (11)—and that people outside Writing Studies would not be persuaded by WPAs’ beliefs or disciplinary authority but instead would need what Richard Haswell calls “solid and ever-strengthening data” to be persuaded (qtd. in Anson 22).

As a scholar, I was and am very much involved in attempting to persuade academics outside my own field. I wish to persuade them that Turnitin.com is not the magic solution to student plagiarism. I wish to persuade them that students’ textual missteps are not a unitary category of unmitigated evil that we can “catch” with a software program and punish under the label plagiarism but are instead a mixed bag of language acquisition, source misuse, incomplete comprehension of US conventions, undeveloped critical reading processes, multicultural textual expectations, and yes, sometimes evil, deliberate appropriation of unacknowledged sources. The responses in my own field to my early scholarship—the stuff with the “intuitive appeal” but “too little empirical evidence”—were mostly positive; I was able to persuade my colleagues to share my beliefs. My articles, especially a 1995 piece from College English,[4] began showing up in TA practicum coursepacks.

Outside my field, however, I was variously called, in print, in the public media, a fool and a villain who was undermining the foundations of Western civilization and its prospects for continuance. In Reader’s Digest, Tucker Carlson declared me a “skillful apologist” for plagiarists.[5] In the National Review, Jonah Goldberg decided that I was either a “tweedy, French-bathed barbarian in pursuit of destroying Western Civilization” or an “idiot.”[6] In U.S. News and World Report, John Leo categorized my work as “politically correct” (remember that witch hunt?) and even implied that I was—gasp—a feminist.[7] Alumni of my university began writing to the Chancellor, demanding my resignation.

Good times.

It was in that environment that I began to code—not as a way of escaping from the firestorm, though coding is indeed a great escape mechanism—but because it was clear, from Anson’s positive testimony and Goldberg’s, Leo’s, and Carlson’s negative attacks, that I was getting nowhere in changing the way the academy and the culture at large respond to too-close use of source language.

At first I didn’t study the scholarship on coding. I didn’t even know such scholarship existed. (It was 2006, but I was still pretty clueless about data-based research methods. They seemed quite obscure and opaque.)

Instead, I studied Pecorari’s works, deriving from them a method for analyzing and categorizing students’ textual moves when they work with sources. For something as scary as data-based research, working from a model came much more easily to me than working from explanatory guidelines like Cheryl Geisler’s excellent book Analyzing Streams of Language.[8] With Tricia Serviss and Tanya Rodrigue, I hybridized Pecorari’s methods, adapting them to the purposes of what eventually became the Citation Project. As we tested the system, I read everything available on citation context analysis, materials from applied linguistics (Pecorari’s field) and information studies.

We coded. Our coding was a simple categorization of intertextual moves in undergraduate research papers, determining whether students are summarizing, paraphrasing, patchwriting, or copying when they cite their sources. Simple question, simple categories.

Or so it would seem. Actually defining and applying this coding system took three years of work before we felt that multiple coders could more or less readily arrive at similar results when they coded a paper. The process was messy: we were working with naturalistic data, not a controlled study. We used no computer programs to analyze the students’ papers; instead, Writing teachers trained in applying our coding system read those papers and interpreted their citations.

When Sandra Jamieson joined the project as co-principal researcher, she saw that we needed to be able to quantify our results. Braver and more energetic than I, she actually enrolled in stats classes and learned how to use SPSS as our database. Since then, she has been producing amazing quantified analyses of our results.

One of those analyses helps with the argument that patchwriting students aren’t trying to “get away with something.” On the contrary, the frequent co-occurrence of patchwriting and paraphrase suggests that patchwriting is something students inadvertently, perhaps unconsciously, slip into when they’re trying to paraphrase. In a forthcoming article in Across the Disciplines, Sandra reports that “78% of the papers [studied in the Citation Project] include at least one incidence of paraphrase, [and] 52% include at least one incidence of patchwriting.”[9] All but one of the papers that include cited patchwriting also include paraphrase, with the student often moving back and forth between the two when presenting material from the same source.

However, as the results continue to roll out of Sandra’s queries to the database, it appears that an important part of my initial “hypothesis,” with its “intuitive appeal,” is wrong—or at the very least, not proven. Since publishing my first article on patchwriting in 1993, I have argued that when students patchwrite, it is not out of an intent to deceive, not out of academic dishonesty, but out of an incomplete comprehension of the source. Armed with a snootful of the social constructionist theories that circulated in the 1980s, I had postulated that students working in unfamiliar discourse communities do not have the vocabulary for successfully paraphrasing the sources with which they are grappling. My audiences, steeped in the same theory-set, readily agreed. And I still believe that it is a useful, insightful hypothesis.

But it is a woefully incomplete one, and it does not get to the heart of the patchwriting issue. Coding and quantifying has, I believe, gotten us there. What Sandra’s initial analyses of the 174 research papers produced in first-year Writing classes at 16 different institutions suggest is that students patchwrite no matter how simple the source, even when the source is crafted for general-audience consumption.

From that observation, which will be explained in detail in our book Struggling with Sources—which we are currently writing and which Parlor Press has pledged to publish—we are generating a new hypothesis: that students patchwrite because they are engaging shallowly with their sources, usually working only with sentences gleaned from the first three pages of the source. In a sense, they don’t understand any of their sources. How could they, when they’re not actually reading them? And when they don’t understand—when they are merely quote-mining their sources—they patchwrite.

These two hypotheses—my 1993 hypothesis deriving from social constructionist theory, and our 2013 hypothesis derived from quantified analysis of teacher-produced coding of naturalistic classroom data—lead to very different pedagogies.[10]

And the data? Even before we publish our book, those data are circulating from conference presentations, from interviews that others have conducted with us, from faculty development workshops, and from our website. They are circulating and persuading not just in our field but outside it. People are changing their curricula based on the Phase 1 findings of the Citation Project.

Now I am a coder, a data-based researcher, and there’s no going back. Now I maintain a bibliography on coding.[11] Now I insist that graduate students read about coding before they start writing their dissertations.

I still read, teach, and am informed and persuaded by Theory. I would no longer, however, use theory as my sole means of interpreting and understanding cultural texts. I hope never again to find myself so persuaded by a widely-accepted theory that I allow it and it alone to guide my analyses. Whether I—and my graduate students—are studying archival documents, community sites, student writing, or interview transcripts, we categorize, we code, and then we interpret.

Just last week a graduate student asked whether I ever direct dissertations that are not data-based. I was startled by the question, and my startled reaction tells the tale of how far I have moved in how few years. Now my considered reaction is, why would anyone want to write a dissertation that does not bring fresh data to the field of study, that does not analyze those data in ways that allow others to reach their own judgments, to be persuaded by the analysis, to use the data for other queries? I guess the answer I would have given that graduate student, if I’d had time to consider the question, would have been, “A dissertation that is not data-based and whose data are not coded? How very dreary that would be.”

 

 

 


[1] Pecorari, Diane. “Good and Original: Plagiarism and Patchwriting in Academic Second-Language Writing.” Journal of Second Language Writing 12 (2003): 317-345.

[2] Murray, Heather. “Close Reading, Closed Writing.” College English 53.2 (Feb. 1991): 195-208. Print.

[3] Anson, Chris M. “The Intelligent Design of Writing Programs: Reliance on Belief or a Future of Evidence.” WPA: Writing Program Administration 32.1 (Fall 2008): 11-36.

 

[4] Howard, Rebecca Moore. “Plagiarisms, Authorships, and the Academic Death Penalty.” College English 57.7 (Nov. 1995): 708-36. Print.

[5] Carlson, Tucker. “That’s Outrageous: Reading, Cheating, and ‘Rithmetic.” Reader’s Digest 161.963 (July 2002): 39-42. Print.

[6] Goldberg, Jonah. “Plagiarism Is Rape?” National Review Online. 15 Mar. 2000. Web. 13 Mar. 2003.

[7] Leo, John. “An Outing Is No Picnic.” U.S. News and World Report. 22 May 2000. Web. 9 Mar. 2013.

[8] Geisler, Cheryl. Analyzing Streams of Language: Twelve Steps to the Systematic Coding of Text, Talk, and Other Verbal Data. Pearson, 2004. Print.

[9] Jamieson, Sandra. “Reading and Engaging Sources: What Students’ Use of Sources Reveals About Advanced Reading Skills.” Across the Disciplines (Fall 2013). Print. Forthcoming.

[10] Howard, Rebecca Moore. “Comp 1 Applications of Citation Project Research.” Chenango Metonymy 22 Feb. 2013. Web. 9 Mar. 2013.

[11] Howard, Rebecca Moore. “Coding.” n.d. Web. 9 Mar. 2013.

Comp 1 applications of Citation Project research

People keep asking me how I teach Comp 1, given the pretty significant findings of the Citation Project. Here’s my first draft of an explanation:

The challenge:

As one of the principal researchers for the Citation Project, I can state briefly the research findings that are described in detail on our website (and that will be described in further detail in our forthcoming book, Struggling with Sources). Our analysis of 174 students’ research papers produced in the required writing course(s) at 16 colleges and universities indicates the following:

  1. Research data strongly suggests that students work hard to avoid plagiarism and to implement the instruction they are receiving in their Writing courses.
  2. Nearly 50% of students’ citations come from page 1 of their research sources. This suggests that when students have found a passage that can be usefully quoted or paraphrased, they quit reading. Only 25% of students’ citations come from page 4 or deeper into the source.
  3. Approximately 50% of students’ citations are to quoted passages from the source. Another 44% are to attempted paraphrase. Thus students are, 94% of the time, working at only the sentence level within their sources; they are not talking about the source as a whole nor trying to summarize significant passages of it.
  4. When students attempt paraphrase of 1-3 sentences, they succeed approximately 66% of the time. The rest of the time, their attempt results in patchwriting, borrowing too heavily from the original phrasing of the source.
  5. The sources referenced in more than 50% of students’ citations are no longer than 5 pages. Many of these are overview reference sources such as WebMD, Wikipedia, and Encyclopedia.com.

These findings pose a significant challenge for first-year writing instruction and for all university courses that ask students to read extensive, complex sources. Our research data makes clear that students are striving to implement what they are taught. The data also make clear that students either do not how to engage with entire sources, or do not know that they should. Instead, the students are doing what the Citation Project researchers call “quote-mining”: linearly reading sources only for quotable or paraphrasable sentences.

I have spent the past three years (Fall 2010, 2011, and 2012) developing a Writing 105 at Syracuse that teaches first-year writers how to engage with complex sources in substantive ways. Each semester’s endeavors have taught me more about how much instruction today’s students need.

My WRT 105 syllabus:

The following sequence of learning objectives aims to help first-semester college writers read sources critically, write about them in depth, and make connections among them:

  1. Differentiating claims from evidence in complex sources. (In the context of my section of WRT 105 at SU, the “complex sources” are articles from publications such as Harper’s, New York Times, and Atlantic.) Students need to be taught this before they’re taught to summarize sources; otherwise, they tend to summarize the evidence rather than the claims.
  2. Evaluating whether a source offers alternative interpretations and multiple viewpoints, and whether it treats these multiple viewpoints as errors to be refuted, or as material for critical thinking. Students quickly become skeptical of one-dimensional arguments.
  3. Identifying key passages in the source and paraphrasing them without patchwriting.
  4. Summarizing the main claims in the source.
  5. Integrating summary into larger discussion of a source.
  6. Learning basic techniques of intrinsic source analysis: recognizing what sorts of evidence are being offered; what type of argument is being made; and what this suggests about the author’s sense of his or her audience.
  7. Learning basic techniques of extrinsic source analysis: identifying the author, publisher, and date of publication in search results from the open Web and library databases; discovering the author’s and publisher’s credentials and agenda; and appreciating the importance of date of publication for appreciating the context in which a source was published, thus avoiding the presentism that would have students read the source as if they, in their present moment, were the source’s only target audience.
  8. Incorporating their source analyses into their written arguments.
  9. Writing extensive evaluations of a single source that include summary and analysis.
  10. Discovering connections between sources, seeing those connections as “conversation,” and seeing themselves as participants in that conversation.
  11. Asking questions as they read sources—questions that they themselves might pursue in small research projects.
  12. Seeing research not as a compilation of basic factual information but as the pursuit of tenable answers to complex questions; and seeing the presentation of research as an opportunity to engage and explain complexity, rather than to “clean it up” and present oversimplifications.
  13. Appreciating the importance of having an argument, advancing a point of view, speaking for themselves with confidence as they write from sources.
  14. Understanding citation not just as a vaccination against charges of plagiarism but as an essential component of a writer’s credibility and authority.

Student reception:

At the beginning of Fall 2012, the students expressed real apprehension about what they were going to be taught. They were candid about the fact that the course would not present them with opportunities to use the writing and reading techniques familiar from high school instruction, and they worried that they would not succeed. Their end-of-semester evaluations, however, express appreciation for the course:

  • “I feel the most significant thing I learned was how to analyze sources and incorporate my own voice. . . .”
  • “We were taught to understand the tone of an article and the word choice. It helped me to learn to speak to my audience better.”
  • The instruction “helped in other courses I took this semester when I wrote papers that required reading and analyzing.”
  • “The level of writing was challenging because it made me rethink my original ideas, and to rewrite my papers to get the best one I could.”
  • “I now have a new writing style that I only obtained from this class.”
  • “With the right technique, I can become a creative, interesting writer.”

I was satisfied with the course, as well. By the end of Fall 2012 I was seeing student writing that I felt had real content and was presented persuasively.

Cross-curricular implications:

Instruction such as this in WRT 105 is only the beginning. If our students are to become skilled at critical, insightful, engaged reading and writing, they need reinforcement throughout the curriculum. Instructors in every discipline, when they give reading assignments, can contribute by taking time to talk about how members of their discipline read (and write) sources. When they give research assignments, they can talk about why and how members of their discipline conduct research, and how research should be presented within that discipline.

Linked sources:

Elder, Cristyn, Ehren Pflugfelder, and Elizabeth Angeli. “Handout: Comparing Policies.” Purdue Online Writing Lab 11 Nov. 2010. Web. 21 Feb. 2013. <http://owl.english.purdue.edu/owl/resource/929/15/&gt;.

“Phase I Data.” The Citation Project. n.d. Web. 22 Feb. 2013. <http://site.citationproject.net/?page_id=224&gt;.

“Researchers.” The Citation Project. n.d. Web. 22 Feb. 2013. <http://site.citationproject.net/?page_id=11&gt;.

Strauss, Valerie. “A Warning to College Profs from a High School Teacher.” Washington Post 9 Feb. 2013. Web. 21 Feb. 2013. <http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/02/09/a-warning-to-college-profs-from-a-high-school-teacher/&gt;.