Research & Criticism

In the last few weeks I’ve had frequent occasion to ponder the role criticism plays in research. I take it for granted that criticism is a part of research, start to finish, but this is something I’m finding I need to be more aware of when engaging those who are coming to research from other paths. No one likes being told that their hard work needs attention or revision, but it’s as necessary to effective research as having sources.

Evaluating a position is built in—it has to be if research is to work. We are fallible, all of us, no matter how much training we’ve had. We will make mistakes. Good researchers, good students, take those and use those mistakes to improve their work. Sometimes that means accepting the hard fact that a line of reasoning we’ve been working on is flawed and should be abandoned. It sucks when this happens. But, this is the way. It’s how we reach a vaccine that works instead of one poisoning us or having no effect whatsoever; it’s how we know the Egyptians built their own pyramids and not some fanciful space visitor; it’s how we know that Fiore, de Saint-Didier, and Aldo Nadi followed the same universal principles of fence.

Giving criticism is difficult, receiving it even harder, but it’s part of the process and so it makes sense to talk about it in more depth. [1] A decent analogy for the process is to consider refining an iron ingot to make steel. The smith heats then hammers it, reheats it and hammers again. In the process the dross drops to the floor and one ends up with steel suitable for work. Fire burns, hammer strokes hit hard, but they’re necessary. The smith doesn’t use fire or the hammer out of hatred for iron, but to improve it. It’s the same with “constructive criticism,” least it should be.

Elements of Research

Framing the Question

To conduct research well requires sufficient grounding in the topic. A student new to a subject is often given a question to answer or issue to explore. This is what teachers do in secondary school and college. With those new to Roman history, for example I used to give them a prompt because I couldn’t expect them to have command of the events, people, and issues of the time. When we covered one of Rome’s most lasting contributions, naturalized citizenship, I selected relevant works for them to read, and then gave them a question that had them use those selections to answer it. For freshmen, this might be relatively straight-forward, but for upper-division students I could take this one step deeper, such as focusing on one text or an aspect of the larger issue. For a graduate student this goes deeper still—they might read Tacitus, portions at least in Latin, and then familiarize themselves with the relevant scholarship about Tactius’ stance on Roman citizenship. Some upper-division students might do this too depending on ability, major, etc. It’s important we meet students where they are and then push them, gently, to the next step—sometimes we meet them at the door, sometimes sitting at the table in a large room. Assessing ability is part of the job, and, it’s something that test scores are next to useless in helping us do. Teaching someone new to sabre, for a different example, one starts with the most basic material, how to stand, move, and extend the arm, not with second intention attacks, compound parry-ripostes, and advanced tactical use of tempo.

Evaluating the Argument

Once that paper is drafted, or more usually now just turned in, the teacher evaluates it. Did the student answer the question? Did they support that answer using evidence, and, did they do so appropriately? Did they show me how that evidence supports their conclusion or merely state that it does? If they used any theoretical frameworks did they do so accurately? Are there logical fallacies or other errors in reasoning? A major portion of this, and easily the most disliked by students, is simultaneously assessing their grammar and syntax. Clarity is the goal, so extra verbiage, three-dollar words sprinkled for effect, and other distractions are important to excise. Just as we correct a fencing student to use only what they will need in a bout, so too do we correct excesses in writing that undermine a researcher’s thesis.

HOW we make this assessment, however, is everything. Effective editing with kindness is, in my view, one of the most difficult skills to learn. With sentences that are awkward, for example, writing the abbreviation “Awk” in red pen next to the offending line isn’t very helpful. Instead, I resort to “Perhaps rephrase? Maybe something like this: ….,” and now I do this even with seasoned writers. This is a gentler way of pointing out a problem area, but it also helps steer them toward what they need to do to fix it. When one of my fencing students moves their foot before the weapon, I don’t shout them down, but point it out and have them do it again. If the student is sensitive, I remind them that this stuff is really hard to do, that they don’t move like this 99.9% of the time, that we all go through this, and have them do it again, and again, until they get it right.

Term Paper vs. Research Article

The way one evaluates a peer’s research paper is similar to the process that one uses with a student paper, but more rigorous and approached with the assumption that the author has learned how to take criticism. This is a dangerous assumption, however, when working with amateur researchers (“amateur” here meaning not professional scholars). It is likely that an amateur scholar is not used to the process. This is one lesson I have learned painfully this year.

I do a fair amount of editing for colleagues, some professional, some amateur, both generally researchers of skill that know that I’m not blasting them for mistakes, but only trying to help them make their work stronger. It’s part of the job. We assist one another. In a talk I’m delivering this very week I have asked the host and a trusted researcher I know to go over my slides and notes. Each has given me useful feedback that will only help me. Now, I could feel bad or embarrassed that I didn’t think of some of these things, but why? This is why I sent them my work. Research, never mind sharing it, is hard! There is no shame in getting help—that goes for any stage of expertise. They are helping me increase the chances that my talk is a success. I am grateful to them for that. I will also be sure to announce their contribution.

Research Reserved is like a Broken Rapier

Research, if it is to have any meaning, must be shared. Even when I was teaching college courses and trying to publish on the side I embraced this idea. It’s one reason that teaching at junior colleges was important to me; it’s also why most of my publishing to date is what too many of my colleagues consider “soft publishing.” [2] It’s why I took the most important part of my dissertation and shared it for free on academia.edu: more people will see it via google than will read it as a monograph collecting dust on the handful of libraries that might buy it. All this work is useless and self-serving if the only people who benefit from it are other academics.

Of course, this is also one reason I don’t have a tenure-track position or why I am not writing reference works at the moment. It’s an imperfect analogy, because this example is more exalted than my own, but when Prometheus shared fire with humanity the gods felt that he had broken the rules. For those academics whose response to attack has been to hole up in the ivory tower and look down on the supposed rubes assailing them, those of us actively passing out research (or worse trying to teach people how to do it better without collecting a cent) are turn-coats. There’s not much they can do but keep us out of the mix and insult us from afar. Dying industries tend to entrench.

Donning the Big-Kid Pants

In sharing research, however, one must be prepared for criticism. If any prospect of that is repellent, then don’t write and share your work. Forget your friends and/or fans, forget the colleagues miles away who are likely to agree with you—they’re easy; consider only the person who wants to see you fail, because sad to say they sometimes exist. Normally it’s not personal, but it will feel that way. If you’re prepared for that clown, one you’re unlikely to meet, then you can handle anything. There was, in the 1990s, a notorious academic in medieval history, who delighted in shredding graduate students at the Huntington. His work is good, and I’m guessing his classes were, but fellow graduate students who delivered papers at that conference dreaded his responses. He took perverse delight in tearing them down, the way a deranged boxer might in punching a toddler. I never had to deal with him, lucky for me (and him—I was a different person in the 90s sad to say), but I ran into his type more than once. [3] Navigating failed humans like that guy, which no one should suffer, will toughen those up who survive it. Today, were I to deliver a paper and receive grief from this loser I’d be only too happy to engage–he attacks because he is weak.

Research & Responsibility

I can only speak for myself, though I think it holds for many people, but the longer one spends in research the harder it is to embrace arrogance or denigrate others who do this too. There is more out there to explore than we have life to live; producing an argument and then sharing it, globally, is not for the faint of heart. We gain nothing in being mean or beating on someone. However, when an argument is weak, flawed, or in some other way deficient it’s not only proper to say so but also to point it out how. Research may be conducted individually, but done right the product is collective—conclusions, right or wrong, affect the whole. They affect us all. By implication this means we have a shared responsibility, to ourselves, to one another, and to those who read us to do our best work. It’s a collaborative process, really.

It is best not to take it personally; after all, it’s rarely personal. More often that not we do not know or barely know others in the field when it comes to “HEMA.” Social media, unfortunately, has proved an ideal vehicle for “trolls,” half-wits, and those who having been bullied at some point feel they can retaliate anonymously and somehow get their own back. It shouldn’t be hard to tell the difference between a troll and someone pointing out a potential issue with our work. Some people are blunt, a personality aspect hard to detect in writing or online, but well-meaning; some are so apologetic they never get to the point; some are kind and just list the issue, and if we’re lucky, provide some help; and then there are the trolls. Analyzing the comments on your paper, blog, Youtube video, etc. will help you figure out who is who, and, whom it is worth listening to. To be honest, on social media and Youtube disabling comments is the best bet–legitimate critics can contact you in other ways.

Just as there are many ways to give a critique, there are many ways to respond. I’ve witnessed most in this context, and while many are acceptable, the best combines listening humbly and responding graciously. If it helps, fake it; pretend. Imagine that Capt. Red-pen is one of your friends just eager to help you improve your argument. In fairness, criticism, where it isn’t a case of right and wrong, sometimes comes down to style and preference. One editor may not like fancier turns of phrase; one may love it. It’s important to distinguish between substantive issues, which we should always consider, and stylistic ones, which we can examine and then decide if it merits further attention. For example, outside of the US there are countries where use of the “historical present” is acceptable if not normal. This is where the author writes about past events as if happening now, e.g. “Colum Cille visits Brude. They talk. The Pictish king is unconvinced, but polite” versus “Colum Cille visited Brude. They discuss various matters, but the Pictish king, while polite, was unconvinced by the missionary’s message.” If the paper you’re editing or reviewing is written this way ask your writer who the audience is and where they plan to publish it. If they are aiming at a North American market, suggest they revise and use a past tense; if for a journal in Europe suggest they check with the editor of that serial as it may be perfectly okay. For a more common example, some writers enjoy a good turn of phrase, some do not. You can suggest that the paragraph-long sentence might be broken up, but the writer doesn’t necessarily have to change a thing—that may just be the way they write. If they’re publishing, then the editor will have better call to make a case for brevity.

If however, your reader points out a misreading, a missing piece of evidence, an important article you haven’t referenced that should be there, or a misstep in reasoning, then set your ego aside and reread your work. Look at it plainly, as if reading someone else’s work, and see if they’re correct. If they are, thank them and revise; if not, and there is a back and forth, explain it. They may not agree, but at least you’ve had the conversation. There are times when we will make the wrong call, put our work out there, and realize that we should have revised. It happens. [4] It can also be avoided, and it gets easier too, especially if you’ve made that mistake and learned the lesson. Some people have to fall before they realize they can get back up. Some, however, will refuse to stand up, remain prone and fight to the death that they were correct when they’re not. [5] People are people.

A Figurative Glove cast in the Hazard

Do your best work. If someone with appropriate training comments on work you’ve shared publicly, have the good sense to consider it—thank them even if you decide to ignore it. You lose no face in being gracious and it will indicate that you are someone who knows how to play nicely with the other kids. A poor defensive response reads a certain way to professional researchers, and if you are going to play their game, then it behooves you to know the rules. They’ll hold you to them whether you like it or not.

I tell my fencing students that they should never, ever underestimate any opponent: treat each one as the most dangerous person that they’ll ever face. We practice, we drill, we train so that under pressure, when it counts, our technique and tactics will be effective. This often means working far more complicated actions in practice—that effort helps refine our game so that when we use the typical, less complicated maneuvers that we do in a bout they are that much crisper.

The same principle works in research—anticipate potential issues and correct them as best you can before sharing your work; read, reread, and verify; share your draft with people trained to evaluate both the material and process. Consider any suggestions. Once that paper is in print or posted online, it is exposed to the world and by extension, so are you. Be prepared for a variety of responses, some great, many more a near silent “meh,” and then a few that seem tailor-made to make you feel as small as possible. Answer each with calm, grace, and confidence—ignore fools, but cultivate a response to legitimate criticism that is measured and open-minded. A lot of researchers fail, some through fraud, some through hubris, some through just being too stupid to listen, and some because they quit when they get a bad review. Every successful scholar abides the dictates of the methodology of research and knows how to take criticism—they use the latter to make their position stronger. You should too.

Notes:

[1] See entry on this site entitled “Dealing with Criticism” 28 Oct. 2019.

[2] Academia is a brutal place. The rewards are few and small, so those lucky enough to find a position or who lick enough boots to land one tend to guard those hard-won prerogatives tenaciously. As with any organization composed of rigid hierarchies, there is a sense that those at the top deserve it, and that those who work below decks, as adjuncts, lecturers, and at junior colleges are where they are because they lack the genius and gifts their tenured peers must possess. It’s bullshit, but between poor pay and the fact that Americans dislike intellectuals their arrogance is unsurprising. What goes for teaching goes for publishing, and even now a monograph and second book are considered “real” work where publishing for the masses is considered less rigorous.

[3] I’ll not name this buffoon, but will say that his two volume medieval reader and work on the Merovingians remains popular. Smart as he might be, important as his work might be, he’s the perfect example of someone who believes their own painful path to full-time teaching entitles him to be abusive. I was far luckier—most of my PhD committee were older scholars, well-respected in their fields, and far too kind and intelligent to indulge in such behavior. The one exception was the guy who tested me on Greek history during my oral exams. It was disgusting enough to prompt my Celtic professor, the wonderful Jószi Nagy (then at UCLA), to ask me the next time I was down there for a class “So, what gives with the macho Greek professor?” He shared the story with the rest of the small class and they were horrified, and this was at UCLA where t.a.s received more comments about how they dress on student evaluations than anything else. Classes with Jószi were some of best I ever took; it didn’t hurt that he is Hungarian and I love sabre either 😉

[4] For a personal example, I contributed to my graduate advisor’s Festschrift, a collection of articles by students and colleagues celebrating his career and contributions to the field. My friend, the late Tom Sizgorich and I, were sort of outliers—Tom focused on early Islam, I focused on early Ireland—but if anything we serve as good examples of just how nimble our advisor, Hal Drake, can be. My submission was on the blending of Mediterranean and Irish narrative motives in the vita of the saint I worked on the most and whose life I translated in my dissertation, St. Áed mac Bricc. The editor requested that I remove the Latin portions of the quotations I used from my notes, and excise some of the examples I used for the nods to Celtic ideas of the “Otherworld” in the paper. I did. The review that came out in Bryn Mawr Classical Review was kind enough, but mentioned in re my paper that more examples from Irish texts would have helped. He wasn’t wrong. Nowadays I would have politely disagreed with the editor and left them in, but we live and learn.

[5] I’d rather not use an example from HEMA here, not after the most recent encounter with this, so I’ll stick to another academic topic. There’s a scholar in my field, a nice enough guy, and well-trained, but whose analysis tends to suffer from a propensity to make tenuous connections. I’ve seen him do this with both linguistics and historical topics. For the latter, he delivered a paper at one conference that would make a decent movie, but which was poor history. Taking three attestations of the name “Patricius,” instances separated geographically and in some degree temporally, he posited that each referred to the same man. Responses were polite, as they usually are at Celtic conferences (it’s a really small field), but they were to the point too. One of the audience asked “—-, a simpler explanation is that these three pieces of evidence refer to three, perhaps two different people, right? Is it likely that the one Patrick we all know traveled this extensively, and, had time and inclination to put his name on a brick?”