Maybe We Shouldn’t Assess Public Archaeology

Following last week’s post, you may sense that I am on the fence about “evaluation.” I am passionate about public archaeology and heritage work, and I believe we should strive to gather more information about how our actions affect other stakeholders. (Side note: you can learn more about how Program Evaluation professionals do this in a reply to my post by Victoria Dekle.)

And yet, “evaluating” ourselves is an uncomfortable concept that brings to mind a universal system for “testing success” when success is subjective. We all have goals for our specific contexts, and stakeholder communities in those contexts who have their own priorities and criteria for “success”. How could we possibly be expected to create a methodology that could address such a tangle of potentialities?

Well, we don’t have to.

For my final post as Guest Editor of the MAPA blog, I am going to explore a few hypothetical arguments against evaluating public archaeology work. (As far as I can tell, there has not been much discussion about the perils of assessment, besides that people probably don’t want to be perceived as tearing down colleagues.) If the budding discussions about public archaeology outcomes continue, these arguments should be considered.

Why not evaluate?

Because peer review is sufficient evaluation

Author walking down the road with his paper, colleagues holding weapons, if he makes it through he reaches "Paper Accepted" sign

Comic by Nick D. Kim (strange-matter.net / lab-initio.com)

The foundation of modern academia is peer review. Funding, publishing, and job opportunities are all mediated by the process. Regularly presenting your work to colleagues for feedback has proven a robust mechanism for developing archaeological theories and ethical frameworks, so where is the compelling evidence that we need a separate process for determining if public archaeologists are doing enough self-assessment? This very discussion of evaluation/assessment in peer-reviewed venues may be the most appropriate path forward.

Because we don’t need anything but authenticity out of public archaeology

Doing public archaeology is often described in terms of its emancipatory effects for both researchers and stakeholders. Underlying these statements seems to be relief that something can be done to address the colonial heritage of archaeology.

Indigenous people hiding their television and appliances saying "anthropologists! anthropologists!"

Anthropologists’ public identities are tied up with notions of authenticity (Copyright 1984 The Far Side, Gary Larson)

With the massive expansion of public archaeology events this decade, a skeptical (cynical?) viewer might say that what practitioners are really after is authenticity. That is, using non-archaeologists and their legitimate claims to the past to make our work more essential. This is a critique levied on other participatory practices when the critic does not believe in the objectives of the enterprise (the first to come to my mind are community based participatory research and participatory art.) It is worth considering that each of our individual public archaeology practices are shaped by our own internal struggles in addition to our judgment about what is appropriate intellectually or ethically.

Because public archaeology exists to challenge economic neoliberalism

Recent Western politics have emphasized the ideal of independent, unregulated individuals and corporations rather than on reinforcing the social bonds within society. Although there are myriad ways in which heritage, archaeology, and history are used in politics, the arguments they are a part of usually are meant to draw people together. De-emphasizing what is good for the collective is almost certainly, in my opinion, not good for archaeological preservation. That message trickles down. For example, collectors often articulate the market value of an artifact or its sentimental value to them as equally as important as preserving material culture in museums or following indigenous heritage management practices.

Our being visible, accessible messengers for archaeological ethics has been noticed. That action is a success. All those questions public figures in our field get in their emails are a sign we are on peoples’ radar. Our excavations making headlines and our colleagues winning grants for their compelling public outreach are signs we are already moving in the right direction. Do we really need to pick apart why or how to move forward?

Because we cannot expect to control how the public uses archaeological knowledge

We don’t get to decide what the people who attend our educational programs do with what we teach them. They can choose how, when, and if they will reconsider their way of interacting with archaeological materials and contemporary people who care about them. (Just look at the science news cycle if you need an example.)

The public does not get to decide what we tell them about archaeological work. We can choose how, when, and if we will listen to their points of view on archaeological materials, including why they care about them. We still hold the power to control a lot of information we obviously think is important, and a lot of non-archaeologists think is important, too. The public knows this and is often very skeptical of our motives.

If we did get more information on public perceptions of our work, it would not necessarily make us more successful in creating more citizen advocates for preservation, because ultimately we cannot control peoples’ use of archaeological knowledge. Is it worth the potential risks if we reconsider the traditional role of scientists as gatekeepers to knowledge? Would we be willing to share any more than we already do if we found out we were not, for example, influencing more people to vote for preservation legislation?

 

Because evaluation would silence voices from underrepresented groups

The standards by which we judge success are built within a Western elite discipline that has deep historical connections to colonialism. If we pursue a form of evaluation, the very form of that assessment will be shaped by Western epistemology.

Indigenous archaeologists have crystallized the notion that there are many theories, methods, fieldwork practices, pedagogical strategies, and interpretive models that are valuable to our understanding of the past but which do not conform to the norms of mainstream archaeological practice (Atalay 2008). Sonya Atalay has emphasized the importance of not only having Native people at the metaphorical interpretive table, but also teaching a tolerance for ambiguity and multivocality in archaeological practice going forward (Atalay 2008:37-38). One way of interpreting this argument is that a multiplicitous landscape of public archaeology assessment is as valid as a singular method.

It is not clear that any questions, methods of asking those questions, or indirect ways of assessing peoples’ thinking about archaeology could be universally applicable, so it may not be appropriate to attempt a unified discussion about assessment. Indeed the very idea of universal applicability is uncomfortably Processual whereas the pursuit of collaborative, indigenous, and even some unidirectional public archaeology communication is rather more Post-processual. But some form of unified, multivocal, debate about public archaeology assessment practices could be a middle ground.

 

I strongly believe we must strive for more extensive self-assessment and peer assessment in public archaeology, but that does not mean there are not legitimate arguments against it. It is exactly this kind of critical self-awareness that lead us toward engaging the public as a discipline, in many different ways, with a huge variety of results. I am looking forward to discussing these issues in my session at the SAA conference next week. I’ll leave you with this:

Public archaeology is not a product.

Whether we are presenting to the public or working with the public, the purpose is not to make that presentation (or whatever it is.) We all know this. But we should explore more about public archaeology as a process, a process in which we are powerful and affecting agents.

5 Comments

  • Taryn Kilbert says:

    Public Archaeology is a wonderful chanxe to educate, with the outcome of public buy-in. Being a’part of’ archaeology on a community by community basis spark more intrinsic motivation to become more embedded in the detective side of archaeology. All good outcomes in my humble opinion.

  • Kellam says:

    I’m pondering your final point (“public archaeology is not a product”) as well as your third one (“public archaeology is intended as a challenge to neoliberalism”). Although we are uncomfortable with the notion, I think public archaeology often IS a product. In CRM we produce “deliverables” (usually a report and a GIS database or shapefiles) to present to clients. These mark an important stage in the contracting process. And much of our educational material ultimately makes its way into a market for “consumption” by a particular kind of public, like other kinds of products. So I think there is an unfortunate conundrum that arises as we seek new venues for public archaeology–those venues are, by and large, embedded in a system that encourages or requires some kind of evaluation of a product.

    But in this post (and your others) you hit on an important shift in emphasis for evaluation from product to process. In what ways do you think the methods and goals of evaluation change when we consider not the outcomes but the practice?

    • Michele says:

      So, I do not intend to speak for Kate, but I thought I would offer my thoughts to your comment and question. I think the final message and Kate’s post of emphasizing process rather than product is because of how public archaeology works. Public archaeology is (or should be?) a lengthy process that requires building a repertoire with the public that does not necessarily accumulate into a “final” presentation or product before moving on to the next project (as is often the case and practice in CRM). Evaluating public archaeology is difficult unless you are able to situate each project in its own particular context (as Kate has already mentioned above, “success is subjective”).

      However, I think the issue with evaluations of products is because often in CRM we are focused on ticking off boxes on our to-do list and evaluations are then focused on how many of those “boxes” were completed (usually by examining the final project, as you mentioned). An example is that Section 106 is required in CRM projects to ensure that engaging the public happens at “some” level, although contextualized to each project. However, there are some problems with only focusing on those guidelines as “endpoints” to check off a list instead of seeing how those can be integrated as “methods”, as part of a process that needs to be present in the project at the beginning. In a sense, evaluation should not focus on whether you have produced something that shows you did do something for Section 106 (as required) but that it focuses on how you carried it out? And the issue with evaluation is that the quality of “how well” a public archaeology project is carried out is often done from our (or our peers’) perspectives (I’m not sure if my comment makes sense; correct me if I’m wrong).

    • Kate Ellenberger says:

      I intentionally framed this as a provocation to get people thinking about potential ways people feel satisfied with the current level of engagement, so I am glad you were inspired to respond!

      Indeed, we do often produce deliverables, but I would suggest that we are responsible to understand how that lecture, article, or event has impacted the public’s perception of issues we consider incredibly important. So while we may produce products, we cannot just check the box of “did the lecture” and expect we have done our part to change archaeological practice. We still have very real economic and sociopolitical challenges ahead of us, convincing people that our field is of value.

      As you said, in these posts I have conceptualized public archaeology as a process. I think the easiest example of why this is useful is teaching; assessment is not only important for improving oneself as a teacher, but is the key mechanism for adapting activities to student needs. Typical assessment tools include pre-lesson activities focused on first impressions, targeted inquiry, and end-of-lesson summary activities, to name a few. The expectation is that consistent and deliberate assessment by teachers results in better learning outcomes for students, and better teachers. Of course, I am talking here about top-down approaches to public archaeology for the most part, but in my experience, archaeologists develop more skills in communication and adaptability when they collaborate with stakeholders.

      For things like community-based archaeology, when I have seen a form of assessment published, it tends to be more personal. I can only think of situations where interviews were done, off the top of my head. There is really so little published and readily available on the subject that I’m not sure what ‘most’ people would find appropriate in these situations. (Surely there are many projects for which these kinds of questions cannot be asked, at least not by anyone inside the collaborative partnership, without compromising the work.) I have observed that these projects usually last only a few years, perhaps because relationships between collaborators is close, intense, and requires a lot of maintenance. I can think of many projects where I have heard archaeologists’ stories about what made them decide to move on, and their perception of what went on was adequate evidence to them of how to conduct future research.

  • Carol McDavid says:

    Kate, I looked at these today (a year late!) and I think you made some terrific insights. Thank you.

Leave a Reply

Your email address will not be published. Required fields are marked *