Sunday, July 6, 2014

Kellert, 2008, Borrowed Knowledge (epistemology)

Prompted by discussions with my son Peter about the limits of interdisciplinary knowledge, (he has just finished a university program in knowledge integration) I wanted to know more about the problems arising from using metaphors outside their appropriate context. I found Kellert a helpful illustration of issues we were discussing.  See also 2013-07-16, notes on Kellert, Harry Collins and the SEE project, the periodic table of expertises, and the expertise space domain. 

The discussion with my son Peter, who is writing about the need for full disclosure of non-epistemic values in research, began with his diatribe against generalist authors like Azar Gat[1] (who as a military historian has no business summarizing arguments about evolutionary propensities for violence) and Steven Pinker (who Peter believes makes broad claims for evolutionary psychology as a cause for the decline of violence, although I read him as emphasizing institutional evolution, institutions in the broadest sense as means of organizing social behaviour). 

What follows is really thinking through some of the issues involved in preparing professionals without a coherent disciplinary foundation (e.g. military and police officers) for real-world problem-solving and practical study while drawing on partial knowledge of a wide range of potentially useful disciplines

Criticism of the Big Picture Generalists
Multi-, Inter-, and Trans-disciplinary work
Dimensions of Expertise

Peter is right that Pinker has provoked criticism. See, for example, John Gray, “Delusions of Peace” ( ) [2] and the authors reviewed by Kitwood (2012) below:

·      KITWOOD, N. "Under the long shadow of Rousseau and Hobbes, scientists debate whether civilization spurred or inhibited warfare—and whether we have the data to know." (2012)
·      Robbins Schug, Gwen, et al. "A peaceful realm? Trauma and social differentiation at Harappa." International Journal of Paleopathology 2.2 (2012): 136-147.
·      Kuckelman, Kristin A., Ricky R. Lightfoot, and Debra L. Martin. "Changing patterns of violence in the northern San Juan region." The Kiva (2000): 147-165.
·      Milner, George R., Eve Anderson, and Virginia G. Smith. "Warfare in late prehistoric west-central Illinois." American Antiquity (1991): 581-603

We don’t have the data to know for sure how small groups interacted – sometimes they were peaceful and sometimes they were violent, so you can’t aggregate numbers and generalize as Pinker does.  

But the more general point concerns claims to knowledge, and Peter’s rejection of multi-disciplinary synthesis (like Pinker). I have deliberately sought out authors like Pinker and Gat, who popularize broad interdisciplinary studies.  From his knowledge-integration background, he believes that they cannot usefully integrate the knowledge as individuals, and only coherent team studies are likely to produce useful results.  If this is true, then it may have implications for how I put the pieces together for the NSP book – less time spent solo and more time spent on network building and checking.  It may inevitably drive me towards edited collections for detailed knowledge (already part of the plan in the context of series…)

See jnl 2013-07-16 claims to knowledge.pptx

Figure 1 interdisciplinarity at various levels of generalization (see below on "esotericity" - this is inverse)

In building networks, you need to seek out people with contributory and interactional expertise.  People like THD have interactional expertise on environmental science and contributory expertise on policy and politics. What are my areas of contributory and interactional expertise? Probably narrower than I like to think…

Part of the problem is that specialists may actively reject the paradigms and evidence, or simply passively ignore or dismiss it. As Bruce Berman said of the MCRI project, there were very few of the academics involved who would actually consent to “play with others”.

Figure 2 specialization (or esotericity) and capacity to assess claims to knowledge

Peter suggested a useful relationship between degree of specialization and capacity to assess claims to knowledge, illustrated in Figure 2 specialization (or esotericity) and capacity to assess claims to knowledge.  The “Einstein problem” is that few people understand the knowledge adequately.  The “Francis Bacon” problem is that few people know enough to integrate effectively.  These two problems go along with Figure 3 – the tiny audience is the Einstein problem, while the wide popular audience is the Francis Bacon problem (or the Stephen Pinker problem) – lots of people read it and believe it to be reasonable, but a much smaller group is in a position to correctly assess it.

The concept of “correct” assessment is a problem.  In the social sciences, much of what passes for knowledge is contested.  Definitions of violence, equality, justice, democracy, war, failed states and all the measures that accompany these are disputed by camps of specialists who are divided by value more than epistemology.

Consider the Annales school of French social and economic historians (e.g. Braudel,[3] Ladurie,[4] Bloch,[5]) in comparison to the World System Perspective which evolved over the same period and a bit later in the English speaking world (Wallerstein,[6] Bergeson,[7] Rubinstein, etc).  Both stressed the interrogation of “evidence”, or the secular cycles[8] described by Peter Turchin’s “cliodynamics,” which looks like a recent application of the “cliometrics” (data-mining applied to economic history) that netted Fogel and North a Nobel prize in economics in 1993.[9]  Each of these broad fields of knowledge consists of hundreds of scholars, thousands of articles and chapters, and scores of journals. Within this broad base defined by approach, I think there is some cross-fertilization based on subject, but there is also a constraint of efficiency: since they can’t read everything, scholars concentrate on the literature within their field, and only cross fields when it offers an advantage to understanding their particular problem.[10] Thus, although all of the authors listed above (Braudel, Ladurie, Wallerstein, Turchin, etc, and the much longer list of lesser knowns) are in the general field of political and economic history, they haven’t necessarily engaged each other or learned from each other – life is too short.

Figure 3 Audience size and specialization

Now if we move from the triangle labeled “political-economic-historical studies” to the field of multiple triangles each representing a discipline, permeability is even less. This raises the problem of disciplines – how are these pyramids of knowledge constructed, and how do you integrate them to solve real-world problems, as Kersbergen et al (2004) amongst many others suggest is necessary? 

Where did the disciplinary silos, and their sub-fields come from in the first place?  In The Chaos of Disciplines, Abbott (2001) describes the fractal nature of social and cultural groups, of which academic disciplines are an example, and posits that this natural fractioning explains how knowledge advances (or changes over time – presumably advancing if we get it right):[11]

“If we take any group of sociologists and lock them in a room, they will argue and at once differentiate themselves into positivists and interpretivists. But if we separate those two groups and lock them in separate rooms, those two groups will each in turn divide over exactly the same issue…Thus cultural structures too may have the characteristic of self-similarity…self-similarity provides a general account of how knowledge actually changes in social science…” (ix-x)

So if Abbot is right, the infinite fracturing of disciplinary silos is a continuous and inevitable process, but the principle of self-similarity suggests that any particular category of scholar in one silo (narrativist, interpretivist, positivist, Marxist, etc) will be able to find fellow-travellers in many other silos, based on the way they think if not on the subject of their study. Humans being what they are, some reach out and some don’t; sometimes it’s planned and sometimes it’s serendipitous.  The various labels surrounding multi-discipline studies are efforts to describe a continuum of the extent to which scholarship is isolated or interconnected across disciplines.
Disciplines “can be identified…by their objects of study (domains, phenomena), by their cognitive tools (theories, techniques) or by their social structure (turf, journals) (Kellert, 2009, 2:7/49, citing Bechtel 1987, 297 and Hayles 1990, 191).  Depending on the elements you use in a definition, “discipline” can correspond more or less to the domains of traditional university departments: math, chemistry, physics, sociology, psychology, history, etc.  But if you focus just on the objects of study, or the social structure, then area studies, legal studies, environmental studies, women’s studies, war studies, etc. can also be defined as disciplines, and indeed have their own journals and sometimes departments.[12]  This brings us back to Abbott (2001) and The Chaos of Disciplines, fracturing and reforming.

We had a discussion multidisciplinarity in the SSHRC-sponsored Major Collaborative Research Project, and I think the outcome was typical amongst traditional disciplines.  It wasn’t a very diverse group – almost all political scientists and historians of one stripe or another, with a heavy dose of normative philosophers.  A simple view is that a multidisciplinary study involves several disciplinary specialists each looking at a problem separately (more or less what we did). An interdisciplinary study involves mutual learning across the disciplines, so (for example) a sociological understanding informs collection of economic data, for analysis within a political or legal framework. Transdisciplinarity is an effort to achieve holistic understanding of reality beyond the boundaries of disciplines. There’s a good Wikipedia entry on transdisciplinarity with reference to Piaget (1970) and the International Center for Transdisciplinary Research (CIRET) established in 1987, which still seems to be in business “The aim of our organization is to develop research in a new scientific and cultural approach - the transdisciplinarity - whose aim is to lay bare the nature and characteristics of the flow of information circulating between the various branches of knowledge.” ( 

But genuinely transdisciplinary work seems to be both rare and problematical. It is rare because both motivation and capability are necessary. Motivation is limited because of incentives to be expert in a disciplinary field in order to have a university job.  Capability is rare because it requires levels of expertise across disciplines that can take decades to acquire, or may be impossible beyond two or three domains. In the absence of that sort of expertise, transdisciplinarity is problematical because it can lead to bad science and conclusions disputed by experts (and Peter’s critique of the generalists). I think one reason that this happens is that transdisciplinarity requires compromises and homogenization of approach.  “Disciplinary pluralism” may be a reasonable alternative.

Kellert (2009)[13] illustrates the problems and value of borrowing from the natural sciences to think about social sciences and humanities by using chaos theory.  Social sciences borrow concepts and metaphors from natural sciences, sometimes applying them appropriately, and sometimes not.  Kellert is a professor of philosophy who has written mainly about chaos and its applications in science. Kellert’s Chapter 2 in Borrowed Knowledge, “disciplinary pluralism,” provides a good explanation of levels of analysis related to the sort of questions asked:  level 0 is the object or phenomena under study; at level 1 are the disciplines that investigate these objects of study; at level 2 are methodology and epistemology – questions about the disciplines in level 1 – how are we to know what is really going on? Are the representations accurate or heuristic fictions? At level 3 we find what Kellert calls questions of meta-methodology and meta-philosophy – e.g. how do we resolve conflicts of modeling, data collection or method? He goes on to justify scientific pluralism for level 1 and 2 questions: “The scientific pluralist stance defends the peaceful coexistence of incompatible theories and approaches within a discipline. Similarly, disciplinary pluralism invites the use of techniques from multiple disciplines to understand the subject matter…” (Kellert, 2009, 2:6/49).

This brings us to the kinds of expertise represented in different fields at different levels of specialization.  Looking for information on claims to expert knowledge, I found a project originating early in the 2000s, Studies in Expertise and Experience (SEE).

The purpose of the project is policy-oriented, as much aimed at limiting technical-elite decision-making (in order to privilege democratic engagement) as improving the integration of expertise. Collins and Evans (2002) seem to be the originators:

“Science studies has shown us why science and technology cannot always solve technical problems in the public domain. In particular, the speed of political decision-making is faster than the speed of scientific consensus formation. A predominant motif over recent years has been the need to extend the domain of technical decision-making beyond the technically qualified élite, so as to enhance political legitimacy. We argue, however, that the `Problem of Legitimacy' has been replaced by the `Problem of Extension' - that is, by a tendency to dissolve the boundary between experts and the public so that there are no longer any grounds for limiting the indefinite extension of technical decision-making rights. We argue that a Third Wave of Science Studies - Studies of Expertise and Experience (SEE) - is needed to solve the Problem of Extension. SEE will include a normative theory of expertise, and will disentangle expertise from political rights in technical decision-making. The theory builds categories of expertise, starting with the key distinction between interactive expertise and contributory expertise. A new categorization of types of science is also needed. ….” [14]

In what appears to be an unpublished paper from about 2011, Collins extends the analysis of different types of expertise in three dimensions.  First, he explores the dimension of specialization.  Figure 4 “The Periodic Table of Expertises” describes a series of dichotomies and continuums that are related mainly to expertise and specialization of knowledge.

Figure 4 From Harry Collins and the SEE project

“The underlying idea of the Periodic Table is that the acquisition of nearly every expertise, if not all of them, depends on the acquisition of the tacit knowledge pertaining to the expert domain in question.  Tacit knowledge can be acquired only by immersion in the society of those who already possess it.  Therefore, the process of moving to the right hand end of the Specialist Expertises line depends on becoming socially embedded in the appropriate groups of experts so that one can acquire `specialist tacit knowledge’ (as indicated in the grey heading).  The process is social though the outcome is real – an ability to do and understand things that one could not do and understand before.

“The two right hand categories of the Specialist Expertise line indicate that there are two kinds of socialisation that can lead to two kinds of specialist expertise.  The rightmost category – contributory expertise – is what is normally thought of as an expertise and it is the practical expertise that enables one to contribute to a domain of practice.  To acquire contributory expertise one must work within the expert domain.  Interactional expertise, on the other hand, can be acquired by deep immersion in the linguistic discourse of the domain alone.  At first thought of as a kind of subsidiary expertise, interactional expertise is now seen, at least by some such as the author, as more and more the essence of human collective practices and social life in general.  For example, it has been argued that without interactional expertise we would all live isolated lives, our understanding bounded by just those things we had practised ourselves…” (Collins, Three Dimensions of Expertise, nd, circa 2011)

Figure 5 from Harry Collins again...

He then goes on to explain how expertise can be described in three dimensions, based on specialization, exposure to tacit knowledge, and accomplishment based on the expertise.

“In [Figure 5 Expertise-Space Diagram], the depth dimension is what used to be the (mostly) single-dimensional model of expertise but now refers to groups and domains as well as individuals.  The horizontal dimension is the extent of exposure to tacit knowledge, once more, referring to either groups or individuals, depending on how the diagram is used.  The vertical dimension is the extent to which the domain is esoteric with ubiquitous domains, such as language-speaking or literacy at the bottom and things like gravitational wave physics at the top.” (Collins, p. 7, ibid)  Collins goes on to explain how the expertise-space diagram can be used. 

What can we say about security expertise in the ESD?  Military and police probably have comparable trajectories, and degrees of esotericity (why not “specialization”?) peak at the tactical/operational level, e.g. gunners, communicators, crime-scene investigators, etc. This probably happens fairly early in a career – Sergeant/Warrant or Captain/Major.  As officers/leaders move up to mid-career, they have to generalize in order to integrate the specialized knowledge of subordinates, and their general expertise is more readily transferable to other domains, hence the normal compatibility of police, military, (and maybe business and government) in management, leadership, planning and execution of operations in the broadest sense.

This seems very familiar, perhaps from a diagram in one of Al Okros’s presentations, or a Training Development Officer (TDO) scheme that illustrates different levels and forms of expertise at different stages in a career.  My conclusion is that synthesizing fields of knowledge is absolutely necessary and occurs naturally in pursuit of applied knowledge, as in the security field, but that Kellert is right - it's often done badly. Collins and the SEE project distinctions probably help us to integrate the knowledge forms that Kellert suggests can trip up applied knowledge. Of course, this discussion is probably too esoteric for mid-career professionals looking for practical tools, although it might be reduced to a conceptual snippet with real-world examples that might serve as a warning.

David Last, 16 July 2013

[1] Gat, Azar. War in human civilization. Oxford University Press, 2008.
[2] Gray, John. "Delusions of peace." Prospect21 (2011).  [John Gray is another historian.]
[3] Braudel, Fernand. La dynamique du capitalisme. Vol. 19. Paris: Arthaud, 1985.
[4] Ladurie, Emmanuel Le Roy. Le territoire de l'historien. Vol. 2. Gallimard, 1973; Ladurie, Emmanuel LeRoy. "Motionless history." Social Science History 1.2 (1977): 115-136
[5]Bloch, Marc Léopold Benjamin. French rural history: an essay on its basic characteristics. Vol. 28. University of California Pr, 1966.
[6] Wallerstein, Immanuel, ed. The capitalist world-economy. Vol. 2. Cambridge University Press, 1979.
[7] Bergeson, Albert. "Modeling Long Waves of Crisis in the World-System." CRISES IN THE WORLD-SYSTEM. London: Sage Publications (1983).
[8] Turchin, Peter, and Sergey A. Nefedov. Secular cycles. Princeton University Press, 2009.
[9] "for having renewed research in economic history by applying economic theory and quantiative methods in order to explain economic and institutional change."
[10] This is my cursory understanding from: Kersbergen, Kees van, and Frans van Waarden. "‘Governance’as a bridge between disciplines: Crossdisciplinary inspiration regarding shifts in governance and problems of governability, accountability and legitimacy." European journal of political research 43.2 (2004): 143-171, and Moed, Henk, Wolfgang Glänzel, and Ulrich Schmoch. "Editors’ Introduction." Handbook of Quantitative Science and Technology Research (2005): 1-15.
[11] Abbott, Andrew. Chaos of disciplines. University of Chicago Press, 2001.
[12] This is suggested by references in Kellert, in these “studies” fields, multidisciplinarity is essential, and interdisciplinary work “has almost become de rigueur…” (Moran, 1997, 155) Moran, Jay P. "Postmodernism's Misguided Place in Legal Scholarship: Chaos Theory, Deconstruction, and Some Insights from Thomas Pynchon's Fiction." S. Cal. Interdisc. LJ 6 (1997): 155.
[13] Kellert, Stephen H. Borrowed knowledge: Chaos theory and the challenge of learning across disciplines. University of Chicago Press, 2009.
[14] Collins, Harry M., and Robert Evans. "The Third Wave of Science Studies: Studies of Expertise and Experience." Social studies of science 32.2 (2002): 235-296.

No comments:

Post a Comment