How Research Methods Textbooks Fail Final Year Project Students

The time is about to come when all those fresh-faced final year empirical project students will be filing through our office doors looking for the study that’s going to give them the first class degree they are craving for.

Unfortunately, as a supervisor you’ll find that their mind isn’t focused on doing scientific research – it’s focused on getting a good mark for their project. This means that most of your time as a supervisor will be spent not on training your undergraduate supervisees to do research (as it should be), but on (1) telling them what they have to do to write up a good project, and (2) reassuring them that they’ve understood what you said is required for writing up a good project.

As an empirical scientist you might believe that the most important part of the training for your undergraduate project students is learning about experimental design and about statistical analysis. Wrong. Absolutely no over-arching information about experimental design will be absorbed by the student – only that they lie awake at night needing to know how many participants they will need to test and – more importantly – how will they get those participants?

Most project students have a small notebook they’ve bought from W H Smiths and in which they write down the pressing questions they need to ask their supervisor at the next supervision session (just in case they may forget). Questions like “Can I do this experiment in my bathroom in my student flat?”, “Can I test my mother’s budgerigar if I’m short of participants?”, “Will it matter if my breath smells of cider when I’m coding my data?”, “Do I need to worry about where I put the decimal point?”, “Will it affect my participants’ behaviour if I dye my hair day-glow orange in the middle of the study?”… and so on.

I believe that project students ask these kinds of questions because none of these questions are properly addressed or answered in standard Research Methods textbooks – an enormous oversight! Research Methods textbooks mince around talking about balanced designs, counterbalancing, control groups, demand effects, and so on. But what about the real practical issues facing a final year empirical project student? “How will I complete my experiment if I split up with my boyfriend and can’t use his extended local family as participants?”, “Where can I find those jumbo paper clips that I need to keep all the response sheets together?”, “Why do I need to run a control condition when I could be skiing in Austria?”

Perhaps we need some new, young, motivated research methods authors to provide us with the textbooks that will answer the full range of questions asked by undergraduate empirical project students. Sadly, at present, these textbooks answer the questions that students aren’t interested in asking – let’s get real with undergraduate research training!

Mental health research: Are you contributing to paradigm stagnation or paradigm shift?

“Normal science does not aim at novelty but at clearing up the status quo. It discovers what it expects to discover.” – Thomas Kuhn.

I was struck by this quote from Thomas Kuhn last week when reading a Guardian blog about the influential philosopher of science. It’s a simple statement suggesting that so-called ‘normal science’ isn’t going to break any new ground, isn’t going to change the way we think about something, but probably will reinforce established ideas, and – perhaps even more importantly – will entrench what scientists think are the important questions that need answering. Filling in the gaps to clear up the status quo is probably a job that 95% of scientists are happy to do. It grows the CV, satisfies your Dean of School, gets you tenure and pays the mortgage.

But when I first read that quote, I actually misread it. I thought it said “Normal science does not aim at novelty but aims to maintain the status quo”! I suspect that when it boils down to it, there is not much difference between my misreading of the quote and what Kuhn had actually meant. Once scientists establish a paradigm in a particular area this has the effect of  (1) framing the questions to be asked, (2) defining the procedures to answer them, and (3) mainstreams the models, theories and constructs within which new facts should be assimilated. I suspect that once a paradigm is established, even those agencies and instruments that provide the infrastructure for research contribute to entrenching the status quo. Funding bodies and journals are good examples. Both tend to map on to very clearly defined areas of research, and at times when more papers are being submitted to scientific journals than ever before, demand management tends to lead to journal scope shrinkage in such a way that traditional research topics are highlighted more and more, and new knowledge from other disciplinary approaches is less likely to fertilize research in a particular area.

This led me to thinking about my own research area, which is clinical psychology and psychopathology. Can we clinical psychology researchers convince ourselves that we are doing anything other than trying to clear up the status quo in a paradigmatic approach that hasn’t been seriously questioned for over half a century – and in which we might want to question it’s genuine achievements? Let’s just take a quick look at some relevant points:

1.         DSM still rules the way that much clinical psychology research is conducted. The launch of DSM-5 in 2013 will merely re-establish the dominance of diagnostic categories within clinical psychology research. There are some who struggle to champion transdiagnostic approaches, but they are doing this against a trend in which clinical psychology and psychiatry journals are becoming more and more reliant on diagnostic criteria for inclusion of papers. Journal of Anxiety Disorders is just one example of a journal whose scope has recently shrunk from publishing papers on anxiety to publishing papers on anxiety only in diagnosed populations. DSM-I was published in 1952 – sixty years on it has become even more entrenched as a basis for doing clinical psychology research. No paradigm shift there then!

This doesn’t represent a conspiracy between DSM and journals to consolidate DSM as the basis for clinical psychology research – it merely reflects the fact that scientific journals follow established trends rather than create new spaces within which new concatenations of knowledge can emerge. Journals will by nature be a significant conservative element in the progress of science.

2.         There is a growing isolation in much of clinical psychology research – driven in part by the shrinking scope of clinical research journals and the adherence of many of them to DSM criteria for publication. This fosters a growing isolation from core psychological knowledge, and because of this, clinical psychology research runs the risk of re-inventing the wheel – and probably re-inventing it badly. Some years ago I expressed my doubts about the value of many clinical constructs that had become the focus of research across a range of mental health problems (Davey,2003). Many of these constructs have been developed from clinical experience and relate to individual disorders or even individual symptoms, but I’m convinced that a majority of them simply fudge a range of different psychological processes, most of which have already been researched in the core psychological literature. I'm an experimental psychologist by training who just happens to have become interested in clinical psychology research, so I was lucky enough to be able to bring some rather different approaches to this research than those who were born and brought up in the clinical psychology way of doing things. What must not happen is for clinical psychology research to become even more insular and even more entrenched in reinventing even more wheels - or the wheels on the bus really will just keep going round and round and round!

3.         OK I'm going to be deliberately provocative here – clinical neuroscience and imaging technology costs a lot of money - so its role needs to be enshrined and ring-fenced in the fabric of psychological knowledge endeavor, doesn’t it? Does it? If that’s the case – then we’re in for a long period of paradigm stagnation. Imaging technology is the Mars Rover of cognitive science while the rest of us are using telescopes - or that's the way it seems. There are some clinical funding bodies I simply wouldn't apply to for experimental psychopathology research – ‘cos if it ain’t imaging it ain't gonna get funded - yet where does the contribution of imaging lay in the bigger knowledge picture within clinical psychology? There may well be a well thought out view somewhere out there that has placed the theoretical relevance of imaging into the fabric of clinical psychology knowledge (advice welcome on this)! There is often a view taken that whatever imaging studies throw up must be taken into account by studies undertaken at other levels of explanation - but that is an argument that is not just true of imaging, it's true of any objective and robust scientific methodology.

Certainly - identifying brain locations and networks for clinical phenomena may not be the way to go - there is growing support for psychological constructionist views of emotion for instance, suggesting that emotions do not have either a signature brain location or a dedicated neural signature at all (e.g. Lindquist,Wager, Kober, Bliss-Moreau & Barrett, 2012). There are some very good reviews of the role of brain functions in psychological disorders -but I'm not sure what they tell us other than the fact that brain function underlies psychological disorders – as it does everything! For me, more understanding of psychological disorders can be gleaned from studying individual experience, developmental and cognitive processes, and social and cultural processes than basic brain function. Brain images are a bit like the snapshot of the family on the beach - The photo doesn't tell you very much about how the family got there or how they chose the beach or how they're going to get home.

But the point I’m trying to make is that if certain ways of doing research require significant financial investment over long periods of time (like imaging technology), then this too will contribute to paradigm stagnation.

4.         When tails begin to wag dogs you know that as a researcher you have begun to lose control over what research you can do and how you might be allowed to do it. Many researchers are aware that to get funding for their research – however ‘blue skies’ it might be – we now have to provide an applied impact story. How will our research have an impact on society? Within clinical psychology research this always seems to have been a reality. Much of clinical psychology research is driven by the need to develop interventions and to help vulnerable people in distress – which is a laudable pursuit. But does this represent the best way to do science? There is a real problem when it comes to fudging understanding and practice. There appears to be a diminishing distinction in clinical psychology between practice journals and psychopathology journals, which is odd because helping people and understanding their problems are quite different things – certainly from a scientific endeavour point of view. Inventing an intervention out of theoretical thin air and then giving it the facade of scientific integrity by testing to see if it is effective in a controlled empirical trial is not good science – but I could name what I think are quite a few popular interventions that have evolved this way – EMDR and mindfulness are just two of them (I expect there will be others who will argue that these interventions didn't come out of a theoretical void, but we still don't really know how they work when they do work). At the end of the day, to put the research focus on ‘what works in practice’ takes the emphasis away from understanding what it is that needs to be changed, and in clinical psychology it almost certainly sets research priorities within establishment views of mental health.

5.         My final point is a rather general one about achievement in clinical psychology research. We would like to believe that the last 40 years has seen significant advances in our development of interventions for mental health problems. To be sure, we’ve seen the establishment of CBT as the psychological intervention of choice for a whole range of mental health problems, and we are now experiencing the fourth wave of these therapies. This has been followed up with the IAPT initiative, in which psychological therapies are being made more accessible to individuals with common mental health problems.  The past 40 years has also seen the development and introduction of second-generation antidepressants such as SSRIs. Both CBT and SSRIs are usually highlighted as state-of-the-art interventions in clinical psychology textbooks, and are hailed by clinical psychology and psychiatry respectively as significant advances in mental health science. But are they? RCTs and meta-analyses regularly show that CBT and SSRIs are superior to treatment as usual, wait-list controls, or placebos – but when you look at recovery rates, their impact is still far from stunning. I am aware that this last point is not one that I can claim reflects a genuinely balanced evidential view, but a meta-analysis we have just completed of cognitive therapy for generalized anxiety disorder (GAD) suggests that recovery rates are around 57% at follow-up. Which means that 43% of those in cognitive therapy interventions for GAD do not reach basic recovery levels at the end of the treatment programme. Reviews of IAPT programmes for depression suggest no real advantage for IAPT interventions based on quality of life and functioning measures (McPherson,Evans & Richardson, 2009). In a review article by Craske, Liao, Brown & Vervliet (2012) that is about to be published in Journal of Experimental Psychopathology, they note that even exposure therapy for anxiety disorders achieves clinically significant improvement in only 51% of patients at follow-up. I found it difficult to find studies that provided either recovery rates or measures of clinically significant improvement for SSRIs, but Arroll et al (2005) report that only 56-60% of patients in primary care responded well to SSRIs compared to 42-47% for placebos.

I may be over-cynical, but it seems that the best that our state-of-the-art clinical psychology and psychopharmacological research has been able to achieve is a recovery rate of around 50-60% for common mental health problems - compared with placebo and spontaneous remission rates of between 30-45%. Intervention journals are full of research papers describing new ‘tweaks’ to these ways of helping people with mental health problems, but are tweaks within the existing paradigms ever going to be significant? Is it time for a paradigm shift in the way we research mental health?

Follow me on Twitter at:

http://twitter.com/GrahamCLDavey

Why don’t neuroscientists want to be read (except by neuroscientists)?

Neuroskeptic asked why Social Scientists don’t want to be read. Fair do’s – it was a pretty flabby abstract that could have been written by a UK science journalist who was trained in Eng Lit and media studies and hadn’t seen science since they were asked to practice some Piagetian conservation tasks in kindergarten (perhaps that might be more common than you think!).

After quoting the abstract Neuroskeptic exclaimed: “Phew. Now I think it's fair to say that this is a typical example of what might be called the "social sciences style" of writing.” Perhaps the issue here is how we differentiate between a badly written abstract and an inaccessible abstract. Psychology is a broad discipline. It claims to span the “softer” areas of the social sciences, through social and developmental cognition to experimental psychology, and then on to neuroscience and the various fields of biology and genetics.

But what about neuroscience abstracts? I’m about to begin a revision of my Psychopathology textbook and had made a decision to include more clinical neuroscience (in an accessible way, of course). But do neuroscientists write their abstracts in a way that gets across even the most basic contribution to knowledge of their studies? Only to some privileged audience.

I had a look at the most recent issue of the Journal of Neuroscience, and chose an abstract at random. Here it is:

ΔFosB, a Fosb gene product, is induced in nucleus accumbens (NAc) and caudate–putamen (CPu) by repeated exposure to drugs of abuse such as cocaine. This induction contributes to aberrant patterns of gene expression and behavioral abnormalities seen with repeated drug exposure. Here, we assessed whether a remote history of cocaine exposure in rats might alter inducibility of the Fosb gene elicited by subsequent drug exposure. We show that prior chronic cocaine administration, followed by extended withdrawal, increases inducibility of Fosb in NAc, as evidenced by greater acute induction of ΔFosB mRNA and faster accumulation of ΔFosB protein after repeated cocaine reexposure. No such primed Fosb induction was observed in CPu; in fact, subsequent acute induction of ΔFosB mRNA was suppressed in CPu. These abnormal patterns of Fosb expression are associated with chromatin modifications at the Fosb gene promoter. Prior chronic cocaine administration induces a long-lasting increase in RNA polymerase II (Pol II) binding at the Fosb promoter in NAc only, suggesting that Pol II “stalling” primes Fosb for induction in this region upon reexposure to cocaine. A cocaine challenge then triggers the release of Pol II from the gene promoter, allowing for more rapid Fosb transcription. A cocaine challenge also decreases repressive histone modifications at the Fosb promoter in NAc, but increases such repressive marks and decreases activating marks in CPu. These results provide new insight into the chromatin dynamics at the Fosb promoter and reveal a novel mechanism for primed Fosb induction in NAc upon reexposure to cocaine.

Luckily, Neuroskeptic was able to translate and rewrite the Social Science abstract, but I have no idea what this study might have made to the understanding of cocaine addiction in particular and substance abuse in general.

A study like this may quite reasonably have been written in the jargonized style understood by all those familiar with neuroscience as it is written in the Journal of Neuroscience – but it is not accessible beyond that audience. The same might be true of the Social Science abstract quoted by Neuroskeptic – it could be argued that much of the language was couched in social science jargon with terms like “thematic analysis” and “hegemonic masculinities” to name but two. I would have liked to have written an unjargonized summary of the neuroscience abstract I’ve quoted (just like Neuroskeptic did with the Social Science abstract). Neuroskeptic was able to do that but with the neuroscience abstract I’m completely stumped!

If there’s a moral here, it’s maybe about neuroscientists belittling social science because of its lack of preciseness, but conversely is neuroscience perpetuating its scientific elitism through the impenetrability of its jargon. How on earth am I able to decide if a neuroscience abstract is a bad abstract (and don’t say go and learn some neuroscience, because if you’re a neuroscientist I might say go and learn some social psychology)?!


Follow me on Twitter at: http://twitter.com/GrahamCLDavey

Extinctions Restaurant Menu

This is a purely hypothetical menu - of course!
But inspired by being served a "Rare Bison Steak" in Marcel's Restaurant in Washington DC some years ago!

The Perfect University

Why doesn’t every student in Higher Education get awarded a first class degree? Is it because they’re not intelligent enough? Is it because they’re not taught effectively? Is it because a majority of students are plain lazy? Is it because they are too spoon-fed? Well no, it’s none of those. It’s because most Universities haven’t yet fine-tuned their assessment and classification systems in a way that will allow all students – regardless of ability and potential – to get a first class degree. A majority of Universities have adjusted their assessment and classification schemes to a point where only the most delinquent of students will attain anything less than an upper-second class degree, but there is still some fine-tuning required to turn out close to 100% first class students.

Here are some tips for those Universities and institutions still striving for this level of perfection. Individually none of these factors is necessarily bad practice or illegal, and indeed many institutions strive to introduce many of these factors as examples of innovative good practice. However, if you put them all together in one scheme, you create an assessment and classification system that can turn the most delinquent and uninspiring student into a first class success. Here are the basic elements of that system:

1.         Always adopt a categorical marking scheme. Make sure that the first class band of marks covers 30% of awardable marks (e.g. 70-100%) whereas other classification bands cover only 10% (e.g. upper second class from 60-69%).  Within the first class band of marks, make sure there are as few categorical marks available to be awarded as possible and that there is a giant leap in awardable marks between a low first and a good first. For example, make the following marks the only ones awardable in the first class band, 75%, 90% and 100%. Then make sure that the assessment guidelines for 90% are as similar as possible to 75%, but with an added factor that all first class scripts would normally possess (e.g. to be awarded 90% a piece of work must have all the characteristics of a piece of work worthy of 75%, but will show “evidence of breadth of reading”).

2.         Always make sure that each piece of work is double marked, and that any discrepancies between markers are rounded up (e.g. if one marker awards 75% and the second awards 95%, then award 95%).

3.         Allow all final year students a resit option on failed papers that is not capped at the basic pass mark. Indeed, also consider allowing final year students the opportunity to resubmit any piece of work where they are not satisfied with the original mark.

4.         Include MCQs as a highly weighted component of every course/module – at both second and final year. Ensure that these MCQs are taken from a limited bank of questions that is recycled every year. Conveniently forget to adjust the marks on these MCQs for the possibility of chance correct answers.

5.         Include as many assessments as possible where the student has the opportunity to score 100% (e.g. MCQs, assessments where there is an indisputable correct answer or answers, etc.)

6.         Have at least one course/module in the final year that is weighted to make up most of the marks for that year (e.g. a final year project/dissertation). Ensure that the credit weighting of this course is excessive (e.g. the equivalent to 4 other courses), but that the work required by the student is nowhere near equivalent to the work required of four courses. Make sure the students are aware that this is a course/module on which they should concentrate their efforts.

7.         Adopt a very liberal classification borderline “bumping up” scheme that will ensure that as many students below a borderline will meet the criteria for being “bumped up” into the higher classification bracket even if they haven’t achieved the required aggregate mark for that higher classification bracket. Make sure that this is a mechanistic “bumping up” process determined by an algorithm (don’t involve the external examiners in this process – they may question it!)

8.         Introduce changes to the assessment and classification processes every year. This will mean that students will usually be simultaneously graded by two schemes – the “old scheme” and the “new scheme”, and all candidates will be classified according to their ‘best’ outcome from either of the schemes.

9.         Encourage students to apply for concessions and submit mitigating evidence. Make this process as simple as possible and do not set deadlines for evidence to be submitted. In particular, allow mitigation to be submitted after the student has knowledge of their degree classification.

10.       Allow external examiners to adjust agreed marks. But only upwards, so as not to unnecessarily disadvantage any student.


11.       No need to make candidates' identities anonymous. Some good students may have an off day in the exams and names on scripts will allow the internal examiner to mark the candidate according to their ability rather than on an exam "off day". Poor students who perform above their expected ability in an exam can be identified and rewarded accordingly.


12.       External examiners need pacifying and domesticating. Make sure that they have comfortable hotels and are given expensive dinners. Always tell them that there have been IT problems in the Registry and a full summary of marks and assessment statistics is unavailable this year. Fabricate at least two admin staff illnesses which have meant that scripts and coursework could not be sent to the external for moderation. Compliant externals should also be appointed for additional years after the end of their term of office. Make sure it is clear to externals that assessment guidelines (and anything else they may query) have been imposed by the University central administration and are out of the control of the Department. Regularly change Departmental Exam Officers so that no one individual can acquire enough knowledge to ensure the assessment period is conducted according to the full set of regulations.


13.      If an external examiner attempts to question the objectivity and validity of an examination and assessment process, the Registry should reply by stating that there was not a critical mass of external examiners across disciplines raising this particular issue to require a change in University policy. University Registries should ensure that the full range of external examiners' reports are not compiled in any single place where they are freely available for general scrutiny.

14.       Finally, make sure that a directive comes down from the University Registry to all examiners to “mark generously and use the full extremes of the marking scales – especially the first class band of marks”. This, of course, is imperative if the institution is to achieve a good grading in forthcoming National Student Surveys!

Please feel free to suggest more practical ideas by which Universities can adjust their assessment and classification processes to generate increasing percentages of first class students. Don't forget, well qualified graduates are our future - we need more of them!

Should You Publish Your Undergraduate Student's Projects?

Most academics and researchers now rely on their undergraduate students’ final year projects as an important research resource. These projects provide opportunities to test out new procedures, methodologies and theories at relatively low cost to the researcher. Nevertheless, no matter how much you might closely supervise this research there is still a nagging doubt that you have delegated important research to a relatively inexperienced individual. How do you decide whether the research they have delivered is worthy of writing up and publishing? Below is a flow-chart that allows the inexperienced junior lecturer to make some decisions about publishing an undergraduate project[1].

---

[1] This flow-chart is designed to ensure optimal career development for junior and mid-career academics and researchers.

Does a Menu Explain a Restaurant? - Clinical Constructs as Potential Category Mistakes

Over the past 20 years or so, clinical psychology researchers have developed cognitive models of psychopathology that have used constructs that appear to capture the beliefs, attitudes and thought patterns associated with psychiatric symptoms. These constructs have been used in many ways: to understand and explain symptoms and to develop new interventions that intervene by attempting to modify the psychological processes implied by the construct. Clinical psychology researchers have never been terribly good at articulating the exact theoretical nature of these constructs, but they are regularly portrayed as inferred states or processes derived most often from the clinical experiences of researchers or clinicians in their interactions with patients (Davey, 2003).[1] The purpose of these constructs is to help understand psychopathology symptoms, to provide a basis for developing interventions to treat the psychopathology, and – in the case of those who advocate cognitive explanations of psychopathology – to link thoughts, beliefs and cognitive processes to subsequent symptoms.

Hypothetical constructs have a long history in the study of psychology and human behaviour (MacCorquodale & Meehl, 1948; Cronbach & Meehl, 1955; Strauss & Smith, 2009), and their main purpose has been to help identify the theoretical mechanisms that underlie performance and behaviour (Whitely, 1983). In clinical psychology research, constructs have played an important part in the development of models of anxious psychopathology – especially in the years since cognitive approaches to understanding anxiety have become prevalent. Clinical constructs are often developed from the researchers own clinical experiences, and they represent hypothetical structures that usually attempt to summarize important aspects of the patient experience and integrate this with one or more theoretically important process that the researcher believes underlies the symptoms. In the past 20-30 years many theoretically influential clinical constructs have been developed during research on the aetiology and maintenance of anxiety disorders. Some of the more influential of these include inflated responsibility (Salkovskis, 1985), intolerance of uncertainty (Dugas, Gagnon, Ladouceur & Freeston, 1998), clinical perfectionism (Shafran, Cooper & Fairburn, 2002), and thought-action fusion (Shafran & Rachman, 2004), to name but a few. There is no doubt that clinical constructs have been influential in the development of theories of anxiety-based psychopathology, and these constructs have a prima facie clinical relevance and respectability by emerging from clinical experience, idiographic assessment, from illustrative case histories, exploratory qualitative methods, or content analysis of patient self-report statements (e.g. Frost, Steketee, Amir et al., 1997; Freeston, Rheaume, Letarte, Dugas & Ladouceur, 1994).

At this point it is important to understand the role that clinical psychology researchers see for the clinical constructs they develop. Without a doubt, in the majority of cases talk of a ‘causal’ or ‘explanatory’ role in the elicitation and maintenance of symptoms creeps into the discussion. For example, Koerner & Dugas (2006) note that intolerance of uncertainty  “is thought to lead to worry directly” (2006, p201); Salkovskis, Wroe, Gledhill, Morrison et al. (2000) write that “the occurrence and/or content of intrusions (thoughts, images, impulses and/or doubts) are interpreted (appraised) as indicating that the person may be responsible for harm to themselves or others. This leads both to adverse mood (anxiety and depression) and the decision and motivation to engage in neutralising behaviours (which can include a range of behaviours such as compulsive checking, washing and covert ritualising)” (2000, p348; my italics); Shafran, Thordarson & Rachman (1996) write that “increased endorsement of dysfunctional beliefs, particularly TAF [thought-action fusion] is likely to exacerbate low self-esteem, depression, anxiety, and perceived responsibility for the event.” (1996, p379). The implication of causation of construct on symptoms is further alluded to in the box-and-arrow schematic models of emotion-based disorders that have become associated with research on some of these clinical constructs (Davey, 2003). There is no doubt that such constructs help us to conceptualize the psychological processes and states involved in a specific psychopathology, but is there any basis for assuming that their role is a causal one?

In order to elevate these hypothetical constructs to the level of empirically verifiable and usable entities the constructs have to become measurable and, in many cases, manipulable – especially if they are to prove useful in clinical interventions. This process usually proceeds with the researcher describing the defining features of the construct and developing an instrument to measure the construct. Once a set of the defining features of the construct has been established, subsequent measurement instruments are developed and validated. Having defined the construct’s main features and developed a measurement instrument, the construct is now experimentally manipulable and objectively measurable according to standard empirical and scientific tenets. Subsequent controlled manipulation of the construct may result in observable changes in symptoms, leading us to conclude that the construct plays a direct or indirect causal role in determining the appearance or strength of the symptoms. These manipulations may be in the form of potential therapeutic interventions or in the form of a controlled experimental manipulation (e.g. the effect of manipulation of inflated responsibility or intolerance of uncertainty on compulsive behaviour). At this point, the construct has become a recognizable explanatory feature of the psychopathology, supported by empirical evidence in the form of its measurable relationship with symptoms (through correlational and regression analyses) and demonstrable effects on symptoms (through experimental manipulation).

The process described above appears to be an admirable attempt by clinical researchers to objectify their clinical experiences and subject them to rigorous, scientific analysis. At the end of this process we have constructs that are measurable and manipulable and can be empirically tested in their relationship with psychopathology symptoms. However, we need to be aware that clinical constructs are not directly observable and need to be inferred from the behaviour and responses of our patients and experimental participants. Inferential techniques, by their very nature, rely on observable behaviour to tell us something about the existence and behaviour of the unobservable psychological mechanisms that underlie performance (Whitely, 1983; Strauss & Smith, 2009). What is important about these inferential processes is that we cannot use the same behavioural anchors to verify the construct and then use them as outcome measures in experiments/interventions to determine whether the construct has an explanatory role or causal effect.

This logical inconsistency appears to be what happens in the research history of many clinical constructs. The confounding factor is that the construct is verified on the basis of patient reports about their psychopathology experiences and their symptoms or on researchers’ assumptions about these experiences (e.g. Frost, Steketee, Amir et al., 1997; Shafran, Thordarson & Rachman, 1996; Chambless, Caputo, Bright & Gallagher, 1984; Dunmore, Clark & Ehlers, 1999). When unpacked, many validated measures of clinical constructs resemble a list of questions about symptoms. It should then come as no surprise that (1) measures of the construct are significantly correlated with measures of symptoms, and (2) manipulating the construct causes concomitant predictable changes in symptoms. This raises serious doubts about concluding that the construct or the psychological states defined by the construct cause the symptoms or are even an explanation of the symptoms. To be fair, there are good arguments for saying that clinical constructs have helped to develop effective interventions for anxiety disorders. But it’s impossible to say that they are effective because they address the ‘causes’ of symptoms rather than the symptoms themselves. If the same behaviours (symptoms) are used to both verify the construct and to explore the construct’s explanatory role in the psychopathology then construct and symptoms are essentially the same thing. Logically, many clinical constructs do not exist other than being extrapolated from the symptoms that they are developed to explain. This relationship between clinical constructs and the behaviours they are developed to explain is reminiscent of what Ryle (1949) called a category mistake. Ryle wrote that:

“..when we describe people as exercising qualities of mind, we are not referring to occult episodes of which their overt acts and utterances are effects; we are referring to those overt acts and utterances themselves” (1949, p26).

 Given that very many clinical constructs are defined in ways that represent mental states of which symptoms are deemed to be their effects then we must seriously consider that the clinical construct approach to explaining psychopathology is also underpinned by a category mistake. In their discussion of constructs in clinical psychology research, Strauss & Smith (2009) distinguish between constructs developed as tools to measure and predict behaviour (constructs based on “nomothetic span”[2]), and those constructs that go beyond the data used to support them and postulate entities, processes or events that are not directly observed but which may underlie behaviour - known as “construct representation” (e.g. Whiteley, 1983; Strauss & Smith, 2009; MacCorquodale & Meehl, 1948). It is arguable that the current approach to clinical constructs in clinical psychology research has generated a culture in which clinical constructs proliferate without being properly theoretically defined – especially in the sense that they might represent constructs based on nomothetic span or construct representation. It may well turn out that many of those clinical constructs that have been researched so avidly in the past 10-15 years are no more than basic redescriptions of the symptoms they are often thought to explain.

REFERENCES

Chambless DL, Caputo GC, Bright P & Gallagher R (1984) Assessment of fear of fear in agoraphobics – the body sensations questionnaire and the agoraphobic cognitions questionnaire. Journal of Consulting & Clinical Psychology, 52, 1090-1097.

Cronbach LJ & Meehl PE (1955) Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.

Davey G.C.L. (2003) Doing clinical psychology research: What is interesting isn’t always useful. The Psychologist, 16, 412-416.

Dugas MJ, Gagnon F, Ladouceur R & Freeston MH (1998) Generalized anxiety disorder: A preliminary test of a conceptual model. Behaviour Research & Therapy, 36, 215-226.

Dunmore E, Clark DM & Ehlers A (1999) Cognitive factors involved in the onset and maintenance of posttraumatic stress disorder (PTSD) after physical or sexual assault. Behaviour Research & Therapy, 37, 809-829

Freeston, M. H., Rhéaume, J., Letarte, H., Dugas, M. J., & Ladouceur, R. (1994). Why do people worry? Personality and Individual Differences, 17, 791–802.

Frost R, Steketee G, Amir N, Bouvard M et al. (1997) Cognitive assessment of obsessive-compulsive disorder. Behaviour Research & Therapy, 35, 667-681.

Haslam, N. (1997). Evidence that male sexual orientation is a matter of degree. Journal of Personality and Social Psychology, 73, 862-870.

Koerner N & Dugas MJ (2006) A cognitive model of generalized anxiety disorder: The role of intolerance of uncertainty. In GCL Davey & A Wells (Eds) Worry & its psychological disorders. John Wiley.

MacQuorquodale K & Meehl PE (1948) On a distinction between hypothetical constructs and intervening variables. Psychological Review, 55, 95-107.

Meehl, P. E. (1992). Factors and taxa, traits and types, differences of degree and differences in kind. Journal of Personality, 60, 117-174.

Meehl, P. E. (1995). Bootstraps taxometrics: Solving the classification problem in psychopathology. American Psychologist, 50, 266-275.

Ruscio J, Ruscio AM & Carney LM (2011) Performing taxometric analysis to distinguish categorical and dimensional variables. Journal of Experimental Psychopathology, in press.

Ryle G (1949) The Concept of Mind. Peregrine Books.

Salkovskis, P. M. (1985). Obsessional-compulsive problems: a cognitive-behavioural analysis. Behaviour Research and Therapy, 23, 571-583.

Salkovskis PM, Wroe AL, Gledhill A, Morrison N et al. (2000) Responsibility attitudes and interpretations are characteristic of obsessive compulsive disorder. Behaviour Research & Therapy, 38, 347-372.

Shafran R, Cooper Z & Fairburn CG (2002) Clinical perfectionism: A cognitive-behavioural analysis. Behaviour Research & Therapy, 40, 773-791.

Shafran R & Rachman S (2004) Thought-action fusion: A review. Journal of Behavior Therapy & Experimental Psychiatry, 35, 87-107.

Shafran R, Thordarson DS & Rachman S (1996) Thought-action fusion in obsessive compulsive disorder. Journal of Anxiety Disorders, 10, 379-391.

Strauss ME & Smith GT (2009) Construct validity: Advances in theory and methodology. Annual Review of Clinical Psychology, 5, 1-25.

Whitely SE (1983) Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179-197.

---

[1] Clinical constructs as latent categorical variables can also be inferred statistically (e.g. Meehl, 1995; Ruscio, Ruscio & Carney, 2011), but these too will usually have their origins in clinical practice or clinical theory and are used to advance the development of causal theories (e.g. Haslam, 1997; Meehl, 1992). This discussion will be limited to those clinical constructs developed to explain (rather than categorize) anxiety disorders.

[2] “Nomothetic span” refers to the pattern of significant relationships among measures of the same or different constructs (i.e. convergent and discriminant validity) (Whitely, 1983; Strauss & Smith, 2009). Thus, the descriptive validity of a construct is established by observing that it is related to measures it should be theoretically related to and not related to measures that it should not be theoretically related to.