GPH: Critical appraisal - Rose using Kuper

Was the sample used in the study appropriate to its research question?

- Size- At first I thought this was possibly too small a sample, however in the methods section the authors’ do say that “recruitment continued until saturation of themes had been achieved.”

- Sampling method: Through advertisements in community mental health centres, local voluntary sector newsletter and care coordinators. I think these are appropriate methods because the study requires participants who would be willing to divulge personal information. Only people who would accept this would volunteer for such a study. However as the paper mentions this causes the problem of selection bias.

- Appropriate persons interviewed – since the question was to ascertain the opinions/experiences of mental health service users and professionals, the study appropriately sampled from these two groups of people.

2. Were the data collected appropriately?

Data was collected through interviews with professionals specialised in conducting such them . I am not sure but I believe the authors used Thick description by providing a series of illustrative quotes from both the service users and mental health professionals, thereby giving me (the reader) a sense of what it was like to be in the research setting.

I believe an iterative process was also used whereby codes assigned to themes were constantly checked and modified appropriately throughout the study.

3. Were the data analysed appropriately?

The interviews were analysed thematically. After reading the in depth description of what this entailed I think this was an appropriate method of analysis. The themes were coded and these codes were cross checked at one point in the study to ensure reliability.

4. Can I transfer the results of this study to my own setting?

Because the authors clearly describe the methodology, in terms of both the interviews and the analysis of data, in a lot of detail I think the results could be transferred to another setting. If not the results the study could definitely be redone for example on an issue such as rape.

5. Does the study adequately address potential ethical issues, including reflexivity?

The paper does not mention any ethical issues and does not discuss whether or not they considered reflexivity.

6. Overall: is what the researchers did clear?

Yes the method and discussion of the results were very clear.

I realised I did the work and didn't submit it! Apologies

Re: Critical appraisal - Rose using Kuper

by Joseph Daniel Jameson - Thursday, 8 November 2012, 1:06 PM

Firstly - nice simple post! Love it. much preferring the numbered/bulleted layout.

However, the BMJ Kuper article does highlight that critical appraisal of qualitative should not be a simple checklist but an 'interprative judgement'. Having said that, for us medics its much easier to understand a checklist - i like tickboxes!. This doesn't mean you can't discuss each tickbox as I have attempted to do here:

1) appropriate sample?

size - appropriate according to saturation of themes.

method - They talk about 'purposive sampling' which I'm not really sure exactly what that means. anyone?

appropriate persons - Users and professionals as yopu said. They also ensure the ethnic composition of the sample is representative of the local population which is likely to reflect the socio-cultural concepts of domestic violence. However, they sample lots more women than men in the users but not the professionals group. (16 women, 2 men in users). They don't state why they did this its probably appropriate as their background explores that domestic violence is more of a problem in female service users.

2) Appropriate data collection?

I don't know if anyone else picked up on the fact that there was financial insentive for the users to take part? could this cause bias? And was the setting of the interview appropriate. Were private interviews, carried out by professionals in a mental health setting conductive to discussion of domestic violence? could this have been too intimidating?

Also, I'm not sure if I have understood this completyely, but is their use of 'interview topic guides' that were checked in consultations with users and experts and then piloted a good example of triangulation?

3) Appropriate analysis

Thematic analysis was effective. Particularly in the cross over between user and professional themes.

4) transferability?

limited, as they state in the study itself, by the specific population the study is researching. Would this mix of ethnic minorities in a London borough reflect same themes as in a white majority population of the west country for example?

5) Reflexivity / Ethical?

I think this may be a true weakness in this study as there appears to be no attempt at addressing the effects of power and gender in the interviews on the research process. These two factors in particular would have a significant effect, i believe, in discussions about domestic violence. Although approval by an ethics committee is stated.... as the BMJ article alucidates: is this enough??

6) overall impression

Personally i was unconvinced. Limited reflexivity on the research process and limited discussion of ethical issues that could arise with discussion of this topic are its major weaknesses. Strengths include clear methodology & analysis and its clinical recommendations!

Re: Critical appraisal - Rose using Kuper

by Patricia Greenhalgh - Sunday, 11 November 2012, 4:28 PM

Joseph this is a really good post.

PURPOSIVE sampling is what we need in a qualitative study. Statistical sampling, stratified or not, is NOT what we need. Purposive sampling means you want to get a really broad sample that will tell you the RANGE of attitudes/experiences. Some people will think X, some will think Y and some will think Z. After you've done your qualitative study on a purposive sample, you might THEN take a statistical sample, perhaps stratified by age and gender, to address questions like "What % of women aged 20-30 think X?"

Does this make sense?