Forensic investigation must be as thorough and objective as scientific research.  Peer review in science ensures that the research is well done.

Unfortunately, peer review in forensic work is not carried out very often.  And when it is there is a risk of bias in how it’s done.  This in spite of strict civil procedure rules governing experts and their reports.

***

Peer review in science is a process of evaluating scientific work, by an expert or a group of experts in the related field.  The scientific worker may or may not know the reviewer(s) – a blind peer review.  Similarly, the reviewer may or may not know the worker. (after Ref. 1 and Dr. Google)

Bias, or unreasoned judgement, can be blatant, or it can be unintended, sneaking up on us out of the ether. (Refs 2, 3 and 4)

Getting rid of bias in peer review is essential to forensic investigation.

***

The penny dropped for me on a better way compared to current practice when I got an e-mail from a reader, Ruth Corbin, after she read a blog I posted on peer review.  (Refs 5, 6)  A quick reply to Dr. Corbin included my initial thoughts on how I think we can get rid of bias.  These are bulleted below including some additional thoughts.

At present, experts investigate the cause of a problem.  They write a report on their investigation and findings.  The expert’s report goes to their client, the insurance claims adjuster or the civil litigation lawyer, and from there to the court or dispute resolution tribunal.  The investigation and report are seldom peer reviewed.  Except in a sense when they are rebutted by the expert for an opposing party.

Just how unbiased is the rebuttal report?  If phraseology in some reports is any indication, not much.

(You’ve all seen this phraseology, I’m sure.  Just to be really sure though, I’ll update this blog in future with some examples)

A peer review doesn’t have to be a crude denunciation or vetting of an expert’s work.  We can get rid of blatant bias.  We’re all here to serve the judicial and dispute resolution processes.  Finding errors and omissions and fixing them serves that purpose.

A peer or rebuttal review a little closer to that in science would help, one that involves the expert as little as possible in the organizing of a peer review of his work.  I think peer review procedures like the following will help, in decreasing order of preference:

• The court or tribunal retains the peer reviewer independent of the expert or his client.  This in the spirit of peer review in the scientific community.
• All the experts engage in “hot-tubbing” session, give their evidence concurrently and agree a joint report. (Refs 2, 7)
• The lawyer or adjuster retains a peer expert to review his work.  Because s/he wants a thorough and objective explanation of issues unfamiliar to the court or tribunal.  But third in preference because the lawyer or claims manager do have their interests.
• The expert retains a peer reviewer.  Because she would like to think they have done thorough work and reasoned well, particularly in the more empirical applied sciences like some medical and engineering specialties.  But fourth in preference because perception is everything in some fields of study.  An expert hiring someone to check his work might not look good.
• The rebuttal expert peer reviews the expert’s work.  I include this procedure because it’s a procedure that is sometimes followed.  It can work if the rebuttal expert pushes back against bias.
• No peer review at all of any kind.  The procedure assumes the expert’s work is thorough and objective and unbiased.

I believe the risk of bias is low when the court or dispute resolution tribunal arranges the peer review and high when the rebuttal expert does this.

Summary

We can get rid of almost all bias in peer review of forensic investigation and expert services.  There are several methods for doing this.  The one we chose will reflect our commitment to change and what we’re prepared to accept.  We’re accepting a lot of potential bias now.

References

1. Merriam-Webster dictionary
2. Biased experts cured with a soak in the “hot tub”.  A blog posted January 31, 2017 at www.ericjorden.com/blog
3. Expert witness forum looks at bias and other touchy subjects in forensic work. Posted March 8, 2018
4. Are experts being broadsided by bias unbeknownst to them?  Posted April 12, 2018
5. A Bundle of Blogs: On the need for peer review in forensic engineering and expert services.  Posted November 29, 2019
6. Corbin, M.Sc., Ph.D., LL.D., Ruth M., Chair, Corbin Partners Inc, and Mediator, Corbin Estates Law, Toronto
7. “Hot-tubbing” experts reduce the cost of civil litigation and ensure objectively.  Posted March 31, 2018

(Posted by Eric E. Jorden, M.Sc., P.Eng. Consulting Professional Engineer, Forensic Engineer, Geotechnology Ltd., Halifax, Nova Scotia, Canada. ejorden@eastlink.ca)   

The phrase “differential diagnosis” caught my eye recently in light of some tendency in forensic engineering for the injured party to take the expert’s initial thought on cause as gospel and run with it.

Yet the forensic expert’s initial thought – an initial hypothesis – is based on just a little evidence and likely, quite subjective evidence.  For example, a briefing by the client, a  read of some documents and perhaps a walk-over survey of the accident or failure site.

Sometimes that thought/hypothesis is subject to revision like happens in the scientific method – and an embarrassment to all concerned if counsel decides to take the case or the claims manager agrees a settlement.

(Scientific method: A method or procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses. Ref. 1)

Differential diagnosis is a medical process.  It occurred to me that forensic experts could learn from medical doctors.

Differential diagnosis is the development of a list of possible medical conditions that might explain a patient’s symptoms.  The list goes from the most likely and urgent at the top to the least at the bottom.  The process involves several phases like forensic investigation involves several stages.  The early phases/stages are subjective in nature.  The process is well developed in medicine as explained by a friend and also Dr. Google. (Refs 1 and 2)

The phases of a differential diagnosis and their similarity to a forensic investigation are a little like the following:

Phase #1 Take history

In medicine, take a history from the patient about what she’s experiencing.  Interrogate and ask questions like a detective.  Try to figure out what’s going on.

In forensic work, take a briefing from the injured party or their counsel or claims manager about their slip and fall accident or the damage to their building.  Ask lots of questions.  Read existing documentation.  Consider different categories of accidents or failures.

Phase #2 Physical examination

Look, feel and listen.  Take the patient’s pulse and measure his blood pressure.  Examine him orally.  Listen to his chest (with a stethoscope).  Do a percussive examination (tap body with fingers and note the sound)

Walk over the site and visually examine the accident or failure scene and the structures there.  Measure and determine the condition of the structure before and after the failure.  Photograph and measure features characterizing the scene.  Take aerial video of the scene from a drone.  Excavate test pits and note the subsurface conditions.  Carry out initial skid resistance tests of the floor at a slip and fall accident. 

Phase #3 Additional investigative tests

Carry out additional tests like blood work, X-rays, MRIs and stress tests.

Take samples and do laboratory tests of the physical properties of materials that failed.  Do field tests and sample and test the physical properties of the soil in the field and laboratory.  If necessary, do additional skid tests of the floor.

Phase #4 Data analysis and interpretation

Analyse data and test results and identify conditions that could account for the patient’s symptoms.  List from most likely to least likely – the differential diagnosis.  Look at the most probable diagnosis at the top of the list then go back to Phase #3 and order more tests to confirm depending on how confidant you are.

Study the data, look at how different pieces of data relate and support one another and relate to possible causes of the personal injury or the crane or building collapse.  Identify the probable cause(s) of the injury or collapse.  If necessary, return to the site to check and confirm earlier findings.

Phase #5 Treatment

In medicine, prescribe a treatment of the condition with medicines, life style changes, diet, etc.  Monitor the condition and if improves good.  If no improvement, consider the dosage of medicines and the extent of other changes.  If none or not much go back to the previous phase and reconsider the differential diagnosis.  Treatment is the prescription, the plan in SOAP. (Ref. 3)

In forensic work, report the most probable cause of the accident or failure.  Recommend how the damage can be fixed and the cause of the accident or failure eliminated.

***

I see the differential diagnosis process as an elaboration of the SOAP process that is also followed in medicine: (Ref. 3)

1. Gather Subjective data.  Take a history from the patient in medicine and a briefing on the problem in engineering.  Reflect on why the patient is hurting and the different categories of structural failure in engineering.

2. Get some Objective data.  Like blood work and X-rays in medicine and measurements and field and laboratory tests in forensic investigation.

3. Analyse the data.  Study, identify and list the different medical conditions indicated by the data that could account for the patient’s symptoms.  And in forensic work, the different causes that could account for the personal injury or the crane or bridge collapse.  The list is the differential diagnosis in medicine and the possible causes in a forensic investigation, going from the most likely cause at the top to the least likely at the bottom.

4. Prescribe treatment.  For example, identify life style changes, diet and/or medications to fix the medical condition and make the symptoms disappear.  In forensic work recommend how the damage can be repaired and the cause removed.

***

Can you imagine the embarrassment to the medical doctor and the pain for the patient if s/he prescribed treatment based on the results of Phase #1 of the differential diagnosis process and he was wrong and the patient dies?  Everybody gets in trouble.

Fortunately, that doesn’t happen often in medicine.  Unfortunately it happens at times in forensic work – the client runs with the expert’s initial thoughts on cause.  A few years ago, an expert noted the occasional pressure on an expert during a forensic investigation to find support for those initial thoughts.

Summary

So, the next time you’re getting a medical check-up think about the forensic expert and hope the doctor doesn’t go with his initial thoughts on the cause of your symptoms.  And if you’re the forensic expert, go out of your way to help your client, the injured party, understand that an initial thought on cause is not necessarily a final diagnosis.  It’s at the front end of the subjective-data-collection-stage and a soft thought.

References

1. Dr. Google
2. Personal conversation with Dr. J. Nasser, Halifax, a retired ear, nose and throat surgeon and a former dentist
3. Using SOAP notes in forensic engineering investigation.  Posted February 6, 2014