Linguistic Evidence and the 2009 NAS Report on Forensics
The National Academy of Science (NAS) report is excellent and it is my sincere hope that the leadership shown by this report will be taken very seriously and followed. The committee who authored this report have all contributed to the handling of forensic evidence in the most reliable and scientifically rigorous ways in their own fields of law, computer science, statistics among others. Working against the status quo is never easy, so I am most grateful to the committee for pushing forward when others are pushing back. But, in my opinion, the report might have pushed a little bit harder, and misses three big points that affect forensic science in the US:
(1) the need for judicial training on how to recognize junk science;
(2) self-legitimatizing and unscrupulous academics who do forensics on the side. Some naive academics may not even be aware that they are aiding and abetting junk science, but others run full-time consulting practices from their academic offices, even while their research agendas are totally different from their forensic practices;
(3) emerging forensic techniques which are taught in self-certification or degree programs, usually by the academics mentioned in (2) who haven’t done any empirical research devoted to validation testing in the areas in which they teach.
These are three big, big, big loopholes that junk can pass through. As the two biggest stakeholders, AAFS and NIJ cannot solve these problems, in my personal experience with both organizations. This is why NAS is right that a new agency is required, but the new agency has to be cognizant of these loopholes or nothing will change. These loopholes can be changed by easily implementable protocols and training programs with a bar set to match the highest scientific standards of replicability and reliability.
Forensic linguistics is a perfect example of these three loopholes.
(1) Judges approach linguistic issues (such as who authored the blog post? is this a real threat letter?) with a non-scientific layman’s approach, not usually aware of linguistics and surely not aware of how unreliable the layman’s approach is for determining these kinds of questions.
(2) Academics whose actual research is not at all related to such questioned are called in to testify because they have academic credentials, and often times no one checks if their research agendas or their research publications actually match up to what they are testifying about. Recently I heard one academic state in a sworn deposition that his article on a Swahili word for honey-wine was actually an article about authorship identification, or “at least the theory underlying it,” because he obviously had done no research at all on authorship identification. The central question regarding the linguistic evidence in the case was “who wrote the blogs?” –authorship identification –but this academic obviously had no trouble trying to delude the lawyers who had hired him.
On the other hand, and just as dangerous, are academics who do publish research-related articles, yet totally eschew any researched methods when it comes to actual casework. The publications and the casework do not line up. While the publications suggest that the author is using a quantitative method, for instance, which suggests objectivity and replicability, the actual case reports are totally subjective, with no quantification at all. Unless an attorney actually reads the publication(s) (usually published in out of the way journals) and reads the case report in light of the publication(s), he or she will not see the mismatch and realize that something really fishy is going on.
(3) One of the quickest ways to attain some kind of status, especially when research takes too long and is too hard to do, is to create a certification program and a teaching component. This is the pattern for all the junk sciences identified by the NAS report –when there is no research, you can be sure to find a self-certifying program and a raft of seminars loosely attached to an academic or certifying organization. Academics who are too busy testifying to do any real research on the reliability of their methods, or academics whose research looks good but whose reports are totally different from their research, are all too eager to rush into self-certification. In a 40-hour class or a six-week course, (conveniently held in the summer), you too can hang up your shingle as a forensic linguist! The NAS report does not address this problem, and might actually make the problem worse since it does emphasize the need for accreditation and certification.