ALCOA – A standard for evidence

In my current role (Clinical Quality Assurance), I support groups that are involved in validating electronic systems for use within clinical trials. That is, they are conducting testing of these computer systems and generating evidence of that testing, its results, problems encountered, and the resolution of those problems.

I used the word “evidence” above very deliberately and with special meaning. The expectation of regulatory bodies (like FDA, EMEA and member agencies, Health Canada, etc.) is that the evidence that is generated in any manufacturing or research endeavor that relates to health products must be strong enough to make a case in a court of law. Furthermore, they assign the burden of proof to those making the claim. That is, they assume that a drug is unsafe, that a device is ineffective, or that data is fraudulent until it is proven otherwise. This is a new paradigm for many who work in Pharma (and related industries) as it runs counter to our “Innocent until proven guilty” sensibilities.

The FDA has been using the ALCOA acronym as a guide to their expectations regarding evidence (both paper-based, electronic, and hybrid) for years and most other health inspectorates have similar expectations. As such, it is immensely useful in developing strategies to prospectively generate strong evidence in both research and manufacturing endeavors. The breakdown and meaning for ALCOA is as follows:

Attributable – It should be clear who created a record and when. Likewise, it should be clear as to who amended a record, when, and why. That last one, the “why” (the reason for change) is an important area in which there are frequently gaps. There is some room for “self evident” reasons for change, but many take an overly broad definition of “self evident”. It is always best to explain why an entry is being changed.

Legible – This would seem to be a common sense requirement. If evidence is illegible, then what good is it, right? Surprisingly, there are some who challenge this point…quite often the challengers are physicians.

I audited a physician once who essentially refused to write his dates in a legible fashion. There was even a note to file that he had “difficulty” in writing the year. When I first read the note, I was thinking he might have some physical problem that made writing difficult for him (e.g., carpal tunnel, MS, etc.). However, after reading it again and meeting with the physician, it was clear that this was not the case. Not to reinforce stereotypes about physician’s handwriting, but jeeze. That was an awkward topic to broach in the audit closeout let me tell you.

Contemporaneous – Alright, now if there’s a fifty-cent word in the mix here, this is it. FDA is fond of this one, and those who know me know that I am as well. It comes off as a bit of a pompous blow-hard word. If there were a simpler word that packed the same meaning, I’d probably use that one instead. So far, I’m unaware of one. Anyway, in terms of evidence, it means the evidence or test results are recorded as they are observed.

The contemporaneousness of records ties very much into the attributability of those same records for the simple reason that FDA has expressed that all signatures or initials must be accompanied by a date that indicates when the signature or initials were appended to the document. It seems to be related to their method of evaluation of data; they want to be able to reconstruct the occurrences around the data. Right in there with that, they expect that all recorded dates be the current date at the time of the entry and that late entries be clearly designated as such. E.g.:
“On 31 Dec 1999, I observed that the refrigerator was broken, however, since the world was due to end any minute, I made no entry. When the world did in fact continue on 01 Jan 2000, I called for service and ensured that sample preparations were moved to another unit.” –TJK 23 Jul 2008

Original – Records are expected to be original; this is a basic scientific principle. In school (where, amongst other things, I studied Chemistry), we had to keep a lab notebook. That notebook was expected to be an original record of our observations within the laboratory. We were not to use scratch paper and then carefully and neatly copy the lab data into pristine lab notebooks….that would lessen their strength as evidence.

In fact, I had a friend back all those years ago who was a laboratory teaching assistant. One of his tasks at the end of semester was to grade lab notebooks. One day, while I was waiting for him to finish up his grading work so that we could head out for some entertainment on a Saturday night; he pressed a stack of lab notebooks into my hand and muttered something about making myself useful.

He then instructed me to look through the notebooks for a stain….something that indicated that the student had spilled something from the lab on their notes while they were recording in them. “Why?” I naively asked.
“Because, the lab is a messy place….if these students (all of whom are on the cusp between two grades) actually completed their notebooks in the lab, they would have spilled something at some point…..something pretty like a cobalt compound would be nice….and therefore can qualify for the “Stain Bonus” and have their grade bumped up the half-point to the next level.

I like telling that story….because FDA seems to think a lot like that.

Accurate – Honesty is first consideration here and thoroughness is the second. Make sure of the information that you are recording is correct and make sure you’re telling the “whole truth”. I don’t have any illustrative stories for this one other than that Regulatory Inspectors are highly trained to detect Fraud. There’s a very fine line between Fudging a result and outright Fraud. Regulatory Inspectors are renowned for not seeing that fine line and “not knowing how it is”. Tell the truth and be sure that your records do too. Transparency is a good thing…in fact it is one of the key goals of ALCOA.

The take home idea here is that the concepts embodied in ALCOA will help provide evidence that is convincing. Without that, claims that your drug is safe and effective, that this particular lot of medicine is safe for human consumptions, etc. will fall on deaf ears.

Hopefully, this is a topic that is helpful and interesting. I’d love to hear any thoughts, both positive and negative, on it. Also, for those who are interested in the background and Regulatory references for this material, you may want to take a look at this page on First Clinical Research. It has a good Q&A section as well as links to some really great presentations from Stan Woollen (formerly of FDA). If the link ever gets broken, let me know and I should be able to get the materials to you.

~TJK