Computer Inspection Checklist from PMDA

I’m not a big fan of using checklists during audits, but it is always good to see which ones are being used by the regulators to better understand their thinking and priorities. This one from Japan came up during the quarterly Society of Quality Assurance CVIC (Computer Validation Initiative Committee) on which I sit.

PMDA Checlist CSV



Computer System Validation (CSV) Audit Mindmap

As a break from writing, I finished the first electronic version of my CSV Audit Mindmap. The intent is for it to be a mental tickler during an audit. Rather than 10-20 pages of checklists, this would be a visual layout of related topics that fit on 1 or 2 pages (or more if you like). Click the link to look at a pdf version…and be sure to let me what you think.

CSV Audit Mindmap




The glass is half full/empty?

As a QA professional I scrutinize the systems, processes, and results (data) of my client groups (the operational areas….i.e., the people who do the real work). This can be very frustrating for people; especially those who work hard, do good work, and aren’t used to having someone doubt them or subject them to the third degree….physicians often have a particularly difficult time with this.

I understand this and strive to make my intrusion on their daily work as unobtrusive as possible. I also try to explain to them exactly what my thought processes are and the rationale for my questions. One of the things I like to tell my client groups/auditees is that I am an optimist who’s paid to be a pessimist. That’s simple and to the point and it seems to convey my position to my client groups/auditees rather well.

More accurately, I’m an optimist who’s paid to be the exact right combination of the guy who sees the glass half full, the glass half empty, and the cracks in the glass. They pay me to see the good, the bad, and the ugly; to take a good system and try and break it…..just to see if someone can.

Below, I’ve assembled a few (real) scenarios that I think illustrate my role:

1) In this case, I was auditing at a Clinical Investigators site (a doctor’s office). At one point in our exit interview he felt that I (a non-physician) was questioning his medical opinion. To which I told him, “I would never question your medical opionion; I’m not qualified…..but based on your source notes, I can’t tell what your medical opinion was….or even that you had one”. Understanding stole across his face…..Documentation. That’s what was missing. He was doing the work in a stellar fashion,  but not documenting it sufficiently.

2) Another physician was less convinced of the need to explicitly document his diagnosis. “Any clinician would immediately have the same diagnosis…its self-evident!”. Again as a non-physician, I told him that I was sure he was right, but that FDA would likely send a “non-clinician”; someone who would look at his documentation in a manner more akin to that of an attorney than of a medical professional….”Oh, I think I understand what you’re telling me now.”

3) A third physician tried to correct me on calling study participants “subjects” rather than “patients”….”subject sounds so cold and uninvolved”. To this, I explained my position that a “patient” is someone who comes to a doctor looking for something tried and true to make them better. A study “subject” is someone who is taking a gamble out of philanthropy, desperation, or a combination of both….someone that, with their informed consent, we are putting in some measure in harms way. They deserve our highest respect and greatest level of care. Patients are very important; study Subjects are doubly so. This doctor seemed a little irritated at my little philosophical tirade (short though it was), but he seemed to understand the point I was making…I guess he thought I was an idealist of some sort…

4) While meeting with a team that was conducting validation testing on  a clinical computer system, some on the team were frustrated by the level of testing that I was recommending (requiring?). “Well in an ideal world, we would test everything…..”….I had to correct him, “In an ideal world, software testing wouldn’t be necessary because everything would work right out of the box in a perfect and unqualified way…We test our software to ensure that it is actually working like we think it is”. He didn’t like that response, but at least he was quiet”.

Are there any others with anecdotes or questions relating to GxP?


ALCOA – A standard for evidence

In my current role (Clinical Quality Assurance), I support groups that are involved in validating electronic systems for use within clinical trials. That is, they are conducting testing of these computer systems and generating evidence of that testing, its results, problems encountered, and the resolution of those problems.

I used the word “evidence” above very deliberately and with special meaning. The expectation of regulatory bodies (like FDA, EMEA and member agencies, Health Canada, etc.) is that the evidence that is generated in any manufacturing or research endeavor that relates to health products must be strong enough to make a case in a court of law. Furthermore, they assign the burden of proof to those making the claim. That is, they assume that a drug is unsafe, that a device is ineffective, or that data is fraudulent until it is proven otherwise. This is a new paradigm for many who work in Pharma (and related industries) as it runs counter to our “Innocent until proven guilty” sensibilities.

The FDA has been using the ALCOA acronym as a guide to their expectations regarding evidence (both paper-based, electronic, and hybrid) for years and most other health inspectorates have similar expectations. As such, it is immensely useful in developing strategies to prospectively generate strong evidence in both research and manufacturing endeavors. The breakdown and meaning for ALCOA is as follows:

Attributable – It should be clear who created a record and when. Likewise, it should be clear as to who amended a record, when, and why. That last one, the “why” (the reason for change) is an important area in which there are frequently gaps. There is some room for “self evident” reasons for change, but many take an overly broad definition of “self evident”. It is always best to explain why an entry is being changed.

Legible – This would seem to be a common sense requirement. If evidence is illegible, then what good is it, right? Surprisingly, there are some who challenge this point…quite often the challengers are physicians.

I audited a physician once who essentially refused to write his dates in a legible fashion. There was even a note to file that he had “difficulty” in writing the year. When I first read the note, I was thinking he might have some physical problem that made writing difficult for him (e.g., carpal tunnel, MS, etc.). However, after reading it again and meeting with the physician, it was clear that this was not the case. Not to reinforce stereotypes about physician’s handwriting, but jeeze. That was an awkward topic to broach in the audit closeout let me tell you.

Contemporaneous – Alright, now if there’s a fifty-cent word in the mix here, this is it. FDA is fond of this one, and those who know me know that I am as well. It comes off as a bit of a pompous blow-hard word. If there were a simpler word that packed the same meaning, I’d probably use that one instead. So far, I’m unaware of one. Anyway, in terms of evidence, it means the evidence or test results are recorded as they are observed.

The contemporaneousness of records ties very much into the attributability of those same records for the simple reason that FDA has expressed that all signatures or initials must be accompanied by a date that indicates when the signature or initials were appended to the document. It seems to be related to their method of evaluation of data; they want to be able to reconstruct the occurrences around the data. Right in there with that, they expect that all recorded dates be the current date at the time of the entry and that late entries be clearly designated as such. E.g.:
“On 31 Dec 1999, I observed that the refrigerator was broken, however, since the world was due to end any minute, I made no entry. When the world did in fact continue on 01 Jan 2000, I called for service and ensured that sample preparations were moved to another unit.” –TJK 23 Jul 2008

Original – Records are expected to be original; this is a basic scientific principle. In school (where, amongst other things, I studied Chemistry), we had to keep a lab notebook. That notebook was expected to be an original record of our observations within the laboratory. We were not to use scratch paper and then carefully and neatly copy the lab data into pristine lab notebooks….that would lessen their strength as evidence.

In fact, I had a friend back all those years ago who was a laboratory teaching assistant. One of his tasks at the end of semester was to grade lab notebooks. One day, while I was waiting for him to finish up his grading work so that we could head out for some entertainment on a Saturday night; he pressed a stack of lab notebooks into my hand and muttered something about making myself useful.

He then instructed me to look through the notebooks for a stain….something that indicated that the student had spilled something from the lab on their notes while they were recording in them. “Why?” I naively asked.
“Because, the lab is a messy place….if these students (all of whom are on the cusp between two grades) actually completed their notebooks in the lab, they would have spilled something at some point…..something pretty like a cobalt compound would be nice….and therefore can qualify for the “Stain Bonus” and have their grade bumped up the half-point to the next level.

I like telling that story….because FDA seems to think a lot like that.

Accurate – Honesty is first consideration here and thoroughness is the second. Make sure of the information that you are recording is correct and make sure you’re telling the “whole truth”. I don’t have any illustrative stories for this one other than that Regulatory Inspectors are highly trained to detect Fraud. There’s a very fine line between Fudging a result and outright Fraud. Regulatory Inspectors are renowned for not seeing that fine line and “not knowing how it is”. Tell the truth and be sure that your records do too. Transparency is a good thing…in fact it is one of the key goals of ALCOA.

The take home idea here is that the concepts embodied in ALCOA will help provide evidence that is convincing. Without that, claims that your drug is safe and effective, that this particular lot of medicine is safe for human consumptions, etc. will fall on deaf ears.

Hopefully, this is a topic that is helpful and interesting. I’d love to hear any thoughts, both positive and negative, on it. Also, for those who are interested in the background and Regulatory references for this material, you may want to take a look at this page on First Clinical Research. It has a good Q&A section as well as links to some really great presentations from Stan Woollen (formerly of FDA). If the link ever gets broken, let me know and I should be able to get the materials to you.


The Lot of the QA Road Warrior

Over the last few years, I’ve been a lot of really neat places for work: Singapore, London, Argentina, Brazil, Mexico, Canada, and countless domestic (US) locales….and the travel destinations of my auditor colleagues would likely make up the difference for the rest of the world. This all sounds very exciting, very jet set, very sexy.

But the truth of the matter is that a little travel goes a long way. When we go, we sit in a conference room…..a conference room that looks like most other conference rooms….and read documents, interview people, make quick notes on legal pads as the auditees look on nervously. Usually, there’s no need for them to worry…we auditors are just making notes so that we can somehow distinguish their conference room (and the work presented there) from so many others we’ve sat in during the course of our careers.

There are many downsides to my chosen profession: justifying expenses to “Corporate Expense Processors” who may or may not know what we do (usually not), waiting to get our money back, sitting in airports, sitting on planes…sitting on planes on the tarmac….waiting. Actually, there’s a lot of waiting and a lot of time away from our families….this last part is hardest.

Don’t get me wrong, there are upsides as well. We do exercise a fair amount of autonomy and have the opportunity to really make a difference with our specific expertise and skill sets. That makes for a very satisfying career, but tonight, I’m in a mood to lament the woes of life on the road. I’m not one to drone on about the hardness of my lot, especially since I’m very pleased with my recent job change, but I think that it is important to mention the downsides to even the best of situations…..I’m an auditor; they pay me to look for the ugly underbelly. And its hard to turn that off.

Well, I’m on the West Coast, but I’m still on East Coast time….so I’ll be going to bed soon. I wish a good night to all.