New Proposed Auditing Course: Considerations for an eData Quality Audit Program

We’ve submitted this course proposal for the April SQA Quality College in Anaheim. Feedback has been good, but even if it isn’t picked up for this venue…it will eventually happen *somewhere*:

Instructors: Timothy J. Kuhn and Conrad Kawaguchi

Target Audience: Seasoned Auditors with little eData/CSV experience or CSV professionals with little auditing experience.

 Course Objectives

• Establish and differentiate between the eData Audit Function and the Operational CSV Quality Roles.

• List the considerations for the GxP eData Audit Program (Contributing to other GxP Audits, Independent Focused Data Integrity Audits, etc.)

• Explore the various roles that the eData Auditor may serve as part of an audit (SME, Co-Auditor, Audit Lead)

• Define and list the desired skill set for the eData Audit Team

• Break down the various types of audits and risk areas for each: GLP Labs, Central Clinical Labs, Central Readers (eCG, Imaging, etc.), ePRO (Electronic Patient Reported Outcomes – AKA Diaries), Acquisition Due Diligence, GMP Manufacturing Sites and Vendors, PV, eSystem Validation, Databases, IT Infrastructure, Cloud Providers, Software Providers, SaaS vendors, Pharma, Device, Combo Product, and others.

Course Description: Electronic Data (eData) is pervasive in the GxP world and auditors in that GxP eData space need to understand what is most important in these areas. This course will explore considerations in establishing an eData audit program, give auditors with little Technology/eData/Computer System Validation(CSV)/IT saavy the understanding needed to assess the eData/eSystems they encounter in the course of their existing GxP audits, and expand the understanding of eData auditors beyond the software vendor audit so as to know what is most important in GxP audits. The course will demonstrate “what to look for” for a variety of audits, both technology-focused and more conventional areas in which technology has “encroached”.




Computer Inspection Checklist from PMDA

I’m not a big fan of using checklists during audits, but it is always good to see which ones are being used by the regulators to better understand their thinking and priorities. This one from Japan came up during the quarterly Society of Quality Assurance CVIC (Computer Validation Initiative Committee) on which I sit.

PMDA Checlist CSV



FDA FY 2010 Inspectional Observation Summaries

The FDA has recently updated their published  483 inspection results here. This is really a great resource as it breaks the findings down by regulation and how often it was cited. The link is to the 2010 483 that I stumbled upon. The parent site has multiple years and in sortable formats:



21 CFR part 11 Update

Folks, it looks like some of our US regulations and guidance documents have been flagged for update by FDA….including everyone’s favorite, 21 CFR part 11 (source Cerulean’s email newsletter). I don’t anticipate major changes, just clarification on their intent.  Everyone enjoy your Friday.

New Guidance Agenda for 2009 Released by FDA

The FDA’s CDER has published its list of expected guidance documents and revised regulations to be issued this year (2009).

Readers of our 2009 forecast will find many items on the list familiar:

· 21 CFR Part 11

· Process Validation

· Adaptive Clinical Trial Design

· Contract manufacturing

· Various marketing, promotional and labeling guidance

· “Dear Healthcare Professional Letters” for recalls


The glass is half full/empty?

As a QA professional I scrutinize the systems, processes, and results (data) of my client groups (the operational areas….i.e., the people who do the real work). This can be very frustrating for people; especially those who work hard, do good work, and aren’t used to having someone doubt them or subject them to the third degree….physicians often have a particularly difficult time with this.

I understand this and strive to make my intrusion on their daily work as unobtrusive as possible. I also try to explain to them exactly what my thought processes are and the rationale for my questions. One of the things I like to tell my client groups/auditees is that I am an optimist who’s paid to be a pessimist. That’s simple and to the point and it seems to convey my position to my client groups/auditees rather well.

More accurately, I’m an optimist who’s paid to be the exact right combination of the guy who sees the glass half full, the glass half empty, and the cracks in the glass. They pay me to see the good, the bad, and the ugly; to take a good system and try and break it…..just to see if someone can.

Below, I’ve assembled a few (real) scenarios that I think illustrate my role:

1) In this case, I was auditing at a Clinical Investigators site (a doctor’s office). At one point in our exit interview he felt that I (a non-physician) was questioning his medical opinion. To which I told him, “I would never question your medical opionion; I’m not qualified…..but based on your source notes, I can’t tell what your medical opinion was….or even that you had one”. Understanding stole across his face…..Documentation. That’s what was missing. He was doing the work in a stellar fashion,  but not documenting it sufficiently.

2) Another physician was less convinced of the need to explicitly document his diagnosis. “Any clinician would immediately have the same diagnosis…its self-evident!”. Again as a non-physician, I told him that I was sure he was right, but that FDA would likely send a “non-clinician”; someone who would look at his documentation in a manner more akin to that of an attorney than of a medical professional….”Oh, I think I understand what you’re telling me now.”

3) A third physician tried to correct me on calling study participants “subjects” rather than “patients”….”subject sounds so cold and uninvolved”. To this, I explained my position that a “patient” is someone who comes to a doctor looking for something tried and true to make them better. A study “subject” is someone who is taking a gamble out of philanthropy, desperation, or a combination of both….someone that, with their informed consent, we are putting in some measure in harms way. They deserve our highest respect and greatest level of care. Patients are very important; study Subjects are doubly so. This doctor seemed a little irritated at my little philosophical tirade (short though it was), but he seemed to understand the point I was making…I guess he thought I was an idealist of some sort…

4) While meeting with a team that was conducting validation testing on  a clinical computer system, some on the team were frustrated by the level of testing that I was recommending (requiring?). “Well in an ideal world, we would test everything…..”….I had to correct him, “In an ideal world, software testing wouldn’t be necessary because everything would work right out of the box in a perfect and unqualified way…We test our software to ensure that it is actually working like we think it is”. He didn’t like that response, but at least he was quiet”.

Are there any others with anecdotes or questions relating to GxP?


Geeky Pharma Plates

I’m not much for vanity decorations on cars. License Plates, Bumper Stickers, Decals, whatever, they usually just annoy me. I don’t really care to know about the political leanings, sports affiliations, or group memberships of my fellow drivers….save through good conversation. I do so love good conversation. So when I am subjected to reading about these things, billboard style, on the back of their cars as I drive to wherever, I’m more prone to be annoyed than amused.

All that being said, I have noticed a recent trend on these car adornments to embrace the rather geeky technical aspects of the Pharma industry. For example, the other day while driving with my wife, we spotted a car sporting Pennsylvania plates with the ID number “21CFR11”. Now for those of you out of the know, 21 CFR part 11 is the US regulation governing the use of electronic records and electronic signatures in FDA-regulated functions (like drug development or manufacture). I can’t think of a Geekier thing to put on one’s car…I mean, that probably tops “I love Linux” as a mantra.

Another plate I saw a while back, this time from New Jersey, read “PHASE4”. That plate was a co-worker’s at a previous employer. Again, for those out of the know,  with Phase 4 referring to studies performed for a possible new indication (use) for an already approved drug. Again, pretty geeky.

Living in NJ as I do with its high concentration of Pharma companies, I suppose it was just a matter of time. Now, I am interested in both Phase 4 and especially 21 CFR part 11, but I think I’ll keep that off my car.

For those interested in these topics, I’ve pulled the following information from the FDA website for your reference:

Phases of an Investigation

An IND may be submitted for one or more phases of an investigation. The clinical investigation of a previously untested drug is generally divided into three phases. Although in general the phases are conducted sequentially, they may overlap. The three phases of an investigation are as follows:

Phase 1 includes the initial introduction of an investigational new drug into humans. These studies are usually conducted in healthy volunteer subjects. These studies are designed to determine the metabolic and pharmacological actions of the drug in humans, the side effects associated with increasing doses, and, if possible, to gain early evidence on effectiveness. Phase 1 studies also evaluate drug metabolism, structure-activity relationships, and the mechanism of action in humans. The total number of subjects included in Phase 1 studies is generally in the range of twenty to eighty.

Phase 2 includes the early controlled clinical studies conducted to obtain some preliminary data on the effectiveness of the drug for a particular indication or indications in patients with the disease or condition. This phase of testing also helps determine the common short-term side effects and risks associated with the drug. Phase 2 studies usually involve several hundred people.

Phase 3 studies are intended to gather the additional information about effectiveness and safety that is needed to evaluate the overall benefit-risk relationship of the drug. Phase 3 studies also provide an adequate basis for extrapolating the results to the general population and transmitting that information in the physician labeling. Phase 3 studies usually include several hundred to several thousand people.


this slide presentation from the 2002 DIA that summarizes Part 11.


ALCOA – A standard for evidence

In my current role (Clinical Quality Assurance), I support groups that are involved in validating electronic systems for use within clinical trials. That is, they are conducting testing of these computer systems and generating evidence of that testing, its results, problems encountered, and the resolution of those problems.

I used the word “evidence” above very deliberately and with special meaning. The expectation of regulatory bodies (like FDA, EMEA and member agencies, Health Canada, etc.) is that the evidence that is generated in any manufacturing or research endeavor that relates to health products must be strong enough to make a case in a court of law. Furthermore, they assign the burden of proof to those making the claim. That is, they assume that a drug is unsafe, that a device is ineffective, or that data is fraudulent until it is proven otherwise. This is a new paradigm for many who work in Pharma (and related industries) as it runs counter to our “Innocent until proven guilty” sensibilities.

The FDA has been using the ALCOA acronym as a guide to their expectations regarding evidence (both paper-based, electronic, and hybrid) for years and most other health inspectorates have similar expectations. As such, it is immensely useful in developing strategies to prospectively generate strong evidence in both research and manufacturing endeavors. The breakdown and meaning for ALCOA is as follows:

Attributable – It should be clear who created a record and when. Likewise, it should be clear as to who amended a record, when, and why. That last one, the “why” (the reason for change) is an important area in which there are frequently gaps. There is some room for “self evident” reasons for change, but many take an overly broad definition of “self evident”. It is always best to explain why an entry is being changed.

Legible – This would seem to be a common sense requirement. If evidence is illegible, then what good is it, right? Surprisingly, there are some who challenge this point…quite often the challengers are physicians.

I audited a physician once who essentially refused to write his dates in a legible fashion. There was even a note to file that he had “difficulty” in writing the year. When I first read the note, I was thinking he might have some physical problem that made writing difficult for him (e.g., carpal tunnel, MS, etc.). However, after reading it again and meeting with the physician, it was clear that this was not the case. Not to reinforce stereotypes about physician’s handwriting, but jeeze. That was an awkward topic to broach in the audit closeout let me tell you.

Contemporaneous – Alright, now if there’s a fifty-cent word in the mix here, this is it. FDA is fond of this one, and those who know me know that I am as well. It comes off as a bit of a pompous blow-hard word. If there were a simpler word that packed the same meaning, I’d probably use that one instead. So far, I’m unaware of one. Anyway, in terms of evidence, it means the evidence or test results are recorded as they are observed.

The contemporaneousness of records ties very much into the attributability of those same records for the simple reason that FDA has expressed that all signatures or initials must be accompanied by a date that indicates when the signature or initials were appended to the document. It seems to be related to their method of evaluation of data; they want to be able to reconstruct the occurrences around the data. Right in there with that, they expect that all recorded dates be the current date at the time of the entry and that late entries be clearly designated as such. E.g.:
“On 31 Dec 1999, I observed that the refrigerator was broken, however, since the world was due to end any minute, I made no entry. When the world did in fact continue on 01 Jan 2000, I called for service and ensured that sample preparations were moved to another unit.” –TJK 23 Jul 2008

Original – Records are expected to be original; this is a basic scientific principle. In school (where, amongst other things, I studied Chemistry), we had to keep a lab notebook. That notebook was expected to be an original record of our observations within the laboratory. We were not to use scratch paper and then carefully and neatly copy the lab data into pristine lab notebooks….that would lessen their strength as evidence.

In fact, I had a friend back all those years ago who was a laboratory teaching assistant. One of his tasks at the end of semester was to grade lab notebooks. One day, while I was waiting for him to finish up his grading work so that we could head out for some entertainment on a Saturday night; he pressed a stack of lab notebooks into my hand and muttered something about making myself useful.

He then instructed me to look through the notebooks for a stain….something that indicated that the student had spilled something from the lab on their notes while they were recording in them. “Why?” I naively asked.
“Because, the lab is a messy place….if these students (all of whom are on the cusp between two grades) actually completed their notebooks in the lab, they would have spilled something at some point…..something pretty like a cobalt compound would be nice….and therefore can qualify for the “Stain Bonus” and have their grade bumped up the half-point to the next level.

I like telling that story….because FDA seems to think a lot like that.

Accurate – Honesty is first consideration here and thoroughness is the second. Make sure of the information that you are recording is correct and make sure you’re telling the “whole truth”. I don’t have any illustrative stories for this one other than that Regulatory Inspectors are highly trained to detect Fraud. There’s a very fine line between Fudging a result and outright Fraud. Regulatory Inspectors are renowned for not seeing that fine line and “not knowing how it is”. Tell the truth and be sure that your records do too. Transparency is a good thing…in fact it is one of the key goals of ALCOA.

The take home idea here is that the concepts embodied in ALCOA will help provide evidence that is convincing. Without that, claims that your drug is safe and effective, that this particular lot of medicine is safe for human consumptions, etc. will fall on deaf ears.

Hopefully, this is a topic that is helpful and interesting. I’d love to hear any thoughts, both positive and negative, on it. Also, for those who are interested in the background and Regulatory references for this material, you may want to take a look at this page on First Clinical Research. It has a good Q&A section as well as links to some really great presentations from Stan Woollen (formerly of FDA). If the link ever gets broken, let me know and I should be able to get the materials to you.