I love how audits can call out in very matter of fact language irrational decisions. "These criteria demonstrate that issues other than strictly public safety played a role in determining the program locations." (p.22) The City Controller of Los Angeles completed an audit on the photo red light program and its effectiveness in reducing traffic accidents caused by running red lights. Best practices state that it is most effective to place cameras at intersections with high traffic and accident rates. The audit found that that placement was the result in part of the need to garner Council support for the program so cameras were located in each Council district regardless of traffic congestion and accident data.

TSA screeners need better training - Really?

An audit done by the inspector general at the Transportation Security Administration finds that TSA screeners need to be better trained. "One lead TSO indicated that he had not accessed the Online Learning Center since 2005. The TSO also explained that staff had limited time to read printed training materials in lieu of going online." You can understand the urgency of getting the program underway but by now there should have been some kind of assessment by the agency itself.

Arghh...

There comes a time when auditing is not as easy as just doing the work and presenting the results. The dynamics of the organization has to be considered and a strategy developed. As an auditor becomes more knowledgeable about each department in the agency, the strengths and weaknesses start to become more apparent. It's like the relationship with a person. At first you see only the good things, then you begin to see the other side. Whether the relationship will remain healthy or not depends on the ability to not personalize the deficits. You have to keep your eye on the goal and work for improvements. But it is not easy.

Facts and stories

When auditees read the first draft of an audit a common response is that we did not talk about anything good that they have done. Our response is usually that our job is to identify problems, analyze why they occurred and how the situation could be improved. We are constrained by standards that require us to be fact-based. Establishing facts takes time. To include in the report an analysis of things that were going right would require the same standards be met. Since audit work is expensive, because of the time it takes to establish facts, it would not be cost effective to also analyze the successful parts of a service or program. However, we do try to add context so that there is an appreciation of the work that is being done to provide the service.

But from the perspective of a journalist trying to find a story that includes protagonist, antagonist, and facts that will capture a reader's interest, our reports are not hard hitting enough. When I have campaigned as an auditor, the common response was always that people had no idea there were auditors doing the kind of work we do. Auditors face a quandary, work for change or work to create an audit report that will capture a journalist's eye so that it will be reported and the public will see that there is accountability. Many of our audits fall under the radar because management has agreed with our findings and our recommendations. Or the audit area is just not interesting or big dollar enough. The choice is usually to work for change and not necessarily news coverage, so many stories never get told.

Short-term engagement

What would I do auditing in an environment that didn't value equity and human rights. Guess I wouldn't last long.

Savoring the Results

This has been a crazy summer so far. At first it seemed like it would never warm up. We planted our garden and then there was 2-3 weeks of hard rain. Seeds didn't sprout and existing plants suffered. Now we have had several weeks of sunshine and things are looking better, although getting to harvest is slow. What are the lessons here?

• You can't always predict when hard rain will come.
• Results might not happen on schedule.
• Sunshine always heals frustration and makes things grow.

I like the last bullet. Could be an analogy for transparency and organizational change.

Auditing by Processing

My thought process too. Like pulling a string or auditing.

Guerrilla Warfare

Thinking about a recent hiring process I was reflecting on possible differences between generations. My motivation in working for government has always been to make it more accessible, accountable and respectful to citizens. It brought to mind a book that spoke to me many years ago, Guerrillas in the Bureaucracy. I think I still have a copy in my office. I googled it and found this interesting book. I also found this article about how do deal with guerrilla bureaucrats. "Create an organization culture that accepts, welcomes, and encourages candid dialogue and debate." Yes!!

Definitely about outliers

I have always been an outlier. From a small town and yearning to be in a city. Coming from an environment that was fairly conservative, but having ideas that were not. Entering the world of art, but not fitting in. Moving on a career path that is not a straight and logical course but one with sharp turns and bumps. But definitely an outlier.

I am working on an audit right now that is also about outliers. But we are looking for outliers because this will provide evidence to evaluate whether management has done its job. What is managements' job? Is it managements' job to make employees all conform?

The real measure for an employer is the quality of the work. Employees may have differing work styles but the work quality may be worth accommodating those styles. That is why an audit should focus on effect. If accommodating different styles does not create a negative effect or actually increases effectiveness, than auditors should be tolerant of outliers in their analysis.

Outliers can also be a symptom of a problem that needs to be addressed. Rather than try to control the outlier, management instead should use it as information about organizational health. When an auditor finds this type of outlier, she can use the information to describe the effect of an organizational problem. Recommendations then would focus on the real problem, not employees who are outliers.

Communicating Results

The nature of audit work is to be methodical. That is our strength. We plan how to analyze whether services are efficient or effective, we execute the plan, we tie our analysis to the plan, and our report to the analysis. And then we write our report.

At that point, the fact we are methodical becomes our weakness. Readers, the public, management, and leadership do not want a recounting of how we got to our conclusions. They want to know what our conclusions are.

Not that they don't want to know that we methodically approached this work. They do. They rely on our professionalism. But they would prefer we keep the details to ourselves.

When writing a report, our job is to communicate clearly and effectively what the problem is, why it is occurring, and at what cost to the agency. To move to this level of communication is always a struggle. But it is our job.

Not statistically significant

The Fair Housing Council of Oregon conducted audits from the citizens' point of view to determine if housing discrimination existed. In Ashland and Beaverton, when African American and white testers were sent to the same location, two-thirds of the black testers faced discrimination in Ashland and 78 percent in Beaverton. Management response in Beaverton was that the testing sample was too small to be statistically significant. But one wonders what the tolerance level is. Even though it was not a large enough sample to generalize to the full population, is it then okay to have some discrimination?

Auditing the food we eat

We rely on our government to regulate the quality of the food that we eat. But recently there have been cracks in that protection. Here is another one. According to an audit of efforts to ensure the quality of meat that is sold in the U.S., high levels of veterinary drugs, pesticides, and heavy metals are not detected or, if detected, still enter the market for us to eat. Additionally, there is no attempt to recall meat, even when tests have confirmed the excessive presence of veterinary drugs. In 2008, when Mexico rejected the entry of U.S. meat for copper levels higher than its acceptable standard, there was no basis to stop this meat from entering the U.S. market since there was no comparable set tolerance level here.

Evaluation and Auditing

When I explain to people what a performance audit is, I usually refer to evaluation practice. Most people understand what an evaluation is while when they hear the word “audit” they immediately think about the IRS. I just finished reading the results of an evaluation of an area that I had wanted to audit. The evaluation was successful in determining the “condition” and the “effect.” In describing the “condition”, it also was able to allude to the “cause.” The recommendations are also on point; however, without addressing the “cause” head on, I wonder how easily they will be to implement.

Performance audits, to be successful, need to get to root causes. In this evaluation, the root cause is the organization. The evaluation looked at similar programs across departments and made recommendations to improve at the program level. However, without effective organization, clear lines of authority and responsibility, the road to improvement will be slow and possibly difficult. In this case, there should be a recommendation to consolidate leadership of programs.

I also was reminded of another difference between evaluation and auditing – the report. In this evaluation, the report language was very academic. Prior to getting to the findings, you had to wade through the methodology of the evaluation and the analysis. Usually the first draft of an audit will also contain writing about the methodology and the analysis, but by the time we get to a draft for our audiences (program, Council, public) that information is removed. We try to describe in simple and concise terms what we found and what it caused.

Entropy and the arrow of time

Here is an interesting interview in the Wired Science blog. Physicist Sean Carroll is studying time. In the interview he talks about the “arrow of time,” the fact that we can know the past but not the future. There are processes that you cannot reverse. An omelet cannot be returned to an egg. Entropy is also discussed, the idea that things become more disorderly over time. The analogy he puts forth is that you are not surprised that an orderly desk becomes messy but you would be surprised if you left a messy desk to return and find it orderly. These two concepts seem related to auditing. Auditing studies the past. In addition, we are not surprised that well-designed services over time morph into inefficient and ineffective services.

Words 2

As I said below, words are important. As auditors we want to speak precisely. But to communicate to our audience we may have to speak more generally and give up some precision. Case in point, is in the new Freakonomics blog post. Economists have a precise definition of "recovery" but the public doesn't understand the same word the same way.

Words

Words are very important in an audit. Auditors cannot afford to be sloppy. At the beginning of an audit when the scope is a concept and lacks specificity, words are important to help narrow the scope. Words used in the audit objectives help focus an auditor's work, keep her on track, and increase efficiency when followed. At the end of the analytical stage of an audit, words become very important in crafting the overarching message. What is it we are really trying to say? What is important and what is not?

The words we use effect the evidence that we need to support the words. If we do not have adequate evidence to support the words, for example "some" as opposed to "few," then we have to change the words. And, finally, when the auditee reads the draft, words are critical. Words that we have used to describe the condition or the effect may have a different meaning to the auditee. We have to be open to substituting words to reach agreement on the recommendations of the audit. As long as we do not change the facts, then changing the words to describe the facts, is acceptable. At the end of the day, these words express our findings to the public and the decision-makers. If we have not written clearly and exactly, our message may not be as effective.

Citizen Point of View "Audit"?

Here's a news story about a Housing Program in the Bay Area. The Program had people pose as persons seeking housing to gauge the response. I did a brief search of the web site to view the report but was unsuccessful. Probably not an audit according to standards but still an approach that could be worth using. Reminds me of the joint housing audit conducted by Multnomah County and the City of Portland where we had students posing as homeless call emergency housing services. The results were amazing and powerful.

Auditors worldwide

I saw this article recently in the NY Times. Again, I'm astounded about audits operating in what seem to be less supportive environments. Reading this led me to investigate. I found the site of the International Organization of Supreme Audit Institutions. Both Zimbabwe and China are members. I first heard about this organization when I received notification from GAO about a presentation by the previous Comptroller General, David Walker.

Believing in the tooth fairy

I just finished The Return of Depression Economics by Paul Krugman. The first edition of the book was written in the 1990's and published in 1999. It was updated in the midst of the current crisis. It is very readable and very instructive. Mr. Krugman reviews the previous crises back to the Depression in an attempt to understand and provide analysis and recommendations for the present. He makes the conclusion that this crisis is like everything we've seen before, all at once. Trading one bubble for another and ignoring the risk of acting outside of the regulated banking system, "the shadow banking system," only led to the failure that had been seen many times before in Asia, Japan, South America, and here.

December

I took December off. I became inhibited by trying to make my random thoughts more complex and complete. My New Year's resolution is to make one post per week. Whoops! Already missed that.