Elements of an Audit

Several times I've written about how the work that auditors and journalists do is sometimes similar. Here is a news story by Henry Esteve that begins with a simple questions. Were the estimates of the cost of a tax credit accurate? Estimates are just that, estimates. Costs may turn out to be lower than higher than estimated. But it is good management to check and adjust a program depending upon what unfolds.

It's called closing the loop. Many audits address this management area. In my auditing experience, government service providers intensely pay attention to the design and organization of services. Just as much work and effort should also be placed in monitoring the service and evaluating whether it was successful.

This lengthy unfolding news story also illustrates a difference between journalism and auditing. News stories are short, not comprehensive, and accumulate over time. Turns out that state energy subsidies have been an ongoing story in 2009 in the Oregonian. The most recent story linked above that caught my eye was followed by at least 10 more stories and commentary including a story that the Governor would roll back subsidies.

Journalism is designed to foster accountability and transparency by shining a light on government practices. So is auditing, but audits are also intended to analyze the source of the problem and offer recommendations for change. Audits have a dual purpose - accountability and improvement of government services. The article cited a potential cause for the under estimated costs - that they had been low balled to gain legislative approval. If I had been conducting an audit I would have wanted to look at management practices that allowed estimates to be manipulated, if that is indeed what happened. What an auditor wants is to shine a light on the problem but also to have management rebuild a system that will protect the accuracy of data and prevent the problem in the future.

I can't imagine

I am signed up for several Google alerts relating to auditing and audits. One such alert recently took me on an interesting journey. The original alert took me to this website (?) blog (?) about the findings of an auditor in Zimbabwe. According to the About Us page, "Zimbabwe Metro online (zimbabwemetro.com) is an aggregated web log that covers a wide-range of topics, including sections devoted to politics, entertainment, media, lifestyle and business.We also feature hyperlinks to various news sources and columnists. Our roster of bloggers includes many people from Zimbabwe’s politicians to our extensive network of prominent writers. We are ranked the most most visited news weblog on Zimbabwe News by Alexa Internet."

Intrigued that an auditor would be exposing government corruption to such a degree, I tried to do some research. Using the NY Times search function I discovered an interesting archive of news stories about Zimbabwe. It seems that the president of Zimbabwe won a run off election and was inaugurated to his sixth term in June 2008. His opponent withdrew from the race because "state-sponsored enforcers were beating and killing his followers." More recent articles document a cholera epidemic and devastating economic and agricultural policies that leave the population starving. In February 2009, the president's opponent was sworn in as the prime minister to try and bring unity to the government. And just yesterday President Obama presented the 2009 Robert F. Kennedy Human Rights Award to publicly recognize Women of Zimbabwe Arise, or Woza, a grassroots effort to speak out against injustice in the country.

And in all of this an auditor is operating. I tried to search for a website for the auditor or the audit but was unsuccessful.

Blink Auditing

I have been reading several books lately that cause me to reflect on auditing skills and why I find auditing so intriguing. (Yes, intriguing!) I have already posted about Freakonomics. Another book that I've recently read and in the same vein is Outliers. That author has a similar perspective as Freakonomics except its focus is on successful people. The premise is that it was no accident or fluke that certain people become successful and that it can be explained by the underlying circumstances.

Another book that I just finished, blink, by the same author of Outliers, Malcolm Gladwell, can also be related to the auditing experience. In the final chapter of this book, Mr. Gladwell realizes that the overarching subject of the book is judgement. The first chapters deal with snap judgements or a conclusion that comes in the blink of the eye. This discussion helped me understand why sometimes judgements in audits seem to be intuitive rather than straight-line rational thought. He then explores when snap judgements can lead you astray. And finally there is a discussion of when snap judgements should be relied on and when not. The case studies and illustrations are very thought provoking.

Accountability, Politics and Independence

It is always difficult to create the best organization to provide government services. Many different structures have been created to remove politics from decisions made about the services. In some jurisdictions sheriffs and police chiefs are elected to give the agencies independence from politics, but this creates a different type of political battle at budget time when these elected officials can make an independent case to the public for a larger share. Another organizational model is to create an independent commission to direct the service such as a development commission or a library foundation.

There is another side to this puzzle. Some of the solutions to keep politics out of decision-making and provide the best service also remove the organization from accountability. There is no question that elected officials are accountable to the voters. When a barrier is created to this accountability, e.g. a commission, that link is broken. While voters may still blame the elected officials for poor decisions, the elected officials may have no control because of the independence of the commission.

During my career I have audited several of these types of organizations. I continue to believe that it is not the structure but the people that make the difference. There is a constant search for the perfect balance between accountability and independence and politics. Here is an audit and news story that illustrates that dilemma. This is a case where the commission is appointed by the governor with the consent of the state senate. The mayor along with the commission oversee police operations.

Why have a fraud hotline?

This is why. I don't think this happens often, but sometimes an anonymous hot line is the only way to find out. Nothing establishes tone at the top like action taken when unethical behavior is detected.

Opaque Transparency

I have written before about transparency and how that is translated into action. In one post, I cited the Government Accountability Office’s report on transparency. To reiterate, transparency is the public’s access to information. I have also written about the presentation of information recently when I compared EPA web information to the New York Times handling of very similar information. Thinking about the citizen point of view is important to designing the presentation.


Today there was commentary in the Oregonian about the opaqueness of government transparency as it is currently delivered on http://www.recovery.gov/ . The GAO has now started to audit transparency. See page 33 of this report. They note as positive that the Office of Financial Stability on its http://www.financialstability.gov/ site has added a usability survey to help them improve the quality of their transparency. The GAO itself uses a similar survey when you search their website and I have to admit I have seen steady improvement in their website and its usability.

Communicating audit findings

This reminds me of what you have do to translate audit findings into a report. LOL

Transparency via the New York Times

I frequently admire graphics in the New York Times. And, I wish many times that audit reports could communicate as clearly. Sometimes I even use ideas I find in the Times to design an illustration for a report. For example this map illustrates my point. The NY Times compiled data on facilities that have permits to discharge pollutants and responses from states regarding compliance and entered it into an interactive map. You can enter your zip code and learn what facilities are near you and the number of violations they have had.

There is much talk now in government about increasing transparency. The federal government has launched http://www.recovery.gov/ to track stimulus funds and we have http://www.usa.gov/ that serves as a gateway to government information including a page of links to government data sources. But flooding citizens with data is not necessarily transparency.
The EPA has a similar feature to the NY Times map that is referenced on its main page. You can enter your zip code and access "MyEnvironment." Here you will have access to information about environmental quality in your area. There is a map with sites that report to the EPA. But the only information for each site is the name and address. It lacks what is of most interest to a citizen - the number of violations. I tried to probe further to see if with more investigation I could find this out at the EPA site but it proved impenetrable to me. Maybe if I had searched more industriously I might have found the information. But you have to admit that a citizen probably would not be as determined.

Four graphs say it all

I continue to admire the NY Times graphics. Here are four graphs that collectively explain why health care reform is necessary. We should try for similar visualization in our audit reports. We could communicate quickly and effectively. What if an audit report was only two pages, one with graphs and the other with recommendations.

Sometimes you just happen upon something

Blogs can lead to interesting places. I started with NYTimes Freakonomics blog. That led to a blog that I hadn't seen yet called frumination. I followed the link because we are currently conducting an audit of transportation outcomes and this link was about what would be the effect if there were no subways in New York. The author of the frumination blog took data available from the Metropolitan Planning Organization of New York (an organization required for federal transportation funding) to do his/her analysis. Another blogger took the data and visualized it in another way. This shows the power of making data available and transparency.

Truly strange

I have worked as a local government auditor for three different governments for the past nineteen years. Governments always struggle with the administration and management of contracts. Every audit that examines contracting always will find room for improvement. But this is as bad as it gets. Jefferson County, Alabama is unable to determine how many contracts it has that are current. Lacking adequate controls, they also don't have a contract compliance officer or an internal auditor. Faced with a budget crisis, the local County commissioners don't have accurate information to make decisions.

But even stranger, this news account, just four days later describes another crisis in the city of Birmingham, Alabama, the County seat. The City Council is requesting an audit from the Alabama State Auditor's Office to determine how much money the City has in the bank. The mayor claims the City will end the year with a $13 million surplus, but the City's financial accounting software shows the City is running a deficit.

Two great audit areas - contracts and the accuracy of financial information.

Chilling Effect in LA

Previously in this blog, I have written about a lawsuit against the LA Auditor who was trying to audit the worker's compensation program. The lawsuit was filed by the elected City Attorney against the elected Controller. The City Attorney claimed that the Controller had no authority to audit another elected official's program. I made the argument that all roads eventually lead to an elected official, so why would that make a program exempt, and also that audits are auditing the program's performance, not the leadership's. Now it appears that a judge has issued a tentative ruling backing the City Attorney's position. The City is now talking about clarifying the charter and perhaps a ballot measure. I can't imagine that the public would not support accountability.

Measuring Audit Results

Auditors regularly measure results of programs and services and also critique the agency's performance measures. Because of this, I have always published an annual report for my office that includes key measures such as reports per FTE, implementation rate, and hours per audit. If we require our agency to report performance, it makes sense that the audit office should also report. One measure I struggle with is outcome. What was the effect of our audits? The most common measure I have seen in other audit offices is annual savings attributed to audits. Since my office conducts performance audits, often we cannot measure cost savings. Generally, recommendations are directed towards improving effectiveness.

I have commented previously on what I consider to be similarities between the fields of journalism and auditing. An Oregonian article last Saturday about last minute legislative action included a sidebar that linked new laws to the Oregonian's investigative reporting. I think that this might offer a potential solution to measuring the effect of audits. Perhaps I can document concrete action taken by a Department or the Council that can be linked to audit findings. So, rather than cost savings, maybe I could categorize actions attributed to audit findings and then quantify them.

Didn't pass the "easy" test

I started with great interest a blog post at Freakonomics and thought that the links to "stat candy" might be good examples for auditing criteria about transparency and providing data to everyone. But after visiting both sites called out in the post, I was very disappointed. I know I haven't finished my coffee and it's early morning, but I was stumped. If I was unable to figure out how to pull data into maps and charts, how would the citizen feel. Obviously these sites are not for normal consumption. Increasing transparency needs to fit two criteria - easy to locate and easy to use.

Assigning blame where it belongs

Steve Duin, a columnist in the Oregonian, wrote today about a settlement with the family of a man who died as a result of his actions and those of public officials after his arrest. Many were involved including the arresting officers, jail staff, and the ambulance personnel. The columnist worries that without the exercise of a trial that will assign blame, there will be no motivation to change. I think there should be accountability in the system, but does assigning blame motivate employees to change?

An audit steps back and looks at the structure that allowed employees to make bad decisions. When I meet with auditees, I always tell them we are auditing a program's or service's performance, not the employee's performance. What was the tone at the top or the climate that allowed police to believe that a man urinating in a trendy neighborhood required a forceful police reaction? Are police trained to recognize that sometimes a person's actions can be the result of mental illness? Do personnel treat all medical emergencies with the same urgency and care? What is the procedure for medical personnel at the jail to make medical decisions? Should medical personnel at the jail and police reassess a person's medical condition before transporting to a hospital to determine the current urgency of the situation? Training, procedures, and the management climate can make for better outcomes. An audit would review all of those and assign blame, but not to the individual, instead to the services that failed. Then maybe there would be change.

More Web 2.0

Found out from a newspaper article that a local police agency is using twitter, MySpace and Facebook. I checked out the FB site for the Tigard Police Department. It is being used to reach out for assistance in ongoing investigations and to provide information.

Here is the most unique and interesting website I have seen for a government. Many interesting features, particularly the search function. Doesn't have the look and feel of the usual government site.

Update to fan of San Francisco on Facebook

Despite the interesting site that increases government accessibility for the San Francisco citizen, don't become a fan. Your "wall" will become inundated by messages from the city. More than you ever wanted to know on a daily basis.

Bus audit creates discussion about purpose

Here is an interesting news article about an audit of the transit system in Houston. A state audit has caused a debate about what the purpose of the transit system is. Is it efficiency? One metric considered was the operating ratio or the percent of the cost that is paid for by the fare. Increasing this metric requires reducing or eliminating routes that serve lower numbers of people. But then this raises the issue of equity or whether the transit authority's purpose is serving the low income, those who have no other means of transportation. Quality can be measured in another way - the comfort of the trip. Some think that using that performance measure light rail transportation is better than bus routes - because light rail is quieter, more comfortable, and more reliable.

A very interesting discussion and illustrates the value of an audit. Audits can improve the quality of the public discussion. This article reminds me of the debate that is taking place in my own jurisdiction, but without the benefit of an audit to clarify issues. See here, here, and here.

I became a fan of San Francisco on FB today

I became a fan of San Francisco on Facebook today. My son invited me to join FB several months ago and I have to admit it has been fun. I am careful though who I let friend me and also what I say knowing that nothing is private. But I have reconnected with some old high school friends that I never would have found without it. Also, I've been able to communicate with my son and his fiance in different ways. However, my daughter still remains appalled that I am a member.

But, if you are a member of FB you should check out the official site of the City and County of San Francisco. I find most government web sites difficult to penetrate or understand. The developers of these sites do not use language that is easily understood by a citizen and also have not thought about what a citizen might be looking for. That old auditing from a "citizen's point of view" thing. But SF's Facebook site was penetrable and actually interesting. It managed to talk about governmental things in a way that was interesting to a citizen.

Well known auditor on NY Times Front Page

I opened the NYTimes web site this morning and saw a familiar face. Mark Funkhouser, previous Kansas City auditor, is well respected by the local government auditing community and was a leader in auditing. I use some of the columns that he wrote for the Local Government Auditors Quarterly to train new staff. He ran for mayor of Kansas City and won. The transition from auditor to mayor has not been easy.

Back in the game

I have been gone for a while. Some of you know the reason. I had many other things on my mind unrelated to auditing. Although, now that I think about it, it was related to outliers. When you have cancer you can look at statistics and play the odds, the outliers. Auditing posts to follow.

Lack of confidence in media

When I read a balanced and informative news article I think many times that auditing and journalism are somewhat similar. I wonder if this trend is related to this one. It makes me remember why it is important to follow audit standards and maintain our independence and objectivity.

Descriptors of an auditor

I just finished Freakonomics. It was an easy read and interesting. It would be a good text for a beginning class on auditing. At the end of the book is some "bonus material." Included is the original NY times article about Steven Levitt written by Stephen Dubner, the co-authors of Freakonmics. Mr. Dubner describes Mr. Levitt as a detective, an intuitionist, and an accidental provocateur. I think these are also descriptors of performance auditors.

Possibilities are opening up

I've followed Web 2.0 and pondered how an auditor might use this technology. Here is an exciting example. The UK Guardian has developed a new service called "Open Platform." (H/T Portland Mercury Blogtown) The newspaper is opening up its news content and data sets. Most interesting is the "Data Store." Take a look. This has great potential as a way to make auditor data sets available for public use. The Portland Mercury wonders if it is the future of journalism. I wonder if it couldn't also be in the future of auditors.

Curiousity

I've started reading Freakonomics. I thought since I found the blog so interesting that I should read the book that started it all. The drive behind this economist is similar to what I find fascinating about auditing. Many times as you start an audit you hear from the program or service some "truth" that influences operations. You test that "truth" and find out how things really work. That is what Freakonomics is about - how things really work. Sometimes it is just more detail behind the scenes that, although the "truth" still stands, what makes it true is different then you thought. But sometimes the "truth" is really wrong. Things are not as they seem. Whatever the finding, the audit is unraveling the ball of string and then trying to craft recommendations to make the program or service better.

Finding data

Auditors look for data to explain audit findings and make them meaningful, e.g. illustrate effect. Sometimes you have to create your own data. Here is an interesting example of how to think creatively and find new data in the NY Times. They used Twitter chatter to capture the conversation during the Super Bowl. (H/T Community Indicators)

Unflinching transparency

Transparency can be uncomfortable. As auditors we are dedicated to several objectives among them transparency, accountability, and creating change. Sometimes these are in conflict. Creating change requires acknowleging management efforts even though they may have fallen short. Transparency requires factual-based reporting and publication. I like to have audit results covered in the news because a larger audience is reached and transparency is increased. But many times the story is written as a conflict. I cringe a little when I read the article. But in the long run I think this is healthy. The public must know there is oversight and review.

Public Records Audit

One of the audits I was most proud of was an audit of how accessible public records were at the county. During that audit we discovered that journalists routinely "audit" public records. Now the League of Women Voters has published a toolkit for a public records "audit" that includes resources and state contacts. This makes it easy for an auditor. Best practices and criteria all in one place.

Strange analogies

Reading my mix of blogs today I was lead to post on a blog that I don't normally read. The blogger describes himself as an "idiot light." He is talking sports and drawing an analogy from the world of car maintenance. He compares himself to the "idiot light" on the dash that reminds you that there are some things that need attention. I couldn't help but think that it was also a good analogy for auditors. So, not only are we "watch dogs" but we're also "idiot lights."

How an audit progresses

This could be a diagram of an audit!

It's the road map thing again!

Two reports have come out recently at the federal level that illustrate one of the biggest weaknesses I have found in government during my career. First, the GAO has issued its second audit on the Troubled Asset Relief Program (financial bailout). I wrote about the first audit here. In summary the audit concludes the Treasury Department should increase efforts to monitor how the financial institutions are using program funds and more clearly articulate and communicate a strategic vision for the program.

The second report is from testimony to the Commission on Wartime Contracting. This Commission is mandated by Congress to study federal contracting for wartime reconstruction and performance in Iraq and Afghanistan. This was after the hard lessons learned in Iraq. In news coverage the deputy special inspector general for Iraq is quoted as saying, "Before we go pouring more money in, we really need to know what we're trying to accomplish" (in Afghanistan)."

It's the road map thing again! The hardest work in solving a problem is establishing the direction you want to go. This includes establishing guideposts, a clear direction to the location you want to arrive at, and a way of knowing when you get there. In almost every audit I've worked on we have findings about lack of clear strategy and performance measures. It's not that management doesn't know that these are important. I think its the lack of resources assigned to these activities. When you're trying to solve a problem, its difficult to spend resources that are indirectly related to that problem. But, again, if you know where you want to go and you have a map to get there, why drive with your eyes closed?

Audit snow?

Recently, Portland was immobilized for several days because of heavy snow. Government was unable to keep motorists moving and the bus system gradually was reduced to a few routes. The rationale is that Portland so rarely has snow that it does not make sense to invest heavily in snow removal equipment. Some mock Portlanders for overreacting. A reporter wondered what Barack Obama would think about Portland. Maybe there should be an audit.

Milestone

It has been over 6 months that I have been blogging. There have been times that I thought I would never have anything more to write about but then some issue or interesting thing appeared. The number of posts each month has declined but I'm not sure that is a bad thing.

I have categorized my blogs now into topics. You can find the list on the right. One thing I learned from that is I seem to be interested in three broad areas: communicating audit results, independence, and transparency. So...maybe my thoughts aren't so random.

Tone at the very top

I've written before about tone at the top, here and here, and how important it is. No matter how many rules are written, the bottom line is people. The people that do the work, from top to bottom, must be committed to ethical behavior. I believe it is in the nature of people to act ethically, but if the leadership creates a climate that does not encourage that behavior, then actions tend to drift to the margins.

Our new president, Barack Obama, took actions in his first days that signaled the new tone at the top. If you look at his first ever, yes first, executive orders and memoranda you see how the tone has changed. On the Freedom of Information Act he issued this statement, "The Freedom of Information Act should be administered with a clear presumption: In the face of doubt, openness prevails. The Government should not keep information confidential merely because public officials might be embarrassed by disclosure, because errors and failures might be revealed, or because of speculative or abstract fears." On transparency he states, "My Administration will take appropriate action, consistent with law and policy, to disclose information rapidly in forms that the public can readily find and use." He also signed several order regarding interrogation and detention of terrorists that puts us back into ethical behavior.

Thank you!

Auditing process and ignoring science

I was led to this report after reading a post by Kari Chisholm on the BlueOregon blog. The post describes the political battle that occurred over a nomination to the Department of the Interior and a hold put on that nomination by Senator Ron Wyden who asked for a report by the Office of the Inspector General (OIG). A year later in mid-December 2008 the report was issued. While the politics were interesting; from an auditor's perspective I found the methodology used to complete the investigation just as interesting.

Many times in audits, we struggle with how to audit something that is less than concrete. This audit reminds me how that is done-through the elements of a finding. The objective of the investigation was to determine if endangered species decisions had been improperly influenced by the Deputy Assistant Secretary for Fish, Wildlife, and Parks.

Criteria: The methodology was to see if policies and procedures were followed. The OIG reviewed 20 Endangered Species Act decisions. The OIG conducted 89 interviews with employees and reviewed over 20,000 emails and other documents and sent out a standardized questionnaire to all of the US Fish and Wildlife Services regions.

Condition: The investigation found a policy void and lack of guidance. One interviewee stated that there was way too much "informal policy" and that he would wake up in the morning and think, "Okay, what's the agency doing today?" The investigation also noted that this intentional failure to clarify policy had been ongoing through several administrations in order to allow the policy du jour.

Effect: As a result, the Deputy Assistant Secretary in this current administration was able to influence decisions and steer the best available science to pre-ordained ends. She caused significant harm to the integrity of the decision-making process and and its reputation with state and local agencies. The report also stated that there will be considerable cost in undoing some of the decisions and that the actions also caused considerable expenditure in taxpayer dollars for litigation.

US 2008 Financial Statement

The GAO released the 2008 financial statement for the US in mid-December. Quoting from the press release - "For the 12th year in a row, the U.S. Government Accountability Office (GAO) was prevented from expressing an opinion on the consolidated financial statements of the U.S. government—other than the Statement of Social Insurance—because of numerous material internal control weaknesses and other limitations." I did a google search on news archives to see if there was any press coverage but came up with none. I did find this coverage on the Public Financial Management Blog. The Citizen's Guide to the Federal Government's Financial Health is only 11 pages and contains highlights. It would be good if this information was reported in the news.

The emotions behind words

There is an interesting post on Freakonomics with many comments. It reminds me how words can evoke emotions. Justin Wolfers is commenting on an Op-Ed piece that concludes that despite the economic crisis. people are still happy. Using daily survey data on life satisfaction based on the Gallup-Healthways Well-Being Index, Mr. Wolfers attempts to show factually that people are not happy. He bases his argument on the results of questions about thriving, struggling, and suffering. The comments are a very interesting read. At one point it devolves into a discussion about Communism.

Several years ago in my career I was part of a team that audited Multnomah County's adult foster care regulatory program. We designed a checklist and performed surprise visits on the foster care homes to determine the quality of the care. One of the criteria we measured was the "home-like" environment. One of the strongest responses I ever got to an audit was because of our finding that some of the homes were not "home-like." It was a very emotional discussion. The manager who objected to us trying to measure that quality felt that we were applying middle-class values and could not adequately understand or measure what home might be like for other cultures.