Shining a light

I've always felt that our job of shining a light had intrinsic value.  At least someone was looking and by our "shining a light" maybe others would too.  Here is another who has the same believe in a different milieu.

Social Media Performance Measures

Here is an interesting story about the Washington Post trying to adjust to the digital age. "Employees are schooled in the lingo of web traffic."  (p.2)  Interesting things to note:

• 35 different daily reports track traffic to different parts of the web site
• Editors receive a performance alert each day telling them whether the site is on track to meet its traffic goals
• Even if traffic is low to some pages, if the online visitor is from a government suffix web site, who they consider a main audience, they will support those pages

Makes me wonder how government should be looking at traffic.

Facts and stories

When auditees read the first draft of an audit a common response is that we did not talk about anything good that they have done. Our response is usually that our job is to identify problems, analyze why they occurred and how the situation could be improved. We are constrained by standards that require us to be fact-based. Establishing facts takes time. To include in the report an analysis of things that were going right would require the same standards be met. Since audit work is expensive, because of the time it takes to establish facts, it would not be cost effective to also analyze the successful parts of a service or program. However, we do try to add context so that there is an appreciation of the work that is being done to provide the service.

But from the perspective of a journalist trying to find a story that includes protagonist, antagonist, and facts that will capture a reader's interest, our reports are not hard hitting enough. When I have campaigned as an auditor, the common response was always that people had no idea there were auditors doing the kind of work we do. Auditors face a quandary, work for change or work to create an audit report that will capture a journalist's eye so that it will be reported and the public will see that there is accountability. Many of our audits fall under the radar because management has agreed with our findings and our recommendations. Or the audit area is just not interesting or big dollar enough. The choice is usually to work for change and not necessarily news coverage, so many stories never get told.

Elements of an Audit

Several times I've written about how the work that auditors and journalists do is sometimes similar. Here is a news story by Henry Esteve that begins with a simple questions. Were the estimates of the cost of a tax credit accurate? Estimates are just that, estimates. Costs may turn out to be lower than higher than estimated. But it is good management to check and adjust a program depending upon what unfolds.

It's called closing the loop. Many audits address this management area. In my auditing experience, government service providers intensely pay attention to the design and organization of services. Just as much work and effort should also be placed in monitoring the service and evaluating whether it was successful.

This lengthy unfolding news story also illustrates a difference between journalism and auditing. News stories are short, not comprehensive, and accumulate over time. Turns out that state energy subsidies have been an ongoing story in 2009 in the Oregonian. The most recent story linked above that caught my eye was followed by at least 10 more stories and commentary including a story that the Governor would roll back subsidies.

Journalism is designed to foster accountability and transparency by shining a light on government practices. So is auditing, but audits are also intended to analyze the source of the problem and offer recommendations for change. Audits have a dual purpose - accountability and improvement of government services. The article cited a potential cause for the under estimated costs - that they had been low balled to gain legislative approval. If I had been conducting an audit I would have wanted to look at management practices that allowed estimates to be manipulated, if that is indeed what happened. What an auditor wants is to shine a light on the problem but also to have management rebuild a system that will protect the accuracy of data and prevent the problem in the future.

I can't imagine

I am signed up for several Google alerts relating to auditing and audits. One such alert recently took me on an interesting journey. The original alert took me to this website (?) blog (?) about the findings of an auditor in Zimbabwe. According to the About Us page, "Zimbabwe Metro online (zimbabwemetro.com) is an aggregated web log that covers a wide-range of topics, including sections devoted to politics, entertainment, media, lifestyle and business.We also feature hyperlinks to various news sources and columnists. Our roster of bloggers includes many people from Zimbabwe’s politicians to our extensive network of prominent writers. We are ranked the most most visited news weblog on Zimbabwe News by Alexa Internet."

Intrigued that an auditor would be exposing government corruption to such a degree, I tried to do some research. Using the NY Times search function I discovered an interesting archive of news stories about Zimbabwe. It seems that the president of Zimbabwe won a run off election and was inaugurated to his sixth term in June 2008. His opponent withdrew from the race because "state-sponsored enforcers were beating and killing his followers." More recent articles document a cholera epidemic and devastating economic and agricultural policies that leave the population starving. In February 2009, the president's opponent was sworn in as the prime minister to try and bring unity to the government. And just yesterday President Obama presented the 2009 Robert F. Kennedy Human Rights Award to publicly recognize Women of Zimbabwe Arise, or Woza, a grassroots effort to speak out against injustice in the country.

And in all of this an auditor is operating. I tried to search for a website for the auditor or the audit but was unsuccessful.

Measuring Audit Results

Auditors regularly measure results of programs and services and also critique the agency's performance measures. Because of this, I have always published an annual report for my office that includes key measures such as reports per FTE, implementation rate, and hours per audit. If we require our agency to report performance, it makes sense that the audit office should also report. One measure I struggle with is outcome. What was the effect of our audits? The most common measure I have seen in other audit offices is annual savings attributed to audits. Since my office conducts performance audits, often we cannot measure cost savings. Generally, recommendations are directed towards improving effectiveness.

I have commented previously on what I consider to be similarities between the fields of journalism and auditing. An Oregonian article last Saturday about last minute legislative action included a sidebar that linked new laws to the Oregonian's investigative reporting. I think that this might offer a potential solution to measuring the effect of audits. Perhaps I can document concrete action taken by a Department or the Council that can be linked to audit findings. So, rather than cost savings, maybe I could categorize actions attributed to audit findings and then quantify them.

Assigning blame where it belongs

Steve Duin, a columnist in the Oregonian, wrote today about a settlement with the family of a man who died as a result of his actions and those of public officials after his arrest. Many were involved including the arresting officers, jail staff, and the ambulance personnel. The columnist worries that without the exercise of a trial that will assign blame, there will be no motivation to change. I think there should be accountability in the system, but does assigning blame motivate employees to change?

An audit steps back and looks at the structure that allowed employees to make bad decisions. When I meet with auditees, I always tell them we are auditing a program's or service's performance, not the employee's performance. What was the tone at the top or the climate that allowed police to believe that a man urinating in a trendy neighborhood required a forceful police reaction? Are police trained to recognize that sometimes a person's actions can be the result of mental illness? Do personnel treat all medical emergencies with the same urgency and care? What is the procedure for medical personnel at the jail to make medical decisions? Should medical personnel at the jail and police reassess a person's medical condition before transporting to a hospital to determine the current urgency of the situation? Training, procedures, and the management climate can make for better outcomes. An audit would review all of those and assign blame, but not to the individual, instead to the services that failed. Then maybe there would be change.

Lack of confidence in media

When I read a balanced and informative news article I think many times that auditing and journalism are somewhat similar. I wonder if this trend is related to this one. It makes me remember why it is important to follow audit standards and maintain our independence and objectivity.

I was right

Front page of the Oregonian today is a second story about Social Security. Yesterday I wrote about this story and speculated that the cause of the incredibly slow system could be lack of funding or over-proceduralization. The headline of the story - "Social Security backlog grows from lack of cash." Another difference between journalists and auditors - auditors get to write their complete findings in one report. Journalists have to write a series over several days to tell the complete story.

Journalists and Auditors

I've always thought that the work of journalists and auditors is similar in many ways. Reading the Sunday Oregonian confirmed that today. Both write public reports that shine a light on a situation that needs changed. Both seek facts and have standards to ensure quality of the report. The auditor is looking for the elements of a finding - condition, criteria, effect, cause, and recommendations. What is happening, what should happen, what is the result, why did it happen, and what should be changed.

First case in point - "Getting disability payments can be a fight to the death" on the front page. The reporters describe the struggle of several individuals over many years to received social security disability payments. A few die before payments are received (condition). Several individuals talk about how they paid into the system and believed that the government would support them when they legitimately needed help (criteria). The length of time of the struggle results in some going on welfare, becoming homeless, and dying (effect). What the story lacks is the cause and recommendations, something an audit would provide. It alludes to one cause - the baby boomer generation and its demand on the system. But perhaps it is also cutbacks in funding or a system over-proceduralized because of concerns about abuse.

Second case in point - "State parole board finds itself up for review" again on the front page of the Oregonian. This has been an ongoing story at the Oregonian and began with a decision by the Parole Board to release an offender who had been convicted of one rape but had admitted several others. In September the Board voted to release the offender and cut his 30 year sentence in half (condition). Because of a mix-up they failed to notify the victim so that she could attend the hearing (criteria). The Board conducted another hearing but again voted for release. The victim sued and another hearing was conducted. Since then several other victims have come forward and there has been a public outcry. The fear is the possibility of future victims. (effect). The series of stories have highlighted what may be some causes - poor notification of victims, hearings that were conducted inside the prison, and a closed-door policy. An audit might take on more systemic causes such as overall lack of transparency and whether the Board should be appointed.

Between the work of the journalist and the auditor, which is more likely to effect real change in government? Both. The work of journalists can put incredible pressure on government to change and have significant results. I think the difference though is time. Journalists face a deadline. Auditors can spend time in the audit process to craft realistic recommendations and to work with the auditee and encourage change. The difference is that found between a surgeon and your physician. One removes a diseased part of your body and the other works to improve habits and prevent or delay disease.