This article by Clare Dallat of the Outdoor Education Group and Preston Cline of the Wharton Center for Leadership and Change Management at the University of Pennsylvania explores how the actual approach taken before, during and after the audit, may go a long way in determining the outcome, and the effectiveness of the report to your staff, clients and organization. It also offers concrete suggestions as to how to structure audits to ensure consistency each time one is conducted.
How do you know, for a fact, that you are providing the services your marketing states you are providing? Internal Risk Management audits are an effective evaluation tool for determining current risk management practices. The recommendations that you enact following a review can provide a clear recordof the measures that your program has taken to improve its risk management practices, in addition to maintaining the integrity of the mission of the organization.
This paper will argue that the actual approach taken before, during and after the audit, may go a long way in determining the outcome, and the effectiveness of the report to your staff, clients and organization. It will also offer some suggestions as to how to structure audits to ensure consistency each time one is conducted.
Start with your language
The very term ‘audit’ can be enough to initiate feelings of being ‘watched’ or ‘judged’ into the minds of the staff involved. Whether the review takes the form of a ‘surprise visit’ or a planned event, the feeling that a timed ‘test’ which needs to be passed after which, things can then get back to ‘normal,’ is not uncommon.
It seems then, paradoxical that the word ‘audit’ itself is defined as “To evaluate or improve safety, efficiency or the like” (Webster’s New Universal Unabridged Dictionary, 1996, Random House). We will offer several suggestions as to how to address this disconnect, and have internal audits viewed by all your stakeholders as an integrated tool in your organization’s overall risk management practices.
Phase One: Identify the Metric
To identify the metric, means to find out what you will be measuring your
*By ecological we simply mean to say: “Being able to see all of the variables within a risk management system and their interactions.”
observations against. Remember to be guided by the primary purpose of an internal audit: “Does your organization do in the field, what it says it does in its documentation?” By having this as your primary question, you have an active standard to measure against.
This standard should be taken from all documentation relating to your program whether it is standard operating procedures, marketing material, website information or program specific brochures. Starting with your mission, these materials should all be aligned so the message delivered in each is consistent and your staff are clear on their principle programmatic goals. Put another way, would a participant’s parent reasonably understand why you are making a particular decision regarding the welfare of their child?
“Collaborative” versus the “Adversarial” or “Punitive” Approach
A key consideration when performing internal audits is to be very clear about what exactly you are auditing. Often the resistance that managers receive from field staff have to do with the fear of the unknown. They feel as though they are taking a test, and no one has told them what to study for, or how they are going to be graded. Do not underestimate the importance of remaining clear and objective throughout the auditing process; ambiguity only increases resistance.
Maintaining Perspective
For many reasons, as managers we can often enter audits with preconceived notions of particular aspects of the program, or the performance of certain staff. Preconceived ideas about what you may find can blind your objectivity, and can result in missing key underlying issues.
When people feel that they are being judged, or tested on their performance in relation to how ‘safe’ they are, the audit and its purpose immediately becomes about them- are they ‘safe?’ Are they doing the ‘right’ things?
A more objective approached is called for if you, as Program Director are to gain maximum information from an internal audit, while also maintaining the support of the staff or volunteers involved in the audit.
Phase Two: The Ecological Approach
This approach takes the person out of the audit and looks at the larger systems of the program’s operations (specific instances where you may witness immediate safety concerns will be discussed further in this article). To do this you first need to identify the larger programmatic variables, such as client, staff, equipment, environment etc. (Adventure Incorporated developed a tool called “Adventure Management Systems®” specifically designed for this purpose.) Once you have identified the variables specific to your program, you need to determine how those variables interact. It is the integrity of these variables and their interactions that will ultimately determine the ecological health of your organization. As you will quickly realize, it is often impossible to look at the entire system at once, therefore you will need to determine the areas that you will focus on, and what questions you will need to answer.
What are the over all questions?
This approach has the same underlying objective—i.e. are you doing what you say you are doing? However, it involves looking much deeper than merely ‘checking the boxes.’ Are the systems that you have in place as an organization, supporting your staff and clients achieve the educational outcomes you are aiming for? This approach actively takes the ‘blame’ or singular responsibility away from the actual staff members being audited.
By instilling a culture in your organization that accepts audits as an essential part of an overall risk management strategy, which are conducted to review the overall systems in place, staff will be much more likely to welcome them and not be overly intimidated by them. This in turn will enable you, as the auditor, to gain more valuable information from them.
Phase Three – Interviewing the staff
By adopting an ecological approach, the question of whether or not the staff member had the required items in their first aid kit, for example, moves away from a simple “Yes” or “No” answer (i.e. it’s all about them), to an “If not, why not?” question (i.e. it’s about the systems in place). In other words, did we have the supplies available? Did staff have time needed to restock their first aid kits? Are the kits being used so often it is hard to restock them? Do we have the right type and size first aid kits for the needs of the program? By approaching audits this way, we are better placed to identify the more complex issues that may get missed by solely utilizing an approach that is based on the “Yes or No” answers provided by the person(s) being audited.
Furthermore, by starting with the assumption that your staff are doing the best they can, you first identify or eliminate program variables as contributors to error. This then allows you to focus on the “human factors.” In other words, if you have identified a staff member as underperforming, you have already ruled out all the other excuses, all that is left then, is their actions. The conversation is less about emotion and more about fact.
Timing is everything – who and when….?
Internal audits should be conducted by staff knowledgeable in the policies and procedures of the organization. The person or persons conducting the audit should have a clear understanding of the type of program and the stakeholders involved. For these reasons, program managers/directors or staff responsible for risk management generally conduct internal audits.
Consider the timing of your on-site visit—is it likely to have a negative impact on the educational objectives of the program? Is it taking the focus of the instructor away from the group? Is your presence distracting the students?
Are they expecting you? Better acceptance is achieved when staff are informed of the upcoming audit immediately prior to the program. Staff are more likely to teach in their natural manner with the element of surprise missing. If you are holding on to the belief that a ‘surprise visit’ might ‘catch’ the staff doing something wrong, you may want examine your hiring practices. If, at some level, you cannot trust your staff, catching them in the wrong, is not the solution.
The auditor needs to have credibility in the eyes of the staff. This is achieved by only questioning the systems in place, not making rash judgments about the correctness of any one person’s actions and always maintaining a respectful attitude.
Accept that you will observe and uncover errors.
When we talk about complex systems, such as organizations involving more than one person, we are going to encounter errors. THIS CANNOT BE AVOIDED. More importantly, however, is the fact that identifying errors actually enables you to identify weaknesses in the system and therefore find ways to make the program even stronger. Your goals for a review should be consistent with this idea.
The Documentation Stage
This stage involves reviewing the documentation surrounding the specific program being audited. Is this documentation consistent with what you say it will be in your program literature? This stage should ideally be conducted prior to the physical on-site part of the audit.
The Interviews
This stage of the review involves speaking with staff members and (if appropriate) participants. Again, the questioning here should serve the purpose of evaluating the level of consistency between what is documented and what is occurring in the field.
It is important to try and keep this process as ‘natural’ as possible—speak with the staff member and let them know why you’re there—would they like to introduce you to the group? By keeping this process as non-invasive and natural as possible, you are better serving your clients and the outcome of the review.
Ethical Issues (what if you witness scary stuff?)
As a staff member, you have an obligation to manage risk to the best of your ability. Due to the human element, mistakes will happen and these can sometimes occur while you are conducting a review. Your first priority is attempting to deal with the immediate risk to your stakeholders. Depending on the situation, this may mean you stepping in and taking charge, or it may mean taking the staff member aside and empowering them to handle the situation. The less intrusive way is obviously favorable.
Phase Four – Writing the Report
This stage of the review involves documenting what you observed in an objective fashion. Having a template that matches up documentation with the questions you asked in the review process will help ensure this. Where were the inconsistencies between documentation and practice? It is essential that the report be written in a way that is easy to read, clearly outlines what was observed and can be made sense of five years from now.
The report should also include a “Recommendations” section—simply put this means, based on your observations during the review and the inconsistencies observed, what do you recommend being done to rectify the issues? Depending on level of risk and exposure to your organization, these should be prioritized in order of importance. It is critical to understand that these recommendations are not based solely on your opinions, but rather need to be backed up with industry standards, best practices and laws or regulations.
Delivering the Report
Depending on your organization and its structure, your report may go to your Risk Management Committee, your supervisor or the Executive Committee. It is important to identify who will view the report and who has responsibility for enacting the recommendations. This will ensure that the process is transparent and that those people identified to enact recommendations are accountable.
External Audits – Where do they fit in…?
Effective risk management should always include auditing of the program by an external reviewer. This allows the organization an opportunity to objectively affirm the successful aspects of their program as well as identifying and questioning components that can be improved. (for more information on professional risk managers see www.outdoor.com)
Conclusion
Having a well-documented, consistent and systematic approach to regular internal audits can assist your organization in managing operational risk. By ensuring that your primary objective in these audits is theevaluation of the actual systems that your organization has in place to support all stakeholders, to achieve your mission, you will be better placed to obtain not just more valuable information from them, but also the willingness of your staff to be a participant and agent for growth, as opposed to a potential blocker due to fear of subjective judgment on their ability to be “safe.”
About the Authors
Clare Dallat is the Head of Risk Management at the Outdoor Education Group, located in southeastern Australia. Her role involves managing operational risk for this large non-profit outdoor education program that provides 3-33 day programs for school students, as part of their integrated school curriculum. Clare has worked in the field of Outdoor/Adventure Education for the past 10 years, spanning roles from adventure trip leader to program director in the eastern United States and Australia. She has led and administered programs for 7 to 84 year old clients. Clare gained a BA (Hons) from the University of Wales, Lampeter, and will shortly commence an MSc in Risk, Crisis and Disaster Management from the University of Leicester, UK.
The Outdoor Education Group(OEG) is an non-profit Outdoor Education provider, serving the independent schools sector in southeastern Australia. OEG is the largest outdoor education provider in Australia, currently working with over 80 schools and 22,000 students a year. OEG works in partnership with each of their client schools to identify and deliver the pre-determined desired educational outcomes, which ultimately focus on the sense of self; others and the natural world, and the interaction between these elements.
The Outdoor Education Group, National Base
www.oeg.net.au
Preston B. Cline Initially trained as wilderness instructor Preston began his career in the late 1980’s leading 60-day wilderness trips with adjudicated youth. Prior to founding Adventure Incorporated he worked as a Risk Manager in both Adventure Education Programs and Summer Camps. In 1999, Preston founded Adventure Incorporated, and as of 2004, had participated in over 60 risk management reviews among a diverse set of clients that ranged from Australia to Alaska. He has a B.S. from Rutgers University, with a minor in Professional Youth Work, as well as a M. Ed from the Harvard University Graduate School of Education, where he completed formal research in the etymology and evolution of risk. Preston founded Adventure Incorporated, a Risk Management Training & Consulting firm focused on the field of Adventure Education. Preston is currently the Associate Director of the Graduate Leadership Ventures Program at the Wharton Center for Leadership and Change Management at the University of Pennsylvania. Preston continues to lecture and publish frequently on the subject of Risk and Risk Management.
