In part 1 & 2 of this series we discussed the following:

• What are internal audits?
• What are the benefits of conducting internal audits?
• What is an audit scope?
• What is usually included in an RTO internal audit?
• Who can be an internal auditor?
• Compliance costs and risks in terms of “risk management”
• The effective internal audit function

In this part, we will cover the following areas:

• Requirements of conducting internal audits
• The quality system of an RTO

The requirement of conducting internal audits

The requirement of conducting an internal audit is not explicitly mentioned in the Standards for Registered Training Organisations (RTOs) 2015 but the requirement is for the RTO to ensure it complies with SRTOs 2015 Standards at all times, including where services are being delivered on its behalf as mentioned in Standard 2 (clauses 2.1 and 2.2) and standard 8 (clause 8.4). This applies to all operations of an RTO within its scope of registration.

The RTO is also required to:

a) systematically monitor the RTO’s training and assessment strategies and practices to ensure ongoing compliance with Standard 1, and;

b) systematically evaluate and use the outcomes of the evaluations to continually improve the RTO’s training and assessment strategies and practices. Evaluation information includes but is not limited to quality/performance indicator data collected under Clause 7.5, validation outcomes, client, trainer and assessor feedback and complaints and appeals.

Clause 8.4. The RTO provides an annual declaration of compliance with the Standards to the VET Regulator and in particular whether it:

a) currently meets the requirements of the Standards across all its scope of registration and has met the requirements of the Standards for all AQF certification documentation it has issued in the previous 12 months; and

b) has training and assessment strategies and practices in place that ensure that all current and prospective learners will be trained and assessed in accordance with the requirements of the Standards.

Systematic monitoring and evaluation is part of conducting internal audits and review processes. The “internal audit” requirements are highlighted several times in the Users’ Guide to the Standards for RTOs 2015.

The quality system of an RTO

A “Quality System” is your organisation’s blueprint.  It identifies your business model and processes, provides details about how your people will work together to get things done, and establishes specifications for performance — so that you can tell if you’re on track… or not.

Components of a Quality System

The International Organisation for Standardisation (ISO) prescribes a minimum standard for the elements of a QMS through ISO 9001:2000. (This part did not change in ISO 9001:2015). To build an ISO 9001 compliant QMS, you must:

• Identify and map processes (administrative, organisational, operational)
• Determine how processes are interrelated (that is, identify and map cross-cutting activities that span organisational boundaries)
• Plan for operations and control of these processes, recognising that the conditions and specifications for control of each of the processes may be different from one another,
• Plan to dynamically allocate resources to accommodate the demands of the operations and control of these processes,
• Apply systems thinking and describe the environment that your interdependent processes are embedded within,
• Identify mechanisms to measure, monitor, analyse and continuously improve the processes in the context of the organisation and its environment
• Establish an Action Plan for proactively deploying the QMS through the organisation, and;
• Ensure that Records are kept that track compliance to the QMS and changes that are made to the QMS itself.

The quality system in an RTO refers to the following:

• Required policies and their accompanying procedures
• Forms, templates, checklists and flowcharts to support the implementation of policies and procedures
• Central registers to track and record your compliance activities
• Compliance matrix that maps how each policy, procedure, form, template etc are related to the Standards.
• Continuous improvement processes and practices

You need a quality system in an RTO to ensure that:

• Policies and procedures are followed
• Compliant records are kept
• Documents are controlled
• Continuous improvement is implemented
• Ensure the organisation meets compliance and regulatory requirements

Stay tuned for more… our coming newsletters will cover the following topics:

• Part 4: Planning for internal audit and considerations
• Part 5: Conducting and recording an actual internal audit
• Special edition on frequently asked questions and answers on internal audits.

In the first part of this series we discussed the following: 

• What are internal audits?
• What are the benefits of conducting internal audits?
• What is an audit scope?
• What is usually included in an RTO internal audit?
• Who can be an internal auditor?

In this part, we will cover the following areas: 

• Compliance costs and risks in terms of “risk management”
• Effective internal audit function

Compliance costs and risks in terms of “risk management” 

Compliance costs for an RTO can be quite high. RTO managers are finding that cooperation across the three areas of risk management  can achieve an integrated risk management solution that is beneficial to the RTO. In risk management, control is the first line of defence; risk and control monitoring is the second line of defence and the third line of defence is self-assurance through the internal audit function. Working on all three defence lines is not new, but the concept has not been widely incorporated into standard risk management practices. In an environment that perceives risks to be increasing and resources to be limited, many managers find that aligning the efforts of the three defence lines can contribute to a systematic and effective risk management process. 

Effective internal audit function 

To be effective, the internal audit must identify a methodology for assessing other defence lines and alleviating common challenges. Barriers to the ability of the internal audit to rely on others include lack of understanding of the first and second lines of defence by the RTO personnel. Without management that is interested to identify and resolve the risks and by not having compliance controls and checklists in place, your RTO cannot assure compliance across all its operations. 

This is when you need someone completely outside the RTO to conduct an internal audit on your operations and systems and provide you with honest feedback and the mechanisms to get your RTO back on track. There is a concern in the RTO sector that relying on others for an internal audit will undermine independence and objectivity. The truth is that an internal audit can help the organisation build a more streamlined risk management process which utilises all available resources in an efficient and effective way. 

Internal audits should be a blended program 

Just relying on consultants for an internal audit is neither a “practical” nor “sensible” solution. You must participate in audit activities. You must understand all the in’s and out’s of the RTO system. You must ask questions and ask for explanations (where is that written, where is that information coming from) and seek to understand how everything is linked back to the regulatory guidelines. 

Blended internal audit program 

To develop and maintain a self-assessment internal audit program, you need to provide training to your RTO staff on internal controls and risk assessment. If you are unsure about anything read the regulatory guidelines, interpret (what does that look like?) and if unsure seek expert advice. To ensure you develop a blended internal audit program involve a professional compliance business and use their experts to develop the internal audit activities that will be performed in order to achieve overall compliance status for your RTO and an enhanced risk-control environment. 

This is Part 3 of the article, where we are discussing the different phases of the validation processes an RTO should be following to ensure they meet regulatory requirements and industry expectations.

In the previous articles, we discussed the following regarding the validation of assessment resources:

• Explanation of assessment validation
• Typical benchmarks used during the validation processes
• Stages of validation (before, during and after the assessment judgements)
• Regulatory requirements for conducting validation
• Assessment system
• Who conducts validation?
• How is validation different from moderation?
• How external consultants can help you with validation of assessment and learner resources?

In this month’s article, we will explore the regulatory requirements around validation of learner resources.

Learner resources

Learner resources are also known as “learning resources”, “training resources”, or “companion guides”. The purpose of these resources is to support learners with the underpinning knowledge required to participate in skill-based tasks. These resources include a range of activities to support the learning including, formative assessments and activities, links to further reading, workplace activities and procedures (where relevant to the qualification) etc.

Why you need to validate your learner resources

The VET regulator, ASQA does not currently prescribe the methodology Registered Training Organisations (RTOs) should use to meet the requirements of the relevant standards, training packages and accredited courses for learner resources.

But their expectations under the Standards for Registered Training Organisations, 2015, is to ensure your learner resources meet the following legislative guidelines:

Standard 1, Clause 1.3 (c): Learning resources to enable learners to meet the requirements for each unit of competency, and which are accessible to the learner regardless of location or mode of delivery.

The guidelines further state that:

Learning resources

• To ensure students are able to obtain and absorb the required knowledge and skills prior to assessment, carefully choose and plan the learning resources you will use to guide them.
• Identify these resources in your strategy to ensure you obtain full coverage of all required areas.

Therefore, we strongly recommend validating your learner resources to ensure your organisation complies with the relevant legislative requirements and guidelines.

The process of validation of learner resources

The validation of learner resources is not very different from the validation of assessment resources. All learner resources must also meet training package requirements and industry expectations.

Who can be involved in validating the learner resources

There are currently no regulatory requirements around who can participate in the validation of learner resources, however, it should be no different from the validation of assessment resources.

It should be a collective team effort and you must include the following people to validate your learner resources:

• Subject matter experts
• Trainers and assessors
• Compliance or administration manager
• Industry experts
• You may also include compliance experts as well as they usually have current and up-to-date knowledge around audit and compliance expectations and requirements.

Stages of validation for learner resources

Stage 1: Validation before using the learner resources

Validation before using the learner resources is to ensure the resources meet training package requirements, how the information is presented and the quality of the formative assessments. This is to ensure the student gains the required skills and knowledge to participate in the summative assessments later. Your review of the learner resources templates in detail ensures they are compliant and meet regulatory standards and Industry requirements.

Stage 2: Validation during or after using the learner resources

Your validation of learner resources during or after use is to ensure:

• Your resources meet client expectations
• Your resources meet training package guidelines and provide all required underpinning knowledge to your students
• Your resources are current and up-to-date in terms of the latest trends, technology and industry guidelines and practices.

In the next and final article, we will discuss:

• Why you need to keep validators information
• Why validation of assessment and learner resources should be systematic and ongoing
• How can you schedule validation
• What is statistically valid sampling
• Validation outcomes

(To be continued in the upcoming newsletter and blogs)

This is our first article in the series regarding “Internal audits”. Our main intention is to provide you with the required knowledge and skills, and equip you with the necessary resources to ensure you can audit your organisation against quality frameworks and standards effectively and efficiently.  

What are Internal audits?

Internal audits are an independent, collaborative, impartial, objective assurance and consulting activity formulated to add value and improve operations of an organisation. It assists the organisation to bring a systematic, disciplined approach to effectively evaluate, monitor and improve the effectiveness of risk management, internal control and governance processes. Internal audits act as a catalyst for a strong risk and compliance culture within an organisation.

What are the benefits of conducting internal audits?

Internal audits act as a catalyst for enhancing an organisation’s governance, risk management and controls by presenting insight and recommendations based on interpretation and examination of data and business practices and processes. There are a number of other benefits, such as:

• Audits assess an organisation’s performance and practices against the regulatory framework, guidelines and legislative/statutory instruments.
• Audits provide management of an organisation with information on the effectiveness of risk management, control and governance processes.
• Audits evaluate achievement of organisation objectives
• Audits ensure assets are safeguarded and secure.
• Audits assess efficiency, effectiveness and the economy of business activities.
• Audits review operations and processes to ensure they are protected from any fraud, malpractice or corruption
• Audits increase financial reliability and integrity
• Audits help to improve the “control environment” of the organisation
• Audits are great learning lessons for all parties involved
• Audits identify the business areas that require urgent attention
• Audits identify opportunities, accountabilities and risks
• Audits help management understand what it needs to know, when it needs to know it and how it needs to be done or implemented.
• Audits identify better ways of doing things by recommending how to improve internal controls and governance processes

What is an audit scope?

The scope of audit refers to the focus, extent, boundaries and range of the activities covered by an internal audit. It includes:

• The objectives for conducting an audit
• Nature and extent of auditing procedures and activities performed
• The organisational units that will be examined
• Location of the audit
• Time-period that will be covered
• Related activities not audited in order to define the boundaries of the audit.

The audit scope, ultimately, establishes how deeply an audit is required to be performed.

What is usually included in an RTO internal audit?

The internal audit is usually a documented process that includes the evaluation of the following:

• Quality framework and standards and legislative guidelines
• Training packages and the companion volume (including the implementation guides)
• Assessment and learner resources
• Training and assessment strategies
• Trainers and assessors
• Industry consultation and engagement and how improvements are made from them
• Recognition of prior learning and credit-transfers
• Transitioning planning, processes and procedures  
• Student certification and completion processes and procedures
• Third-party agreements and monitoring processes and procedures
• Student support, progression and welfare processes and procedures
• Student and staff Interviews and questionnaires
• Student records, student files and student data analysis
• Enrolment and pre-enrolment processes and procedures
• Marketing and advertising practices and procedures
• Regulatory compliance and governance practice
• Policies and procedures an organisation uses
• Other organisational practices and systems

Who can be an internal auditor?

An Internal auditor can be anyone who has the required knowledge, skills and experience to objectively, professionally and unbiasedly evaluate your organisation’s processes and procedures to identify opportunities for improvements.

It can be an internal staff member or an external person such as a compliance consultant.

The required knowledge, skills and experience of internal auditors will be discussed in our next edition.

(To be continued in the upcoming newsletter and blogs)  

This is Part 2 of the article, where we are discussing the different phases of the validation processes an RTO should be following to ensure you meet regulatory requirements and industry expectations.

In the previous article, we discussed the following regarding the validation of assessment resources:

• Explanation of assessment validation
• Typical benchmarks used during the validation processes
• Stages of validation (before, during and after the assessment judgements)

In this month’s article, we will explore the regulatory requirements around assessment validation.

Regulatory requirements for conducting validation

According to the Standards of Registered Training Organisations (RTOs) 2015, you are required to implement a quality review process (Clauses 1.8, 1.9, 1.10 and 1.11).

Clause 1.8a requires that the RTO’s assessment systems comply with the assessment requirements of the relevant training packages or VET accredited courses.  

Clause 1.8b requires RTOs to ensure that the evidence gathered is valid (one of the Rules of Evidence) and that assessment processes and outcomes are valid (one of the Principles of Assessment). 

These requirements must be met and demonstrated in all assessment policies, procedures, materials and tools of the RTO. Clause 1.8 primarily relates to the development (or purchase) of the RTO’s assessment resources.

Assessment validation has been strengthened in the Standards for RTOs and the requirement is to:

• Develop and implement a comprehensive plan for ongoing systematic validation of assessment that includes all training products on the RTO’s scope of delivery (Clause 1.9)
• Validate the assessment practices and judgements for each training product at least once every five years with at least 50% of products to be validated within the first three years of each five-year cycle (Clause 1.10)
• Ensure that validation is conducted by one or more suitably qualified persons, who are not directly involved in the delivery and/or assessment of the training product being validated. (Clause 1.11).
• These clauses relate primarily to the actual delivery and outcomes of the RTO’s assessment systems, including the performance of the RTO’s assessors.

Assessment system

Documents required for conducting an effective validation session, in the RTO’s assessment system, includes but is not limited to:

Validation related documents:

• Validation plan
• Validation schedule
• Validation record or validation form
• Validation register
• Validation report form
• Continuous improvement form
• Continuous improvement register
• Pre-assessment validation documents

Assessment resources:

• Unit assessment pack/student pack
• Trainer assessment pack/assessor pack
• Mapping document
• Assessment evidence according to a sample size

Other documents:

• Training and assessment strategy
• Feedback forms
• Unit of competency
• Companion volume/implementation guide
• AQF framework
• ACSF framework

You will be required to evaluate if the assessment resources meet:

• Training package requirements (application, elements and performance criteria, foundation skills, performance evidence, knowledge evidence, assessment conditions)
• Principles of assessment; fairness, flexibility, validity and reliability
• Rules of evidence; valid, sufficient, authentic and current
• The appropriate level of difficulties (AQF Level)
• Provide sufficient and clear instructions
• Record any appropriate adjustments

Who conducts validation?

Validation is a collaborative process. The team must hold collectively:

• Vocational competencies and current industry skills relevant to the assessment being validated
• Current knowledge and skills in vocational teaching and learning
• The TAE40110 Certificate IV in Training and Assessment (or its successor) or the TAESS00001 Assessor Skills Set (or its successor).
• Validators can be employees of your RTO, or you can seek external validators.

The trainer and assessor who delivered/assessed the training product being validated:

• Can participate in the validation process as part of a team
• Cannot conduct the validation on his/her own
• Cannot determine the validation outcome for any assessment judgements they made
• Cannot be the lead validator in the assessment team.

It is important to keep the records of all validation activities and validators as auditors might ask for it during audit activities and for managing continuous improvement processes at an RTO.

How is validation different from moderation?

Moderation is a quality control process aimed at bringing assessment judgements into alignment.

Moderation is generally conducted before the finalisation of student results as it ensures the same decisions are applied to all assessment results within the same unit of competency.

The requirement in the Standards to undertake validation of assessment judgements does not affect your RTO’s ability to undertake moderation activities, or any other process aimed at increasing the quality of assessment.

(ASQA, 2018)

(To be continued in the next newsletter)

This is Part 2 continuing from the previous newsletter. As discussed in Part 1 of this article, there are a number of assessment-related issues that may affect your audit outcome. You should ensure your assessment resources, therefore, meet the following criteria:

• The context and conditions of assessment. For example, an assessment tool could be developed to cater for a particular language, literacy and numeracy requirements, the learner’s workplace experience or other learner needs that require reasonable adjustment.
• The context of the assessment may also take into account assessments already completed, and the competencies demonstrated in these assessments. By looking at the context, you can consider the conditions under which evidence for assessment must be gathered.
• All activities are conducted adequately using the required:
  • equipment or material requirements
  • contingencies
  • specifications
  • physical conditions
  • relationships with team members and supervisors
  • relationships with clients/customers
  • timeframes for completion.
• Assessment methods or tasks are suitable to the requirements of the units of competency and students are assessed on the tasks and activities according to the requirements of the training package.
• The language used is simple English
• The evidence required to make a decision of competency is clearly outlined
• The types of activities and tasks student need to perform are clearly outlined
• The level of performance required for each assessment activity is clearly outlined
• Adequate exposure to workplace conditions, including appropriate simulated environments
• Sufficient knowledge-based assessment tasks and activities such as written questions and case studies etc.
• Sufficient practical based assessment tasks and activities such as projects, role plays, workplace tasks and observations etc.
• Assessment resources are error-free and free from any grammar, copyright or plagiarism issues

It is a wise decision to get your training and assessment strategies and resources validated by independent industry experts to get honest feedback and an unbiased opinion.

In this article, we will discuss different phases of validation processes that you should be following in your RTO to ensure you meet regulatory requirements and industry expectations.

Validation of RTO assessment resources

You must validate all your assessment resources to ensure they meet the principles of assessment, rules of evidence, training package requirements, regulatory guidelines and Industry expectations.

Explanation of assessment validation:

Validation is a process of checking that the assessment tools, methods, judgements, evidence and processes to ensure that the training product meets:

• ​Principles of Assessment – i.e. valid, reliable, flexible and fair
• Rules of Evidence – i.e. valid, authentic, current and sufficient
• The judgment made by the trainer/assessor is benchmarked with colleagues or industry experts
• There is sufficient evidence to support the judgment of the trainer/assessor
• Whether the requirements of the Training Package or accredited course have been met.

Typical benchmarks used during the validation process include:

• National training package which are developed by Skills Service Organisations (SSOs)/ Industry Reference Committees (IRCs) and can be found on the training.gov.au website.
• Units of competency which consist of competency standards and need to be unpacked so that those validating the assessments can compare the actual competency against the tools being validated.
• Industry standards and consultation will vary, and these standards form the basis of the skills and knowledge required to perform work roles.
• AQF Guidelines and Framework
• Information provided to candidates, assessors and third parties
• Legislation relevant to the assessment such as privacy, health and safety, anti-discrimination, copyright law and so on.

Validation occurs through different stages:

Stage 1: Validation before assessment judgements i.e. pre-validation of assessment resources

Validation before assessment judgements are made; look at the design of the assessment activities, if it meets the training package requirements, how the instructions for tasks or questions are presented and the benchmarks against the learner performance. This is where the mapping is undertaken. You review the assessment tool templates in detail to ensure they are compliant and meet regulatory standards and Industry requirements.

Stage 2: Validation during assessment

Validation during assessment is looking at the actual benchmarking answers or performance the learner has provided, and making a judgement with another assessor, either together or separately. This often is coordinated within assessors who undertake training and assessment of the same vocational area. This process was also known as moderation and always remember prevention is always better than the cure, therefore, any issues identified at stage 1 or stage 2 should be eliminated as soon as possible and gaps should be filled with gap-analysis to ensure your organisation is bullet-proof.

The requirement in the Standards to undertake validation of assessment judgements (post validation) does not prohibit your RTO from undertaking moderation activities, or any other process aimed at increasing the quality of assessment.

Stage 3: Validation post assessment (i.e. post validation)

Validation post assessment concentrates on the learners’ performance and their responses to questions, the actual assessment decision that was made, the task and processes that align to the assessment, any feedback from students, and the reporting processes.

The purpose of this post-assessment validation is to verify the validity and consistency of assessment decisions to bring assessment judgements and standards into alignment.

It is a process that ensures the same standards are applied to all assessment results within the same Unit(s) of Competency. It is an active process in the sense that adjustments to assessor judgements are made to overcome differences in the difficulty of the tool and/or the severity of judgements. It aims to ensure assessors have a common understanding of the unit requirements

It involves checking that your assessment tools have produced valid, reliable, sufficient, current and authentic evidence, enabling RTO to make reasonable judgements that the training package requirements have been met.

(To be continued in the upcoming blogs) 

It is important to look into ASQAs 2017 report that shows:

• Around 72% of RTOs FAIL audit on Assessment
• Approximately 50% of those FAIL to be able to rectify their assessment tools on resubmission under the OLD audit mode

According to the new audit model:

• There may be NO opportunity to rectify critical non-compliances
• Initial registration clients with critical non-compliances are unlikely to get an opportunity to rectify and potentially would be unable to reapply
• Registered RTOs risk sanctions, conditions, or even worse cancellations for critical non-compliances on the first audit

There are a number of assessment-related issues that may affect your audit outcome. You should ensure your assessment resources meet the following criteria:

• Assessment resources have sufficient and clear information regarding what, when, how, where, why for your assessment template and all assessment tasks and activities. 
• Assessment resources have robust benchmarking and/or trainers guide. 
• Assessment resources are allowing the trainer/assessor to assess the skills and knowledge of students through different assessment tasks over a period of timeto ensure “consistency” and “sufficiency” 
• Each and every question and assessment task has very clear guidelines around what is expected from the students in terms of both “quantity” and “quality” 
• You have customised the off-the-shelf resources according to your RTO needs and requirements and not using them “as-it-is”
• Your assessment resources are written by Industry experts with subject matter experts and are “Industry-relevant” and “current”
• Your assessment resources address all requirements of the training packaging rules
• Your assessment resources have detailed and valid performance checklists/observation checklists for assessing and observing the students before, during and after any skill assessment activity or workplace task 
• Your trainers and assessors gather sufficient, valid evidence for competency assessment
• Your organisation offers appropriate simulated environments for conducting assessments 
• The authenticity of assessment, particularly in distance and online delivery is established and maintained