This is Part 3 of the article, where we are discussing the different phases of the validation processes an RTO should be following to ensure they meet regulatory requirements and industry expectations.

In the previous articles, we discussed the following regarding the validation of assessment resources:

• Explanation of assessment validation
• Typical benchmarks used during the validation processes
• Stages of validation (before, during and after the assessment judgements)
• Regulatory requirements for conducting validation
• Assessment system
• Who conducts validation?
• How is validation different from moderation?
• How external consultants can help you with validation of assessment and learner resources?

In this month’s article, we will explore the regulatory requirements around validation of learner resources.

Learner resources
Learner resources are also known as “learning resources”, “training resources”, or “companion guides”. The purpose of these resources is to support learners with the underpinning knowledge required to participate in skill-based tasks. These resources include a range of activities to support the learning including, formative assessments and activities, links to further reading, workplace activities and procedures (where relevant to the qualification) etc.

Why you need to validate your learner resources
The VET regulator, ASQA does not currently prescribe the methodology Registered Training Organisations (RTOs) should use to meet the requirements of the relevant standards, training packages and accredited courses for learner resources.

But their expectations under the Standards for Registered Training Organisations, 2015, is to ensure your learner resources meet the following legislative guidelines:

Standard 1, Clause 1.3 (c): Learning resources to enable learners to meet the requirements for each unit of competency, and which are accessible to the learner regardless of location or mode of delivery.

The guidelines further state that:

Learning resources

• To ensure students are able to obtain and absorb the required knowledge and skills prior to assessment, carefully choose and plan the learning resources you will use to guide them.
• Identify these resources in your strategy to ensure you obtain full coverage of all required areas.

Therefore, we strongly recommend validating your learner resources to ensure your organisation complies with the relevant legislative requirements and guidelines.

The process of validation of learner resources
The validation of learner resources is not very different from the validation of assessment resources. All learner resources must also meet training package requirements and industry expectations.

Who can be involved in validating the learner resources
There are currently no regulatory requirements around who can participate in the validation of learner resources, however, it should be no different from the validation of assessment resources.

It should be a collective team effort and you must include the following people to validate your learner resources:

• Subject matter experts
• Trainers and assessors
• Compliance or administration manager
• Industry experts
• You may also include compliance experts as well as they usually have current and up-to-date knowledge around audit and compliance expectations and requirements.

Stages of validation for learner resources
Stage 1: Validation before using the learner resources
Validation before using the learner resources is to ensure the resources meet training package requirements, how the information is presented and the quality of the formative assessments. This is to ensure the student gains the required skills and knowledge to participate in the summative assessments later. Your review of the learner resources templates in detail ensures they are compliant and meet regulatory standards and Industry requirements.

Stage 2: Validation during or after using the learner resources
Your validation of learner resources during or after use is to ensure:

• Your resources meet client expectations
• Your resources meet training package guidelines and provide all required underpinning knowledge to your students
• Your resources are current and up-to-date in terms of the latest trends, technology and industry guidelines and practices.

In the next and final article, we will discuss:

• Why you need to keep validators information
• Why validation of assessment and learner resources should be systematic and ongoing
• How can you schedule validation
• What is statistically valid sampling
• Validation outcomes

(To be continued in the upcoming newsletter and blogs)

This is Part 2 continuing from the previous newsletter. As discussed in Part 1 of this article, there are a number of assessment-related issues that may affect your audit outcome. You should ensure your assessment resources, therefore, meet the following criteria:

• The context and conditions of assessment. For example, an assessment tool could be developed to cater for a particular language, literacy and numeracy requirements, the learner’s workplace experience or other learner needs that require reasonable adjustment.
• The context of the assessment may also take into account assessments already completed, and the competencies demonstrated in these assessments. By looking at the context, you can consider the conditions under which evidence for assessment must be gathered.
• All activities are conducted adequately using the required:
  • equipment or material requirements
  • contingencies
  • specifications
  • physical conditions
  • relationships with team members and supervisors
  • relationships with clients/customers
  • timeframes for completion.
• Assessment methods or tasks are suitable to the requirements of the units of competency and students are assessed on the tasks and activities according to the requirements of the training package.
• The language used is simple English
• The evidence required to make a decision of competency is clearly outlined
• The types of activities and tasks student need to perform are clearly outlined
• The level of performance required for each assessment activity is clearly outlined
• Adequate exposure to workplace conditions, including appropriate simulated environments
• Sufficient knowledge-based assessment tasks and activities such as written questions and case studies etc.
• Sufficient practical based assessment tasks and activities such as projects, role plays, workplace tasks and observations etc.
• Assessment resources are error-free and free from any grammar, copyright or plagiarism issues

It is a wise decision to get your training and assessment strategies and resources validated by independent industry experts to get honest feedback and an unbiased opinion.

This is Part 2 of the article, where we are discussing the different phases of the validation processes an RTO should be following to ensure you meet regulatory requirements and industry expectations.

In the previous article, we discussed the following regarding the validation of assessment resources:

• Explanation of assessment validation
• Typical benchmarks used during the validation processes
• Stages of validation (before, during and after the assessment judgements)

In this month’s article, we will explore the regulatory requirements around assessment validation.

Regulatory requirements for conducting validation

According to the Standards of Registered Training Organisations (RTOs) 2015, you are required to implement a quality review process (Clauses 1.8, 1.9, 1.10 and 1.11).

Clause 1.8a requires that the RTO’s assessment systems comply with the assessment requirements of the relevant training packages or VET accredited courses.  

Clause 1.8b requires RTOs to ensure that the evidence gathered is valid (one of the Rules of Evidence) and that assessment processes and outcomes are valid (one of the Principles of Assessment). 

These requirements must be met and demonstrated in all assessment policies, procedures, materials and tools of the RTO. Clause 1.8 primarily relates to the development (or purchase) of the RTO’s assessment resources.

Assessment validation has been strengthened in the Standards for RTOs and the requirement is to:

• Develop and implement a comprehensive plan for ongoing systematic validation of assessment that includes all training products on the RTO’s scope of delivery (Clause 1.9)
• Validate the assessment practices and judgements for each training product at least once every five years with at least 50% of products to be validated within the first three years of each five-year cycle (Clause 1.10)
• Ensure that validation is conducted by one or more suitably qualified persons, who are not directly involved in the delivery and/or assessment of the training product being validated. (Clause 1.11).
• These clauses relate primarily to the actual delivery and outcomes of the RTO’s assessment systems, including the performance of the RTO’s assessors.

Assessment system

Documents required for conducting an effective validation session, in the RTO’s assessment system, includes but is not limited to:

Validation related documents:

• Validation plan
• Validation schedule
• Validation record or validation form
• Validation register
• Validation report form
• Continuous improvement form
• Continuous improvement register
• Pre-assessment validation documents

Assessment resources:

• Unit assessment pack/student pack
• Trainer assessment pack/assessor pack
• Mapping document
• Assessment evidence according to a sample size

Other documents:

• Training and assessment strategy
• Feedback forms
• Unit of competency
• Companion volume/implementation guide
• AQF framework
• ACSF framework

You will be required to evaluate if the assessment resources meet:

• Training package requirements (application, elements and performance criteria, foundation skills, performance evidence, knowledge evidence, assessment conditions)
• Principles of assessment; fairness, flexibility, validity and reliability
• Rules of evidence; valid, sufficient, authentic and current
• The appropriate level of difficulties (AQF Level)
• Provide sufficient and clear instructions
• Record any appropriate adjustments

Who conducts validation?

Validation is a collaborative process. The team must hold collectively:

• Vocational competencies and current industry skills relevant to the assessment being validated
• Current knowledge and skills in vocational teaching and learning
• The TAE40110 Certificate IV in Training and Assessment (or its successor) or the TAESS00001 Assessor Skills Set (or its successor).
• Validators can be employees of your RTO, or you can seek external validators.

The trainer and assessor who delivered/assessed the training product being validated:

• Can participate in the validation process as part of a team
• Cannot conduct the validation on his/her own
• Cannot determine the validation outcome for any assessment judgements they made
• Cannot be the lead validator in the assessment team.

It is important to keep the records of all validation activities and validators as auditors might ask for it during audit activities and for managing continuous improvement processes at an RTO.

How is validation different from moderation?

Moderation is a quality control process aimed at bringing assessment judgements into alignment.

Moderation is generally conducted before the finalisation of student results as it ensures the same decisions are applied to all assessment results within the same unit of competency.

The requirement in the Standards to undertake validation of assessment judgements does not affect your RTO’s ability to undertake moderation activities, or any other process aimed at increasing the quality of assessment.

(ASQA, 2018)

(To be continued in the next newsletter)

It is important to look into ASQAs 2017 report that shows:

• Around 72% of RTOs FAIL audit on Assessment
• Approximately 50% of those FAIL to be able to rectify their assessment tools on resubmission under the OLD audit mode

According to the new audit model:

• There may be NO opportunity to rectify critical non-compliances
• Initial registration clients with critical non-compliances are unlikely to get an opportunity to rectify and potentially would be unable to reapply
• Registered RTOs risk sanctions, conditions, or even worse cancellations for critical non-compliances on the first audit

There are a number of assessment-related issues that may affect your audit outcome. You should ensure your assessment resources meet the following criteria:

• Assessment resources have sufficient and clear information regarding what, when, how, where, why for your assessment template and all assessment tasks and activities.
• Assessment resources have robust benchmarking and/or trainers guide.
• Assessment resources are allowing the trainer/assessor to assess the skills and knowledge of students through different assessment tasks over a period of timeto ensure “consistency” and “sufficiency”
• Each and every question and assessment task has very clear guidelines around what is expected from the students in terms of both “quantity” and “quality”
• You have customised the off-the-shelf resources according to your RTO needs and requirements and not using them “as-it-is”
• Your assessment resources are written by Industry experts with subject matter experts and are “Industry-relevant” and “current”
• Your assessment resources address all requirements of the training packaging rules
• Your assessment resources have detailed and valid performance checklists/observation checklists for assessing and observing the students before, during and after any skill assessment activity or workplace task
• Your trainers and assessors gather sufficient, valid evidence for competency assessment
• Your organisation offers appropriate simulated environments for conducting assessments
• The authenticity of assessment, particularly in distance and online delivery is established and maintained

In this article, we will discuss different phases of validation processes that you should be following in your RTO to ensure you meet regulatory requirements and industry expectations.

Validation of RTO assessment resources

You must validate all your assessment resources to ensure they meet the principles of assessment, rules of evidence, training package requirements, regulatory guidelines and Industry expectations.

Explanation of assessment validation:

Validation is a process of checking that the assessment tools, methods, judgements, evidence and processes to ensure that the training product meets:

• ​Principles of Assessment – i.e. valid, reliable, flexible and fair
• Rules of Evidence – i.e. valid, authentic, current and sufficient
• The judgment made by the trainer/assessor is benchmarked with colleagues or industry experts
• There is sufficient evidence to support the judgment of the trainer/assessor
• Whether the requirements of the Training Package or accredited course have been met.

Typical benchmarks used during the validation process include:

• National training package which are developed by Skills Service Organisations (SSOs)/ Industry Reference Committees (IRCs) and can be found on the training.gov.au website.
• Units of competency which consist of competency standards and need to be unpacked so that those validating the assessments can compare the actual competency against the tools being validated.
• Industry standards and consultation will vary, and these standards form the basis of the skills and knowledge required to perform work roles.
• AQF Guidelines and Framework
• Information provided to candidates, assessors and third parties
• Legislation relevant to the assessment such as privacy, health and safety, anti-discrimination, copyright law and so on.

Validation occurs through different stages:

Stage 1: Validation before assessment judgements i.e. pre-validation of assessment resources

Validation before assessment judgements are made; look at the design of the assessment activities, if it meets the training package requirements, how the instructions for tasks or questions are presented and the benchmarks against the learner performance. This is where the mapping is undertaken. You review the assessment tool templates in detail to ensure they are compliant and meet regulatory standards and Industry requirements.

Stage 2: Validation during assessment

Validation during assessment is looking at the actual benchmarking answers or performance the learner has provided, and making a judgement with another assessor, either together or separately. This often is coordinated within assessors who undertake training and assessment of the same vocational area. This process was also known as moderation and always remember prevention is always better than the cure, therefore, any issues identified at stage 1 or stage 2 should be eliminated as soon as possible and gaps should be filled with gap-analysis to ensure your organisation is bullet-proof.

The requirement in the Standards to undertake validation of assessment judgements (post validation) does not prohibit your RTO from undertaking moderation activities, or any other process aimed at increasing the quality of assessment.

Stage 3: Validation post assessment (i.e. post validation)

Validation post assessment concentrates on the learners’ performance and their responses to questions, the actual assessment decision that was made, the task and processes that align to the assessment, any feedback from students, and the reporting processes.

The purpose of this post-assessment validation is to verify the validity and consistency of assessment decisions to bring assessment judgements and standards into alignment.

It is a process that ensures the same standards are applied to all assessment results within the same Unit(s) of Competency. It is an active process in the sense that adjustments to assessor judgements are made to overcome differences in the difficulty of the tool and/or the severity of judgements. It aims to ensure assessors have a common understanding of the unit requirements

It involves checking that your assessment tools have produced valid, reliable, sufficient, current and authentic evidence, enabling RTO to make reasonable judgements that the training package requirements have been met.

(To be continued in the upcoming blogs)

In the first five parts of this series we discussed the following: 

• What are internal audits?
• What are the benefits of conducting internal audits?
• What is an audit scope?
• What is usually included in an RTO internal audit?
• Who can be an internal auditor?
• Compliance costs and risks in terms of “risk management”
• The effective internal audit function
• The requirement of conducting internal audits
• The quality system of an RTO 
• Planning for internal audit and considerations 
• Conducting and recording an actual internal audit

This is our special edition on frequently asked questions and answers on internal audits. We have selected the top 10 questions from the list of questions sent to us by educational institutes. 

Q1: Why are RTO internal audits important?  

Answer: Internal audits are a significant element of an RTO quality management system to ensure RTO practices and procedures meet the regulatory and legislative standards and requirements. These audits can help to monitor the RTO system and to check that compliance and norms are complied with. The aim of an internal audit is to collect data on the quality system’s performance and effectiveness. Internal audits also increase productivity, detect non-compliance and non-conformities, and evaluate the RTO’s internal control including its corporate governance and processes.

Q2: Why should I have an external audit to review my RTO’s quality management system? 

Answer: There are a number of benefits of organising an external audit by expert RTO consultants: 

External auditors are independent of the organisation and review the systems and processes based upon their extensive experience and auditing background. They are impartial and unbiased in their approach and follow documented processes and procedures to provide opinions and advice on RTO’s quality management system.

The benefits of organising an external audit includes: 

• Advice on critical RTO non-compliance, efficient controls and compliance procedures, identification of best practices, reduction of operational costs, and the realisation of possibilities for profit enhancement.
• Potential important savings on internal audit expenses, in particular for organisations with multiple offices and courses, high internal audit resource turnover or different levels of internal audit activities.
• Access to the correct abilities, in the correct position, in the correct location, at the correct moment.
• Shifting expenses to the consultancy company to develop and maintain the internal audit capacity and freeing capital and resources for key company reasons.
• Overcoming difficulties for human resources-attracting and retaining talent, maintaining expertise on changing hazards and developing value-making abilities.
• Alignment of strategic goals of the internal audit function with important business processes.
• Overall risk management review, tracking of compliance and corporate performance.

Audits should be conducted by RTO experts that have extensive and current ASQA and other regulatory experience and who do not have any conflict of interest with the organisation. 

Q3: What should be the qualifications and work experience of an RTO consultant? 

Answer: We suggest that an RTO auditor should hold the Diploma of Quality Auditing, Diploma of Vocational Education and Training, Diploma of Training Design and Development or have at least 5 years of experience in RTO audits and administration. 

You must also ask and verify the success rate of the RTO consultant to ensure you receive the best advice and consultancy services. 

Q4: How long does an external audit takes? 

Answer: It depends upon your RTO’s scope and requirements to conduct an audit. Usually, two days are recommended for an RTO with two to ten qualifications on their scope. 

Q5: How should an RTO audit be conducted? 

Answer: The RTO audit should be conducted using a proper checklist and documented procedures and processes. 

Preparation of a plan that shows how your audit activities are systematic, independent and that you have a documented process for obtaining audit evidence is critical for a successful and valid internal audit. 

Q6: What is included in an RTO’s quality management system?

Answer: The RTO’s quality management system must include, but is not limited to the following:

• The required policies and their accompanying procedures
• Forms, templates, checklists and flowcharts to support the implementation of policies and procedures
• Central registers to track and record your compliance activities
• A compliance matrix that maps how each policy, procedure, form, template etc is related to the Standards.
• Continuous improvement processes and practices

Q7: What is the usual cost of conducting an internal audit by external auditors and/or consultants? 

Answer: Different consultants have different prices depending upon their expertise and experience. CAQA auditors and consultants charge $5200 plus GST for a two day audit. This price is valid at the time of printing – 1 July 2019. 

Q8: Why you need a CAQA auditor to conduct a health check of your RTO or educational institute? 

Answer: We are experts in RTO audits. We have highly trained and experienced compliance consultants who have worked in the VET sector for more than 20 years.

A VET health check is crucial in ensuring quality RTO systems and procedures are in place and are ready for an audit for registration, re-registration, continuous improvement or internal annual audit purposes. Every RTO must comply with the Standards for Registered Training Organisations 2015:

• Compliance with the pre-enrolment and enrolment requirements
• Marketing and advertising
• Third-party services  
• Maintenance of trainer and assessor currency
• Compliant assessment system 
• Complaints and appeals 
• Validation of assessment resources 
• Management of continuous improvement
• Student support and welfare
• Responsive to industry and learner needs
• Quality assurance
• Secure and accurate certification
• Accessible information about services
• Informed and protected learners
• Fair complaints handling
• Effective governance and administration
• Legal compliance
• Fit and proper person requirements
• Financial viability
• Business planning and direction etc.

Q9: What legislation or standards do CAQA auditors and consultants have experience in? 

Answer: 

• Standards for RTOs 2015
• AQTF and VRQA Guidelines (VRQA registered RTOs)
• The VET Quality Framework (ASQA registered RTOs)
• The ESOS Act and the National Code (CRICOS)
• VET Student Loans (Higher Education Support Act).
• Skills First (Victoria), Smart and Skilled (NSW),  and other state funded contracts
• ISO 9001:2015 – world’s leading Management System Standard
• English Language Intensive Courses for Overseas Students (ELICOS)  
• NEAS (accreditation and quality assurance services in English language teaching)
• Australian Nursing and Midwifery Accreditation Council (ANMAC)
• Worksafe 

Q10: How do CAQA auditors and consultants conduct an internal audit?

Answer: The process involves the following: 

• Meet with you to discuss your specific requirements and needs
• Scope what you need to do and recommend a path forward
• Provide qualified auditors to advise on compliance issues
• Work with you to ensure your systems, processes, materials and practices are resilient enough to withstand the test of an audit
• Advise you of all the issues we find as a result of our audit
• Advise you in writing and implementing your quality policies and procedures
• Develop a plan to address all your non-compliant issues with a suitable timetable
• Provide a report for you detailing the above.