This is Part 3 of the article, where we are discussing the different phases of the validation processes an RTO should be following to ensure they meet regulatory requirements and industry expectations.

In the previous articles, we discussed the following regarding the validation of assessment resources:

• Explanation of assessment validation
• Typical benchmarks used during the validation processes
• Stages of validation (before, during and after the assessment judgements)
• Regulatory requirements for conducting validation
• Assessment system
• Who conducts validation?
• How is validation different from moderation?
• How external consultants can help you with validation of assessment and learner resources?

In this month’s article, we will explore the regulatory requirements around validation of learner resources.

Learner resources
Learner resources are also known as “learning resources”, “training resources”, or “companion guides”. The purpose of these resources is to support learners with the underpinning knowledge required to participate in skill-based tasks. These resources include a range of activities to support the learning including, formative assessments and activities, links to further reading, workplace activities and procedures (where relevant to the qualification) etc.

Why you need to validate your learner resources
The VET regulator, ASQA does not currently prescribe the methodology Registered Training Organisations (RTOs) should use to meet the requirements of the relevant standards, training packages and accredited courses for learner resources.

But their expectations under the Standards for Registered Training Organisations, 2015, is to ensure your learner resources meet the following legislative guidelines:

Standard 1, Clause 1.3 (c): Learning resources to enable learners to meet the requirements for each unit of competency, and which are accessible to the learner regardless of location or mode of delivery.

The guidelines further state that:

Learning resources

• To ensure students are able to obtain and absorb the required knowledge and skills prior to assessment, carefully choose and plan the learning resources you will use to guide them.
• Identify these resources in your strategy to ensure you obtain full coverage of all required areas.

Therefore, we strongly recommend validating your learner resources to ensure your organisation complies with the relevant legislative requirements and guidelines.

The process of validation of learner resources
The validation of learner resources is not very different from the validation of assessment resources. All learner resources must also meet training package requirements and industry expectations.

Who can be involved in validating the learner resources
There are currently no regulatory requirements around who can participate in the validation of learner resources, however, it should be no different from the validation of assessment resources.

It should be a collective team effort and you must include the following people to validate your learner resources:

• Subject matter experts
• Trainers and assessors
• Compliance or administration manager
• Industry experts
• You may also include compliance experts as well as they usually have current and up-to-date knowledge around audit and compliance expectations and requirements.

Stages of validation for learner resources
Stage 1: Validation before using the learner resources
Validation before using the learner resources is to ensure the resources meet training package requirements, how the information is presented and the quality of the formative assessments. This is to ensure the student gains the required skills and knowledge to participate in the summative assessments later. Your review of the learner resources templates in detail ensures they are compliant and meet regulatory standards and Industry requirements.

Stage 2: Validation during or after using the learner resources
Your validation of learner resources during or after use is to ensure:

• Your resources meet client expectations
• Your resources meet training package guidelines and provide all required underpinning knowledge to your students
• Your resources are current and up-to-date in terms of the latest trends, technology and industry guidelines and practices.

In the next and final article, we will discuss:

• Why you need to keep validators information
• Why validation of assessment and learner resources should be systematic and ongoing
• How can you schedule validation
• What is statistically valid sampling
• Validation outcomes

(To be continued in the upcoming newsletter and blogs)

This is Part 2 continuing from the previous newsletter. As discussed in Part 1 of this article, there are a number of assessment-related issues that may affect your audit outcome. You should ensure your assessment resources, therefore, meet the following criteria:

• The context and conditions of assessment. For example, an assessment tool could be developed to cater for a particular language, literacy and numeracy requirements, the learner’s workplace experience or other learner needs that require reasonable adjustment.
• The context of the assessment may also take into account assessments already completed, and the competencies demonstrated in these assessments. By looking at the context, you can consider the conditions under which evidence for assessment must be gathered.
• All activities are conducted adequately using the required:
  • equipment or material requirements
  • contingencies
  • specifications
  • physical conditions
  • relationships with team members and supervisors
  • relationships with clients/customers
  • timeframes for completion.
• Assessment methods or tasks are suitable to the requirements of the units of competency and students are assessed on the tasks and activities according to the requirements of the training package.
• The language used is simple English
• The evidence required to make a decision of competency is clearly outlined
• The types of activities and tasks student need to perform are clearly outlined
• The level of performance required for each assessment activity is clearly outlined
• Adequate exposure to workplace conditions, including appropriate simulated environments
• Sufficient knowledge-based assessment tasks and activities such as written questions and case studies etc.
• Sufficient practical based assessment tasks and activities such as projects, role plays, workplace tasks and observations etc.
• Assessment resources are error-free and free from any grammar, copyright or plagiarism issues

It is a wise decision to get your training and assessment strategies and resources validated by independent industry experts to get honest feedback and an unbiased opinion.

This is Part 2 of the article, where we are discussing the different phases of the validation processes an RTO should be following to ensure you meet regulatory requirements and industry expectations.

In the previous article, we discussed the following regarding the validation of assessment resources:

• Explanation of assessment validation
• Typical benchmarks used during the validation processes
• Stages of validation (before, during and after the assessment judgements)

In this month’s article, we will explore the regulatory requirements around assessment validation.

Regulatory requirements for conducting validation

According to the Standards of Registered Training Organisations (RTOs) 2015, you are required to implement a quality review process (Clauses 1.8, 1.9, 1.10 and 1.11).

Clause 1.8a requires that the RTO’s assessment systems comply with the assessment requirements of the relevant training packages or VET accredited courses.  

Clause 1.8b requires RTOs to ensure that the evidence gathered is valid (one of the Rules of Evidence) and that assessment processes and outcomes are valid (one of the Principles of Assessment). 

These requirements must be met and demonstrated in all assessment policies, procedures, materials and tools of the RTO. Clause 1.8 primarily relates to the development (or purchase) of the RTO’s assessment resources.

Assessment validation has been strengthened in the Standards for RTOs and the requirement is to:

• Develop and implement a comprehensive plan for ongoing systematic validation of assessment that includes all training products on the RTO’s scope of delivery (Clause 1.9)
• Validate the assessment practices and judgements for each training product at least once every five years with at least 50% of products to be validated within the first three years of each five-year cycle (Clause 1.10)
• Ensure that validation is conducted by one or more suitably qualified persons, who are not directly involved in the delivery and/or assessment of the training product being validated. (Clause 1.11).
• These clauses relate primarily to the actual delivery and outcomes of the RTO’s assessment systems, including the performance of the RTO’s assessors.

Assessment system

Documents required for conducting an effective validation session, in the RTO’s assessment system, includes but is not limited to:

Validation related documents:

• Validation plan
• Validation schedule
• Validation record or validation form
• Validation register
• Validation report form
• Continuous improvement form
• Continuous improvement register
• Pre-assessment validation documents

Assessment resources:

• Unit assessment pack/student pack
• Trainer assessment pack/assessor pack
• Mapping document
• Assessment evidence according to a sample size

Other documents:

• Training and assessment strategy
• Feedback forms
• Unit of competency
• Companion volume/implementation guide
• AQF framework
• ACSF framework

You will be required to evaluate if the assessment resources meet:

• Training package requirements (application, elements and performance criteria, foundation skills, performance evidence, knowledge evidence, assessment conditions)
• Principles of assessment; fairness, flexibility, validity and reliability
• Rules of evidence; valid, sufficient, authentic and current
• The appropriate level of difficulties (AQF Level)
• Provide sufficient and clear instructions
• Record any appropriate adjustments

Who conducts validation?

Validation is a collaborative process. The team must hold collectively:

• Vocational competencies and current industry skills relevant to the assessment being validated
• Current knowledge and skills in vocational teaching and learning
• The TAE40110 Certificate IV in Training and Assessment (or its successor) or the TAESS00001 Assessor Skills Set (or its successor).
• Validators can be employees of your RTO, or you can seek external validators.

The trainer and assessor who delivered/assessed the training product being validated:

• Can participate in the validation process as part of a team
• Cannot conduct the validation on his/her own
• Cannot determine the validation outcome for any assessment judgements they made
• Cannot be the lead validator in the assessment team.

It is important to keep the records of all validation activities and validators as auditors might ask for it during audit activities and for managing continuous improvement processes at an RTO.

How is validation different from moderation?

Moderation is a quality control process aimed at bringing assessment judgements into alignment.

Moderation is generally conducted before the finalisation of student results as it ensures the same decisions are applied to all assessment results within the same unit of competency.

The requirement in the Standards to undertake validation of assessment judgements does not affect your RTO’s ability to undertake moderation activities, or any other process aimed at increasing the quality of assessment.

(ASQA, 2018)

(To be continued in the next newsletter)

It is important to look into ASQAs 2017 report that shows:

• Around 72% of RTOs FAIL audit on Assessment
• Approximately 50% of those FAIL to be able to rectify their assessment tools on resubmission under the OLD audit mode

According to the new audit model:

• There may be NO opportunity to rectify critical non-compliances
• Initial registration clients with critical non-compliances are unlikely to get an opportunity to rectify and potentially would be unable to reapply
• Registered RTOs risk sanctions, conditions, or even worse cancellations for critical non-compliances on the first audit

There are a number of assessment-related issues that may affect your audit outcome. You should ensure your assessment resources meet the following criteria:

• Assessment resources have sufficient and clear information regarding what, when, how, where, why for your assessment template and all assessment tasks and activities.
• Assessment resources have robust benchmarking and/or trainers guide.
• Assessment resources are allowing the trainer/assessor to assess the skills and knowledge of students through different assessment tasks over a period of timeto ensure “consistency” and “sufficiency”
• Each and every question and assessment task has very clear guidelines around what is expected from the students in terms of both “quantity” and “quality”
• You have customised the off-the-shelf resources according to your RTO needs and requirements and not using them “as-it-is”
• Your assessment resources are written by Industry experts with subject matter experts and are “Industry-relevant” and “current”
• Your assessment resources address all requirements of the training packaging rules
• Your assessment resources have detailed and valid performance checklists/observation checklists for assessing and observing the students before, during and after any skill assessment activity or workplace task
• Your trainers and assessors gather sufficient, valid evidence for competency assessment
• Your organisation offers appropriate simulated environments for conducting assessments
• The authenticity of assessment, particularly in distance and online delivery is established and maintained

In this article, we will discuss different phases of validation processes that you should be following in your RTO to ensure you meet regulatory requirements and industry expectations.

Validation of RTO assessment resources

You must validate all your assessment resources to ensure they meet the principles of assessment, rules of evidence, training package requirements, regulatory guidelines and Industry expectations.

Explanation of assessment validation:

Validation is a process of checking that the assessment tools, methods, judgements, evidence and processes to ensure that the training product meets:

• ​Principles of Assessment – i.e. valid, reliable, flexible and fair
• Rules of Evidence – i.e. valid, authentic, current and sufficient
• The judgment made by the trainer/assessor is benchmarked with colleagues or industry experts
• There is sufficient evidence to support the judgment of the trainer/assessor
• Whether the requirements of the Training Package or accredited course have been met.

Typical benchmarks used during the validation process include:

• National training package which are developed by Skills Service Organisations (SSOs)/ Industry Reference Committees (IRCs) and can be found on the training.gov.au website.
• Units of competency which consist of competency standards and need to be unpacked so that those validating the assessments can compare the actual competency against the tools being validated.
• Industry standards and consultation will vary, and these standards form the basis of the skills and knowledge required to perform work roles.
• AQF Guidelines and Framework
• Information provided to candidates, assessors and third parties
• Legislation relevant to the assessment such as privacy, health and safety, anti-discrimination, copyright law and so on.

Validation occurs through different stages:

Stage 1: Validation before assessment judgements i.e. pre-validation of assessment resources

Validation before assessment judgements are made; look at the design of the assessment activities, if it meets the training package requirements, how the instructions for tasks or questions are presented and the benchmarks against the learner performance. This is where the mapping is undertaken. You review the assessment tool templates in detail to ensure they are compliant and meet regulatory standards and Industry requirements.

Stage 2: Validation during assessment

Validation during assessment is looking at the actual benchmarking answers or performance the learner has provided, and making a judgement with another assessor, either together or separately. This often is coordinated within assessors who undertake training and assessment of the same vocational area. This process was also known as moderation and always remember prevention is always better than the cure, therefore, any issues identified at stage 1 or stage 2 should be eliminated as soon as possible and gaps should be filled with gap-analysis to ensure your organisation is bullet-proof.

The requirement in the Standards to undertake validation of assessment judgements (post validation) does not prohibit your RTO from undertaking moderation activities, or any other process aimed at increasing the quality of assessment.

Stage 3: Validation post assessment (i.e. post validation)

Validation post assessment concentrates on the learners’ performance and their responses to questions, the actual assessment decision that was made, the task and processes that align to the assessment, any feedback from students, and the reporting processes.

The purpose of this post-assessment validation is to verify the validity and consistency of assessment decisions to bring assessment judgements and standards into alignment.

It is a process that ensures the same standards are applied to all assessment results within the same Unit(s) of Competency. It is an active process in the sense that adjustments to assessor judgements are made to overcome differences in the difficulty of the tool and/or the severity of judgements. It aims to ensure assessors have a common understanding of the unit requirements

It involves checking that your assessment tools have produced valid, reliable, sufficient, current and authentic evidence, enabling RTO to make reasonable judgements that the training package requirements have been met.

(To be continued in the upcoming blogs)