THE SKILLS DEVELOPMENT AND ASSESSMENT
A Practical Approach to Corporate Training and Assessment
Raymond A. Talke, Jr.
Todayís workplace requires an ever increasing emphasis on skills development. Existing job roles and professions are changing rapidly, reflecting the advances of technology and the demands of a more competitive marketplace. Employees are pressured to do more on their jobs, with less personnel resources, less time, and less guidance. Employers are faced with the challenge of recruiting and maintaining personnel with the skills required to enhance their competitive position in the marketplace. Developing employee skills has become more critical as the marketplace has become more competitive. Corporations and organizations in the United States spend over $300 billion a year training employees. Yet few organizations are aware of the effectiveness of their training expenditures.
Considering the large amount of funds that are allocated for training, corporations and organizations should ensure that those expenditures contribute to the success of the business or organization in some tangible, measurable way. Whereas childhood education is geared towards the development of heterogeneous knowledge and skills, with little emphasis on application of that set of knowledge and skills, effective adult training and education emphasizes the development of a fairly homogenous set of skills and knowledge that can be applied directly to the individualís work environment. Indeed, the basic premise of corporate or organization sponsored education should be:
In addition, in order to be effective, adult education, training, and assessment must offer value to both the creator of the training (corporation or organization), as well as the recipient of the training (employee). Skills learned through the training program must have relevance to the recipientís job responsibilities, and must contribute to the organizationís goals and objectives. In order to be effective, corporate or organization delivered training and assessment must meet the following criteria:
In order to ensure that this set of criteria is met, a training and assessment model must be developed and followed, to ensure the success of any organization or corporate skill development and assessment initiative. The Skills Development and Assessment Octagon is an effective model, that when used appropriately, ensures the success of training programs in meeting organization or corporate goals.
THE SKILLS DEVELOPMENT AND ASSESSMENT OCTAGON
The Skills Development and Assessment Octagon is a skills development and assessment model that approaches the development of corporate training programs as a strategic process that is integrated into all other facets of the business. It is made up of eight stages that, when implemented as an integral part of a business initiative, identifies training needs, develops a program to address those needs, and measures the success of training against established business goals and objectives.
The eight stages of the Octagon are:.
Each of the Octagonís stages support the attainment of the next stage. In addition, the Octagon is a closed-loop process; for example, the effectiveness of training, as measured in stage #8, may be used to refine the actual business goals of the corporation. However, in order to be successful, the stages of the Octagon must all be completed as part of the development of any training program. Implementation of the entire Octagon will yield better results than just implementing individual pieces of the model. In addition, the Octagon will be most successful when it is fully integrated as part of the general business process, rather than having training stand alone as its own discreet process.
STAGE #1: DEVELOP BUSINESS GOALS AND OBJECTIVES
The first stage of the Octagon involves the development of business goals and objectives. After all, how can the success of a training program be measured if the underlying business goals and objectives have not been defined?
The development of business goals and objectives occurs at the start of a business initiative. A business initiative is simply a business program that attempts to accomplish one or more actions. For example, a manufacturer may want to establish a reseller channel or distribution network. Or, a company may be introducing a new product. Or, an organization may wish to prepare new employees to work effectively at their assigned jobs. Each of these are examples of business initiatives.
Goals should then be established for each business initiative. Goals are a description of what should be accomplished. Often, the goal is the same as the business initiative. Each of the above examples of business initiatives are also examples of goal statements. For larger business initiatives, multiple goals may be required to support the business initiative. Goal statements describe the business actions that must be accomplished to support the business initiative. Goals should be documented and communicated to all individuals responsible for the attainment of the goals.
Once the business initiative and goals have been established, the objectives of that initiative should be developed, quantified, documented, and communicated. An objective is a yardstick used to determine whether or not the initiative was successful. The objectives are used as the measurement criteria for the initiative. For example, objectives of the above three initiatives may include:
Establish a reseller channel.
Goal: Establish a reseller channel to market a newly developed product.
Objectives: Create 400 resellers by year end.
Introduce a new product.
Goal: Successfully introduce a new product into an established marketplace.
Objectives: 23% market share by end of second quarter.
Prepare new employees for job assignments.
Goals: Prepare newly hired employees for their new job assignments.
Objectives: Train 30 employees by year end.
Note that the objective statements are measurable and quantifiable, and that they include a discreet time frame in which the objective is to be accomplished. Objectives should be as specific as possible, and contain valid and objective measurement criteria.
Once the goals and objectives have been established, a mechanism for measuring success must be established. For example, the locations of data that are used to conduct the measurement must be identified, and procedures for accessing and manipulating the data must be instituted. A baseline, that is, the current state of the initiativeís objectives, must be established. These procedures will be used later in the Octagon to determine the success of the initiative.
The establishment of business goals and objectives is the critical first stage in initiating a training and assessment program. The goals and objectives provides a framework and point of reference for understanding the corporate focus, and serves as the measuring stick for all activities, including training, in support of the initiative.
STAGE #2: MAP ENGAGEMENT AND DEPLOYMENT MODEL
Once the goals and objectives for the business initiative have been established, the actual process for fulfilling the initiative must be created. This stage of the Octagon requires that the entire engagement and deployment model be mapped.
Each step required to fulfill the goals and objectives of the initiative is documented in the order in which the steps occur. For each step, the personnel responsible for completion of the step are identified and criteria for successful completion of the step is documented. The following is an example of what an engagement and deployment model map for a new product initiative might look like:
Introduce a new product.
Goal: Successfully introduce a new product into an established marketplace.
Objectives: 23% market share by end of second quarter.
Engagement and Deployment Model Map
For the purpose of brevity, this is an abbreviated example of an engagement and deployment model map. Real business initiatives may have many more steps than are listed here. Yet, it is important to list every step in as granular a fashion as possible. In addition, the completion criteria identified for each step must support the attainment of the business initiative goals and objectives. A clear understanding of the process used to complete the initiative and knowledge of the completion criteria expectations for each step will help ensure the success of the initiative.
STAGE #3: IDENTIFY TARGET PROFESSIONS AND JOB ROLES
The engagement and deployment map will identify each of the steps required to complete the business initiative as well as the personnel required to perform the tasks. Completion criteria for the tasks have been established. During stage #3 of the Octagon, the jobs or professions that require training to carry out the initiative are identified and the expectations and objectives of the training are established.
In most cases, it is not required that all individuals involved in the success of the initiative be trained. For example, a task in a given engagement and deployment model map may be similar or identical to tasks that the identified individual has performed before. In this case, training an individual on skills that he or she already possesses would be a waste of time or money.
Yet, there may be certain job functions that require new skills or knowledge in order to complete the task to the identified completion criteria. Each of the personnel requirements within each step of the engagement and deployment model must be analyzed to determine if additional training is needed or desirable for the individuals charged with completing the task. Training should be provided to those personnel who possess a significant skill gap between their current skills and the skills required to complete their assigned task to the specified completion criteria. Although the decision criteria used to determine whether or not training is required differs between organizations, the following questions may help determine if a given job role is a candidate for additional training:
If a yes answer is provided to one or more of the questions, the job role may be a candidate for training.
For each job role that is identified as a candidate for training, a role description statement for that job role should be created and documented. The job role description statement serves as the cornerstone for all training and assessment activities designed for the individuals working within the job role. The job role description statement should include an overview of the tasks that the individual in the job performs, the environment in which an individual in the job works, and the level of performance expected of that individual.
For example, the following might be a job role description for a Senior Sales Representative:
The job role description should provide a clear description of the primary job responsibilities of the individual who will be the subject of the training. This job role description forms the base of the next four stages of the Octagon.
Once the job role description has been created, methods of marketing the training and assessment program should be established. The corporation should determine how to incent individuals to attend the training program. For example, one incentive to ensure participation of individuals in the training program is to make training a condition of continued employment. Yet, this may have the effect of fostering "malicious compliance" and predispose the individuals to fear or dislike training. More constructive approaches may be used to incent people to attend training, such as an educational marketing campaign, financial incentives, gifts, etc. When training is developed for individuals not employed by the company offering the training (i.e. resellers or suppliers), the need to create incentives to attend training becomes even more critical.
Methods of assessing the job performance for individuals targeted for training should also be established during this stage. Methods of measuring job performance before and after training should be designed and documented, and access to the required data needed to make these measurements should be arranged. Remember, the measurements of job performance should support the level of performance statement in the job role description, the completion criteria for the task that is documented in the engagement and deployment model, and the goals and objectives of the business initiative.
Measurement criteria to assess the contribution of the targeted job role to the success of the business initiativeís goals and objective should also be established. In other words, did a trained individual contribute more to the success of the initiative than an untrained individual, and if so, by how much? Usually, this measurement will use a benchmark set of data to determine the values of the measurements before training, and then compare the same data to see if the measurement values changed after training. While there may be a correlation between training and improvements in business measurements, it is often difficult to establish training as the sole cause of the improvement. Other factors, such as advertising, changes in the marketplace, etc. may have independently accounted for the improvements. To further assess the contribution of training to improved business measurements, the measurement criteria may also contrast the performance of untrained individuals and trained individuals in the same job role, and determine the contribution each group made to business results. Most training measurements will include both a method to establish correlation of training to business results, as well as a contrast between trained and untrained individuals. Again, access to the data required to perform business measurement analysis must be arranged and the methods of performing the analysis must be established before any training is actually offered.
Finally, the administrative mechanisms for delivering training and assessment vehicles, and providing administrative fulfillment, must be considered during this stage. Although the specific procedures may not be finalized until the actual method of training and assessment is established, certain tasks requiring long lead times (such as ordering supplies, contracting course developers, etc.) should be started early in the process.
STAGE #4: PERFORM JOB TASK ANALYSIS FOR EACH TARGETED PROFESSION OR JOB ROLE
Now that the job roles requiring training have been identified, the skills and knowledge that must be possessed by individuals in these job roles must be determined. Training delivered to support the identified job roles must be based on a documented set of objectives that are relevant to the job responsibilities of the individuals taking the training, and must support the attainment of the business initiativeís goals and objectives.
Numerous methodologies exist for conducting a job task analysis--observation, surveys, focus groups, interviews, and workshops, to name just a few. Yet all methods require that the actual job responsibilities of the targeted job roles be mapped by individuals familiar with the job and the focus of the business initiative. Participants of the job task analysis are generally job incumbents, whose job responsibilities closely match the job role that is the focus of the training program, and subject matter experts, who are familiar with the business initiative that precipitated the need for training.
In order to support an effective skills development and assessment program within the model of the Octagon, the job task analysis process should yield the following outputs:
A list of tasks performed within the targeted job role that support the attainment of the business initiativeís goals and objectives.
Task Process Maps: For each identified job task, a detailed map of how the individual within the job role performs the task.
Critical Tasks Identification: Within each task process map, steps in the process that are critical to attaining the business goals and objectives.
Skill Gaps Identification: Within each task process map, steps in the process that can not be performed by the typical job incumbent without additional training or experience.
Terminal Objectives: For each identified task, an objective that describes the task, the environment in which the task is performed, and the successful completion criteria expected of an individual in the job role. The terminal objective for each task must ultimately support the attainment of the business initiative.
Enabling Objectives: Enabling objectives, describing the skills, knowledge, and attitudinal requirements of an individual performing the job role, are developed for each step in the task that is identified as a critical task or skill gap. These objectives form the base of the training deliverables.
Assessment Test Objectives: Participants in the job task analysis identify the objectives that should be used for assessment testing. These are the knowledge and skill requirements that are used to validate that an individual is competent to perform the tasks identified as part of the job role.
The outputs of the job task analysis form the base of development of the training and test vehicles.
STAGE #5: DEVELOP AND DELIVER TRAINING VEHICLE
Each terminal objective developed during the job task analysis will serve as the foundation for a unit training. The enabling objectives developed during the job task analysis will serve as the outline for the content that is to be included in the training.
The type of training to be developed and delivered will be determined during this stage of the Octagon. Based upon the needs of the organization and of the training audience, the appropriate training vehicle will be selected. Traditional instructor led training may be selected, or any of its alternatives, including computer based training, on-the-job training, distance learning, self-study, etc. The selection of the training vehicle will be determined by financial resources, the culture of the business, the geographic diversity of the training audience, training development and delivery resources, scope of the content, and the preferences of the training audience. Legal factors, such as the Americans with Disabilities Act, may require that specific, or multiple, training vehicles be created to meet the requirements of the potential training audience. In many cases, multiple training vehicles will have to be developed to address the needs of the business.
Once the type of training vehicle has been determined, training developers will create the training content. It is recommended that the training program undergo a systems assurance process to ensure the training meets the requirements of all the objectives and is accurate. One or more pilot implementations are suggested before the training program is released into wide distribution.
As part of the training development, a Level 1 evaluation mechanism (based on the Donald Kirkpatrick education evaluation model) should be created. Level 1 training evaluations are designed to measure the training audienceís reaction to the training. Typically, Level 1 evaluations are designed as training evaluation surveys, also known as "Happy Sheets" or "Smile Sheets". Level 1 evaluations should not be used as the sole evaluation of the effectiveness of a training program. However, they are useful in identifying problems with the content or delivery of training. Poor measurements on Level 1 evaluations should be rectified as early in the process as possible.
Administrative details relating to the delivery of the training should also be addressed during this stage of the Octagon. For instructor led training, instructors must be recruited, and possibly trained. Enrollment procedures must be developed and communicated to the potential audience of the training. The training marketing program should be reviewed and deployed. Issues of expenses and travel, if required, should be addressed during this stage.
STAGE #6: DEVELOP AND DELIVER ASSESSMENT OR CERTIFICATION VEHICLE
Kirkpatrickís Level 2 educational assessment gauges the amount of knowledge and skills that an individual has retained from training. Level 2 assessments may be used to help determine the effectiveness of the training program.
There are three types of Level 2 assessments that are generally used:
Assessment examinations are usually tests delivered to training participants sometime after the completion of the training program. The primary purpose of assessment exams is to determine the individualís retention of knowledge and skills learned during the training exercises. It is generally recommended that assessment exams be offered no earlier than one week after the conclusion of the training program. Since most retention loss occurs in the first 72 hours after the conclusion of the training, this would better reflect the true scope of skills and knowledge that the individual would retain and use on his or her job.
Qualification Exams: Qualification examinations possess all of the characteristics of the assessment exams, but also are used as a prerequisite before that individual is allowed to do something. For example, an individual may have to pass a qualification exam before he or she is allowed to take a new job position. Or, a company may require its reseller channel companies to pass a qualification exam before they may carry a certain product. Passing a qualification exam, however, does not necessarily imply that an individual possesses all of the requirements to enter the activity for which the qualification exam is a prerequisite. It merely assesses the knowledge and skills that the individual possesses at the time the examination is taken. Care must be taken to construct qualification examinations carefully, and assess the legal liability assumed by making the qualification exam a prerequisite to enter into an activity. The skills and knowledge tested in the qualification exam must directly relate to, and be a requirement of, the activity for which the qualification exam is a prerequisite. If the activity directly relates to the business initiative for which this training program was developed, and valid processes were followed for all of the stages of the Octagon, legal liability issues will be considerably lessened.
Certification Exams: As with the other two assessment vehicles, certification examinations assess the retention of skills and knowledge. However, certification exams are primarily used to determine if an individual is competent in a given profession or job role. Certification implies that a certified individual will be able to perform the tasks required in a given job role or profession in a safe and effective manner. Since certification implies competence in a given job role or profession, the creator of the certification program assumes a substantial amount of legal liability. The certification examination must be proven to be relevant and essential to the performance of the given job role or profession, and must be free of any racial, cultural, gender, or age bias. Also, completion of the actual training program is usually not a prerequisite to taking the certification examination. An individual who can pass the certification examination is assumed to possess all of the skills required to perform the tasks of his or her job role or profession, whether or not training has been taken. As with qualification exams, the legal liability of a deliverer of certification exams may be lessened if the stages of the Octagon are followed, and sound, accepted, psychometric and testing procedures are used.
The appropriate assessment method should be selected based upon the needs of the sponsoring company. In addition, incentive mechanisms should be established to ensure that individuals that have participated in training take the assessment exam. This is particularly important if the training program is delivered to individuals not in the employ of the organization that created and delivered the training. The assessment, particularly qualification and certification assessments, must be perceived by the target audience as having substantial value.
Once the type of assessment has been selected, the assessment vehicle is created, and arrangements are made for its delivery. Most assessment exams take the form of traditional tests. However, practicums in which individuals are required to demonstrate proficiency, proof of work in their professions, simulation exams, and board reviews may all be valid assessment methods. Remember that the assessment method chosen must lend itself to an objective assessment of the individualís skills and knowledge. Subjective assessment criteria increases potential legal risk and decreases the validity of the assessment vehicle.
Delivery issues relating to assessment vehicles must be addressed in this stage. Administrative details, such as how and where the assessment is to be delivered, scoring mechanisms, marketing, incentives, etc. must be considered during this stage of the Octagon.
As is the case with Level 1 assessments, Level 2 assessments can not be used alone to determine the success of the training program. Retention of skills and knowledge does not necessarily correlate to success on the job or achievement of business goals and objectives. However, poor Level 2 assessments may indicate a potential flaw in the training program or in the business initiative that fostered the training program.
STAGE #7: MEASURE JOB PERFORMANCE
Once the training and assessment vehicles have been delivered to the target audiences, measurements are made to determine if the trained individuals are actually using the skills and knowledge they obtained during training on their jobs. Usually, an assessment is made of an individualís job performance before training has commenced, and again, after training has been completed. The conditions used to assess job performance should be based upon the performance level criteria defined in the initial job role description.
During stage #3 of the Octagon, sources of measurement data were identified and a benchmark measurement was established. During this stage of the Octagon, measurements are actually taken to determine if, in fact, the training was effective in altering the behaviors of the individuals involved in the training. Numerous methods can be used. Objective data measurements, if available and applicable, are obviously useful. However, certain job performance standards may be difficult to quantify in a numerical fashion. In this case, observation, or surveys delivered to the training participants, their supervisors, and their peers may prove helpful in establishing a performance delta before and after training.
The measurement of the effect training has on job performance follows the Level 3 assessment in the Kirkpatrick model. If training is found to have no effect, or a negative effect, on job performance during the Level 3 evaluation, the previous activities performed in the Octagon will have to be reassessed. The relevance of training, and its adherence to the objectives described in the job task analysis output should be reviewed. If the training is found to be congruent to the objectives derived from the job task analysis, and there are no obvious problems identified in the Level 1 and Level 2 assessments, the accuracy of the role description and job task analysis findings should be reviewed. If the role description and job task analysis findings are found to be accurate, the business initiative and its goals and objectives should be carefully examined.
Conversely, positive Level 1 and Level 2 evaluations, combined with a positive Level 3 assessment, indicates a training program that succeeds in transferring relevant, useful skills to the audience. Although a direct correlation to business results is not yet established, the training can be said to be effective in achieving its designed purpose.
STAGE #8: MEASURE BUSINESS RESULTS
The final stage of the Octagon is used to determine if the training and assessment program actually contributed to the success of the business initiative. Measurements are taken to determine if individuals who attended training have contributed to a successful attainment of the initiative's goals and objectives.
Generally, two measurements are taken in this stage. First, the contribution of individuals efforts to the achievement of the initiativeís goals and objectives is measured before and after training. In addition, the entire initiativeís success is measured before and after the training program is offered. An improvement in the identified business results may indicate the effectiveness of training in helping to achieve these goals and objectives.
Yet, other factors may contribute to the success of the business initiative that are independent from training. Changes in the marketplace, a major advertising campaign, and other factors may influence success of the business initiative. As a result, additional analysis must usually be performed to determine the actual effect training had on the success of the business initiative.
The second business results measurement should contrast the contribution to the business results between trained and untrained individuals. If trained individuals are found to contribute to the attainment of business results significantly greater than those who have not been trained, it may be assumed that the training and assessment program was successful in accomplishing its mission.
Conversely, if no difference is found between the contributions made by trained and untrained individuals, the training program must be reassessed. Perhaps the training program is inadequate, the proper roles were not targeted, or even the business goals and objectives were not really relevant. This information may then be reinserted into the Octagon model, ensuring that the next iteration is more accurate and targeted to the correct factors.
The assessment of business results encompasses Kirkpatrickís Level 4 assessment. A thorough assessment of Level 1 through Level 4 results may be used to justify the training program expenditures, or to indicate where improvements may be made.
The Octagon is a tool that should be followed to ensure that skills development efforts are integrated into the business plan and that the training efforts contribute to the positive attainment of business results. This skills development and assessment model is designed to allow for continuous process improvement, by identifying, early in the process, potential flaws and potential areas of success in the training program.
The Octagon may also be effectively used to justify the costs and efforts required to implement a training program. It is far easier to justify budget expenditures if clear measurements can be offered to validate the expenditures.
The Octagon contains eight stages that must be implemented consistently to ensure an effective training program that contributes to the goals and objectives of the sponsoring company. Yet it allows the flexibility required to adjust to specific markets and situations. Consistent implementation of the Octagon will help ensure that the training programs created using the model will meet the needs of both the sponsoring organization and the individuals targeted for training.
Send mail to
Minds in Action with
questions or comments about this web site.