ISSN 0798 1015

Vol. 40 (Number 2) Year 2019. Page 7

A study on factors influencing training effectiveness

Un estudio sobre los factores que influyen en la efectividad del entrenamiento

Paresh C MOHANTY 1; Madhusmita DASH 2; Manoranjan DASH; Subhankar DAS 3

Received: 30/07/2018 • Approved: 10/11/2018 • Published 21/01/2019

Contents

1. Introduction

2. Review of Literature

3. Objectives & research methodology

4. Results & its Analysis

5. Limitations

6. Conclusion

Bibliographic references

1. Introduction

Research on training exposes the positive and negative sides of training outcomes. On the positive side, continuous training and retraining are required to keep the human assets performing (V. Sessa and M. London, 2006). A lot of training effort, the world over, is directed towards training people to learn and carry these learning to the point of use (V.Gautamand S. Gautam, 2011). Motorola, a fortune 500 company, estimates that every dollar spent on training generates $30 in productivity gains within three years (Davis and Yi, 2004). A study by the American Society for Training and Development ( ASTD) of more than 500 publicly traded U.S.-based companies found that companied that invested the most in training and development had a shareholder return that was 86 percent higher than companies in the bottom half and 46 percent higher than the market average ( L.Bassi, J. Ludwig, D. McMurrer, and M. Van Burren, 2000) But there is the flip side too.  Though billions of dollars are spent and numerous human hours are sacrificed, only few training programmes deliver the desired results (B. Pfau, and I. Kay, 2002). This has induced us to find out what makes a training programme effective and what does not.

1.1. Measuring Training Effectiveness

Evaluation of training deals with any attempt to obtain information on the effect of training programme and to assess the value of training in light of that information. It provides a justification of training expenditure and creates a rationale for allocation of resources. Evaluation of training is done during training exercises and subsequently, at the place of its use in the organization (V. Gautam and S.Gautam, 2011).

One of the original frameworks for identifying and categorizing training outcomes was developed by D. Kirkpatrick (with four level frameworks) for categorizing training outcomes (D. Kirkpatrick, 1996).

• Reactions –  Trainee satisfaction
• Learning – Acquisition of knowledge, skills, attitudes, behavior
• Behavior – Improvement of behavior on the job
• Results – Business result achieved by trainees

In addition to Kirkpatrick model, there are several other models proposed for evaluation of training effectiveness as mentioned below:

• A.C. Hamblin’s Model of Evaluation – Reaction, Learning, Job behavior, Functioning
• Peter Warr’s CIPO Model – Context, Input, Process, Outcome (Immediate outcome, Intermediate outcome, Ultimate outcome)
• B.R.Virmani &Premila Seth – Pre-training context, Training input, Post-training reaction, Learning, Job improvement plan, On the job, Follow up and transfer
• Peter Bramley’s Model of Evaluation– During the event , After the event, Learning, Behavior, Effectiveness
• David Reay’s Three Phase Model –Trial, On–going and Final

However, Kirkpatrick’s model continues to be the most popular method of representing training evaluation criteria because of its simple and practical approach (M. Leach and A. Liu, 2003). In this study, we evaluated the training at reaction and learning levels only. Generally evaluations at these levels include items related to trainer’s preparation, delivery, ability to lead a discussion, organization of the training materials and content, use of visual aids, presentation style, ability and willingness to answer questions and ability to stimulate trainee interest in the course. We selected reaction and learning stages because of the following reasons:

• Long et al.’s finding states that trainee reaction has a positive correlation with further levels of evaluation as well as future learning and attending future programmes (Long, L.K., DuBois, C.Z. and Faley, R.H. (2008) .
• The hierarchical nature of Kirkpatrick’s framework suggests that higher level outcomes should not be measured unless positive changes occur at lower level outcomes. For example, if trainees do not like a course no learning will occur (D. Kirkpatrick, 1996)
• Conventional wisdom suggests that trainees who like a training programme learn more and are more likely to change behaviors and improve their performance (transfer of training). Recent research results suggest that reactions have the largest relationship with change in affective learning outcomes (T. Sitzmann, K Brown, W. Casper, K. Ely and R. Zimmerman, 2008)
• Kirkpatrick’s contribution, which was studied by ISTD, established that around 60% of the organizations studied, were using this model for evaluation purpose. Of these, approximately 77% organizations gauged participants’ reaction, 38% measured learning, 14% assessed job behavior and only 7% measured final result (Bassi, Ludwig, McMurrer and Van Buren,2001)

This reflects the acceptance of Kirkpatrick’s model as well as the use of reaction and learning level evaluations for training effectiveness. This paper tries to evaluate the effectiveness of the training programme at reaction and learning level, employing the eight chosen factors.

The search for factors effecting training have grouped researchers into two segments. One group has focused on the methods and settings that maximize the reaction, learning and behavior changes of trainees (Tannenbaum & Yukl, 1992), while the other side has concentrated on the learners’ characteristics like their intelligence and motivation to learn (R.A. Noe, 1986). Still, some have indicated that a variety of organizational, individual, training designs and delivery factors can influence training effectiveness before, during and after training (Salas & Cannon-Bowers, 2001). This research has concentrated on the factors taken by the first group to measure training effectiveness and out of them; we have focused on the major eight factors as narrated below :

i. Training Need Analysis : A need for training arises when a gap is identified between competency required to perform the job ,and existing level of competency in employees (M. Casey and D. Doverspike, 2005). Once we identify training gaps within the organization, it becomes easy to design appropriate training programmes. Training Need Analysis could accomplish several important things. If it’s not properly conducted, any one or more of the following situations may occur:

• Training may be incorrectly used as a solution to a performance problem
• Training programmes may have the wrong content, objectives or methods
• Trainees may be sent to training programmes for which they do not have the basic skills, prerequisite skills, or confidence needed to learn
• Training will not deliver the expected learning
• Money will be spent on training programmes that are unnecessary (V. Gautam and S. Gautam, 2011)

ii. Setting Training Objective: The objective of a training programme must be cleared before designing and conducting it. It helps each segment of training in the following ways - the training objective is beneficial to the trainer because it helps to measure the progress of trainees and make the required adjustments in case of need. Also, he/she comes into a position to establish a relationship between objectives and particular segments of training. The training objective is beneficial to the trainee because it helps in reducing the anxiety up to some extent. It helps in increasing concentration, which is the crucial factor for making the training successful.  The training objective is beneficial to the training designer as well. If the designer is aware what is to be achieved in the end, only then he/she will buy the training package accordingly. It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainee (E.A. Locke, L.M. Saari, K.N. Shaw and G.P. Latham, 1981).Training objective is an important tool to judge the performance of participants (R.A.Noe, 1986).

iii. Programme Design: A design document can be used to guide the development of training and explain the training to managers, subject matter experts, reviewers and other trainers (George M. Piskurich, 2006).At this stage, a sketch is made to give justice to training. This includes selection of trainees & trainer, time plan for preparation & delivery, course content to give justice to training, scheduling training sessions, content of the learning activity, total budget available for training, allocation of priorities, selection of training methods, time allotted to the programme, administrative arrangements, the physical infrastructure available etc.

iv. Faculty/Resource Person/Facilitator: In adult learning a trainer is normally introduced as a faculty , facilitator or the resource person.  There is no substitute for the diligence of the resource person or faculty. The most effective design can lead to incomplete results if the facilitator, who has coordinated the activity, does not come across to participants as a sincere person who means business in discharge of his/her training applications (V. Gautam and S. Gautam, 2011).The actual transfer of training depends a lot on the trainer because it is the trainer only who can remove the mental block of trainee, motivates the trainee to learn, deletes the negative perception of the trainee regarding the training(A. Towler and R. Dipboye, 2001).

v. Audio-Visual Aids: Based on researches on learning medium, it is accepted that out of the five senses, people learn the most via sight (80%). Through experience, it has been noticed that audio-visual support to the learning process has facilitated the grasp of an idea much better. It generates interest to learn in participants, and creates a sense of involvement in the entire learning process (H. Dolezalek, 2004). Audio-visual aids have, the world over, become an important support to the trainer for transferring knowledge to the learners.

vi. Methodology: Since training is directed towards improving precise knowledge and specific skills, a choice of strategy has to be made to convince people and influence their behavior. This is about appropriate use of communication media, by the training enabler, for the accomplishment of a training session- from all time lecture method to role play, exercises, brainstorming, group work, management games, etc etc. Amongst a host of methods available, the trainer selects the techniques to integrate – a) the objectives of the programme; b) the learners’ competencies; c) the resources available and d) his/her own competencies to handle a technique (V. Gautam and S. Gautam, 2011).

vii. Non-academic &Academic Environments: Non-academic environmental features include the travel comforts, hotel accommodation, and food etc. for the trainees. If not taken care of, such factors generate irritation that may disturb learners and ultimately, affects their learning appetite. At the place of training, the classroom arrangement – seating pattern, light arrangements, AC/ fan placements, audio system, materials provided, food, tea, snacks etc. and their timing of delivery among others, are all important and should be meticulously taken care of (R.M.Gagne and K.L. Medsker, 1996).

viii. Learning Outcome: The ultimate goal of a training programme is acquisition of knowledge or skill-sets or attitude by the learners (J. Marquez, 2006). The participants are keen on gaining knowledge, skill or behaviour ,definite take-away from the training and much emphasis is given to the learning that can be implemented back into their jobs. Even though a training programme is liked by the participants but adequate amount of learning has not taken place, it is a failure and trainees have nothing with them to carry to the place of their work (M.London,1989).

Through this research we have tried to explore the following two findings:

i. Factors having effect on conducting  result oriented training programmes

ii. Opinion of participants, from various sectors, on these factors

In our study, we have tried to obtain the reaction of the learners on the training programme conducted and measured their level of learning, after completion of the programme. We conducted a managerial training programme for the private sector and government sector separately and obtained their opinion.

i. Measurement of training effectiveness on “learning” both at entry and exit levels and their comparative results :

To measure learning, we carried an entry level test of the participants, via multiple choice questions, on the concepts to be discussed in the managerial training programme. On completion of the training programme, the participants were re-tested with the same questions. While the average score of the public sector employees at entry level was 13.6%, they scored an average of 86.5% after completion of the programme. Similarly, private sector employees’ average score was 12.82% at the entry level and 90.42% upon the programme’s completion. The results confirm that training was effective so far as generating learning was concerned and participants took keen interest in this training programme.

ii. Measurement of training effectiveness at Reaction level :

We also prepared a questionnaire to obtain feedback of the participants and simultaneously interviewed the trainees to confirm their viewpoints. Along with the Questionnaire, the trainees were also asked to tell the reasons for assigning their individual ranks to the aforesaid factors.

The questionnaire comprised eight factors that result in a training’s success and the participants were individually asked to rank these factors in order of their importance. Rank 1 to be assigned to the most important factor and Rank 8 to the least. The trainees were explained the importance of each factor in steering a training programme and were then asked to rate these factors accordingly.

Below are the results of the Survey:

BATCH 1
(GOVERNMENT. SECTOR)

 -----

BATCH 2
(PRIVATE SECTOR)

A common method of trying to measure training has been through simple questionnaires (completed by participants at the end of a training session).This basic approach, however, relies on subjective judgments. It can be the source of misleading conclusions. As per Bedinham (K. Bedinham, 1998), the questionnaires rarely take into account of the complexity of the topics covered in the training session or the difficulties which different classes might have with unfamiliar subjects. However, we have not come across any better instrument to gauge the trainee experience in evaluating a training programme. The purpose of this study is to employ the qualitative research method in order to evaluate the training effectiveness, explore the differences among the factors (if any) which contribute to the training effectiveness through a comparable sample from government and private sector firms and examine the effectiveness of the training.

The Public and Private sector employees ranked the “faculty” as the most important factor for a training program’s success and the “training methodology” as 5th important factor. Except these two similarities, the private and public sector participants differed on the importance of other parameters. While 91% of private sector trainees’ response was in favor of importance of faculty (to make a training programme successful), 80% of the government sector participants preferred the same. Both sectors rated this factor the highest because the trainer:

• Explained the programme objectives, topics to be covered and expected benefits
• Encouraged them to be involved in the learning process
• Illuminated the concepts and clarified confusions
• Substantiated arguments with practical examples
• Suggested solutions for our work related problems.
• Excellent presentation style
• Ability and willingness to answer questions raised by us
• Judicious use of training materials and audio-visual aids
• Seemed to be genuinely interested in us

Training Need Analysis is considered as the basic design based on which any training effort is launched (V. Gautam and S. Gautam, 2011),but ranked 6th by the public sector employees and 3rd by the private sector trainees. Setting training objective is least important (ranked 8th) to the public sector trainees, while given 4th rank by private sector counterparts. However, Locke, Shaw, Saari, and Latham (E.A. Locke, K.N. Shaw, E.A. Saari and G.P. Latham, 1981) have given it utmost priority. Programme design, which is given maximum rating by Sacks and Belcourt (A. Sacks and M. Belcourt, 2006), is given a higher rank (2nd) by the public sector employees, whereas the private sector trainees ranked it 6th.Use of training methods by a trainer, is intended to establish a rapport with the learners and earlier research (M. Broadwell and C. Dietrich, 1996) has given it maximum importance for training effectiveness. Surprisingly, both the groups have placed it at 5thposition inorder of importance.

 Through experience it has been noticed that audio-visual support to the learning process has facilitated the grasp of an idea much better (J. Barbazette, 2007). Audio visual aids is given 4th rank by public sector, while ranked 7th by the private sector employees in our study. Learning environment is ranked 3rdby the public sector employees, while for private sector staffs, it is least important (ranked 8th),but maximum importance is assigned to it by researchers like Smith and Delahaye (B.J Smith and B.L. Delahaye, 1998).  Learning must occur for training to be effective (R.M. Gagne and K.L. Medsker, 1996). Even though private sector employees deemed it very important (ranked 2nd), govt. sector participants assigned it rank 7 .

All trainees (considered in our research) have stated their strong reasons for grading each segment.

This paper weighs the training programme at “reaction and learning” levels only , without an evidence of transfer of knowledge to the workplace. Further, we have focused on the methods and settings that maximize the reaction and learning, which is in the domain of the organization and the faculty. The intelligence and motivation of the learners that influence training effectiveness greatly, is not measured in this research. The authors are planning to   conduct further research, comprising same set of participants, after a gap of six months to explore their behavior and result levels’ outcomes from this training. Even if we tried to prove that trainees in public and private sectors vary in their perception and approaches, such is not full-proof. It is likely that piloting the same programme in any of the sectors for the two batches, the feedback could be the same. In addition to these, we have a plan to test how individual and situational characteristics influence training motivation and learning of the participants.

We analyzed training at reaction and learning stages (based on Kirkpatrick’s module on evaluation), which was imparted to two groups of participants from private and government sectors. For both sectors the training programme, its design, delivery and resource persons were the  same.  The eight factors identified though this qualitative study - training need analysis, setting training objectives, programme design, faculty/ resource person, audio-visual aids, learning environment, methodology and learning outcome, were found to be vital and relevant factors which influence the effectiveness of any training programme. So far as learning is concerned, the programmes were successful as participants from both groups scored an average more than 88 % in a post training test when at entry level they scored average 13.21% only. Out of the eight factors that have an effect on training, both groups agreed on “faculty or resource person” being the most important factor in making training successful. They have not denied the impact of other seven factors on training effectiveness, but have not agreed on their importance equally either, except training methodology (which both groups placed at 5th position). This research has implication for the HR managers, participants and the organizations that decide to conduct a training programme. This research will also add value to the training managers who are coordinating and designing the programme, instructional designers who are into content development, the trainers who deliver a defined learning and the managements of both government and private sector organizations who are investing their time and money with a hope to have definite return on investment from training.

Barbazette, J. (2007) Managing the Training Function for Bottom Line Results: Tools, Models and Best Practices , books.google.com

Bassi, L.J., Ludwig, J., McMurrer, D.P. and Van Buren, M. (2002) Profiting from Learning: Firm-Level Effects of Training Investments and Market Implications. Singapore Management Review, Volume 24, No.3.

Bedinham, K. (1998) Providing the Effectiveness of Training. Education +Training. Vol. 40, No. 4, 166-167.

Broadwell, M. and Dietrich, C. (1996) “How to Get Trainees Into the Action.” Training, February 1996, 52-56.

Davis, F.D. and Yi, M.Y. (2004) Improving computer skill training: Behavior Modelling, Symbolic Mental Rehearsal and the Role of Knowledge Structures.

Dolezalek, H. (2004) Industry Report 2004, 33-34.

Gagne, R.M. and Medsker, K.L. (1996) The Conditions of Learning: Training Applications.

Gautam, V. and Gautam, S. (2011) Training and Development- Towards an Integrated Approach. Indian Society For Training and Development. 6th Edition.

Kirkpatrick, D. (1996) “Evaluation” in the ASTD Training and Development Handbook, 2nd.ed. R.L. Craig (New York: McGraw –Hill, 1996), 294-312.

L. Bassi, J. Ludwig , D. Mcmurrer, and M. Van Buren , Profiting from learning : Do Firms’ Investment in education and training pay off ? ( Alexandria VA: American Society for Training and Development , September,2000)

Leach, M. and Liu, A. (2003) Investigating Interrelationships among Sales Training Evaluation Methods. Journal of Personal Selling & Sales Management, Vol. 23 No. 4, 327-339.

Locke, E.A., Shaw, K.N., Saari, L.M., and Latham, G.P. (1981) Goal Setting and Task Performance: 1969-1980. Psychological Bulletin, 90, 125-152.

London, M. (1989) Managing the Training Enterprise: High-Quality, Cost-Effective Employee Training in Organizations.

Long, L.K., DuBois, C.Z. and Faley, R.H. (2008). Online Training: The Value of Capturing Trainee Reactions. Journal of Workplace Learning, Vol. 20 (1), pp. 21-37

Marquez, J. (2006) “Building Knowledge,” Workforce Management.

Noe, R.A. (1986) Trainees' Attributes and Attitudes: Neglected Influences on Training Effectiveness. Academy of management Review, 11, 736-749.

Pfau, B. and Kay, I. (2002) HR Playing the Training Game and Losing. HR Magazine, 49-54.

Piskurich, G.M. (2006) Rapid Instructional Design: Learning ID Fast and Right.

Sacks, A. and Belcourt, M. (2006) An investigation of Training Activities and Transfer of Training in Organizations. Human Resource Management, 45, 629-648.

Salas, E. and Cannon-Bowers, J.A. (2001) The Science of Training: A Decade of Progress. Annual Review of Psychology, 52, 471-499.

Sessa, V. and London, M. (2006) Continuous Learning in Organizations: a living systems analysis of individual, group, and organization learning. Emerald Group Publishing Limited, 123-172.

Sitzmann, T., Brown, K., Casper, W. and Zimmerman, R. (2008) A review and Meta-Analysis of the Nomological Network of Trainee reactions. Journal of Applied Psychology 93 (2), 280-295.

Smith, B.J. and Delahaye, B.L. (1998) How to Be an Effective Trainer. Handbook of Training and Development for the Public Sector.

Tannenbaum, S.I. and Yukl, G. (1992) Training and Development in Work Organizations. Annual Review of Psychology, 43, 399-441.

Towler, A. and Dipboye, R. (2001) Effects on Expressiveness, Organizations, and Trainee goal orientations on training outcomes. Journal of Applied Psychology, 86, 664-673.

1. Research Scholar. Siksha ‘O’ Anusandhan University (Deemed to be University) Bhubaneswar, India. alchemyforu@gmail.com

2. Ph.D. Associate Professor. Dept of Humanities and Social Sciences. Siksha O Anusandhan (Deemed to be University), Bhubaneswar, India. madhusmita71@gmail.com

3. Ph.D. Associate Professor. Dept of Humanities and Social Sciences. Siksha O Anusandhan (Deemed to be University), Bhubaneswar, India. manoranjanibcs@gmail.com

4. Research Scholar in Faculty of Management Studies, IBCS. SoA (Siksha ‘O’ Anusandhan University), Bhubaneswar, India. info.subhankardas@gmail.com