Re-Thinking Learning and Development. Part 1: Thinking about the Numerator

Because of our place as a provider of education products, we interact with many clients on the design and delivery of corporate learning programs.  Based on our experience, it is time to think differently about how we think about corporate learning.

Everyone in the Learning and Development field is trying to determine how to measure Return on Educational Investment.   ROEI = (derived benefits from training and development) / (education and training cost.)  As you can see from this 15-year-old article from Jack Phillips, we’ve been at this for some time.

Or, you can check out the ASTD Handbook for Measuring and Evaluating Training, which also attempts to address the issue.

As another alternative, here is a brief video from a training expert (Lynn Johnson).  See if she makes a convincing business case from your perspective.  (I surely didn’t think so).

How to measure the benefit to the organization of its learning and development effort IS the right question, but most would agree that measuring the benefits is difficult at best.

I have three degrees in engineering and science, and consider myself to be pretty strong analytically.  I believe I can develop a way to measure learning benefits. I can suggest ways to produce data spreadsheets and metrics.   But, if I am completely honest, while I can produce a report measuring such benefits, I’m not sure even I would believe that report.

As the CEO of my own business, I can tell you we spent a lot of money on training that I didn’t think had any measurable impact on our bottom line.   So, as CEO I tolerated it, largely because I didn’t want to be seen as being opposed to learning (that’s like being against puppies and clean air).   However, I allowed money to be spent without a high expectation about the outcomes.   I don’t think I am alone among C-suite inhabitants.

To combat this feeling that training just isn’t “worth it,” in many companies the L&D people are under increasing pressure to manage the DENOMINATOR of that ROEI equation – something we at least have control of.   With this approach you end up offering classes for more people, in 3 days instead of 5, on-line vs. live etc.

However, and in spite of the incumbent problems in measuring educational benefits, it is way too soon to give up on ways of increasing the NUMERATOR.

I am surprised by how often we encounter L&D professionals who ask us to teach a topic, and when we ask “Why do you want to teach this?” or “What is it that you want your employees to think, feel or do differently as a consequence of the training?”  they often have no good answer.

So we may get asked to teach a class on, say Meyers Briggs.   It is perhaps a 1.5 – 2 hr module.  The participants fill out their MBTI survey and then in class we can explain the various dimensions and how to interpret their results.    Now you may be pretty confident you have a facilitator who can teach this and receive very high student evaluation scores.   BUT . . . I would challenge you to survey the participants 2 months after the module, and ask 4 questions:

1)      Can you recall your MB Type indicator classification (like INTJ)?

2)      Can you explain what the dimensions mean (what is I, or E, or T?)

3)      Can you GUESS the MB type of your closest co-workers, based on your observations of their behavior?, and

4)      Given a particular scenario, can you describe how your human interaction strategy could be tailored to match their MBTI classification?

When I did such an evaluation in my own company, we found that only about 30 % could answer question one successfully, and the success rates dropped from there on.   0% (out of 25 participants) could answer all 4 questions.  Hopefully your results would be better than these, (but I would be surprised).

So, if you permit me the luxury of cynicism for a moment, if that is the measure of success for this workshop…  why do it at all?

Here is another way you could approach it.

If we wanted to teach MBTI in an impactful way, this might be a possible approach:

  • For admission to the class the employee first must have a co-worker or 2 with whom they had difficulty interacting – or who challenges them interpersonally.  They should be asked to write a description of the kind of situations that are problematic.   Their REASON, then, to come to the class is to gain some new ideas about how to interact better with that person.
  • Then, in the first 2-3 hr module, we ask participants to describe their problem scenarios, and challenge the class to discuss why these interactions are problematic.  We ask the group how to approach such individuals differently – and see what the group can come up with on their own.
  • Then, in the next 2-3 hr module, we administer the MBTI survey and explain the concepts behind it.
  • In the next 2-3 hr module we invite the class to consider what MBTI profile their personal antagonist has, and then invite them to reconsider the scenarios discussed in session 1 to arrive at a better way to handle interpersonal communications.
  • In between module 3 and 4, we challenge participants to  go back and experiment with their new skills with their problem co-worker
  • In the final 2-3 hr session, participants report back on what they tried, and what happened.  Their fellow participants and the facilitator can offer coaching suggestions as needed.

The problem, of course, is that this approach is 12 hours, rather than 2, and therefore costs 6X as much to deliver. . . In an era of budget constraints, who is looking for a 6X budget increase?   So this latter approach is most likely to be rejected.

But perhaps we should change the structure of training overall to do Fewer trainings… while Going Deeper. Our next post covers just that.

What about you? What’s the most impactful training you’ve ever experienced–and why?



Filed under Innovation

2 responses to “Re-Thinking Learning and Development. Part 1: Thinking about the Numerator

  1. Wayne Fisher

    The most impactful training I’ve ever experienced is the “Train-the-Trainer” workshop by Ed Jones. As a newly minted PhD, the only training model I knew was traditional classroom training. I had developed a Friday afternoon seminar series on design of experiments (DOX) and process optimization for my department. Like your Meyers Briggs example, it was very well received, and helped build my credibility within the group. But it had no lasting impact in the organization.

    After taking Ed’s workshop, a colleague and I transformed the same material into an actionable workshop with very measurable outcomes. After initial success within my business unit, I moved to a corporate role to deploy the DOX workshop globally. We demonstrated $100 million in incremental profit in year 1 on a shoestring training budget, and went on to train thousands of managers at all of our global technical centers.

    The point of this story is that we started the DOX workshop design by clarifying the business benefit we wanted to deliver. Specifically, how is it going to make money for P&G? As you said, focus on increasing the NUMERATOR. This gave tremendous focus to our DOX workshop design. In this case, it was straightforward – lower costs, higher throughput, increased product satisfaction, reduced product cycle times. 20 years later, these measures are still business relevant, and my colleague is still managing this popular workshop globally.

  2. In education, we tend to think about “learning outcomes” which defining what we want you to KNOW when we finish teaching, We design our class experience and lectures based around that, ask you to practice with a problem set (homework), and then assess (give you a quiz).

    It is very different when you think not about learning outcomes, but BEHAVIOR OUTCOMES. It is the same idea, but our work is not done, even if you get it (intellectually), we need to think about application of the learning so that your own behavior is modified.

    This is why companies like Toyota do not start with teaching, they start with a problem they are trying to solve, and then teach around it.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s