My review entailing Viplav Baxi's comments

I like the style Viplav explained on the difference between metrics/numeric measurement, and paths and patterns in analytics. I was trying to explain it in words for quite some time but I just couldn't, even with a clear vision in my head on how it looks like...

Re: What about learning analytics in the corporate sector? by Viplav Baxi - Sunday, 16 January 2011, 05:49 PM

One of the areas that greatly interest me is simulations for corporate training needs. I have done work in the IT and Financial sectors that show the power of simulations to bring together some complex information about how a learner navigated a simulated job situation. Scores are too simplistic to do justice to such complex tracking of learner progress and competence. Consequently, learning & knowledge analytics become more complex as well. For example, let us consider a scenario that has multiple decision points (connected like in a graph) and multiple paths to the correct outcome. Let us assume that there is an ideal path (not hard to imagine in a highly disciplined process training). A learner's decision making trail or actions trail could be compared to the ideal path/trail and analytics could be programmed to infer from deviations to get a better and more comprehensive picture of learner performance. (also not unlike the notion of knowledge analytics being used to compare competency levels in a discipline). If you know cricket, you would be familiar with a graph that shows runs scored vs overs for both teams with circles denoting fall of wickets, resulting in what are popularly called "worms" that deviate from each other on the graph as the match progresses. Point is, these analytics move from being comparisions between numbers (Peter spent 5 more minutes than Pan on the google group), to being comparisons and analytics based on patterns and paths. Viplav

My comments:

Thanks, Viplav... Your explanation really put my understanding in words, quite clearly (about the patterns and paths versus numbers).

Tailing this, I wonder:
- How can this pattern be presented to senior management (assuming they are not at 'our level of understanding') in a form that they appreciate?
- How can we be certain that the paths the employee takes in getting to the final point are something they learnt and contributing to the final result, or merely a waste of time until they found the right 'nodes'?
- How should the evaluation be designed to make it fair for every employee (because some people do take time to understand after a long-winded paths, etc...)?
- Does this mean an experienced 'wanderer' could achieve the KPI better than newbies? I don't think so too.

There's still further way to go after this, and I believe corporate sector is very particular about 'measurement' in evaluation.

Hope I'm in track with my points,
- Shazz
Kuala Lumpur 1:33AM

Still wonder if I'm in the same track,

- Shazz @ LAK

19 Jan 2011