It is a mistake to confuse methodology with objectivity

People use a variety of evaluative processes in business to ensure objectivity. But the dangerous fact is that these evaluative processes, these methodologies, even the most rigorous, are woefully insufficient to guarantee objectivity.

Why? There are three basic dangers in relying on method to ensure objectivity.

1. The normative dangers

The overarching danger attaching to evaluative models is the belief that we can use methodology as a legitimating device, even if our evaluative methodology has flaws. Furthermore, if any kind of injustice has resulted from the application of a methodology how can it even be called an injustice, if the "test" for justice is the methodology?

2. The epistemic dangers

I use staffing as a paradigmatic example because of its situation, which cross-cuts the human and the operational domains of management.

To ensure transparency and effectiveness, it is assumed that a method which permits the definition, measurement and testing of people's "competencies" will do the job.  First, who guarantees that the definition of competencies is appropriate? What sanctions the applicable criterion?  It depends on the motive - are the criteria sanctioned in terms of the business needs that the hire will meet or the efficiencies that the hiring process will meet? (If you have to think more than 5 seconds on that one, you're in trouble.)

Second, competency is not a feature of potential hires like the number of doors on a car. People are living, learning beings with potential and we can't screen for traits we haven't identified but might have found very valuable.

Competencies mean little without relationship to motivation to develop them further and opportunities to use them.

We have a collective Western habit of freeze-framing states and confusing the ability to label the frame with the ability to recognize the meaningful stuff in phenomena. Nothing living stands still, including the level of the abilities of your potential staff.

3. The ontological dangers

Empiricism is at pains to distinguish itself from teleology - the identification of ends. The underlying belief at stake is the idea that a grand purpose or design to everything is an anthropomorphic projection. This may or may not have clear import in certain areas of scientific research, but to carry that principle of wariness over into the types of planning whose purpose is to improve quality of life is ludicrous.

For example, in staffing (the most clear examples come from that domain, as I explained earlier), operationally defining merit criteria in terms of a screening methodology puts the methodological cart before the ontological horse. A competency is not an exhausively definable capacity, but a set of potentials that evolves over time. Efforts should be made to ensure fairness and objectivity, and to ensure thereby that the right person is selected for the job (i.e., the "telos" or end is that the essential criteria are connected to the needs of the job and the person's fitness for the role overall is assessed according to those ends). This connection is lost when the specific criteria become disconnected and itemized, and especially when they become too narrowly defined due to a different objective - the objective to manage the "empirical" screening process efficiently. Of course, divorced from its ends, efficiency has no meaning. This is neither fair nor effective.

I may have said this before, but it is clear that no matter how much “empirical evidence” we have, in the end it is our capacity for meaningful valuation that allows us to identify worthwhile goals and our capacity for reasoning that allows us to collect the relevant evidence in support of those and assess it. These two capacities are not separate, but deeply connected in us and contribute equally to the quality of our lives. My examples tend to be drawn from staffing because they are the most straightforward, but the underlying principles could apply to many areas public admin.

Going against the grain of the contemporary Zeitgeist, to ensure objectivity, and by extension, fairness,  in people or program evaluation, we should be sure to use our capacities for meaningful valuation and reason to the best of our abilities.

Employee Engagement: Competence, Motivation, Opportunity

In his Contrarian Thinking blog, Etienne Laliberté posted a link to a discussion paper entitled "An HR Director's Guide to Employee Engagement". I think the authors really underscore the limitation of the competency approach here:

The long-standing focus on ‘skills’ as key to increasing productivity is too limited. The ‘people and performance’ model identifies the three requirements for releasing employees’ discretionary effort as ability, motivation and opportunity. The skills agenda addresses only the first of these – employee engagement addresses all three. There is an historic opportunity for the Government to demonstrate that it understands the drivers of...performance.

Their Main Point: You not only need the relevant competencies, you need opportunity and motivation.

But here's the tricky question: Which is more important?

First, consider whether relevant competencies are something people are born with. Generally they are learned. They are abilities that people learn both formally and informally, in the classroom, and more often on the job.

Second, consider what are called "generic competencies". Who cannot learn them? Apart from the tragic cases, people have capacity to learn most generic competencies, and it's a matter of motivation and opportunity whether they realize these capacities or not.

I can understand that for jobs in which professional certification and/or very well-developed technical skills are required, these requirements would be included in a job description or a job posting. But it is very important to understand that competencies are not like standard features on a car, like horsepower, or doors, or seating. They are not static givens that people are born with, they are things that, given the opportunity and motivation people will learn, develop and display.

The fact that competencies are learned, not givens, raises questions about the value of hiring on the basis of lists of generic competencies. We should be looking for potential and providing opportunities, not looking for static givens thereby ruling out potential.

If I were hiring someone for for a job, I would certainly ensure they had the key professional technical expertise required, but on the more generic competencies, I'd forgo the standard list, except perhaps for communication skills, and look for what they enjoy:

What did they do or what ideas do they have that make their eyes light up when they tell you about them? Are they interested in the fine details, or are they interested in the big picture? Do they prefer action or contemplation? Do they feel anxious if things are not perfectly clear and stable, or can they tolerate a certain level of ambiguity? Are they more comfortable dealing with "just the facts" (task masters) or do they explore possibilities (experimenters)? Are they motivated primarily to climb the ladder (careerists) or to express creativity (innovators)? Do they enjoy working alone or in groups? What balance of direction/discretion are they most comfortable with? In what way do they really want to make a difference in their professional lives?

Understanding people's preferences, their motivations, would tell me most of what I'd need to know, if not more, than checking off the standard lists. And given the opportunity to act in accordance with their preferences and motivations, they would continually develop the relevant competencies.

So motivation and opportunity do not merely "complete" the trio of elements of employee engagement necessary to improve performance, they actually are the basis for learning, developing and displaying competence.

The Logic of Quantophrenia

Here's a crash course on the reasonable and unreasonable (quantophrenic) use of indicators (business metrics):

Reasonable:

Given a reliable indicator (sign) of a concrete problem, there is a concrete problem.
If the problem is resolved concretely, the indicators should represent (track/signal) that.

I>P (If I then P)
Therefore,
~P > ~I (If not P then not I)

This is a logically valid form of reasoning. It spurs committed people to remedy the concrete problem.

A simple example for illustrative purposes: Imagine that an unexpectedly high number of employees have been on long term acting assignments in different parts of your organization. What does that indicate? Let's say you do some research and your investigation reveals that, although there are a number of bona fide developmental activities occurring, the problem is due to cumbersome staffing processes and a weakness in integrating business and HR planning, making assignments a quick remedy. So what do you do? You reason that if you streamline staffing and pay more attention to resourcing with respect to projected business requirements, over time, your numbers should diminish to levels that would be expected given developmental activities.

Quantophrenic version:

Given some accurate indicator (sign) of a concrete problem, there is a concrete problem.
If the indicators have changed for the better, the problem has been concretely resolved.

I>P
Therefore,
~I > ~P

IS NOT a valid form of reasoning (it commits what logicians call "the fallacy of denying the antecedent"), and in itself a sign of a problem; i.e., tick box management.

To illustrate, using the previous example, an unexpectedly high number of employees on long term acting assignments indicates problems with cumbersome staffing processes and a weakness in integrating business and HR planning. So what do you do? To meet your indicator target, you remove everyone from their acting assignments. Obviously, you have met your indicator target 100%, but you have not solved the concrete problem, in fact you have quite clearly exacerbated it.

This is a very simplistic example, but it illustrates the potential dangers of metrics as business drivers.

Question: what drives quantophrenia?