Todos los objetivos deben estar reflejados o traducidos a resultados financieros. La siguiente fórmula nos permite determinar el nivel de valor de una actividad, tarea, proceso, producto o servicio en términos financieros desde la perspectiva del cliente:

siendo para el caso:

V = Valor. Es la efectividad y/o productividad en términos de rentabilidad o utilidad de un objetivo.

Q = Calidad. Grado en el cual el producto o servicio cumple con las expectativas.

s = Servicio. Nivel de satisfacción del cliente por la calidad, precio y oportunidad del producto o servicio

recibido.

c = Costo. Insumos requeridos para generar el producto o servicio.

t = Tiempo. El grado de oportunidad en que se recibe el producto o servicio.Antes de comenzar el proceso de construcción de un Balanced Scorecard, se necesita relevar información estratégica que servirá como entradas para los primeros pasos de la construcción del tablero. Se sugiere preparar una checklist de fuentes de información estratégica, incluyendo cosas como planes estratégicos, planes financieros, planes de mercadotecnia, planes operativos, reportes anuales, programas de mejora de la calidad, análisis de los consumidores, entrevistas con la gerencia ejecutiva, otros documentos de planeamiento, análisis competitivos, análisis de tendencias de la industria, análisis de tendencias tecnológicas, análisis de tendencias en mercadotecnia y otros análisis de la industria.

Y hasta aquí van las cuestiones introductorias que quería mencionar para entrar en el planteo inicial del Balanced Scorecard para el área de Tecnología Informática. Yo diferenciaría las cosas en función del tipo de organización. Si se trata de una empresa que no es específicamente de base tecnológica pero posee un área en tal sentido, pensaría en un Balanced Scorecard transversal a toda la organización, tratando las cuestiones de tecnología como objetivos dentro del mapa completo. Ahora, si la empresa fuera de base tecnológica, me orientaría hacia alguno de los modelos específicos, como por ejemplo:

El Balanced IT Scorecard (BITS), propuesto por el European Software Institute (ESI)que provee una

nueva versión de las cuatro perspectivas originales agregando una quinta en relación con la gente

—«People».

El Balanced Scorecard de Advanced Information Services Inc. (AIS), que considera a los empleados

—«Employee»— como una perspectiva distintiva, expandiendo así el análisis a cinco elementos.

Metrics Identified and DiscussedProductivity Metrics

1. In Sprint Cycle Time (4) - measuring the turnover of stories throughout

the sprint as measured by when a story was started vs. when it was

completed. A shorter cycle time is a good indicator. A longer cycle time is a

poor indicator.

2. In Sprint Work in Process (3) - measuring the continuous work in process

flow during a sprint relates to the flow of the team. More work in process is a

poor indicator. Less work in process is a good indicator. This assumes the

lean principle that less work in process leads to higher throughput.

3. Sprint Completion Bar - measuring sprint points using a stacked bar with

4 elements of Done (green), Not Tested (yellow), Not Coded (orange), and

Waste (red). Each element presented in a separate color. The stacked bar

shows work sprint over sprint. Longer Done bars is a good indicator. Longer

Not Tested or Not Coded are a poor indicator. Any waste (as defined by work

that was subsequently removed due to not needed) is a very poor indicator.

Productivity Myths1. Completed vs. Commited % (a.k.a. Earned Value)- measuring the

complete vs. the original sprint plan commitment expressed as a

percentage. Used to say if > 80% then the team succeeds the sprint, less

than that the team fails. This is often misused because if pushed, teams will

tend to either be safe in their commitment (e.g. lower productivity) or hide

unfinished work.This metric can be used effectively if the term "failure" is

removed and the team is not pushed to meet a certain percentage.

2. Velocity - measured by number of points completed per sprint. If used as a

productivity measure, it may initially drive the team to increase productivity,

but if pushed, will eventually lead to story point inflation. Avoid using

velocity to compare different teams. Rather, velocity should be considered

as a predictability measurement.

3. Individual Performance Reviews - Most companies use performance

reviews to measure individuals for compensation. These tend to drive

individual (anti-team) behaviors that work against team productivity.

4. Resource Utilization - Measuring individual percentage busy-ness factor.

Teams that are pushed to utilize each resource fully tend to locally optimize

their teams - which means their whole system is sub-optimized. This is a lean

principle trying to increase team throughput requires sub-optimizing

individuals.

5. Business Value measuring performance - Attempting to measure

productivity through delivery of business value counters a typical prioritized

backlog approach. If properly prioritized, higher valued stories will be at the

top of the product backlog and thus early sprints will deliver more value. If

value is used to measure productivity - it will lead to dysfunction when lower

valued, but higher complexity stories enter the backlog. Rather, this metric

should be used to validate the prioritization of the backlog - not performance

of the team.

6. Source Lines of Code (SLOC) - measured by the number of lines of code

to determine productivity. A higher number does not necessarily translate to

higher productivity - only more code. Using this as a measurement will go

against the principles of "simplest thing possible" and "re-factoring" to keep

code simple. A productive story may actually reduce code, not increase it.

 

Quality Metrics1. Technical Debt Points (12) - measuring the volume and throughput of

technical debt to determine the quality evolution of a product. A higher

number of points is a poor indicator. A lower number of points is a good

indicator. This can be used with a technical debt limit - if the debt exceeds a

certain value, the team will be put into a forced debt reduction format. This

can also be used in a stacked bar sprint over sprint showing technical debt

vs. new stories.

2. Running Automated Tests (4) - measuring unit and functional automated

tests that are passing each sprint. A higher (and growing) number is a good

indicator. A lower (or flattening) number is a poor indicator. This metric was

detailed by Ron Jeffries. Paired with code coverage metrics, it can drive both

productivity and quality behaviors.

3. Post Sprint Defect Arrival (3) - measuring the number of defects that are

found after the sprint they were initially developed. A higher number is a

poor indicator, a lower number is a good indicator. This is a leading indicator

of quality in that it attempt to predict released quality based on incremental

sprint quality. Reminder - this is a team metric - not a QA only metric.

4. Post Release Defect Arrival (1) - measuring the number of defects found

after release to customers. A higher number is a poor indicator, a lower

number is a good indicator. This is a lagging indicator of quality meaning

that quality is not measured until after release. Reminder - this is a team

metric - not a QA only metric.

5. Root Causes Fixed - measuring the number of defects that defined and

fixed a root cause. A higher number is a good indicator, a lower number is a

poor indicator. This works well in a sustaining team environment to drive

fixing core issues, not just the symptoms.

 

Quality Myths1. Number of Test Cycles (1) - measured by how many times a story or

sprint iterates between development and test. As presented, it was indicated

as a lower number is a good indicator. This metric, however, could lead to

fewer feedback loops.

2. QA Story Points - measure stories in separate engineering and quality

points. This allows QA teams to better predict their productivity and

throughput. The concern is that this separates the engineering and quality

teams going against the Scrum principle of cross-functional teams. It also

limits the engineering/test conversation which is critical to effective Scrum

teams.

3. Quality Number of Defects - measuring Quality and Quality productivity

by the number of defects they find. A higher number is thought to increase

quality and measure productivity, however it goes against the Scrum team

principle of whole team. It is better to encourage both QA and developers to

work towards story completion - not

4. Quality Checked After Development - not really a metric, but not a good

thing to do.

 

Predictability Metrics1. Enhanced Burndown Chart (12) - measuring both velocity and scope

change throughout a release as a measure of meeting a release goal -

predictably. This metric can be combined with cost metrics (below) to

measure predictability in project costs.

2. Velocity (4) - essentially a subset of the metric above.

3. Cost per Story Point (2) - measuring the $ per story point by knowing the

number of people on the team, the sprint length and the story point velocity

to determine the average cost per story point. This can also be used to

measure capitalized vs. non-capitalized expenses in IT firms by separating

the story points which are capitalized and non-capitalized on projects. It is

recommended to use a average number for both costs and velocity by using

a number (3) of sprints.

4. Hours per Story Point - measuring the average number of hours estimated

per story point. Used to determine if the estimated labor to implement story

points is changing over time. For example, a team that starts out at 10 hours

per point on average migrates to 20 hours per point shows that the point

value is decreasing over time. This metric can easily be misused either by

the team in estimating points using the reverse of this metric or by leaders

pushing teams to decrease their estimated hours per point (assuming they

are making them more efficient). This should only be used to evaluate

predictability. Other methods are to use a "reference" story, or to create a

estimated labor hour range per story point rather than a specific number.

 

Value Metrics1. Business Value Delivered (7)- measuring the value delivered each sprint

based on assigning a business value to each item in the backlog. A higher

number is a good indicator.  This can also be an employee motivator

because teams want to deliver value and feel good about being able to. As

mentioned above, this can be dysfunctional if teams attempt to sustain

business value over the course of multiple sprints because if the backlog is

prioritized correctly, business value should be lowering over the course of

the release.

2. Customer Satisfaction Survey (6) - measuring feedback quantitatively

and/or qualitatively from customers and other stakeholders through a

regular (quarterly) survey. Various feedback can be gathered on quality,

predictability, productivity of delivery, support, appropriateness of new

features, etc.

3. Employee Satisfaction Survey (3) - measuring the qualitative and

quantitative feedback internally from employees with a regular survey

(quarterly) and tracking the results over time. This can measure

effectiveness at roles, quality of work/life balance, teamwork, product

definition, process adherence, feedback, happiness value, etc. A happy

workforce tends to increase productivity due to people wanting to be at work

and contribute to team results.

Metrics In Agile Home Metrics Metrics in Agile

Christian Miles Thursday, August 06, 2015

I've been meaning to write a blog post all about metrics for a very long... It's a really

important subject and one that doesn't get spoken about anywhere near enough - I

also plan to come back and add to this as time allows.

Metrics in Kanban as opposed to Scrum is perhaps more common place, Kanban is

much more a case of bench marking where we are, Conducting 'experiments' and

measuring the result before rolling back or continuing.

But, Good metrics not only allow you to understand the effects of experiments

made but also allow you to start planning with at least something approaching a

scientific methodology! Although admittedly it does take more effort than the old

finger in the air, double it + a safety margin trick that!

Below are some of the main metrics I use regularly.

Flow efficiency

I love this little metric - It gives a quick indication as to how efficient your process is.

The exact way you record it might differ based on what you're trying to measure...

but roughly, Record the date a story is requested, record the date it's released and

record the number of days it was actually worked on.

So if a story takes 20days from being moved from the backlog to the todo column

and is worked on (developed/tested/etc) for 5 days you have a flow efficiency of

5/20= 25% Which although looking low is probably a really good figure!

Cumulative flow diagrams

I love a good CFD... It's a really good visual way to track a project. Most software

tools will allow you to generate a Cumulative Flow Diagram - Otherwise you can

export the data to Excel and generate one.

Generally speaking the closer together the lines the lower the WIP (Work In

Progress) and shorter the lead times should be and the more work should get done

(green)

You can see from the graph a very flat section early on where no work was getting

done - That was before I joined as Scrum Master ;-) And another slight decrease in

work 'done' and a longer lead time about 3/4 along... That could be a cause for

concern.. It was actually the Christmas/New Year period but it shows graphically

how well a team is working and how much work is left - These are great to take into

retrospectives.

Lead Times

Lead times.... Are simple, How long from requesting a story (Moving from the

backlog to todo) until release (or 'Done')

However you probably want to start recording lead times by story/ticket type.....

If your using story points starting plotting lead-times against story points using a

histogram.

You start to get some really useful information back,

For example - You might be able to say that 85% of stories estimated at 13SP are

delivered in under 5 days but up to 5% take longer than 10days.. However 50% are

delivered in 3 days or less.

Having powerful metrics like this allows you to really plan... You can pick the right

time to start doing work and know with a level of confidence when it will be

complete.

You can prioritise stories more precisely - Knowing you can leave that really

important legal change story until 10days before it's required as it's estimated at

13SP story which than means you can work on the 20SP story first as it'll save X-

thousands every day!

Cycle Times

Cycle times are very common in Kanban and Scrumban...

It's a measure of the number of tickets in progress and the number of tickets

completed daily - You can sample at different rates but I tend to run Scrumban in

weekly cycles but different teams will work differently.

So... To keep the numbers simple, If you have 10 tickets in progress and complete 2

a day (average) - You cycle time is 5. 10/2

You should find if WIP increases so will the cycle time and the opposite too!

Start plotting cycle times and you'll begin to see trends... And trends are how you

know it you're improving... or not! It's also useful to start plotting different types of

tickets together - So below we can see that the cycle tine in May/June increased

from the norm... But so were the cycle-times for bugs... You can't draw a conclusion

from this but you can ask questions and understand.

Happiness/Satisfaction

This probably gets overlooked more than any other measurement. But measuring

the happiness of your team and customers is probably one of the most important...

Reality is important but perception probably more so!

This can be especially important if running an agile transformation... Bench mark

how happy the team is! Understand your customers issues/concerns and overall

level of happiness with the service.

You could create a mood wall, questionnaires, etc. The The mood wall in particular

are really good for discussing in retrospectives.

Velocity!

Well I couldn't write a blog covering metrics without mentioning it...  But I don't

really want to discuss it too much on this blog, Velocity is fine for an internal team

metric and helping to predict burn-down against a backlog or for allowing

meaningful negotiation about story sequencing, etc.. But don't get carried away

with it.. Try not to chase it and don't let people abuse it!

And if you do want to know more..... Velocity 

Burn down

The burn-down chart - These are very common in Scrum, both for tracking progress

in a sprint and for measuring against the backlog.

I must admit... I'm not really a great fan of the Sprint Burn-down - It seems to have

become another chart that management overuse (Like velocity).

You should be able to track progress of stories completed via the Kanban/Task

board.

If you are doing a sprint burn-down - Perhaps rather than looking at hours against

tasks and burning down perhaps looks at the number of stories in the sprint and

burn down against that measurement - It's actually recording what's important

(Completed stories)

I do however like a burn-down against a backlog (Although if you're using the CFD -

this is less important) If you do have a well estimated well 'refined' backlog and you

have a stable velocity figure these can be useful to either predict delivery or to alert

early on that you're going to miss a release point.

In Conclusion 

Metrics are important...  They allow you to safely change the workings of a team

and understand the effect. Management typically prefer's this way of working, it's

safer and you have evidence to back up decision making.

At the planning level metrics are invaluable - And allow you to plan with some

confidence.... Although planning when you have actual metrics suddenly becomes a

much more complex and tiring process than just guessing it!

But you have to them use metrics responsibly and remember Goodhart's law

- "When a measure becomes a target, it ceases to be a good measure."

Try and keep away from absolute numbers and monitor trends instead - The CFD's

are brilliant for this - Look at the general trend not specific numbers. Don't chase a

number but try and understand why those numbers are changing.

Agile Velocity Home Velocity Agile Velocity

Christian Miles Friday, September 19, 2014

I was in a meeting recently explaining some ideas on how we could track velocity

within a team I was leading. When the ops manager I was talking with finally came

clean and admitted he had no idea what velocity was! And why was everybody

talking about it and could I just explain in simple terms what it was all about!

Well that's one of the problems with IT and especially agile people... we do love our

buzz words.

Planning Poker being played

So what is velocity?

Well - There's probably lots of subtly different definitions.. but put simply - It's a

measure of how quickly the team deliver's 'done' items of functionality over a

sprint.

I've just read that back and to be honest there is nothing simple about that

explanation! So no wonder he still looked confused afterward!

So perhaps if I explain how velocity is calculated it will help!

So how do you calculate the team's velocity?

With every sprint a number of user-stories should be selected and committed too..

Each of these stories should be estimated with a number of story points, typically

these are based more or less on a Fibonacci number sequence scale (1/2,

1,2,3,5,8,13,20,40,100) - Fibonacci number sequences are fascinating being found

in nature and used in computer science (Fibonacci search technique and Fibonacci

heap to name just two uses)   - The numbers we use in planning poker don't exactly

stick to this sequence.... 1/2 has no place whatsoever in a Fibonacci sequence and

40 should be 33!

There are various methods of estimating work - planning poker is just one and I've

blogged about several techniques in the past - but for now suffice to say that every

story (and perhaps task) of work should be estimated with a relative amount of

effort to complete.

At the end of the sprint you sum up all the story points which are 'done' (remember

it's important to enforce that definition of done) and that's the team's velocity!

So - If you have 10 stories and via planning poker or some other estimation

technique you decide that each task is worth 8 units of effort and you complete all

10 stories the velocity for the team in that sprint is 80.

You can now start to use this figure in planning - Both for helping to establish how

much work the team should commit to within a sprint - The best logical order to do

work and for producing burn-down or burn-up charts (my favourite is a burn down

chart)

I tend to use an average velocity figure for planning - You should expect that the

velocity figure will probably change over sprints (hopefully upwards)

In the past I used to calculate the average from the 2nd sprint onwards (I always

assume the 1st Sprint might be a little different as the team beds in) and than use

standard deviation to exclude the unusual sprints! more recently I've been

calculating the average on just the last 3 or 4 sprints - I don't have definitive proof!

but this does seem to give a more realistic velocity figure and tends to allows for

improvements to filter through quicker to the average and be reflected in the burn-

down charts

Hopefully that's explained Velocity!