I had this idea today. I'm studying some scrum and agile project management with several of my comrades-in-arms here at Object Mentor. We were talking about tracking three numbers: goals, estimates, and commitments.
The following is for entertainment purposes only. I don't have any kind of statistics gathered to prove or disprove any of these points. If you follow this as if it were "advice" or "professional opinion" then I will not be responsible for your success or failure. I don't know that any of this is true.
So it dawned on me that there is an interesting possibility that arises if you track all three items. If you find, over time, that the goal is always the commitment you can optimize by not doing any more estimation (since nobody is listening to them anyway). That gives you more time to work on making the commitment.
If the goal is always the commitment, it may show that marketing and/or management are running roughshod over the group -- that they are inflexible people demanding flexibility from those around them. Sort of like a boor who demands everyone around him shows respect and good manners.
OTOH, if the estimate always becomes the commitment, it shows develpment may be running roughshod over the marketing timeframe, and that's not so good either.
Maybe the graph of these items will show how good your development managers balance the concerns of the development staff against the concerns of the marketing staff. Maybe evidence of compromise might make us appreciate our project managers a bit better.
If the goal and estimate converge maybe it is meaningful in some other way... if only to show that we listen to each other (or browbeat each other into submission).
But who knows, really? It's just an idea.
!commentForm
- Goal - what marketing & management wants to see to meet their schedule
- Estimate - the time you honestly believe it will take (plus or minus some amount) to do the work.
- Commitment - when you agree to provide it, whether it is long or short of the goal, and whether it is long or short of the estimate.
The following is for entertainment purposes only. I don't have any kind of statistics gathered to prove or disprove any of these points. If you follow this as if it were "advice" or "professional opinion" then I will not be responsible for your success or failure. I don't know that any of this is true.
So it dawned on me that there is an interesting possibility that arises if you track all three items. If you find, over time, that the goal is always the commitment you can optimize by not doing any more estimation (since nobody is listening to them anyway). That gives you more time to work on making the commitment.
If the goal is always the commitment, it may show that marketing and/or management are running roughshod over the group -- that they are inflexible people demanding flexibility from those around them. Sort of like a boor who demands everyone around him shows respect and good manners.
OTOH, if the estimate always becomes the commitment, it shows develpment may be running roughshod over the marketing timeframe, and that's not so good either.
Maybe the graph of these items will show how good your development managers balance the concerns of the development staff against the concerns of the marketing staff. Maybe evidence of compromise might make us appreciate our project managers a bit better.
If the goal and estimate converge maybe it is meaningful in some other way... if only to show that we listen to each other (or browbeat each other into submission).
But who knows, really? It's just an idea.
!commentForm
To quote Dennis Miller, I don't want to get off on a rant here, but....
I don't understand what the point of an estimate is, if not to set the commitment. Why would you have a bunch of experienced software developers waste time estimating how long something is going to take if you're just going to chuck it out the window? Aren't you just setting false expectations about what will be delivered and when?
In my experience, if the estimate and the commitment are not the same, bad things happen. Maybe everyone will stay in denial about getting everything done on time, only to finally admit 2 days before the trade show that it's only 80% complete. Maybe the developers will try to take "short cuts" by not writing tests, or by ignoring bugs they find...leading to an unstable product that (might) work in a demo, but certianly not in the hands of an actual user. Maybe the team will put in month after month of 80 hour weeks in order to get it done, and then quit to go work at more reasonable companies that understand the value of talented developers. However it manifests itself, there's no such thing as a free lunch.
I suppose if you know that going in, and that's what you're trying to do, that might be OK...but those side effects are always much harder to measure and predict than a delivery date. I can tell you (with a fair degree of accuracy) how long I think it's going to take to implement feature XYZ. If you ask me how many 60 hour weeks Bob will work before he starts looking for a new job, I won't have the foggiest idea. Why replace a low risk activity with a high risk one?
As a developer, I think that agreeing to meet a goal, without making sure that all the stakeholders understand my honest opinion as to the risks involved, is dishonest. As a project manager (I wear multiple hats at my company) I would be upset with a developer who gave me an intentionally inflated (or deflated) estimate, and I take great pains not to let the temporal desires of management and marketing interfere with the estimation process. Otherwise, we (as a company) will not have a realistic idea of the risk we're undertaking, which usually leads to disaster.
Marketing and management goals should never be used to set the budget, the scope, <i>and</i> the delivery date. One of those items is always a function of the other two, and pretending that it isn't doesn't change that fact. I have never understood the philosophy of giving people more work than they can handle to try to make them work faster. Maybe it's because I've never seen that approach be successful. The work usually gets "done", but you always wind up paying for it in other ways. So again...you're just taking on lots of risk! Are software projects really not risky enough?! Is a 70% industry-wide failure rate really so low that we can afford to heap all this additional risk of top of what we already have?
...of course, that's just my opinion, I could be wrong.
I don't understand what the point of an estimate is, if not to set the commitment. Why would you have a bunch of experienced software developers waste time estimating how long something is going to take if you're just going to chuck it out the window? Aren't you just setting false expectations about what will be delivered and when?
In my experience, if the estimate and the commitment are not the same, bad things happen. Maybe everyone will stay in denial about getting everything done on time, only to finally admit 2 days before the trade show that it's only 80% complete. Maybe the developers will try to take "short cuts" by not writing tests, or by ignoring bugs they find...leading to an unstable product that (might) work in a demo, but certianly not in the hands of an actual user. Maybe the team will put in month after month of 80 hour weeks in order to get it done, and then quit to go work at more reasonable companies that understand the value of talented developers. However it manifests itself, there's no such thing as a free lunch.
I suppose if you know that going in, and that's what you're trying to do, that might be OK...but those side effects are always much harder to measure and predict than a delivery date. I can tell you (with a fair degree of accuracy) how long I think it's going to take to implement feature XYZ. If you ask me how many 60 hour weeks Bob will work before he starts looking for a new job, I won't have the foggiest idea. Why replace a low risk activity with a high risk one?
As a developer, I think that agreeing to meet a goal, without making sure that all the stakeholders understand my honest opinion as to the risks involved, is dishonest. As a project manager (I wear multiple hats at my company) I would be upset with a developer who gave me an intentionally inflated (or deflated) estimate, and I take great pains not to let the temporal desires of management and marketing interfere with the estimation process. Otherwise, we (as a company) will not have a realistic idea of the risk we're undertaking, which usually leads to disaster.
Marketing and management goals should never be used to set the budget, the scope, <i>and</i> the delivery date. One of those items is always a function of the other two, and pretending that it isn't doesn't change that fact. I have never understood the philosophy of giving people more work than they can handle to try to make them work faster. Maybe it's because I've never seen that approach be successful. The work usually gets "done", but you always wind up paying for it in other ways. So again...you're just taking on lots of risk! Are software projects really not risky enough?! Is a 70% industry-wide failure rate really so low that we can afford to heap all this additional risk of top of what we already have?
...of course, that's just my opinion, I could be wrong.
"In my experience, if the estimate and the commitment are not the same, bad things happen. Maybe everyone will stay in denial about getting everything done on time, only to finally admit 2 days before the trade show that it's only 80% complete. Maybe the developers will try to take \"short cuts\" by not writing tests, or by ignoring bugs they find...leading to an unstable product that (might) work in a demo, but certianly not in the hands of an actual user."
In my experience, these are all functions of tying the estimate to a commitment. If there is an implicit commitment in the estimate, developers tend to avoid raising the visibility of lessons they learn along the way, sucking it up in the false hopes that it will all magically come together, like the disastrous dress-rehearsal the night before the breathtaking opening night.
If estimates and commitments are not tied - if inaccurate estimates are not viewed as a failures to meet a commitment - then developers are encouraged to address issues early on, giving managers more time to make reasonable decisions.
In my experience, these are all functions of tying the estimate to a commitment. If there is an implicit commitment in the estimate, developers tend to avoid raising the visibility of lessons they learn along the way, sucking it up in the false hopes that it will all magically come together, like the disastrous dress-rehearsal the night before the breathtaking opening night.
If estimates and commitments are not tied - if inaccurate estimates are not viewed as a failures to meet a commitment - then developers are encouraged to address issues early on, giving managers more time to make reasonable decisions.
What he said. :-)
If the estimates follow the goals or the commitments there seems to be a problem. It is hard to tell from a graph which followed which. Admittedly, in a "proper" agile project the developers make the estimates and the product owner accepts them and schedules accordingly. In such a system, the estimate will always determine the commitment. However, there are reasons that maybe they will not.
I remember reading a wise man (I believe Tom Demarco) saying that a real estimate is as likely to be short as long. Think about that for a minute. Most of the time managers asking for an estimate don't really want to know what they're into. They want to know what is the soonest that they can receive a feature. The less seasoned developer gives just that. If all goes well, you can expect the feature in X days.
I'm not talking about padding. I'm just talking about knowing what you're in for. If we report the minimum or near-minimum time, then we make our bosses happy for today, but angry next week. We certainly can just accept other people's goals as our estimates if we really want to, but I think it's self-defeating. Telling people what they want to hear may be good politics, and might be good comedy. It might sell some pop songs. But none of those things are engineering.
If the estimates follow the goals or the commitments there seems to be a problem. It is hard to tell from a graph which followed which. Admittedly, in a "proper" agile project the developers make the estimates and the product owner accepts them and schedules accordingly. In such a system, the estimate will always determine the commitment. However, there are reasons that maybe they will not.
I remember reading a wise man (I believe Tom Demarco) saying that a real estimate is as likely to be short as long. Think about that for a minute. Most of the time managers asking for an estimate don't really want to know what they're into. They want to know what is the soonest that they can receive a feature. The less seasoned developer gives just that. If all goes well, you can expect the feature in X days.
A: How long will it take for this feature?
B: Four days.
A: Four days? That's pretty quick. Could it run over that estimate?
B: Of course it *could* run over.
A: How much do you think? A week?
B: Of course not! Maybe two or three days.
A: Ah. Three days. So if things go wrong it could be seven days?
B: That's what I said.
A: Hmmm... Could it be done three days early?
B: NO WAY! I said it would take four days. Why do you think I could do it in one?
A: How early could it be?
B: <snidely> In a ''perfect world'' maybe in three.
A: Ah. If the minimum is 3 and the max is 7 then the ''real'' estimate is 5 days. It's as likely to go under as over.
B: But I think I can do it in four!
A: Right you are! You can commit to doing it in 4 but the real estimate is 5. I don't know why you think you'll come in earlier, but it's your time.
I'm not talking about padding. I'm just talking about knowing what you're in for. If we report the minimum or near-minimum time, then we make our bosses happy for today, but angry next week. We certainly can just accept other people's goals as our estimates if we really want to, but I think it's self-defeating. Telling people what they want to hear may be good politics, and might be good comedy. It might sell some pop songs. But none of those things are engineering.
When we talk about estimates, I don't think we're referring to the same thing. For me, a real-time estimate is incomplete unless it accurately addresses the issue of risk.
I completely agree that telling people what they want to hear is bad mojo. I would always rather have a little argument today that a big one tomorrow. I also agree that using "if nothing goes wrong" time as a basis for an estimate is...let's say...inadvisable.
For these reasons (and others), I really don't like to give estimates for individual features in real days. Use story points, or ideal time, or gummy bears or whatever you want but please...for the love of God...no real time estimates! There's just too much noise in that signal. For me, estimating work is always about relative difficulty and past history. If you don't have any past history, make some! Or else realize that your estimates will probably be very different in a few weeks. I don't want to sound like an infomercial for the XP planning game here, but estimating in real time just opens up the door to speculation, argument, and misunderstanding. There's a better way.
Tim, we agree on another point...you said that an estimate is just as likely to be short as long. Very true. I try not to make the mistake of reporting mean velocity to a customer who doesn't really understand what it is. That's because they might make the mistake of taking that mean velocity, dividing the size of the backlog by it, and reporting to their boss that the project will be done in X days. That's why I report my velocity like this:
90% confidence velocity : 29.4
70% confidence velocity : 32.1
50% confidence velocity : 38.7
Now, the 50% confidence velocity is mathematically the same as the mean velocity, but presenting it in this format exposes it's true nature...a coin flip. And I'm not talking about padding here either. These numbers come from statistical analysis of past history...not just some developer giving me his gut feel. That means when the commitment is made, we all know exactly what we're commiting to: We plan to deliver X functionality by Y date because we believe there is a 90% chance of success.
Maybe the whole statistical thing is too much for your customers. Maybe they wouldn't know a normal curve from the Coney Island Cyclone. Fine. Do the math yourself and give them more descriptive terms like "Possible", "Likely", etc. The point is this: The basic unit of measure for an estimate is not hours, days, weeks or months, but risk. Talking about estimates without talking about risk is like playing poker without looking at the cards. It's up to the developers to assess the risk, and it's up to the business facing stakeholders to translate that risk into dollars (that's what an MBA is for anyway). If we do that, everyone gets what they need. The bean counters get beans, the geeks get clear goals, and everybody knows what the deal is from day one.
Maybe we should be careful not to agree too much. People are going to think I am just saying things you want to hear. +1 re: story points. - Tim
I completely agree that telling people what they want to hear is bad mojo. I would always rather have a little argument today that a big one tomorrow. I also agree that using "if nothing goes wrong" time as a basis for an estimate is...let's say...inadvisable.
For these reasons (and others), I really don't like to give estimates for individual features in real days. Use story points, or ideal time, or gummy bears or whatever you want but please...for the love of God...no real time estimates! There's just too much noise in that signal. For me, estimating work is always about relative difficulty and past history. If you don't have any past history, make some! Or else realize that your estimates will probably be very different in a few weeks. I don't want to sound like an infomercial for the XP planning game here, but estimating in real time just opens up the door to speculation, argument, and misunderstanding. There's a better way.
Tim, we agree on another point...you said that an estimate is just as likely to be short as long. Very true. I try not to make the mistake of reporting mean velocity to a customer who doesn't really understand what it is. That's because they might make the mistake of taking that mean velocity, dividing the size of the backlog by it, and reporting to their boss that the project will be done in X days. That's why I report my velocity like this:
90% confidence velocity : 29.4
70% confidence velocity : 32.1
50% confidence velocity : 38.7
Now, the 50% confidence velocity is mathematically the same as the mean velocity, but presenting it in this format exposes it's true nature...a coin flip. And I'm not talking about padding here either. These numbers come from statistical analysis of past history...not just some developer giving me his gut feel. That means when the commitment is made, we all know exactly what we're commiting to: We plan to deliver X functionality by Y date because we believe there is a 90% chance of success.
Maybe the whole statistical thing is too much for your customers. Maybe they wouldn't know a normal curve from the Coney Island Cyclone. Fine. Do the math yourself and give them more descriptive terms like "Possible", "Likely", etc. The point is this: The basic unit of measure for an estimate is not hours, days, weeks or months, but risk. Talking about estimates without talking about risk is like playing poker without looking at the cards. It's up to the developers to assess the risk, and it's up to the business facing stakeholders to translate that risk into dollars (that's what an MBA is for anyway). If we do that, everyone gets what they need. The bean counters get beans, the geeks get clear goals, and everybody knows what the deal is from day one.
Maybe we should be careful not to agree too much. People are going to think I am just saying things you want to hear. +1 re: story points. - Tim
Add Child Page to OptimizingEstimates