Inkwell: Authors and Artists

    
Ari: It totally resonates. PMs, working a dedicated capacity, have the
advantage of perspective over the engineering team. They don't have to
understand every last detail about technical issues, and can instead
frame questions and conversations is contexts that are more useful or
easier to follow for other people.  Some programmers are flexible
enough to know when to switch gears: "I'm talking to a customer now who
doesn't know Java, so I need to talk about these things in a different
way", but many can't, or don't enjoy this kind of context shifting.

Active listening and faciliation get a bad wrap - they're seen as
fuddy duddy warm and fuzzy new age type things. Yeah, they're nice, but
it's not writing code.   

The problem is that once you have 5 or 10 people who are really good
at writing code, your biggest problem is no longer about finding people
that can write really good code - your problem is getting them to talk
to each other, and to customers, or testers, or marketers, in a way
that results in the best work being produced. The best understanding of
each others needs. The best exchange of ideas. The skill set for
facilitating the best work to happen has very little to do with knowing
how to write code. 

So as soon as you have a team of people, facilitation and
communication become crticailly important, and people in leadership
roles, whether they're called lead engineers or PMs, have the critical
role in making good cross team communication possible.
  

    
Scott, I've turned your 6 points from #44 into 9 points.
And in thinking further about this I do still thing you are correct
in saying these are all things a Project Manager does, and I also
think that XP/agile may create changes in how a Project Manager
does some of these things.

Perhaps a process that is inherently less centralized than the current
norm will change how groups behave, and how projects are managed.
If not, I'm interested in why not?

The Internet, with email and the web decentralized access to information,
and has had widespread, although often subtle, changes in how management
works - the managers are not always able to be the sole gatekeeper of
information. On the other hand, as you say, some very fundamental things
do not change much.

I am not at all an expert in XP and have not yet worked in a team that fully
utilized, but of course, like all experienced managers and developers I
recognize all the parts as things I've done at one time or another.

Here are the 9 points and how I think XP relates to each.

 1. Handing disagreements between team members.
    I don't think XP offers anything new here, except that
    via pair-programming two people in a pair are perhaps
    more likely to unbderstand the others' point of view, having
    coded a mile in their shoes.

 2. Handling drops in quality of work.
    Perhaps with XP these drops become know much more quickly,
    so that earlier intervention is practical.

 3. Watching and supportng overall team morale.
    I don't think XP offers much new here.

 4. Keeping an up-to-date map of overall current and future work.
    XP provides better current data (constant builds/constant testing)
    For example, see the Mozilla project's Tinderbox:
      http://tinderbox.mozilla.org/showbuilds.cgi?tree=SeaMonkey

 5. Leading the introduction of new development practices.
    XP doesn't specifically offer something new here, but it might
    make it easier to evaluate new practices, because of the
    methodology of XP/agile - the stories and tests might give
    objective results, faster.

 6. Evaluating team performance.
    XP doesn't specifically offer something, except, again, maybe better
    data for the objective parts of the eval.

 7. Evaluating individual performance.
    Same story as team performance.

 8. Representing the project to people further up the power hierarchy.
    Again, mainly, but perhaps importantly, better data.

 9. Co-ordination and communication with other project teams.
    I believe this is easier if one or more of the projects are
    XP projectrs, but I am really guessing here.
  

    
Hmmm. Interesting points. PMI would claim (and I'm not sure I disagree) 
that 90% (or is it 95%) of project management is tied up in 
communication, which is different enough from coordination (which is 
relatively trivial) that it needs to be, imho, continually stressed.

We just continued discussing this issue (what does a PM, or even a 
business analyst do, to ensure that specs are quickly, accurately, 
thoroughly captured) at lunch, and it comes down, imho, very much to 
listening, and then having the ability/knowledge, need to ensure that 
things really connect: if we have a widgitizer, where does it put the 
widgets? how does that fit in with how we currently do things....
  

    
In my experience on a couple of XP projects, the project manager
becomes more important, not less.  

In the projects I did before XP, I had only a hazy idea of what
project managers did, and I think the feeling was mutual.  The project
manager seemed to be only one of many players deciding what we should
do next, and their work mostly seemed to be "project tracker" as
described above, which didn't seem all that important as schedules were
mostly works of fiction.

Now I think a good project manager is absolutely essential.

On an XP project, at least as I've seen it done, the project manager
is considered the Customer and is the person who represents all the
stakeholders.  If a pair is implementing some functionality and there's
any ambiguity about a business requirement, the pair asks the Customer
right then and there what to do.  One of the XP tenets is that the
programmers need to have access to the Customer as much as possible. 
Ideally you want the Customer sitting in the same room so that
programmers aren't kept waiting too long when they want an answer.

So far I've found that project managers really like the control they
get in an XP project.  However, XP tends to generate a lot of work for
project managers because they're answering questions about the details
of what's happening this week, while at the same time, they're on the
hook for coming up with a solid plan and resolving external
dependencies for the following week, and they also need to be thinking
further ahead.  It's very easy to fall behind and it sometimes seems
like too big a job for one person.
  

    
To answer the questions from above:

1)  If there's a question about a business issue, the Customer
decides.  On the other hand, if it's purely technical, there's no
particular resolution mechanism other than discussion and an appeal to
various XP tenets.   Often you can lower the stakes by emphasizing
simplicity and that we can always come back to it later.

2) Most quality decisions (such as how completely to implement a
feature and which corner cases matter) are left up to the Customer.  A
story card is done when the Customer says it's done.  On the other
hand, the Customer isn't supposed to care about things like how pretty
the code looks and suggestions to skip writing tests and just implement
the feature will be resisted - that part of the quality issue is left
up to the professionalism of the programmers.

3) XP emphasizes "coaching" as a way for more experienced programmers
to help keep things on track.  Also, part of the weekly meeting is a
retrospective about how things are going and thinking about
improvements.  I don't know what happens if a project goes completely
off the rails, but I guess that's ultimately up to whoever signs the
paychecks.

4) the Customer keeps the story board up to date - it's their primary
tool for keeping track of the project.

5) For smaller things, a developer can introduce a new development
practice by convincing his or her partner that it's a good idea.  For
bigger decisions, the developers might talk it over in a larger group.

Individual performance is very obvious among the developers on an XP
project, and this takes some getting used to.  Even experienced
developers can be uncomfortable about having someone else see all the
typos they make.  But after seeing everyone else make mistakes, you get
used to it.

Team performance is also very obvious - each week you demonstrate what
you've finished.

If a problem is one of education, XP handles it pretty well - junior
coders get all the help the need.  Personality conflicts are tougher
and XP doesn't really help there, other than by changing pairs.

6) The Customer represents the stakeholders to the team and
vice-versa.
  

    
A question that has been going around for the last week or so at work
revolves around Firefox.   Not too long ago, people were saying that it was
incredibly secure because it was Open Source, many eyes, etc...   Now, more
and more critical bugs are being found and some critics are going as far as
saying they were mistaken in their statements and people shouldn't trust
Open Source software.

How does Project Management of these large open source projects differ from
that of large closed source projects?   Do you think that, say Microsoft,
would have avoided these defects due to the differences in Project
Management?
  

    
Matisse: I like your 9 point list, better than my 6 :)  There are a
bunch of other good questions I think that follow on. Makes me think
there should have a been a chapter in the book, or an appendix, about
"important questions to ask", that cut across all the different
philosophies and methedologies, cutting to the core of the issues. (I
need to think about this some more - obviously it's too late to make an
appendix :)

Brian: Thanks for that list - good stuff. But there are some parts of
the questions that you didn't cover - in particular #2. You mention
that the customer makes the call on questions of quality - This seems
fine in the abstract, but what happens when the customer continually
makes decisions that the programmer thinks are very bad. Half
implementated features. Crappy usability. The hacker/programmer ethic
is about craftsmanship, and I have great difficulty imagining them
responding well to any situation that forced them to make work they
though was bad. Drawing the line helps divide authority - but that
division does not obsolve programmers from caring deeply about the
impact of the customers decisions on the quality of what they are
making.

(And just to be fair. Of course in many cases the customer and the
programmers will sit down and negotatiate through their differences.
But the ability to negotiate through differences is something anyone
can do in any methedology. If this ability is the crux of how
differences are settled, they and stand with my claim that the secret
sauce in this instance isn't XP, but the ability and willingness of two
parties to negotiate, and hopefully, collaborate).

So getting back to my original point in that 6 item list - There
always needs to be someone in a position of authority to be the
tiebreaker, to see conflicts of personality, style or interest, and
step in to resolve them.  I agree that in better functioning teams,
this role is needed less - but it's always needed. 

To be entirely blunt, there's always shit that happens that no one
wants to deal with. The nasty, ugly, unpleasant, emotional, complex,
stuff. The stuff they tend to skip over in books about methedologies.
This includes confrontational situations (telling someone they're
wrong, or that there are problems with their work), situations of
responsibility (dealing with a 50% budget cut, firing people,
distribution of resources, etc.) and situations of negotiation (where 2
or more smart, but entrenched, parties are at a mutual roadblock, or
to be blunt again, hate working with each other). 

The list I made tried to get out how difficult it is to distribute
ownership of these kinds of things. For indivuduals in any system there
is rarely enough reward motivation to take on the burdens of solving
these kinds of problems.  I agree that many of the approaches in
XP/Agile make it easier to solve these things than more
evil-uber-hierarchical-Borg like systems of work, but you can't get
away from the need for clearly definied leadership roles.
  

    
cchoffme: Good question.

I hope you won't see this as a dodge (Hey! look over there!) but I
don't think is smart to evaluate Open source software on the basis of
one single project. I happen to think Firefox is great - but it's also
very new to such a large audience. When small projects hit a large
audience there is a big learning curve in figuring out how to deal with
relentless scrutiny the work will now be under.

When I worked at Microsoft, I learned early on about being careful and
dealing with that kind of pressure - people gun for you from day one,
and you never get the benefit of the doubt - you have to be ready for
it. The firefox folks are new to this level of success, and the high
scrutiny and expectations that come with it - it changes the game, and
it will take them a little while to get used to it.
  

    
cchoffme: To answer the second part of your question, about leading
open source projects.

First, a disclaimer: I've never worked on an open source project. I've
been around them, I've done standards work with the W3c which is sort
of open source-ish (at least in spirit), but I've never been a PM on an
open source work.  (Same for XP, I'm familiar, I've talked to people
about it, but have little first hand experience with it).

That said - I think project management of open source projects is more
difficult. I think any time you work with heavily distributed teams,
the work of managing and organizing is harder. It's much easier to go
down the hallway to talk to a programmer to clarify a complicated
problem, than it is to try and sort it out over email. Amplify that by
teams of people, and it gets much trickier. Open source software is
almost always highly distributed, so there you go.

You also have the problem with open source that many of the people are
working on a volunteer basis. Their commitment level will fluctuate
over time. As a PM on a dedicated team I can rely on their full time
commitment to the project/organizaztion/job/salary to help make things
happen. For open source, it's fuzzier. I don't have as many resources
to use to motivate them. 

But on the positive side, for open source projects folks are working
because they want to - most often because they believe in the project
and enjoy the kind of work they are doing. So as an open source PM, if
I define the project correctly, and set up the right value system
("ease of use is job #1", or "security is more important than
features"), I will attract only people that share those values and
interests, and have an easier time keeping the project in line with
those goals.

I think most open source projects, despite the media's habit of
describing them as autonomous collectives of programmers, often center
around a small number of highly dedicated leaders that set the tone for
everyone else. Often these folks are the founders of the project, or
people who develop high reputations among others in the group through
the quality of their work, and they act as the spine for the project -
reminding people of the goals, teaching and mentoring other people in
their work, defining responses to new issues. So open source projects
do have hierarchies, they're just formed more organically, are more
fluid, and are de-emphasized. 
  

    
Brian - you wrote:

"So far I've found that project managers really like the control they
get in an XP project.  However, XP tends to generate a lot of work for
project managers because they're answering questions about the
detailsof what's happening this week, while at the same time, they're
on the hook for coming up with a solid plan and resolving external
dependencies for the following week, and they also need to be thinking
further ahead.  It's very easy to fall behind and it sometimes seems
like too big a job for one person."

This is exactly what all project managers *do* - there's no way around
it. If you're a leader of any kind you end up in this position - it's
exactly where you should be: running back and forth between the future
and the present. 

Lots of smart well meaning people fail at leadership roles because
they can never get comfortable with the high frequency context shifts
they have to make - some people can't shift easily between thinking
about a coding problem, a test problem, a business problem, a personel
problem, and a planning problem in the same afternoon (or sometimes,
the same conversation). Then there's the dimension you point out about
context shifting between futuring planning and dealing with the now.
But this is what leadership roles are like - being brilliant isn't as
important as being merely smart, but very flexible (and do use an
abused term, agile).  There are 3 or 4 dimensions to every situation,
and there's no mathematics for figuring out which one to deal with
first, or how to deal with it. It's a very human and intuitive process
- which is why I'm waaay more interested in the art of project
management, than I am in the science of project management.

A big bet in the book that i make is that success in leadership roles
depends heavily on communication and relationships - knowing how to
delgate, how to build trust, how to explain things clearly and knowing
how to translate across different kinds of expertise. The book has two
chapters on this (9 & 12), written from this very perspective. There's
no way around the challenges you described - it's exactly what leaders
do. So the solution isn't to find a process that avoids those kinds of
challenges (none exists) and instead to build out the value of ways, or
approaches, for thriving in those challenges.
  

    
re: "what happens when the customer continually makes decisions that
the programmer thinks are very bad. Half implementated features. Crappy
usability."

My take on this is that there's a learning curve for customers too,
and you can try to talk them out of a bad decision, but sometimes you
just have to have faith that they'll figure it out eventually.  The
idea here is to make problems visible and decisions reversable as much
as possible.   Rapid iterations resulting in demoable software mean you
always have working software to try out, and if there's obviously a
problem with a feature and it's important enough, a card can be added
to next week's schedule.  This lower the stakes somewhat and makes it
easier to go along with "wrong" decisions temporarily.

re: "If this ability is the crux of how differences are settled, they
and stand with my claim that the secret sauce in this instance isn't
XP, but the ability and willingness of two parties to negotiate, and
hopefully, collaborate":

Well, I don't think it's fair to expect a methodology to resolve every
dispute!  However, some methodologies encourage collaboration more
than others.  To take a strawman example, the waterfall model doesn't
exactly encourage customers and programmers to talk to each other very
much.
  

    
Brian wrote "Well, I don't think it's fair to expect a methodology to
resolve every dispute!  However, some methodologies encourage
collaboration more than others.  To take a strawman example, the
waterfall model doesn't exactly encourage customers and programmers to
talk to each other very much."

I think we're talking past each other here. I entirely agree!
Methodologies cannot resolve every dispute - In fact methedologies
don't resolve ANY disputes - people do. No methedology resolves the
really tough issues people face when making things - there are too many
variables. People always have to work at interpreting methodolgies to
fit the unique things about any given team, and any given project.

So my point is that methedologies are really not the most important
thing - people are. You give me good experienced people who ask the
right questions, pay attention to what's working and what isn't, and
have convinction in trying to make things better, and they will
eventually turn the crappiest methodology into something good. You give
me people who spend all day flipping through methedology books
searching for advice on which bug to fix, or how to make a decision,
and who are placing their faith in what some person wrote in a book,
the methedology is bound to fail them. 

The focus should always be on the players, the people, and not the
system they are or are not using. 

I'm not a fan of waterfall development either, but there has never
been anything written that says a customer can't sit with a development
team during the entire waterfall process, collaborating with the
engineers on decisions where the customer's input would be valued. Any
wise programmer or project manager using waterfall or any other method
could, if so moved, involve customers as much as they wanted to. I
think the entire field of usability engineering has thrived in a
largely waterfall software development world because it solves the
problem of getting customer data at various intervals throughout long
monolithic development projects.

Sure, XP makes customer involvement more explict and gives structure
to it, but if the programmers and project managers believe customers
add value to the process, they'd be involving them regardless of what
methodology they were using. 

So I'll say what I said before - The thing I want my team saying is
"We are making good software" not "we are using methedology X,Y or Z". 
If a methodolgy helps a given team make good software, great.
Outstanding. Wonderful. But the methodlogy isn't doing the heavily
lifting, and doesn't deserve the credit. The people do.
  

    
I definitely see that on some teams now -- many groups are trying out
"agile development", the whole thirty day sprint thing and so forth. 
And some people are buying it, working to scale their features
appropriately so they're "shippable" by the end of the 30 days, doing
all of the "scrums"  and other things that are there to improve the
clarity of communication... and some people are just shoehorning their
old habits into the new system, with features that should take more
than 30 days to complete, but instead require every waking hour of the
dev's time to complete in 30 days.

One of the reasons people seem to be turning to it is to deal with
"internet time", to be able to churn out products in faster cycles, 60
or 120 days instead of the 18+ months that's normal.  But along with
the "agile development" there seems to be a lot of "let's not really
plan anything in detail, let's do it all by the seats of our pants."  I
have yet to see if that's inherently bad or not -- it seems like it
is, but maybe not.

In any event, I'm wondering if this kind of "internet time"
development always means less planning -- does the project manager's
task shift away from planning and more towards chaos management? Or
should planning still be emphasized?
  

    
> The focus should always be on the players, the people, and not the
> system they are or are not using. 

This is possibly the mjaor underlying thesis of your book Scott, and I think we
all agree with that. I was re-reading chapter 9 last night (about communication),
and that's something that will never go out of style.

Are you familar with the book "The Timeless Way of Building"? Your book might be
sub-titled, "The Timeless Way of Project Management"

ABout "Internet time" - I'd generalize that to any situation where the scheduling
expectation is for results much more rapid than has been the case in the past.
  

    
David wrote:

"In any event, I'm wondering if this kind of "internet time"
development always means less planning -- does the project manager's
task shift away from planning and more towards chaos management? Or
should planning still be emphasized?"

I think this is a misnomer about agile methods - there is just as much
planning going on, it's just distributed differently. 

Imagine taking a 4 week trip to Europe - you could plan this two
different ways. 1) Spend time before the trip figuring everything out
and making decisions about where to stay, or 2) Wake up every morning
and spend a half hour figuring out the next 24 hours. 

In both cases a good chunk of time is spent planning, it's just that
in #2, the more agile way of planning, you've planning efforts are
distributed, and the time horizons you plan on are shorter.  

Agile or XP shouldn't mean no planning - hell, a good chunk of XP
explained is about the planning game, the session where everyone
decides what work will be done in the next iteration. 

One of the points i tried to make in Chapter 2 (The truth about
scheduling) is that there is no law that says you must plan like #1 or
#2 - there is a spectrum of different mixtures of up front and short
term planning that you could do.  There's also no law that says 30 days
is the magic planning number for sucessful projects. 

The truth is that each project is different, and the emhpasis on early
planning vs. responsive/"plan today over breakfast" should follow the
needs of the project. 

I advocate in the book that you always need some early planning just
to figure out some of the boundries and high level goals, but after
that the length of milestones (aka XP iterations) should be long enough
to offset the expected volatility of the project. The more volatility
(market, team, management, etc.) the shorter the milestones. 

Also, I think part of your point is whether dealing with chaos is a
kind of twisted planning - when there's no planning at all there's
often a crazy amount of management work that needs to be done puting
out fires and recovering from mistakes that, even though it's not
planning, it's work done by the same people that would have done the
planning had the team chosen too. So there is a leadership cost when
teams don't plan or organize well. 

So the joke is that for people who say "planning? We don't need no
stinking planning!", you need to look how they're spending their
non-planning time. If it's 50% chaos, panic, and flailing about, then
odds are good some planning, or at least some strucure, would have made
for a smoother ride.
  

    
Matisse wrote: 

>> The focus should always be on the players, the people, and not the
>> system they are or are not using. 
>
> This is possibly the mjaor underlying thesis of your book Scott, 
> and I think we all agree with that. I was re-reading chapter 9 
> last night (about communication), and that's something that will 
> never go out of style.

Excellent! Glad that comes through.  

Though what's interesting is how, like the discussion in this forum,
it's irrestible to talk about methods. Many people seem more intereted
in debating methods and trying to define rules/systems/processes,
rather than talking about people, how people tend to behave, and how
different projects and people are. It's fine of course, and it's fun,
but it's curious why the conversations often shift in that direction.  

Is it because we're engineer/builder type people? Is it because the
tech-sector is male dominated (at least population wise) and we tend to
externalize problems rather than internalize them? It's just curious
to me.
  

    
Scott: I agree that people are the most important thing, but they're
not the only thing.  I've seen too many organizations where smart
people's time is wasted trying to solve problems that don't need to be
solved, or fighting the organization to be allowed to work on the
problems that do need to be solved.  There is a big difference between
an organization where talking to customers means fighting institutional
barriers, versus one where it's expected and encourages.

Furthermore, I don't care how smart you are, you can still be tripped
up by bad work habits.  Smart people in particular tend to be
distracted by interesting problems when time could be saved by avoiding
them.  And most people have a tendency to avoid the tough issues by
taking refuge in tasks that are more in their comfort zone.

So I'm skeptical of arguments that say, in effect, "if we hire smart
people, we don't need any methodology."  (That's probably not what you
meant, but it's a side-effect of playing down methodology so much.) 
The value of a good methodology, properly applied, is really just a way
to encourage good work habits.  It's an agreement among people who are
working together about what good habits should be.  If we have shared
ideals we can help each other to live up to them.

I'm also skeptical of the dichotomy between people-centered and
methodology-centered thinking.  In no way does XP downplay people
skills.  Introducing XP into an organization requires leadership,
because changing habits is hard.
  

    
I think it's because we are engineer types, and more generally, we are people
(unlike say, therapists) who think of methodologies as a way of dealing with
tools, processes etc., when in fact, as you do in your book, there are
methodologies foir dealing with people, relationships, communications, etc.

I was talking with a development/consultant a week ago about this, how the
*technologies for communication* are something that we humans are in the infancy
of. It would be sheer hubris to believe that no new technologies for
communication and cooperation will be created or are worth creating.

On the other hand, I do believe that the essential, emotional and spiritual core
of relationships probably doesn't really ever change, and you are quite right to
keep bringing the focus back to that. That is, i think, an important part of
improving our methods for dealing with people - acknowledging and working with
feelings as part of the process. Feeling are always part of the process, and we
do better to pay attention to that.
  

    
Brian slipped in with good comments about "smart people", which in some way
emphasizes the different types of "smart" - logical smart, emotional smart, etc.
  

    
Scott, I do agree that there's a tendency to be distracted by finer
points of methodology.  I don't think XP is all that bad in that regard
- nobody is memorizing rules or consulting rule books.  I've never
been in an argument where someone said we should do something because
Kent Beck said we should do it that way - the emphasis is always on how
to adapt to local conditions.  There's an idea that some things are
"XP" and others are "not very XP", but there's also a fair amount of
flexibility in the details.  

I *have* been in disputes over the finer details of how to count story
points, which is something I try to discourage.  But even a
methodology that has an explicit goal of minimizing process and
paperwork can be misinterpreted.

I'm somewhat preoccupied with methodology because I've been working in
organizations that badly needed to improve.  Maybe this preoccupation
seems strange to someone who comes from an effective organization.  As
the novelty wears off, hopefully this will simply be "the way we do
things" and our attention will be on other issues.
  

    
It might be worth noting that the "Agile Manifesto" top item is

  "Individuals and interactions over processes and tools"

and that it explicitly places more value on 'Individuals and interactions'
than on 'processes and tools'

http://agilemanifesto.org/
  

    
This is priceless:
  http://hestia.typepad.com/flatlander/2005/05/the_parable_of_.html

> And the manager Ahimelech and the manager Balazar did meet, even at a sushi bar,
> and did contend mightily in faith. For both purported to follow the way of the
> profits, but could not agree how best to follow the teachings. And they agreed
> to go forth, and to tend their vineyards, and to meet again on this day to see
> who wouldst be the victor.
  

    
verily
  

    
Brian: We're in agreement - it takes both. I totally understand your
focus on methedology given the situation you're in. 

Chapter 10 on how not to annoy people is largely about process, and
anyone that read it's will probably call me a process advocate, since
the way I wrote about it implied process is needed and has to be
adjusted all the time. 
  

    
Matisse: uh, oh... now we're quoting manifestos :)