Paul Bissex (biscuit) Thu 1 Mar 07 17:19
Welcome, Ted! 12,000 words in the topic so far -- I don't envy you the catch-up reading (not that you're obliged).
Ted Leung (ted-leung) Thu 1 Mar 07 18:28
Paul, I summarize the traffic on the cosmo-dev mailing list every week. 50 messages is short work ;-)
Scott Rosenberg (scottros) Thu 1 Mar 07 19:43
Hi, Ted -- glad you're here! For what its worth, I'm happy you're joining us, but the idea came from the fine hosts of this Well conference, not from me or Salon. Christian, I think the "climb the ladder, then kick it away" approach you describe is probably closest to what Andy Hertzfeld was advocating in Chandler's early phases. It's worth pointing out that from a very early phase of the project OSAF actually did have "running code," if by running code you mean something that you could download and execute. The problem was more a disconnect between the high expectations people had for the project and the extremely limited functionality that running code offered for a very long time. You could download and "run" Chandler 0.3 and 0.4 and so on, but you couldn't do a lot with it. One lesson I took from that -- and it's a lesson that is now commonly embraced by the builders of Web applications, and also one that most people at OSAF would, I think, agree with -- is that you should try to roll out a new application or service by finishing at least one feature at a time in a fashion that users can actually use. (In the 37 Signals playbook this is defined as "build half an application, not a half-assed application.") That gives you a chance to begin to build a community of users and loop their feedback into your further development work. From what I can see Chandler is now on the verge of that sort of dynamic, with the forthcoming "preview" release. But I think it would plainly have benefited from arriving at that point sooner.
Ted Leung (ted-leung) Fri 2 Mar 07 00:31
Hi Scott, Thanks for the clarification on the invite. We are definitely on the "build a slice of an application path". The particular slice for us is the calendar. I arrived about halfway through the story in the book, so I'm not well qualified to comment on what happened before I showed up. But one thing that has been difficult has been to balance the "platform" and "application" aspects of Chandler. Along with that, we were definitely trying to build out all areas of the application at the same time. Since we decided to focus on the calendar, we've been able to get some traction.
virtual community or butter? (bumbaugh) Fri 2 Mar 07 06:52
Happily, I embrace "build half an application" in my real life, but in this context that seems a bit of a paradox: does that really make the other barriers to software development that we've been discussing go away? Or is it just that they become small enough you can drown them in the tub?
Scott Rosenberg (scottros) Fri 2 Mar 07 07:53
I don't even think you get to drown them in the tub at that point, but perhaps you have a fighting chance of wrestling them to the ground.
Scott Rosenberg (scottros) Fri 2 Mar 07 09:47
BTW, in my morning reading I see that Ted has an excellent post up on his blog all about this basic question we're beginning to identify, about how much efficiency or software-development productivity the web-based software approach (or what Ted calls "rich internet applications (RIAs)") wins us. The post is at http://www.sauria.com/blog/2007/03/01/adobe-wants-to-be-the-microsoft-of-the-w eb/ It's maybe a tad technical for non-developers, but the basic point is that, the more ambitious you get in terms of creating a rich environment for a Web app, the harder the development gets, unless you're willing to say, from the start, that your app only works on one browser. It's a fascinating dilemma. My browser of choice is Opera, and boy, life is getting harder out there, because a lot of Web-based apps are developed to support (a) Firefox, because it's good and oopen, and (b) IE, because you can't ignore those gazillions of people and (c) everyone else needs to fend for themselves. To me, the larger point is this: the move to Web-based development will solve one set of problems that the previous generation of software wasn't very good at (coordinating data on multiple machines, sharing data among multiple users, backups and managing data so the user doesn't have to be a sysadmin, keeping the application code patched and up to date). But it will bring along a whole nother set of problems that we can already see the outline of -- these includee cross-browser compatibility and the development nightmare Ted explains, security and privacy issues.
Brian Slesinsky (bslesins) Sat 3 Mar 07 13:32
Paul Bissex (biscuit) Sat 3 Mar 07 18:57
Do we really have to worry? Part of the lesson of Scott's book for me was that software developers almost never adopt that mindset about their own projects, no matter what prior or current evidence seems to point to. "All programmers are optimists" and all that.
Cogito? (robertflink) Sat 3 Mar 07 19:10
>If you have the right kind of laziness then you will avoid hard problems when you can. If you're impatient you'll release early and often and instinctively avoid taking large bets that won't pay off for years. Hubris means that despite using minimal effort, you still want to impress people with something great.< I like that the above would tend to militate against linear and hierarchical thinking.
Howard Berkey (howard) Sat 3 Mar 07 23:20
slipping back to mz: >Actually, software is easy. Coding is easy. And fun. And challenging in a problem-solving way. A lot of us are good at it. The hard part of software development for many of us lies in the human factors completely unrelated to actual coding. Playing nice with others does not come easily, for some reason, to a lot of people in this profession.
Howard Berkey (howard) Sat 3 Mar 07 23:21
(should mention I have not had the chance to get the book yet, but I look forward to it!)
Ludo, Ergo Sum (robertflink) Sun 4 Mar 07 05:16
>The hard part of software development for many of us lies in the human factors completely unrelated to actual coding.< There is even some (scriptural) "evidence" that human factors have caused some difficulty in the "intelligent design" of the universe. Those pesky people creatures are even getting in the way in Iraq. (maybe a little playing nice?........Naw!) It is a very interesting outlook when the producers and users of systems are viewed as unrelated to the actual process. BTW, in MBA education, business case study solutions have concluded with the comment; "The problem is solved, all that remains is implementation".
Howard Berkey (howard) Sun 4 Mar 07 10:48
>It is a very interesting outlook when the producers and users of >systems are viewed as unrelated to the actual process. Oh yes indeed. Or "unrelated to each other in the actual process". There have been various attempts in the software industry to address this, with Agile/XP/etc being the latest (and maybe the most adopted, but that is hard to say). But you are completely right with that point.
If gopod's on our side s/he'll stop the next war (karish) Sun 4 Mar 07 14:09
Ted's blog post summed up one set of my answers to Earl's question. Our challenge in multi-platform support has moved from hardware to operating systems to windowing platforms to browsers, and now back to hardware as we try to get Web programs to run on pocket telephones. We converge on a platform definition but the definition keeps changing. I don't think that's the whole story about why software is hard, though. Looking at the design side rather than at the implementation side, there's constant expectations creep. The goal is to produce a product that is great when it's delivered, not when the the first requirements are conceived. Our product will be compared to other products that are released while ours is built. I'm not convinced that it's more difficult now than used to be to estimate the work required to implement to a set of requirements. The state of the industry, though, is that we can't count on requirements to stay put. Agile methodology provides a way to allow the development process to chase the requirements. I don't think there's an adequate equivalent on the business side of the software industry.
Scott Rosenberg (scottros) Mon 5 Mar 07 09:34
Brian wrote: >The trap seems to be thinking that in >order to do something ambitious, you >have to solve hard problems head-on, >rather than dodging them. >I'm a bit concerned that this discussion >(and I suspect the book as well, though >unfortunately I haven't read it yet) is >promoting the idea that software is supposed >to be hard and that train wrecks and >disasters are the inevitable price of doing >something ambitious, but that you should be >ambitious anyway. That doesn't seem like a >good mindset for the people on a software >project to have because it doesn't >teach good avoidance techniques. The book certainly doesn't aim to promote any particular idea; it's one story, surrounded by a lot of background research, that you can draw different conclusions from. I'm with Brian until he says "...but that you should be ambitious anyway." That's certainly not my conclusion! For me, it's more like "train wrecks and disasters are inevitable and no one's found a sure way to avoid them, but there are some things you can do to minimize their likelihood." Those things include working incrementally, limiting (or at least staggering) your ambition level, and not making unrealistic assumptions about schedules or fixed requirements. I think I understand what Brian means by "avoidance techniques" and "dodging" hard problems but I also think that terminology is guaranteed to raise the hackles of the people (project sponsors, business people, program managers, etc.) who need to be on board with the developers' plan. It sounds like you're saying, "let's punt." I think the same point can be expressed by saying, "Let's be realistic, let's acknowledge that there are swamps here, and let's agree we don't want to get sucked into them." <karish>, that's an interesting suggestion about finding the business equivalent of an agile methodology. The thing is, as I understand it, agile arose as a developer-side reaction to the churn of the business side (changing requirements, etc.). Usually what I hear from developers is a demand that the business people they work with get more disciplined and rigorous about what they want -- in essence, that they move a little *less* fast!
Christian Crumlish (xian) Tue 6 Mar 07 14:37
Coming into the home stretch here I'm going to queue up a few more questions. Scott you can tackle some or all of them at your leisure. One funny thing: I've been re-reading D in C and set it down face up in one of the coffee bars here on the Yahoo campus and a developer started reading the cover and turned to me and said "I have nightmares in code." (He was smiling as he said it.) ok. first of all: early on you explain a bit about how the term geek was embraced by computer nerds and it got me thinking again about the different connotations between those two words. To me, geek has become a term of pride (as you discuss), whereas nerd is still somewhat negative sounding, although it is also embraced ironically by some. When I was writing my book I spent a lot of time talking to Craig Newmark (of craigslist) and he proudly describes himself as a nerd and points to his pocket-protector days in high school as a formative influence on his desire to make software inclusive (which I thought was an interesting insight into the world of social software). So, any thoughts on nerds vs. geeks and how they relate to software? I'll put my next question/musing in a separate entry.
Christian Crumlish (xian) Tue 6 Mar 07 14:45
My next question has to do with patterns. That's a particular interest of mine because my current job involves managing a design pattern library. Several times you talk about Christopher Alexander's "A Pattern Language" and there's a whole interesting digression about the Portland Pattern Repository (Ward's Wiki). I find it interesting how the pattern metaphor has migrated from architecture to software design to interaction design (OK that last jump is less of a stretch). Any thoughts on how pattern-awareness has impacted software development? Ward's wiki devotes a lot of energy to noting antipatterns, kind of warning people away from "worst practices" so to speak. Is this accumulated lore of any use or does it fall victim to the whole "this is a special case" thinking you document at discouraging length in your book?
Christian Crumlish (xian) Tue 6 Mar 07 14:45
ok, that's plenty for now...
Paul Bissex (biscuit) Tue 6 Mar 07 18:49
I have one to throw in there too, and maybe Ted can even chime in on this one -- I gather that the Chandler team is a bit more distributed geographically these days than they were during the time you were writing your book, though you wrote about some of the steps in that direction. Do you think there's any particular significance to this change? Is it a good sign for Chandler?
Scott Rosenberg (scottros) Wed 7 Mar 07 12:26
Paul first: Yes, the OSAF team's more far-flung these days; I know that Brian Kirsch moved to Hawaii, Ted is in Seattle, Mike "bear" Taylor is back east somewhere, Phillip Eby is in Florida... there are probably others too. I don't know how specifically significant this is, except that it forced OSAF to move more of its process out from meetings rooms onto mailing lists and such, and that's gotta help them as they continue to try to get more buy-in from "bazaar"-style open source developers. Christian: Geeks vs. nerds -- I did a little research on this and the best I could come up with was that "geek" has tended to be more of a west coast term, "nerd" an east coast one. Nerd I think has a pedigree that is pretty much from school (a la "Revenge of the Nerds") where it originally had nothing to do with technology per se and meant any social misfit with an overdeveloped brain. "Geek" is even older (as I mention in the book it even appears in Shakespeare!), and of course it long meant the mentally impaired person who bit the head off the chicken at the sideshow. Both words have to some extent been co-opted by the tech subculture as terms of pride in self-labeling, and Neal Stephenson wrote a wonderful op-ed piece in the NY Times a year or two ago that defined "geek out" as any sort of over-the-top passion for the arcane details of any endeavor (so that you can "geek out" on train trivia or knitting or whatever). To my ears as well, right now, "geek" feels more fully redeemed than "nerd," which still sounds unappealing. But these judgments are, I think, highly malleable by time and usage. The first time I heard the word "blog" I went "bleaugh!" It sounded grotesque. Now it's just common, don't-bat-an-eyelash parlance, and that happened in <5 years. As far as the patterns movement goes, I think what's important there is that in each case -- beginning with Alexander in architecture and moving to the "Gang of Four" in object-oriented programming and now, as you say, interaction designers as well -- you have fields that are trying to figure out how to pass along learned experience in the absence of fundamental laws or scientific principles. In other words, the use of patterns is a sign that you're dealing with a craft; the pattern is a distilled bit of craft experience, a way of streamlining the communication of any piece of knowledge of the form "In these sorts of situations in the past, this has helped and that has not." In that sense, I think that the pattern movement is a huge success, in helping solve one of the software field's biggest problems, which is how to transfer know-how. But it also represents an implicit acknowledgment that there is no rigorous, universal set of software engineering principles at the core of the field -- that it remains very much a craft.
Ted Leung (ted-leung) Wed 7 Mar 07 13:35
Paul, Yes the team has become much more distributed. In addition to the people Scott listed, Reid Ellis lives in Canada, Bryan Stearns has moved to Oregon, Morgen Sagen lives far enough away from San Francisco that he is also remote. On the Cosmo team, 2 of the 5 developers live outside the Bay Area. That has definitely made an impact on the open source aspects of the projects. Late this summer, an engineer working at another company produced an entirely new and much higher performance storage engine for Cosmo. This is probably the most significant contribution to come from people not employed by OSAF. Also, one effect of forcing process out of meetings and into mailing lists is that there is a record of the rationale for decisions, and people who were not in the meetings are still able to contribute when relevant. I personally regard this trend as a good one.
Paul Bissex (biscuit) Wed 7 Mar 07 14:03
That was my intuition too. Thanks for the confirmation.
Paul Bissex (biscuit) Wed 7 Mar 07 18:32
Since we're getting close to the end, Scott, I wanted to squeeze in one more question. My perception, and feel free to correct me, is that your book is fairly unique in the world of software writing in being aimed not specifically at the practitioner, but at the interested layperson. This book feels very much a part of the legacy of the Whole Earth Catalog (which we're soaking in right now of course), operating from the assumption that the world is a complex but ultimately comprehensible place, and that interested non-specialists have as much place learning how a particular corner of it works -- and how the work in that corner might improve the world -- as anyone else. What I'm trying to get around to here is asking you how you view your place in the world of software. It's clear that you have a deep interest in it; that was really driven home by your story about collecting tiny snippets of different programming languages. You are not just an anthropologist of an alien culture. And yet you remain a writer of sentences and paragraphs, not functions and procedures. Do you have any idea where this interest will take you next? And do you have any models or precedents you keep in mind to help you explain, to yourself or others, what it is you are doing and what your relationship is to the world you are chronicling?
Scott Rosenberg (scottros) Thu 8 Mar 07 09:15
Paul, I'd be honored and proud to be viewed in some way as part of the Whole Earth legacy, and certainly, from my years of presence on the Well and my general marination in the writing and ideas of people like Stewart Brand, Howard Rheingold and Kevin Kelly, it's fair to see some influence there. But I can't say that it was conscious in any way. I think I was more directly aware of some antecedents that we've discussed and are sort of obvious (Tracy Kidder, Steven Levy, etc.) and maybe some that are less obvious. One point of comparison for me -- or, to be more precise, maybe, a target of aspiration! -- is John McPhee's writing on geology, where a knowledgeable outsider takes you inside an extremely complex field and the issues the people in it confront in their work. Making the book comprehensible and absorbing for the non-specialist was absolutely my aim. It was, you could say, one central requirement of the project -- and I feel comfortable that I at least partially fulfilled it. But there's no question that it was tough to aim for that while also knowing that the book's "early adopters" would, inevitably, be programmers who would read it from a very different vantage. I spent the first 10 years of my writing career as a theater critic, and that sort of writing involves a primary responsibility to the general reader. Most readers of your reviews are never going to see the play you're writing about, but maybe, if you write a lively and engaging enough piece and you find ways to connect what the event was all about with something else in the readers' world, you can make the 10 minutes they might spend reading your review valuable to them. It was also always important to me -- but in a secondary way -- that I write knowledgeably enough to earn and retain the respect of the people in the field I was writing about, the directors and actors and playwrights. So this sort of insider/outsider balancing act is one that I've long been familiar with. And my status in the theater world was similar to my relationship to programming -- not something I've been actively involved in since teen years (the last time I appeared on stage!), but have worked on the edge (I spent a couple years reading new plays for the American Repertory Theater in the early '80s) and have some knowledge of the field from the inside. Where it's all going to take me next I have no real idea. I've been back working at Salon since early last year, trying to add some new interactive features to the Web site. I thought it would be a way I could take some of what I'd learned in writing DREAMING IN CODE and bring it home to help the company. But it's also true that everything I've attempted to do since returning to Salon has taken a really long time! While that's been frustrating, it has, at least, confirmed the book's perspective -- that making software really is hard, and that it will usually find unexpected ways to trip you up.
Members: Enter the conference to participate