Someone on the D newsgroup today proposed a small feature that would help make the language more orthogonal, but in the end dismissed the issue as “perhaps not important enough”. The yardstick for importance in D since I’ve been around has always been whether the addition would allow you to do things that are impractical or impossible today. That’s a fine yardstick for measuring importance. But it’s essentially a very brutal kind of feature triage. There are scores of small issues that lay festering because, individually, they are not critical. They are like mere grains of sand in the way of progress, easily side-stepped.
But enough grains of sand and you start to have a sizable obstacle. And D has a fair number of such grains. Pretty soon D needs to move out of feature triage mode and start working on polish, completing or cleaning up the features that do exist, and resolving issues with the tool chain. I thought that phase was about to begin now that const is basically settled, but now the goal posts seem to have moved again. Now it appears that cleanup won’t be a focus until until after the multicore problem is solved. And given that multicore is not a solved problem in general, I think that could be a while.
Maybe the problem is just that Walter is not, and never will be, interested in working on small details. One can always find some new shiny holy grail feature on the horizon that’s more interesting to work on than, say, package visibility rules. Features in D seem to get to the stage where they’re basically usable, then stagnate. Like __traits, or like the non-overrideable $ in indexes (or rather overrideable only with the undocumented __dollar hack), or like associative arrays, or uniform call syntax that works only on built-in arrays. If you’ve used D for any amount of time, I’m sure you’ve got your own favorite examples. Language features in D never quite seem to reach that “insanely great” stage.
Here’s my pet theory about why that is so. I think it comes down to dogfood. Walter and the majority of the “D inner circle” basically don’t write code in D. They spend their days writing C++, not D. So shortcomings and annoyances in day-to-day D usage just don’t show up as major blips on the radar with these guys. No associative array initializers? Eh, no big deal, you can still initialize them in a loop. So it’s not critical. That’s easy to say, but when you’re the guy who actually has to write that loop, it starts to get annoying after the 10th time. I think it’s universally recognized that to make great products it helps to eat your own dog food. Unfortunately the core team working on D doesn’t.
With one exception — Andrei A. seems to have been writing a lot of D code. And the result shows. Things like fixing the limitation with IFTI came about directly because Andrei complained that it was too hard to write his libraries in the way they should be written with IFTI as it was. Unfortunately I’m worried that Andrei’s influence in the D community will be drastically reduced now that he has left Seattle. Walter seems to have trouble maintaining lines of communication when face-to-face meets at the coffee shop aren’t possible.
But back to dogfood. In contrast to the core D folks, the Tango team seems to be deep in their own dogfood. And really I think that right there explains the existence of Tango. The Tango folks are actually using D and they got fed up trying to work with stuff that wasn’t built with that kind of in-the-trenches knowhow.
The current situation leaves me uneasy about D’s future. How do we make sure D not only survives but becomes insanely great? I have no silver bullet, but it seems to me like Walter is always going to be more interested in new and shiny than solid and complete. That’s fine, really. Somebody does need to be looking at the next big thing. So I think what D really needs is a #2 guy who is both a heavy D user and a compiler writer and who’s main interest is making D polished. I’m hoping that LLVMDC will somehow help there. Lindquist and Kamm have been making great strides on that alternative D compiler, and have already proposed fixes for things like cross-module symbol visibility. Hopefully they will be able to communicate effectively with Walter about these things, otherwise the ultimate result may be a forked language. I know no one really wants that to happen, but it’s happened with the runtime already, and I don’t see any evidence that the environment that gave rise to that fork has significantly changed.
[update Sept 9, 2008]
Don’t count Andrei out of the running yet! A day after I wrote the above, Andrei came out with a proposal for for iterators/ranges in D, what could become std.range for D2 Phobos. What’s more, to implement it he has requested that Walter implement reference return values, and Walter has apparently agreed. Well, this is exactly the kind of practical, day-to-day bread-and-butter programming stuff that D desperately needs. Exactly the kind of polish and completeness issue I was talking about above. The deficiencies of opApply and the lack of reference return values have long been on the top of my D peeves list. So I’m really happy to see this development. Maybe there’s still hope for D after all.
