Experts vs. opinions

For those who don't know him, Aaron Seigo is one of the leading KDE developers and community leaders. (KDE doesn't have a "lead developer" position, just as Drupal does not, but my understanding is if you merge Earl Miles and Angie Byon you sort of have Aaron's role within the KDE community.) He also blogs far more than is probably healthy, but his posts, while long, tend to be very spot-on.

His latest article is one that is of particular interest to the Drupal community, I believe, because as a large, minimally-structured, Open Source development community we face many of the same challenges that other such projects do, such as KDE. In particular, the challenge of who to listen to.

How many issues on Drupal.org have been derailed or lengthened unnecessarily by one voice? How many times have we striven for agreement of everyone who cares to be involved rather than consensus of those who understand the subject matter? More often than we care to admit to ourselves, I'm sure.

Part of this issue is, as Aaron notes, that we do a poor job of separating expertise from opinion. As the saying goes, opinions are like assholes: Everyone's got one. That doesn't make those opinions of equal value, however.

Yep, I went there: Not all opinions are created equal. And the more ill- or un-informed they are, the less value they hold. (In general; if a vast sum of uninformed people interpret something the same way, that could be a sign that the message is being poorly presented in the first place.)

Unfortunately, we don't always admit that to ourselves. An expert analysis, with the scientific or experiential rigor that it entails, is worth easily ten times as much as casual opinions. So why do we, as the Drupal community, try so hard to value all casual opinions equally?

Or do we? Do we just pretend to and not admit to ourselves or each other that "some opinions are more equal than others"?

I'll give a personal example: To be perfectly honest, I am not a fan of the new D7 menu structure and organization. I find it cumbersome for the work I do, it's too hard to get to the areas that I actually do need to get to in my day to day work, in many cases it is not self-evident to me, etc. And yeah, I gripe about it at times. But honestly, I don't expect it to change because of me.

I have a bachelor's degree in Human-Computer Interaction, from 2002. However, Mark Boulton and Leisa Reichelt have been working in the usability and IA field for the past several years while my experience has been in software engineering, site architecture, and software architecture. Quite simply, their substantiated expert analysis of what Drupal 7's IA should be is worth more than my personal opinion of it. Sure, my opinion matters somewhat, but when you get right down to it if it comes down to my opinion vs. their analysis of our IA, they should win. That's one of the reasons they were hired, first by the Drupal Association (to redesign Drupal.org) and then by Acquia (to redesign Drupal 7): They had that expertise that we, honestly, did not.

But it took their status as being paid to give them that added clout, and even then both were quite frustrated at the number of opinions that were thrown against their expert analysis.

Conversely, I've been developing software large and small for various platforms for over a decade, and have both a bachelor's and masters degree in software development. To me, input on how to architect a piece of software internally from other highly-experienced developers or people with software engineering training and experience is more valuable than input from a UX designer like Leisa or a casual weekend-PHP-jockey hacker who learned PHP from an online tutorial about how to hack up a Wordpress template. If I have a question about extensible code structure, I pay attention to Earl Miles' input more than I do Leisa Reichelt's or someone who just joined Drupal last week. That's neither a slight against Leisa nor against new people; it's a recognition that different people have different backgrounds that make their input more or less valuable in different circumstances, and the balance there will change for different people over time.

Do we, Drupal, do that? Do we do it enough? Could we do it too much?

Personally I believe we do it but then lie to ourselves and say we don't. And that's a problem.

That doesn't mean opinions are invalid or irrelevant, to be sure. It means the message must be critically examined to separate opinions from expert analysis, and the messenger, and his delivery, does influence the message.

So what shall we do about it?

Comments

Getting more (high quality) contributors

What I see starting to happen as random J webshop picks up Drupal is the "WordPress model" -- where there is a shrinking group of core contributors. There is no feeling that one should be contributing (oh yeah, we use this tool for free, those core guys will fix it).

I was focusing on the aspect of getting people involved and contributing. ABSOLUTELY someone off the street that comes in and posts a core issue / patch is not going to get a lot of attention.

So how do we separate someone who DOES have the experience (outside of Drupal) from someone that doesn't? It's a tough one, and there has been a high barrier to proving oneself.

How do we welcome more expertise into the Drupal community? How do we get more people contributing with one offs that move the project forward, without having to be on a (mythical or not) core team?

Domain experts

Part of it, I think, is that Drupal has gotten too big to just wander in and find a niche. It's too big, and in many ways too complicated, for that to be as viable as it used to be.

I think of how I ended up taking over the DB layer. After 2 years of being a casual occasional contributor, I happened to be pondering a sneaky way to push PHP 5 adoption at Drupalcon Sunnyvale when Dries Himself sat down next to me on the floor to ask what I was doing and talk about it briefly. That really energized me, and Dries gave me several pokes of encouragement over the next few months. Really, without that I'd never have taken it as far as I did.

But there's too many people for Dries to sit down on the floor next to these days, and too much code for someone to "just read and grok it", and too many other things that Dries is doing besides sitting on the floor at conferences. And it's unclear who else you can sit on the floor next (literally or figuratively) to to get that sort of direction and prompting, and just as importantly to know that such prompting is going to come to something.

Part of it is documentation, part of it is marketing, and part of it is better recognition of there being more, different people to sit on the floor with. Part of it is, as noted further down by another commenter, providing better explanation of the direction Drupal is moving so that people know what one-offs that mythical core team cares about. Let's be honest, we're all more likely to review patches that relate to something we're working on than something that doesn't. That's why, for instance, Berdir got a lot of attention from me because he started doing some seriously awesome DB patches.

Part of it is a sense of ownership.

We do need to attract more people to Drupal, but not just warm bodies. We have been saying for years we need to attract more designers, but we also need to attract more development veterans and software engineers, not just casual programmers. We need to attract more project managers and managerial types. Those are all professions with expertise. We need to bring them all in.

I am not entirely sure how, and it's no doubt different for each of them.

Now I'm rambling. :-)

Hmmm...

So, I think you're spot on here Larry, and personally, I think it'd be rather useful if certain community members were granted that... clout as you say. Perhaps this could be done by tagging various users on d.o as being experts in various fields, and then letting that (small) group of people include others into their expertise as they interact with them in the issue queues and on IRC/etc. I don't know if that's a practical idea, but during core development, it helps sort the experts in an issue from the as..er I mean opinions.

Just a thought,

Eclipse

I like the idea of

I like the idea of identifying "experts" in various areas of drupal. I know it would help me understand who's giving opinion vs who's giving expert advice. I would take it a step further though and say those "experts" should also be helping / training others to get them more involved. If we're not getting the involvement that we'd like with core contributions, let's mentor some folks who have an interest. Let's get more people to have the ability to carry that "expert" tag within the community.

Defining clout

Well, that's part of the problem. We actively avoid acknowledging that stratification (I won't necessarily say hierarchy). The one and only place we do so, aside from core-commit-access, is subsystem maintainers.

As a subsystem maintainer for a year and a half now, I still have no idea what it means to be a subsystem maintainer. Every time I ask, I get brushed off or given a touchy-feely useless answer. (With one notable exception: Webchick referred to it as having a "Batphone" to her to talk about DB-related stuff, which is great but still very amorphous.)

If even those on the "inside" who have "clout" don't know what it means, how can anyone else?

And even then, that only works if you know who they are (really, how many people read MAINTAINERS.txt?), and if your area of expertise fits neatly into one of the existing silos. We've no way to recognize people who have skill that should be leveraged cross-silo.

It took us until this past fall to have a design/usability lead position at all, and it's barely there at that. Bojhan and Yoroy are now mentioned in MAINTAINERS.txt, which is awesome, but how many people noticed? How many people found out about that from this comment? How many people, er, know what that even means? Honestly, I don't. :-)

I don't claim to have a solution here at the moment. I just think it's important that we acknowledge the problem.

I'm probably one of those

I'm probably one of those weekend-PHP-jockey but I'm following Drupal since 4 years. I do just a few websites per year and my skills are mostly html and css (and enough PHP to horribly tweak some templates, obviously). I have experience of UI just because I did a little of that with Drupal for all my clients. In fact I'm an architect not a web designer or a programmer. (and unfortunately there's not much job for architects in these days)

Anyway, I have been following drupal for 4 years, reading drupal planet everyday and following recent debates on development strategies. As such, even if I'm not an expert, I agree with your point of view. Opinions have different values and we can't go on with this big lie. As a matter of fact there's a hierarchy. So why don't we make this thing official? It would only speed development.I don't think that open source has to neglect every aspect of private companies. Some sort of hierarchy is necessary and healthy.
Moreover, we need a roadmap to focus developer job. At this point of complexity is impossible to manage such a huge project and community and drive development on irc or private talks. If you really want to include more casual people like me it doesn't worth nothing to say that drupal is an open community and everyone count. What I need is transparency.

Here's an example
Personally, as you with the new menu structure and organization, I'm having huge problems with the Overlay. In fact I believe it's a huge mistake. But this is a not so worthy opinion because I'm not an expert. Still what bugs me more is not the results but the way it became part of the roadmap.
A private company (Aquia) hired two insanely great designer and experts (I really appreciate Mark and Leisa works), but unfortunately they didn't have any previous experience with cms and especially with that weird beast (half framework, half product) called Drupal and they've been overwhelmed by this daunting job (Leisa wrote this herself somewhere I can't find, now). In fact, they came out with a solution that required an incredible complex job in terms of development (I've never seen anywhere such an extensive use of overlays) and still is far from been practical and his slowing the release of D7 a lot, for what I see.
But what can you do when the company of the creator of Drupal kindly donates such a great opportunity to the community. Can you say :- No, thanks the results aren't good enough? I'm 100% sure that Aquia intentions where good but as everybody repeat only code is gold. Just providing a few mockups and a UI concept hasn't been enough and in this very special case it just made things more complicated. Has anybody made UX tests comparing the old UI and the new one before starting to code?
It's a mistake, it happens. But if we want to avoid those in the future we need more transparency, some sort of hierarchy, someone with specific responsibilities, someone to lobby if me or my company things a feature is good for the future of Drupal. Even code sometimes has more or less value and I want to know why.
I hope this doesn't sound too rude, it's just an example. It's not an attack to Dries or Aquia. It's just an alert to make people think on how to manage such a delicate situation where the project is growing a lot and there are some companies that make profits around it and we should be really careful to deal with the influence they have on development to not leave smaller developer behind.

Forgive my english.
Trying to learn how to write in english I recently realized how long my sentences are. This is normal in italian, but I guess it's hard to follow in english (especially if there are a lot of other mistakes!)

For the records... Mollom

For the records...
Mollom thought my previous comment was spam. Is it because of my english or because the word "Dries" has been detected by the 'evil' mollom filters? ;)

So what shall we do about

So what shall we do about it?

[warning: rambling ahead] Probably nothing imho. The current system rewards those with the best (easily understood is important) arguments, who've earned the most trust, and who are actually willing to do the work to turn their pontifications into real interfaces, apis, and features. I believe the current setup is disorganized, chaotic, and prone to error. At the same time, I believe its a very human system that comes naturally -- its not a democracy, or a technocracy - either would be stupid. If we ruled by the sways of opinion than drupal would never be able to try the daring things have allowed us to get where we are. The chief motivating factor would be to make sure that updates required no effort on the end users part.

Conversely if the rule of "experts" ruled who knows -- I will tell you this however: Drupal experts are a lot like mullahs in that they tend to be conservative, unwilling to accept new ideas, and often have a huge stake in there being minimal changes in the way drupal works or in dramatically altering it ways that will be painful for the majority. Of course, the mullahs are still the ones who fix the nasty bugs in the new versions too... and they do the majority of work, and i insist this is why they have power, and gained the honor of being an expert.

I think the problem is (and i don't know i'd call it a problem per se) not that we lack a system for the experts voice to rise above the others, but that many experts don't raise their voice nearly often enough or don't give enough background on why they think the way they do. I dunno -- its just my opinion, to a large degree i see your point of view, but what i don't see are ways that would change that wouldn't be much worse in practice than the status quo.

You state the problem well

Nick, I think you hit the nail on the head:

The current system rewards those with the best (easily understood is important) arguments, who've earned the most trust, and who are actually willing to do the work to turn their pontifications into real interfaces, apis, and features.

That is, we reward those who can shout-down others and whoever code's first. The "talk is silver code is gold" line is, I think, harmful, because it makes decision making a matter of whoever throws together a patch first, not whoever has the most robust or thought-out plan. "Roadmap" is an ugly word which means we have no direction, and different parts of Drupal wander off in different directions.

We do have "experts" raising their voices. And we also have, in the exact same thread, one single voice manage to block the issue by simply filibustering it. I've seen it happen several times (although out of politeness I will not give examples; if you've been around a while I'm sure you've come across one).

I am not necessarily proposing a structural change, but a cultural change. Whether that includes a structural change or not, I don't know.

I think you're also missing my point; I'm not saying that newbies should shut up and let the experienced developers work. I am not talking about "Drupal experts". I am talking about trusting usability experts with usability, design experts with design, IA experts with IA, software architects with software architecture, etc. Just as we periodically say "screw this guessing what the user wants, let's actually do a usability study", we need to do the same in other areas as well.

Heh -- yeah i did miss the

Heh -- yeah i did miss the point. What happened was a constructed a narrative for the article before was I done reading it. Ironic no? :-D

"Premature cognitive closure" i think is the name, which I learned from an article I coincidentally read recently: http://www.boston.com/bostonglobe/ideas/articles/2010/01/17/think_differ... One line that struck home in that article for me was "Third, conclusions often rest on assumptions that are not readily testable, and may even be immune to disproof."

I guess there are two ways to think about this: how instigate a shift in culture (hard, basically leading by example, and occasional spanking of offenders when they its obvious enough that its indefensible). In either case, "proactive" cultural shifts require those who care to be vocal and to set an example.

Second way to think about it is that instead of fighting what are natural tendencies its better to change our systems and tools in ways that minimize the damage they may do -- for example: shorter release cycle keeps everyone closer to the end product vs the product they work for, so its harder to lose track...

Big subject here. Mind is spinning. After saying that, the irony about conclusions based on assumptions that are untestable was not lost on me. :-D

Nice!

Very good article! It definitely dovetails into this question.

Slippery Slope: Checks and Balances

I am sure most everyone is aware of this, but its it's still important to point out, that this is a slippery slope. As soon as we create a system where we designate expertise, we also designate power. This is fine, but there needs to be checks and balances. Expertise and authority is not always the best opinion, it should simply be considered more. Expertise does not negate personality, and a bad personality with expertise is very dangerous. The process of choosing expertise needs to be very open and transparent, and criteria defined up front. And, "experts" needs to be held accountable.

So, as I do think designating expertise is actually very valuable, it requires a detailed and well thought-out system of governance, otherwise, Drupal will become the project of a few, instead of a project of the many. (and yes, who currently has power of the project can be debated :)

So having been involved for

So having been involved for the past two years, I think we are running into serious issues.

And its not really this issue, at the end of the day the many factors that make you an expert are inferior to the movement of open source. We are often faced with issues in UX, where being an expert also means you are going against certain community trends (because those are which got us to a bad UX at the first place). Where there community will attack you on this, to me means that the "expert" tag often doesn't go beyond the small group of core developers. We need to find ways for this Maintainers.txt to propel more weight towards new members, but also a community responsibility towards maintainers - to get new people involved.

I have been feeling strongly the past few months, that our number one issue is not the code/design process. Its the lack of people, as we proud our selfs on being flexible and not having anyone working fully paid on Drupal core. Sooner or later it is going to break us up, mostly in terms of motivation. Assuming we all still have the ambition to change the world :)

Paid Positions?

Paid positions is a tough subject. I would love to see some people paid to work on Drupal. Some projects are already heading that route. 75% of Linux developers are paid. But, other projects have not had the same success.

For example, look at Joomla. There are now two paid Joomla! developers which brought about a certain amount of drama. Who decides who is paid to do the development? Why are those people chosen? These are questions people will ask.

Ideally, I would like to see the core committers who work alongside Dries be paid for by a foundation or the association. This would provide them with more time to do what they do. We talk about culture change, wrangling people, helping other architect the internals of Drupal, and so forth. How many people can put in the time during the evenings and weekends? Who of the experts can be the next core maintainers alongside Dries and have day jobs?

Lullabot for Webchick and Acquia for Dries and Gabor have been great at sponsoring time. Who else can do that?

Beyond them, I don't know that we would get paid developers to work on Drupal. At least not initially. Maybe help getting Drupal shops to be more efficient so they can put some more time into working on Drupal itself.

How Do We Encourage Experts

I'm left with two rather difficult questions. How do we encourage experts in the Drupal community to speak up and get involved? And, how do we encourage experts from outside Drupal to come join us?

One of the solutions is to pay them (like Mark and Leisa were). But, that will not work in all cases. This is where I get stumped in the process of changing culture.

My first step is to someone recognize the people who engineer solid solutions. But, how do you do that within the Drupal eco-system and larger world?

Scaling an Open Source Community

Great article and conversation.

Drupal, the software, scales pretty well. Scaling Drupal the community, on the other hand, is a real challenge. And it's totally natural that it is so hard. This is a social experiment on a grand scale. I'm just disappointed that there isn't a track at DrupalCon SF about the process of building Drupal.

Some observations:

  1. Drupal is, at its core, hierarchical. Go to Drupal.org and you see Dries' name at the bottom of every page. In the case of the core co-maintainer, that authority is clearly delegated. I think Dries and the co-maintainer should get involved earlier and more often in important issues in the queue. And if they don't have enough time, Dries should delegate more people with that level of authority. And as Larry mentioned, it is extremely motivating to be given authority by Dries.
  2. Drupal's modularity allows people who can't deal with the process of developing Drupal core to, nevertheless, make a huge difference. I heard John Albin on a Lullabot podcast talk about his huge frustrations in working on core. What happened? He became the lead developer for Zen, has made huge improvements to it and is a highly respected and influential player in the community.

If you are turned on by the potential of lots of people collaborating to make something really powerful and useful, you can find a way to make a difference in the Drupal ecosystem.

That said, we could all benefit from more reflection and collaboration on figuring out how to scale the community. I've made one small proposal here ( adding more core co-maintainers). But I've got a lot more. And it is not clear where the forum is to to make such proposals.

Coda: I think the launch of the new Drupal.org could have profound and positive impacts on how we work together.

All opinions not equal! From poor thinking to good thinking.

Great! Finally starting out with something essential - that all opinions are not equal. Otherwise, we wouldn't ever go to a second doctor to get a second opinion. And otherwise the opinion that all opinions are not equal would be equal to the opinion that they are.

The all-opinions-are-equal fallacy is one of the big stumbling blocks to good thinking currently so prevalent in the West - a nutty attempt to not think about opinion, culture, religion and so on.

The stuff about Drupal, very interesting for me.

The stuff about opinions - that's something I've written (and thought) a lot about - as a college prof most of whose students have been trained to believe that thinking about opinion, and many other things, is taboo:

http://www.elsas-word-story-image-idea-music-emporium.com/not-thinking.html

Experts vs. Opinions not at the core of the problem

I know that Elsa isn't really responding to me, but since it's threaded that way (Garland isn't great at meaningfully distinguishing between "add new comment" vs "reply"), and I have an opinion, I'll respond.

It seems to me that a lot of the thorniest core-related issues have experts on both sides of the issue.

I think the part of this that we need to put the most brain-power into solving is how to provide more structure within the community when so much good stuff has proceeded from scratch-your-own-itch. So we need to figure out ways to add more structure but keep the environment inviting for innovators who can make a difference.

Back to experts v. opinions... Occasionally, I've seen weird anomalies where the core co-maintainer is advocating along with not-very-knowledgeable new folk for maintaining a certain status quo. I think that has more to do with work overwhelm on the part of the core co-maintainer and a need the slow down change than it has to do with Drupal culture valuing opinions over expertise.