Sunday, 14 September 2008

StackOverflow - CrackOverflow or StackOverblown?

logo link to homepageStackOverflow is finally due to come out of private beta this Monday (15/09/2008), so I thought it was about time I got my thoughts down about the new programming resource/community that is the brainchild of Jeff Atwood of CodingHorror.
StackOverflow has caused quite a stir within the programming community, already being dubbed "CrackOverflow" due to its addictive nature, but has some of the hype really just made it "StackOverblown"?
Note: Any links to user profiles or questions on StackOverflow are not intended to be personal attacks against them, they simply reinforce some of the points I am trying to make here. Also, links to StackOverflow will require access to the beta if it has not yet been released.

What is StackOverflow?

Straight from the "About" page on the StackOverflow blog:
It’s a collaborative programming community venture between Joel Spolsky and Jeff Atwood.
In my opinion, it's a great idea, and one that is much needed by the programming community as a whole. We have tons of forums around, all doing their own thing. But we need a centralised location with good quality guys/girls there. Most of the forums I contribute on seem to be rapidly deteriorating in quality.
StackOverflow aims to do this by having some "unique" features:
  • A rep system which allows users to vote on questions and answers given.
  • "Badges" that are awarded for users, their questions and answers meeting certain criteria.
  • Members can edit and work with all content (rep permitting) similar to that of a Wiki.

The Positives...

So, lets start with the nice stuff so I have unlimited rant space for the negatives ;)
The "Honeymoon" Period was Awesome
I have to admit, when I first got into the StackOverflow beta, I was proper "loved up". I was singing its praises to every geek I knew, and even the "normies" (who stared back with glazed eyes). I had a real good feeling about it and liked the way it was going. I thoroughly enjoyed what was happening and thought the CrackOverflow title was well deserved. :)
It's Quick and Easy
Much like the drunk girl at the party, it's easy to pick up, doesn't take much talking to to get down to business and is easily accessible. Authentication is done via OpenID so there is not another password to remember. Profile information is minimal and the site speed is, well, speedy. As well as the actual site being speedy, it's also very fast to get answers from the community.
I <3 the Rep System
I thought the rep system was a great idea. I am a firm believer that software is becoming, and should be a socially-aware affair. Humans are social. Software is used by humans. Ipso facto, software is social. The rep system allows the "cool" (maybe not a good choice of words for a geek community) to be elevated above Joe Average.
I <3 the Badges System
A fun little edge to the rep system, the ability to get "badges" or awards for doing certain things is cool. Inspired by the XBOX 360 GamerCard system, it gives you something else to "work" for.
I Like Jeff & Joel
Most geeks find it nice to see geek startups, especially geek startups doing well. I remember the good 'ol days when Google was this cute little search engine that not many people knew about and it worked. Yeah it kinda sucks they are now bent on world domination, but hey-ho. It was equally real nice to see Jeff &  Joel's ideas unfolding and seeing the site pick up speed. No matter what I say here, I always hold these two gents in high regard.

... And The Negatives

Sadly there are a few more of these, I guess I get more caught up in concerns with StackOverflow rather than doing my StackOverflow-sunshine dance. :(
Note: Some of these are merely usability/feedback points.
The Rep System
While the rep system is a huge plus, there are a huge amount of issues with it. It's a simple concept, but a complex ecosystem.
Farming/Seeding
There have been many questions that were created to simply get answers. Each answer and upvote means rep. This user in particular went through a phase of pretty much posting "What is <programming term>?". The user did not actually need to know the answer, they already knew it. They were doing so under the guise of "trying to build the library". This to me just reeks of "rep seeding/farming". Now, much like spam, if only one of these questions is upvoted, then the user has received "free" rep. So why not create a bot to simply start posting all the computer science terms I can muster? Yes, it may help the content of StackOverflow, but does that mean the user should get the rep?
It should be noted that this user did seem to stop after being flamed for it, however, the issue remains. This behaviour was only really picked up because they posted several questions in quick succession.
"The Bob Munden" (Fastest Gun in the West)http://www.shootingusa.com/LATEST_UPDATES/BobMunden1.jpg
Another problem that has been highlighted is the "fastest gun in the west" (who is currently Bob Munden). This is basically the "fastest fingers first" issue.. People that answer quickly tend to get upvoted and even accepted even if their answer is wrong. This problem occurs for three main reasons:
  1. The person answering does not read the question properly in their haste.
  2. The person asking the question most likely doesn't know they are wrong.
  3. The people upvoting don't read the question properly either, or they don't know and "have a hunch" (or both!).
This to me is a real serious issue. I have seen many answers that are outright wrong. Some have actually been upvoted way past the actual correct answer! My question on type enforcement for generics is a prime example! The answer posted by Jon Limjap actually led me down the right path (so I of course accepted) only received 3 upvotes. However, the answer by Lars (who was the first to respond) received 6 upvotes and was clearly wrong! He failed to read my clarification and even edited later to "correct" and still didn't address my issue. I know people then did exactly the same and upvoted. If you read the edit history on the question, even he points out he read it too quickly!
"Community Mode"image
It was raised that if the community contribute to a question and improve its readability/content etc, then the original user should not continue to get the rep for it. Now, IMO this is completely wrong. I had originally posted my thoughts (which a large number of members agreed with) as an answer to a question airing my thoughts. In short:
  • The original poster should always maintain true ownership of the question.
  • Jeff actually agreed with this at one point, but now seems to have u-turned on this as users cannot reclaim control of it.
  • If the question is such a mess that the community needs to work so much on it, then it should be requested that the original asker fixes it (or it is fixed once). If that much discussion is required on a question then it is a badly worded question.
"I Want the Rep, the Whole Rep, and Nothing but the Rep"
This is a classic attitude issue. There are too many arrogant geeks out there that just want to look good and somehow think a rep of 4000 makes them cooler than the other geeks. The problem is that geeks like this are bad geeks. These people just want to get as much rep as quickly as possible. They answer posts quickly and incorrectly, ask crap questions just to pull in little bits of rep here and there and generally just degrade the whole experience.
Question Quality
An obvious one really. We want good questions. StackOverflow is not Yahoo! Answers where anyone can ask anything. It's a programming community.
To get it into context, I ended up losing my rag one day and posting this sarcastic question: "What is a programming question?" (this has since been deleted). It had a wide range of interesting answers. The majority saying we want good, tough programming questions. Equally many said pretty much anything that remotely relates to programming.
Stupid Questionshttp://www.leeds.ac.uk/classics/resources/theory/questions.jpg
Now, we all like to think there are no stupid questions. Well, there are. I am a firm believer that any programmer worth anything should be able to research quickly and effectively. There is simply too much going on in programming to remember everything, so you need to find stuff, fast. There have been many questions getting posted to StackOverflow that you could Google (faster) and actually get most of the answer from the summary in the search results! I personally think [for the majority] this is just laziness and falls into the "rep farming/seeding" category as well we accentuating "The Bob Munden". Of course, I welcome questions from actual newcomers to the geek community.
"Good" vs. "Good for Me"
This is something I picked up on quite recently. I find that I tend to upmod questions not that I can answer, but those that I can't due to their difficulty. To me, a "good question" is one that makes me think, makes me curious and want to find an answer. However, often simple questions get upvoted more simply because they know the answer. IMO, this makes no sense! That's not a "good" question, it's just an "easy" question! For example, I had a real bitch of a question on "serializing large, complex objects to XML". I, and others found this tough, yet it only received 3 upvotes. Yet this question on "use of the var keyword", which is pretty subjective and requires no real thought at all got 21 upvotes! Can you say WTF?
Organisation
Since its launch, the rate at which questions are being posted has increased massively. In the mess of questions we need a way to be able to find ones that answer our question, or we can answer quickly!
Search Support
The search functionality is improving. However, is it still limited (for example, no OR search). It also has limited filtering options. One major problem for me is that it displays searches the answers as well as questions. So, you can end up with a page of results that point to one question (which may not help you). Tag searching is also improving but still limited and even misunderstood by its creator (see the comments).
Finding Your Stack
I am a C# developer. I work on Windows and ASP.NET applications. I know nothing about Java, Python, Ruby and the many other languages out there. I can offer limited advice on architecture and design. Now, currently, it is bloody difficult for me to find questions with the appropriate tags so I can assist. I propose:
"Smart Lists" - these should be lists that each user can create that you can specify tags to search for. For example, I could create three "Windows" (which searches for items tagged "C# WinForms"), "Web" (tagged "ASP.NET") and Architecture (tagged "architecture"). Now, a web developer who works on the LAMP stack may have a "Web" tab, but entirely different tags.
I am currently getting around this by having Launchy shortcuts set up for my stacks.
Sort Order's Applied to Questions
I find these can be kind of "weird".. I question their real use. This may be largely due to the problems described, for example the "by Votes" sort. Highly voted questions tend to either be rubbish questions (the first is the unofficial FAQ, 3rd is Bob Munden) or questions already answered. Do I want to see them? Nope. "Hot"? May not be on my stack, do I care? Nope. "Active"? Well, I don't see the real difference between this and hot so, nope.
Moderation
I think one of the biggest issues with user-driven content is managing to keep it under control. One problem the Internet has had since hitting the masses is anonymity. People love it because they can become an absolute asshole and no-one will ever know! Now, I am certainly not suggesting that StackOverflow users are asshole's, far from it in fact. But there needs to be some control.
Contested Answers
To help combat the problem noted above about the incorrect answers, I think people past a certain rep should be able to "contest" the answer (rather than just comment) this makes is very obvious to visitors that the answer is in debate, much like Wikipedia does with its maintenance policies/guidelines.
"Hit and Run" Questions
Questions that have been asked, but the user asking does not accept an answer. Yes, while "Votes" view will pushed voted answers to the top, I feel that experts should be able to accept that answer on behalf of the user who asked the question (so the person contributing gets credit). This should be obvious it has taken place, and of course the user can always "unaccept".
Comments Help, But Are Easy to Miss
Comments were recently added, which enable users to add notes to questions and answers. While these help a lot, they are hidden by default and they are not obvious when shown.. So they can often go missed.
Lack of "Solid" Guidelines
The guidelines set in the StackOverflow FAQ are really non-specific. There is no real attention made to moderation policy which I find deeply worrying, sometimes those with moderation powers can be worse than the users! I think this really needs to be addressed so those that are moderating are all working towards a common goal.

In Summary

I started out incredibly positive about StackOverflow.. This has now changed. Don't get me wrong I am still positive about it, but lets just say I am not holding my breath. There is still a lot of work to be done, and it is only going to get harder following it coming out of beta. Watch this space I will blog more and keep on this.
In my opinion StackOverflow could be the absolute best thing for the programming community, but has the potential to be one of the greatest failures, in that it's a great idea but implemented badly.
However, Jeff & Joel are smart guys, and I do have faith that they won't let it get to that. But, we are all human and no one is infallible. :)
What are your thoughts on the above? Do you have any points you would like to add? I know there is a lot here and I am bound to have missed something!
Share:  digg it! del.icio.us Live Technorati Facebook

8 comments:

Ivan Hamilton said...

Nice piece. I too am ambivalent toward SO.

I think guidelines and the ability to bring them to bear are important to produce quality.

The rather simple voting system encourages popular, not helpful questions and answers.

SO discourages down-voting. I think that's the wrong approach. I believe the best answer is not the "most good", but the "least bad".

The Wikipedia approach encourages labeling the flaws in information. The SO approach leaves incorrect information unscathed.

Popularity is not a good way to gauge the accuracy of something. Unfortunately, it may be the best we've got.

Sam Hasler said...

"My question on type enforcement for generics is a prime example! The answer posted by Jon Limjap actually led me down the right path (so I of course accepted) only received 3 upvotes. However, the answer by Lars (who was the first to respond) received 6 upvotes and was clearly wrong!"

It looks like you asked that question before comments were implemented. If you had been able to add a comment that it wasn't addressing the question I doubt it would have been upvoted as much.

You've got over 3000 rep, you should edit Lar's post to put a note on it saying it's the wrong answer, otherwise people seeking an answer to the same question might assume it's correct if they don't read any further.

Rob The Geek said...

Hi Guys, first off, thank you for the comments.

@Ivan
Nice to see we are on similar lines. I am very much in agreement with you on your points. I think there are some interesting challenges facing StackOverflow.. I think at this stage, we just need to sit tight..

Today was the first day StackOverflow was released to public beta. And TBH, it has not been too bad, but its still very early days!

@Sam
I think you perhaps missed the point.. Commetns shouldn't be required to correct the problems I mentioned, they are down to people actually being of quality and wanting quality.. It seems like its [too] easy for people to upvote easy/mundane items rather than actually drive the quality of posts up (I have an example which I will blog about sooner - watch this space).

As for the issue with Lar's post. At the time, I never had 3000+ rep and again, I shouldn't need it. The problem was not my rep, but the fact that he didn't read the question properly!

Sorry for not making this clearer.

Chris said...

Just to be clear, you can do an AND search for multiple tags (though Jeff was wrong when he explained how to do it in that comment).

Rob The Geek said...

@Chris
Did you read the comments? I know you can do AND searches on tags, the problem is that you cannot browse (i.e. search for JUST the tags). You need to enter the URL manually, which then of course needs special chars escaped. The bookmarklet I threw together allows you to do that.

Mendelt said...

Hey Rob! Great way to get people to your blog through stackoverflow :-) I'll have to try that sometime. It seems we had the same idea last sunday to blog about stackoverflow. (http://blog.mendeltsiebenga.com/post/Another-Stack-Overflow-review.aspx) you seem to have put a bit more thought into your post though. I agree with most of your points. I suspect the problems with stackoverflow go deeper though. By combining the rules from a simple system like a wiki with the sometimes contrary rules from a somewhat more complex system like a forum and then bolting on the reputation system the rules governing interaction with stackoverflow get a bit too complex. I like the post-ownership and I also like the community-mode but I don't think they work well together. This makes it hard to see if the community aspects of the site will scale. I hope the "problems" we're seeing now can be resolved. But I'm not sure.
That said, I don't think I could have done things (much) better.

See you on SO!

Alan said...

The handful of questions I posted are ones I knew the answer to but which I struggled with and had to research to find the answer. I was trying to contribute to the knowledge base and save someone some time should they run into a similar situation.

There are too many subjective questions that are related to programming but are not true to the intent of the site of providing specific answers to specific questions.

What's the point of accepting an answer if the accepted answer may not be the best or even correct? Should the highest voted answer be the accepted answer?

Is the author the only one that can accept an answer? And he can't azaccept his own? Why encourage one to answer his own question?

j_baker said...

Great piece. I agree 100% on the issues you see (provided that they are in a "no particular order" format). Personally, I see over-moderation being the biggest challenge the stackoverflow community must overcome. But I think this is just an area where procedures and conventions must be developed.