Archive for January, 2009

C# Subversion ChangeLog Generator

Monday, January 19th, 2009

Ever wanted to generate a change log from all your (dubious) Subversion repository comments?  For some reason want to do it in C# or just by calling an exe?  Now you can!  You can even tie it in to continuous integration!

About 6 months ago I spent an evening porting the Subversion Change Log generator to C# in order to integrate Change Log generation it into our Cruise Control.Net nightly builds at work.

It’s not a direct port, more a C# reimplementation, but it produces the same output as svn2cl (which in itself was inspired by cvs2cl).  The gist is, that it calls svn.exe on the command line and retrieves the full history of comments for whichever svn path you request.

It’s not the most elegant thing I’ve ever written (it pretty much just spawns the svn.exe and pipes the output through an XSL transform) and requires you having a command line svn.exe on your system (which if you use subversion on windows is pretty likely), but if you do you should be able to generate change logs from your repositories by calling the compiled executable like so:

SubversionReportProducer.exe –style=ChangeLog.xsl –outputLocation=out.txt –repositoryPath=svn://repo/trunk

If you donít want to pass the subversion exe path and repository path through on each execution you can specify them by modifying the values in SubversionReportProducer.exe.config.  Those values will always attempt to load from configuration, and then attempt to load from a command line parameter (overwriting any app.config settings) allowing for maximum flexibility in batch scripts and automation.

I’ve bundled it with a couple of XSL transform files to produce some output, mostly taken from svn2cl and one subtle adaption that we use in our build process (ingeniously named changelog.xsl).

Your mileage by vary, but this could well be of use to someone.

Source code provided, feel free to contact me about any bugs but Iím pretty much just kicking this out into the wild..

Download Source
Download Compiled Executable
Requires .Net 2.0+

SEO Lies & Real Solutions (Or Why SEO Experts Are All Hacks) – A Case Study

Thursday, January 15th, 2009

People just love to go on about search engine optimisation these days.  They LOVE it.  SEO experts are put on pedestals as the salesmen that just keep selling, even when they’re not at work.  If they’re not going to do it, their competitors will, after all.

Except SEO is a mugs game, the “experts” are full of shit and it’s pretty much all lies.  I’ll give you this one for free.  By the end of this post I’ll have walked you through how to search engine optimise your web presence and with any luck, you’ll see the benefits.  Before I do that, I’m going to give you a little background information.

The person that “manages” the SEO in your company (if the job listings for “SEO Managers” are to be believed) are sharp, sales focused individuals with a good grasp on technology. As such, these individuals are relatively well paid, for a general sales job anyway.

I did a very quick sweep of some job sites and came across the following job listings.  These represent a fair sample and are pretty much the average (source http://www.totaljobs.com/JobSeeking/SEO.html).

SEO Manager £35000 – £45000 per annum + Bonus + Benefits
SEO Manager £35000 – £41000 + benefits
SEO Marketing Executive £30000 (SEM / PPC / Adwords / Analytics)
SEO Web Leader £30000 – £45000 per annum + Benefits  (Featured job)

Other Supporting Figures (http://www.itjobswatch.co.uk/jobs/uk/seo.do)
UK excluding London average salary    £30,267
% change on same period last year          +5.27 %

There are multiple pages of results that look the same as that cross section.  The average amount an “SEO Manager” pulls in as a base salary appears to be around £35000.  That’s not mathematically accurate, but it’s a reasonable estimate.

So what does an “SEO Manager” really do?  Well, interestingly one of the job listings above really gives the game away. “SEM / PPC / Adwords / Analytics”.  To decompose the acronyms “search engine management”, “pay per click”, Google Adwords and Google Analytics.

Lets dissect each of those job roles one by one and work out what your SEO Manager is doing for you.

Search Engine Management
This is actually a reasonably interesting category because it doesn’t mean very much. The real wins in “SEM” will be done by your programmers or your middleware CRM systems.  I’ll explain some of the valid techniques for “SEM” below, however, it’s important to note that “search engine management” doesn’t really mean anything at all and should be something to happen naturally as your website grows.

Pay Per Click Submissions
Daily, this role tends to involve the exporting of a product list from a database, and importing it to a number of PPC sites, such as Kelkoo and PriceRunner.  Often a very simple task to automate, despite PPC sites having a tendency to ask for data in weird and wonderful formats.  This bit of the job description probably covers nagging the PPC partners when they invariably don’t list your products in time.

Google AdWords
Daily, logging into your Google account, monitoring the Estimated Cost / Day on some keywords (and when I say monitoring, I mean “stopping bidding when they look too expensive”) and using the very friendly Traffic Estimator Sandbox to pick new links to bid on.  Not a hard task for a seasoned marketer of any kind.

Google Analytics / Analytics Software Of Your Choice
Adding a 2-3 line piece of JavaScript into your website and then logging in to your analytics package, to monitor the progress of your site, broken down into graphs and exportable data.

You could almost be fooled into believing that anyone could be an “SEO Manager” with roughly a days worth of reading the Google help documents and learning how to use a few very simple tools.  Oh, you were thinking that too?  This is a job your marketing department should be doing.  Don’t have a marketing department?  It’s not hard, it’s your job, and I can help you with some very simple tips.

You see, the good thing about Google is that they’re on your side. Google thrive for two reasons.

  1. Search is their thing – Google is the internet’s home page, they drive almost 90% of the worlds traffic. You don’t need to play them, they want to know who you are.
  2. Google survive by selling adverts.

Google want to list you as much as you want to appear at the top of page 1 (and, seeing as nobody reads past the first two pages of search results, better keep off page 3+).

So lets make it easy for Google to find you.

In the dark ages of the internet (or for the sake of argument, the early 2000’s) people were always trying to sell tips related to manipulating your Google “page rank”, the magical algorithm that Google uses to determine who appears at the top of the search page.  There were even some tips that worked.  That was 2002. Before we go any further I want to make one thing explicitly clear:

You can’t fool Google’s search results for your website.

SEO That Works

Got it?  Excellent.  So what can you do?

Lets take a fictional product in a reasonably established market.  You’re a company that develops interesting light fittings and lamps, and you’re about to launch a new range of touch-sensitive-base lamps.  Sounds plausible.

Luckily for you, Google is really really good at indexing websites, but there are a few simple tricks that help Google index you thoroughly and quickly.

  1. Content is king

    If you’re trying to get highly listed for a keyword, ensure your site has content related to it.

  2. The Holy Trinity Of Keywords

    One of the simplest and more effective ways to get indexed and ranked highly is to ensure that there’s a good relation between your URI and the topic you’re hoping to rank highly for.

    Make sure your URL, page title and keyword densities all say the same thing.  Ever wondered why if you search for anything on Google the Wikipedia link is always in one of the top three places?  This is why.

    Google indexing puts a lot of weight in to the relationship between the URI of your website, the title of your page, the header tags (H1) and the keyword density of your content.  To work using the above touch-sensitive light example a URL of http://www.touchlamps.com/shop/Our-New-Touch-Lamp.html would rank higher for the search term “touch lamp” than http://www.lighting.com/shop/touchlamp which would rank higher than http://www.homeware.com/products/123.  Likewise, if there were two shopping sites www.lamps1.com/shop/lamp and www.lamps2.com/shop/lamp if the page title of the former was “buy items” and the page title of the latter was “buy touch lamps” the latter would rank higher.  Having the H1 and H2 tags on your page relate to lighting or touch lamps would further endorse your page rank.

    What should you take away from this?  Ensure that for each page of your site, the keywords that you want
    t
    o be linked to for are frequent in the top 1/3rd of the page.

  3. Use a sitemap

    Commonly missed, nothing helps a search engine better than a good sitemap.  You’d probably like to dynamically generate this for a large shopping site, but if you don’t, Google offer some nice webmaster tools to help youSitemaps act as signposts for search engines, enabling them to efficiently spider and index your website, allowing you to dictate the way the search engine behaves related to your content along the way.

    Sitemaps help you tell the search engines how often you expect any given content to change, and suggest a frequency of re-checking to allow their indexes to be permanently up to date. 

    There’s no use in trying to see a new product if the search don’t know you’re trying.

  4. Reputation Reputation Reputation!

    It’s all about who you know.  The internet is brilliant for spreading word of mouth so make friends with other sites that share either your interest, or interest in your products.  Nothing works better than a little positive PR. 

    Do you sell a product that has a rabid fan-base?  If so, consider searching for user forums or big sites in the user community and throw them some freebies.  This is a common trick in the ultra-competitive market of technology resale.  Supply fan sites with free samples in exchange for links and you’ll find that your page rank improves as a consequence. 

    Google and other search providers give “bonus points” to your ranking if you’re linked to by other locations that also rank highly for the same keywords.  Thus, you might take a financial hit for sending out products (worst case) or just be willing to help out other resources by linking to other useful content (if you’re not a retail entity), however the SEO-bonus and goodwill produced from such an effort will be worth far more. 

  5. Sanitise Your URLs

    Having a URL schema that looks something like http://shopping.netsuite.com/s.nl/c.851077/n.2/sc.10/category.285/.f is the quickest way to punish yourself and your website for no reason.  I picked that URL at random, it’s a website based on Netsuite’s web platform and their CRM produces some of the worst URLs I have ever seen (I just searched for “netsuite /s.nl” due to pre-exposure to their URLs in search of an example, I couldn’t comment on the actual content of that URL).

    If your developers or your CRM aren’t producing human-readable URLs, get new developers or get a new CRM.  It’s really that simple, it’s taking part in the online market with a self induced handicap.  Url rewriting is very simple, and doesn’t even have to be perfect to be good enough.  It’s well supported using mod_rewrite in Apache or a handful of ISAPI filters on IIS up to IIS6, and supported natively with HTTP modules from IIS6 onwards.

    Look at it this way, what do you prefer: http://www.hotlamps.com/shop/123/touch-lamp or http://www.hotlamps.com/shop/s.nl/c.851077/n.2/sc.10/category.285/123 or http://www.hotlamps.com/shop/123.  I would always vote for the first URL, it reinforces what you’re attempting to sell, plays into your page rank bonus for Url to content relationship and just makes far more sense to a visitor.

    Use URL rewriting or you’ll regret it.

You’ve done all that and you’re still not getting traffic?  You’ve got a brand new touch lamp you’re trying to sell and just can’t get the page impressions?  Thinking about hiring an “SEO Manager” to make all your problems go away?  Stop.  Buy your visitors.

I’m not a marketer (and that’s probably obvious by now) but if you’re doing all the right things and people just aren’t seeing your product, it’s time to look into your cost-per-acquisition of a new customer.

You’ve just decided you need to go through with an online marketing push for your new flagship range of lamps.  A quick Google search (13th Jan 2009) leads with two sponsored links. One for “Touch Sensitive Lamps” and another for “Touch Lamps at Amazon”.  In addition to that, the sidebar (sponsored links) point to “M&S Lighting”, “Touch Lamps at shop-com.co.uk”, “Touch Lamps? at dealtime.co.uk” (a price comparison engine), “Lamps Touch”, at supaprice.co.uk (also, presumably a price comparison engine), Ask.com, and Shopzilla.co.uk/ComparePrices.

Two things strike me about those results.  The first is that most of them are price comparison engines buying sponsored ad space for the minimum buy price.  The second is that there aren’t many players in the “touch lamp” market past M&S so buying links should be easy. 

Regardless of your hiring an “SEO guy” or not, you’re going to buy buying Google ads.  It’s a fact of life in a world dominated by a single search provider and even more dominated by a single Ad provider.  You have to do two things, buy the right keywords, and play the price comparison engine game.

Buying The Right Keywords

The really great new is that Google has a fantastic tool available to you to predict how well buying clicks will go for you.  To maintain the current example, I ran a few phrases through the sandbox (find it yourself at http://adwords.google.com/select/TrafficEstimatorSandbox) and got the following results:

 

Keyword Est. Cost Per Click Est. Ad Position Est. Clicks/Day Est. Cost/Day
“fancy lamp” £0.00 – £0.65 1 – 3 0 £0 – £1
lamp £0.80 – £1.08 1 – 3 3,214 – 4,021 £2,580 – £4,330
lighting £1.05 – £1.52 1 – 3 5,981 – 7,484 £6,270 – £11,410
“touch lamp” £0.66 – £0.85 1 – 3 17 – 21 £20
“touch lighting” £0.81 – £1.01 1 – 3 1 £1
bedside £0.57 – £0.74 1 – 3 238 – 297 £140 – £220
“bedside lamp” £0.80 – £1.14 1 – 3 6 – 7 £5 – £9
“touch light” £0.55 – £0.72 1 – 3 4 – 5 £2 – £4

Using readily available tools, it doesn’t take any kind of expert to recognise trends when provided with tabular data.  You basically want to buy a good spread of keywords with the lowest CPC vs. the highest Clicks/Day, but more likely, meeting somewhere in the middle.  From the above predicted results (and remember, they’re just predicted trends) the two sure-buys seem to be “bedside” and “touch lamp”.  Which makes perfect sense. 

My one word of warning when buying traffic is to be careful not to get carried away.  I’ve worked for a company that crippled itself buying really obvious keywords and costing itself more to purchase some clicks than the profit margin on the item.  That’s commercial suicide.  Know what your recommended cost-per-acquisition is and don’t go over it, however good an idea it seems.

Playing The Comparison Engine Game

I don’t have very much to say about the comparison engines.  They often charge you for submission and position, and they index so many products that they have instant high ranking on search engines.  They minimum bid on any keyword that you can buy and are very prolific.

I’d pick just one or two to participate in, and closely monitor the ROI on any payments you make to them.  Don’t let them hold you to ransom, just remember that any links you’re paying for on comparison engines are placed side by side with the links of your competitors.  So make sure you’re the best OR the cheapest, but don’t presume the links have much value.

So Should I Hire That SEO Manager Anyway?

That’s totally up to you.  But as a rule of guidance, if you were to be selling a new range of touch lamps, you’re SEO Manager would have to bring in 28500 potential sales per year to make hiring him anywhere near as efficient and worthy as just buying 90 days worth of Google ads for “bedside”.  You could save your £35k, ensure your developers are doing the right thing and get the same results, for nothing.

Tricking Google

Earlier I said that “you can’t fool Google” and I really meant it.  Over the years tonnes and tonnes of tricks have been developed to exploit the way page rank works. 

A few of the most common tricks were:

  • Flooding your page with huge collections of keywords the same colour as the background of your website to increase page rank via keyword density.
  • Blog link spam
  • Websites full of links to your own content

You’ll be pleased to hear that these techniques, along with being the digital equivalent of fly-postering, not only don’t work, but negatively effect your rank.  Google have something like 4,500 developers.  A large portion of them will be dedicated to stopping people tricking and abusing their ranking algorithms.  Don’t waste time trying.

Footnote

I used to work as a developer on a large eCommerce website specialising in IT equipment and hardware.  I was responsible for writing all the code that dealt with search engine optimisation and wrote the majority of the web-facing code.

I met many many people who claimed to have some secret source, I read lots of aging documents on Google page rank (whatever you read is out of date, page rank changes daily) and sat through too many meetings with brain dead SEO consultants who couldn’t advise anything concrete past their smoke and mirrors.

Worse than that, I saw some of these consultants hired to “improve” our page ranks in a number of unsuccessful and unquantifiable ways.  I’ve seen consultants cost tens of thousands of pounds delivering nothing when that money should’ve been spent on traditional advertising and content creation.  I’ve seen companies try and buy so many links it forced them to near bankruptcy.

I wouldn’t call myself an expert on SEO, but I certainly know what doesn’t work.  After three months of working on “SEO” the only things that ever worked for us, were good content, good links and purchasing Adwords.

I hope you take a few good practices from this, but more than anything, I hope you don’t buy into the SEO bullshit.

If you want proof that regularly updating content works, go search for “David Whitney” on Google.  My humble website ranks above any other (source http://www.google.co.uk/search?hl=en&q=david+whitney&btnG=Google+Search&meta=).

Good luck!

Late Night Therapy

Tuesday, January 13th, 2009

I’ve had quite a busy night this evening.  I’ve been trying to finish watching the third season of Alias (about 5 years too late on that one), ended up starting an article on best practices for build automation and integration testing in the .NET framework, worked my way towards watching the double episode premiŤre of the new season of 24, got tired, and now can’t sleep.

So a little bit of lazy doodling took it’s place.

UntitledSea 

Trying to clean my head of oceans and deep dark red so I can move on to other things.  I’m not really sure if I like anything about this apart from the reds in the sky.

I still have a few huge pieces stewing both on paper and in my head which I should get around to finishing but working entirely in red is exceptionally soothing, and thus, distracting.

Bed time.

Now Playing: Nine Inch Nails – In This Twilight

Microsoft, thankfully, standing firm on BluRay and the 360

Friday, January 9th, 2009

Another day another slightly amusing interview snippet.

So apparently Microsoft are taking a hard stance on including a BluRay drive for the Xbox 360. To quote Kotaku quoting Robbie Bach:

“Itís not a feature we get a ton of requests for. We really donít. When you ask people the list of things they want to see us spending time creating in Xbox, Blu-ray is way, way down on the list.

The second thing is, from a technical perspective, it doesnít help us in the core of what Xbox does, which is in gaming. We canít have publishers produce games on Blu-ray disc. Because then they wonít play on the 28 million Xboxes weíve already shipped. So it doesnít help us in the core gaming space.

The third thing, and this maps to all three of those, is that it costs a lot of money. And so the scenario is, OK, let me get this straight: Iím going to add something to the product thatís going to raise the cost, which means the price goes up, consumers arenít asking for it, and by the way, my game developers canít use it.”

It’s a move I entirely agree with, but I can’t help but feel that they’d be cutting off this nose to spite their face if they didn’t end up offering it as an optional extra?

There’s a clear and good reason not to include a BR drive as internal. It makes absolute sense, it’d further fracture the already slightly divided 360 user base (really, who buys an arcade?).

That said…

Stating that there isn’t demand sounds like conjecture and a little bit of FUD to me. Microsoft look like they’re using it to try bolster the movie download business for themselves.† If they felt that HD-DVD was demand worthy of a console addon when far less people were interested, then it only makes sense that there is more interest now that a format has been decided.† Avoiding BluRay out of interest makes it seem like backing HD-DVD was actually a move to undermine Sony.

The drives aren’t even so expensive to make these days (combined BR/HD-DVD/DVD-RW drive for about $99 / £50 on general sale), but there would be considerable development time porting a java virtual machine to the 360 (presuming one doesn’t already exist), and lets face it, Microsoft don’t have a great history with the JVM ;)

That said, I’m not aware of the market realities, but the critic in me would suggest they just don’t want to feed any licensing revenue into Sony and they’re more than just a little bit bitter.

That and they probably want to avoid inferiority comparisons when Sony go on their next “ah ah ah! we have Blu Ray included! that’s why we’re more expensive!” rant.

Personally? I’m not interested in a blu ray drive for me 360. They’ll likely use one on the next xbox as it’ll be the dominant format for high capacity disks by then, it’s only logical. Either that or they’re going to take one hell of a gamble on worldwide infrastructure. It’d be a fun bold move but probably one that’d cripple them as much as choosing BR crippled Sony for the first half of this generation. Not exactly a good business move.

At least they’re saying no categorically.

A Few Of My Favourite Things (Of 2008)

Wednesday, January 7th, 2009

Everyone loves lists.  Everyone.  So much so that entire music TV channels appear to exist solely to tell me what the top X of Y are.  So, to do the blogging equivalent of locking the stable after the horse has bolted, here are a few reasons why I enjoyed 2008.

Music

I really hate picking my favourite music of a period because there are never quite enough places to note everyone.  Below are the CDs I really remember enjoying this year.  They’re in rough order and all of them albums below pleased me in very different ways.  Some of them are predictable choices for me, A Perfect Circle side projects, Porcupine Tree front-mans solo album, Opeth, Nine Inch Nails, but amongst them are a couple of bands who were both new to me and won a constant rotation.  Textures deserve a special mention for being by far my most enjoyed CD this year, while the Ashes Divide CD, whilst flawed in a few places, was my most played.

  1. Textures – Silhouettes
  2. Steven Wilson – Insurgentes
  3. Ashes Divide – Keep Telling Myself It’s Alright
  4. Opeth – Watershed
  5. Slipknot – All Hope Is Gone
  6. Meshuggah – Obzen
  7. Cult of Luna – Eternal Kingdom
  8. Nine Inch Nails – Ghosts & The Slip
  9. Zimmers Hole – While You Were Shouting At The Devil… (We Were In League With Satan)
  10. 3 – The End Is Begun

Honourable mentionsFrost* – Experiments In Mass Appeal, Metallica – Death Magnetic, Made Out Of Babies – The Ruiner, Ayreon – 01011001, Cynic – Traced In Air
Biggest disappointmentTomahawk – Anonymous
Best live performance: Ministry

Games

I’ve had a big year of gaming.  I finally quit World of Warcraft in the latter half of 2008 which afforded me plenty of time to play just about everything that came out.  This year was slightly off balance with a most of the games that seemed to be worth playing arriving in the second half of the year.  Nothing this year grabbed me like Mass Effect did last year, but of the games I played this year, the following are worth a mention:

  1. Fallout 3 (I enjoyed Fable 2, but I’m a Fallout kind of guy)
  2. Left4Dead (my first online shooter for about 5 years, awesome coop fun)
  3. Braid (the first ever XBLA game to convinced me to buy points)
  4. World of Goo
  5. Rock Band / 2 (for drunken fun and Christmas relief)
  6. Dreamfall (The Longest Journey) (not all that gamey, but an excellent sequel)
  7. Zack & Wiki – The Quest for Barbaros Treasure (the only reason to own a Wii this year)

Honourable mentions: WoW (for the good times), Gears 2, Fable 2, Mirrors Edge, Prince of Persia – all games that deserve their strong reputations, all second half of the year games.  I’ve probably forgotten the first half of the year now.
Biggest disappointment: That Sonic Unleashed was terrible, that Nights2 was mediocre.
Most anticipated for 2009: Heavy Rain (despite not owning a PS3), Mass Effect 2 (with any luck!)

Films

I somehow saw far fewer films this year than I normally tend to.  I’m not sure if that’s indicative of the quality of the films or just my lack of attention.  I have an unlimited cinema pass yet seemingly I’ve just seen nothing at all.  That said, I remember really enjoying:

  1. In Bruges – Not what you expect at all, and a really quite human story.
  2. The Dark Knight – Lets face it, everyone loved it.  I still enjoy Nicholsons Joker equally however..
  3. Sweeny Todd – Very standard Tim Burton, but stunning to look at regardless.
  4. Be Kind Rewind - Utterly charming and out of character for Jack Black
  5. Hellboy 2Just pure fun
  6. Vexille - Exceptionally stylish, but I perhaps enjoyed it for the visuals alone.
  7. Wall-EStandard Pixar, but very endearing.

Honourable mention: Zack and Miri Make a Porno (oh so Kevin Smith, but very funny for it)

Technology

Surprisingly the software that’s impressed me this year has all been very mainstream.  Certainly a year of lots of software maturing to a point where it’s excessively stable and usable.

  1. Microsoft Surface – Surface computing black magic
  2. Opera 9 – Polished web browsing
  3. ReSharper 4.0 – C# / Visual Studio refactoring tool.  I don’t know what I’d do without R#.
  4. NHibernate 2.0 – .NET port of the popular java ORM.
  5. Windows Vista / Server 2008 (finally shaking off it’s dubious reputation, at least with the informed)
  6. Google Maps Mobile
  7. Windows Live Mail
  8. Notepad++ – Great alternative text editor that finally replaced context in my heart.
  9. Google Reader – Upsettingly better than all
    t
    he other syndication services.
  10. Wacom Bamboo (graphics tablet)
  11. Windows Live Writer
  12. XMBC going multi-platform with the Atlantis port – Hands down the best home theatre application.

TV

I’ve been watching less and less TV, and seemingly slipping towards more light comedy than anything over this past year.  Too many long evenings writing software leads to shutting off your brain eventually.  I have been watching a few shows religiously however.

  1. South Park
  2. Terminator – The Sarah Connor Chronicles
  3. How I Met Your Mother

Pretty much in that order.  South Park is the one piece of TV satire that’s consistently on the mark and funny, the Terminator series is a lot more grown up than Fox would like you to believe and How I Met Your Mother, while stealing from Friends and Scrubs in an odd hybrid, is very entertaining and endearing.  Also an honourable mention goes to The Big Bang Theory for making me feel at home with television personalities.

I should really get around to catching up watching Heroes and Lost and Dexter, but they require a little more engagement than the above.

So there we have it.  I’d list books, but the books I’ve read this year have been all the “obvious” computer science books that people always say they’ll get around to reading but never do (books like The Mythical Man Month and The Pragmatic Programmer) so I’ll not bore you with yet another predictable list of the best programming and computer science books ever written.  I’m fairly certain you can find that list on CodingHorror.

I enjoyed those things in 2008, and I hope you check at least a few of them out.