Posts Tagged ‘ AI

Artificial Intelligence

Our firm is very successful at helping Google identify which sites should rank highest for high volume search terms. Those sites, of course, belong to our clients. Though far from the only tactic we use, link building is one of the most important. Our lead Search Engine Optimisation (SEO) analysts identify the type and quality of sites that they would like to link to our clients and how they would like them to be linked; our team of link builders then find suitable sites that are willing to link to our clients. Finding sites that meet the lead SEOs criteria that are also willing to link to our clients is a time consuming, laborious task with many dead ends. About half the company is employed looking for links.
We’ve looked at a few ways of automating the process. We’ve used backlink searches on competitor sites to generate hit lists, automated quality checking to reduce the size of these lists, Amazon’s mechanical turk to ‘automate’ searching for contact details, spreadsheet after spreadsheet of innovative macros to identify high quality sites. And, yes, we have got more efficeient – but not signifcantly so.
To frame the problem here’s a quick recap on how Google works. When you type a search phrase into Google it uses a reverse index to bring back a list of sites that are relevant to your query. These sites are presented in order of quality. Google mainly uses the Page Rank algorith to calculate quality (basically an Eigen vector calculation of cross linked web sites); Page Rank is modified using various top secret quality indicators such as domain age, perceived ‘spaminess’ of the site, load speed and so on. Some of the factors published by Google may or may not contibute towards the ranking but, in general, make the Internet a better place. We have, however, no way of identifying which factors are actually used by Google, which are false indicators or what the weighting is behind each factor. We can’t even calculate the Page Rank though the algortithm itself hads been in the public domain for a decade. As Page Rank is an iterative algorithm it relies on being able to iterate regulalry over vast numbers of websites – to do this and approach and level of accuracy we’d need infrastructure of similar size to Google’s.
We need a proxy for Page Rank (including all the other quality weightings in use). Every one of our SEO team can look at a site and decide very quickly whether or not it’s a good site and if a link on that site is likely to help or hinder our clients’ quest for rankings. I would assume that they are using their skill, experience and intelligence to make this judgement. Could we use an artiifcially intelligent system to make this judgement call?
There are three ways I will be investigating: a rule based system using Prolog, a neural network and a statistical model. Uniquely amongst SEO agencies we’ve captured a few terrabytes of data about websites, backlinks and rankings over the last few years providing plenty of data to teach an intelligent system; we can test results against live data.
Over the next couple of weeks I’ll publish details of how we’ll set these systems up and in a few months I’ll summarise the results.