Grant Midwinter http://www.grantmidwinter.com Technology by Design Thu, 03 Jul 2008 12:47:12 +0000 http://wordpress.org/?v=2.5 en New Clients! http://www.grantmidwinter.com/2008/07/02/new-clients/ http://www.grantmidwinter.com/2008/07/02/new-clients/#comments Wed, 02 Jul 2008 08:59:02 +0000 Grant http://www.grantmidwinter.com/?p=175 So, we have been busy. Nose to the grind. Grafting. We have a few new clients:

  1. We are extremely happy to announce that we will be working with Leeds Film Festival on their upcoming rebrand. Please keep a look out for posts as we develop their new site.
  2. Elite commerce are a project management resource company who specialise in delivering corporate objectives in the IT sector. And they’re good, as we should know from working with them helping Sky to launch and re-brand their new sites in the gaming sector.

These are just a couple of the projects we’re currently working on. There are a few more in the pipe line that we’ll be talking about in the near future.

]]>
http://www.grantmidwinter.com/2008/07/02/new-clients/feed/
Google Browser Sync is Open Sourced http://www.grantmidwinter.com/2008/06/25/google-browser-sync-is-open-sourced/ http://www.grantmidwinter.com/2008/06/25/google-browser-sync-is-open-sourced/#comments Wed, 25 Jun 2008 17:06:37 +0000 Midwinter http://www.grantmidwinter.com/?p=174 ‘Jimdog’ brings us news over on our (at last count) 123 comment post over the state of Firefox 3 and Google Browser Sync’s lack of compatibility. GBS has been open sourced very quietly by Google which means that the server side code should also now be available!

There is a fair amount of work to do be done converting this to Firefox 3’s places system and then of course there’s the issue of a remote server to sync to. If anyone is interested in taking this and developing for FF3 then please let us know and maybe we can contribute in some way.

It seems like Google expected the ruckus over this issue generated by our and other, lesser, articles to die down on release of FF3. As our pretty traffic graphs show though the community was anything but prepared to stop sending them emails and clogging up their search listings.

So thanks for the heads up Jimdog, FF3 and browser sync are sure to be here soon.

]]>
http://www.grantmidwinter.com/2008/06/25/google-browser-sync-is-open-sourced/feed/
Googlers Googling For Ideas? http://www.grantmidwinter.com/2008/06/03/googlers-googling-for-ideas/ http://www.grantmidwinter.com/2008/06/03/googlers-googling-for-ideas/#comments Tue, 03 Jun 2008 22:29:05 +0000 Midwinter http://www.grantmidwinter.com/?p=173 As always, I was over on reddit, grabbing some mostly useless news and friffery to wind my brain down. Things didn’t go to plan though when I came across this post and was immediately struck by a resemblance to one I’d written myself not so many months back.

Uncanny coincidences are going to happen when you get a company the size of Google employing highly intelligent (for a part) graduates in computer sciences as if they were going out of style (they were never really in style). However, short of finding your code in an apps, could you ever really tell if a lone employee were to raise an idea as their own… and then release it with a big fat G on it?

I did some Googling and Snapping. It wasn’t hard to find a few interesting links where coincidence has crossed the line into severe uncertainty.

I think I like Microsoft.

]]>
http://www.grantmidwinter.com/2008/06/03/googlers-googling-for-ideas/feed/
Google Browser Sync for Firefox 3 http://www.grantmidwinter.com/2008/04/02/google-browser-sync-for-firefox-3/ http://www.grantmidwinter.com/2008/04/02/google-browser-sync-for-firefox-3/#comments Wed, 02 Apr 2008 19:32:24 +0000 Midwinter http://www.grantmidwinter.com/2008/04/02/google-browser-sync-for-firefox-3/ UPDATED on 17th June, Firefox download day so scroll for the latest

One of my favourite extensions for my preferred choice of web browser (Mozilla Firefox) has to be Google Browser Sync.

This nifty extension lets you synchronise not only your bookmarks, but visited URLs, passwords and history across all of the browsers you install it on. For example I run it in the office at work, then shut down the PC for the day and head home. I load up Firefox on my home PC and all of my tabs are there from when I left work.

I also use it to synch my browsers across all of my workstations and servers, it beats having to copy paste through Remote Desktop or VNC. As it’s a Firefox extension, there’s no problem when I switch from Windows to Mac and then to Linux.

The only slight problem at the moment is that when I come to upgrade to Firefox 3, it’s no longer supported for that browser version. To be honest it would be already be installed it would work with Browser Sync.

Google have been extremely non responsive on this so far - maybe they just haven’t noticed or perhaps it was someone’s 20% project? If everyone who, like myself, is interested in getting Browser Sync back for FF3 can send them a nice email then maybe I can selfishly migrate!

UPDATE

I’ve been trawling the web a little more to find other people who have commented on this issue, and there’s not an inconsiderable number of others who have noticed. If you go to this thread over on Google groups there’s already a fairly extended number of posts asking for the update.

Adrianm suggests:

“I’ve sent an email to the support alias (labs+browsersync@google.com), so we’ll see what comes back..”

That was on the 19th March, maybe it’s something we should all try? I think I’d rather not suggest a mass email campaign to be honest. What’d be great though is if just one person with a contact at Google can let them know we’re interested! A response either way is just fine by me, then I can tackle the unknown territory that is Mozilla’s Weave.

UPDATE

Just today(April 3rd) Google Groups user mirage posted this:

“Let me just add to this thread and ask for the same thing: please port browser sync to firefox 3. It’s a life saver, and it’s really a bad predicament to have to choose between firefox 3 and browser sync.”

Which exactly mirrors my sentiments. We’re getting a surprising amount of traffic through on Google now with people literally typing in ‘Google Browser Sync Firefox 3′ to try and find any more information on the subject. Let’s see if we can take this momentum and see if we can draw an official response from Google on the matter.

If anyone has any suggestions then drop us a comment or send me an email. I have noticed our server isn’t playing ball with all the traffic coming through on this but we’re in the middle of moving our hosting across to a new dedicated box so hopefully we’ll improve in service a little! I’ll go fire up wp-cache now.

UPDATE

It’s April 9th, I’ve emailed Google’s Press email address after receiving no response at the labs+browsersync address. I think that we need to spread the word about this a little and try and get some support. I’m going to see if some larger tech news sites are interested in giving us a hand.

UPDATE

I received a reply from Google’s Press team (April 10th):

Hi Phil,

Thanks for your interest in Google Browser Sync. This is the first time this has been noted. I am looking into this but it may take a little while.

Best,
C.

So, we were the first to actually make them aware of this interest? It must be difficult to have so many products (and good ones at that) that you can actually lose track of them. Now we know that Google have our pleas noted though and that’s the only response I was looking for.

I’ve also been looking into other avenues - namely the technical challenges to get Google Browser Sync to work correctly with FF3. I suspect I’ve hit upon a snag:

Bookmarks & History

If your extension accesses bookmark or history data in any way, it will need substantial work to be compatible with Firefox 3. The old APIs for accessing this information have been replaced by the new Places architecture. See the Migration Guide for Places for details on updating your existing extension to use the Places API.

This is from Mozilla’s developer documents for extensions. Essentially it looks like the structure of the data itself may have changed significantly enough that the API at Google’s end would also need updating. If this is the case then it means that a 3rd party form of the extension would need to host it’s files away from Google’s servers. Then again maybe not if the new Google App Engine can be used.

UPDATE

It’s the 20th of May - a lot of people have asked if I can update here with the latest information I’ve got on this topic. Unfortunately Google seem to have stopped replying to my emails, I can only assume we’re not a priority for them right now.

Some people have suggested things like online petitioning to try and garner more of their attention but you’d think that with the FF3 release candidate coming under full steam they might of noticed already? Apparently not though, if anyone has any great ideas of how to bring our cause to their attention then please leave a comment or send us an email and we’ll proliferate it amongst our fellow Browser Sync hunters as much as possible!

UPDATE

Today is Firefox 3 download day, an event which hasn’t been as widely publicised as I originally envisioned. Unfortunately I’m not downloading Firefox 3 today - I already have RC2 running on one computer and for now don’t see a particular need to update.

The other 3 or 4 systems I use on a regular basis (I know I’m a geek) are all syncing just fine on GBS thanks. I know that by now most of you have seen the comments that GBS is not going to be released for FF3. Obviously no one is able to fact check that source (it was in an email from Google PR) but I’m inclined to think it’s the real deal and it’s about time one of us managed to get a straight response at least.

If you, like me, are tired of getting told to use Foxmarks or Weave - that DONT do everything GBS does despite anyone who tells you different - then have heart. We’d like to hear from you as to whether you’d be interested in a similar extension made by our team here? Also let us know of the features that are key - of course secure encryption, speed and reliability are big focuses but anything outside of that then slap down a comment and we’ll come up with a feature list.

We’ll go one step further though and be sure to open source the server, as well as the client software!

]]>
http://www.grantmidwinter.com/2008/04/02/google-browser-sync-for-firefox-3/feed/
Scary looking walking dog thing http://www.grantmidwinter.com/2008/03/18/scary-ass-walking-dog-thing/ http://www.grantmidwinter.com/2008/03/18/scary-ass-walking-dog-thing/#comments Tue, 18 Mar 2008 16:29:23 +0000 Grant http://www.grantmidwinter.com/2008/03/18/scary-ass-walking-dog-thing/ Not sure how this relates to the development of the web or our business but man is Inspiring and terrifying in equal measure. Yo Phil can we build one of these please?

]]>
http://www.grantmidwinter.com/2008/03/18/scary-ass-walking-dog-thing/feed/
Search Technology Is A Precursor To Artificial Intelligence http://www.grantmidwinter.com/2008/02/09/search-technology-is-a-precursor-to-artificial-intelligence/ http://www.grantmidwinter.com/2008/02/09/search-technology-is-a-precursor-to-artificial-intelligence/#comments Sat, 09 Feb 2008 11:40:51 +0000 Midwinter http://www.grantmidwinter.com/2008/02/09/search-technology-is-a-precursor-to-artificial-intelligence/ Many of you who read our content reguarly, especially those of you who subscribed to Surrch.eu; know that Grant Midwinter has their own search technology. Our search crawlers and engines are highly adapatable and a lot of our work focuses on taking search functionality and adapating it for a useful purpose other than the straight Google or Yahoo style user search engine.

Google has the right idea, I believe, when they profess a goal of indexing all the world’s information. However, they seemed to have missed the part where they utilise the world’s information for a higher purpose. As the old adage goes, “it’s not what you have, it’s what you do with it that counts”. The main use of having large quantities of automatically gathered data is of course for pattern analysis. This may sound pretty boring until you realise that the kind of patterns we’re talking about are those that the human brain is able to spot so easily and we take for granted in the course of our daily lives. An interesting pattern that the human brain is able to observe (some better than others) is the social factors that make someone popular, or well liked. This is why search crawling, and analysis, is so important a step to generating a true Artificial Intelligence.

I theorise, that anything associated with humanity, is actually down to the advanced pattern recognition available within our brains. Even learning language is a repeated exposure to stimuli until we understand the associative context of a word such as ‘chair’ to its inanimate object - or for a more difficult scenario; the word love to the associated chemical batter our brains and bodies are flooded with in fine balance.

It is to this end that Grant Midwinter is now in the business of creating more advanced applications based on search and tending towards AI. Our first publically available sample will be an anti-piracy tool unlike any existing application on the market. Let me give you a bit of background here:

With the new trend of uploading copyrighted videos of tv shows and movies to open websites, such as Stage6.com and Veoh.com, a new breed of internet forum has emerged. These forums have lurkers just like any other but they also have ‘link hunters’. The link hunters find links to the copyrighted videos and post them on the forum for others to view. So technically they are breaking no laws, and the onus is on the video hosting site to remove the infringing content.

Link hunters have the advantage though, they have an army of human brains who are good at spotting patterns, knowing what is and isn’t popular content. They’re also pretty darn good at knowing what is infringing on copyright and how to post links to obfuscate their true locations from simple machines. So we’re going to use our semantic inference engine and crawler technology, adapt it, so that it will allow you to list all the links currently online of infringing content on the major video sites. The MPAA, RIAA and other such entities can then use it to post their DMCA (Digital Millenium Copyright Act) takedown notices and have the infringing content removed. Of course they’ll have to automate the process if they want to take down the material more quickly than it can be put up by the legions of link posters out there. It’s unlikely we’ll provide an API for this because it’s a test subject for us of the capability of our unique technology.

If you’re interested in how our engines work, or any of our thoughts on search and AI then as always just ask and we’ll do our best to give you answers. Look for the Link Hunter Articifical Intelligence on release day [UPDATE: Release delayed, more soon]

]]>
http://www.grantmidwinter.com/2008/02/09/search-technology-is-a-precursor-to-artificial-intelligence/feed/
So… valentines day coming up http://www.grantmidwinter.com/2008/02/05/so-valentines-day-coming-up/ http://www.grantmidwinter.com/2008/02/05/so-valentines-day-coming-up/#comments Tue, 05 Feb 2008 14:49:31 +0000 Grant http://www.grantmidwinter.com/2008/02/05/so-valentines-day-coming-up/ samurai armour

Something special for the love of your life (honey I hope your reading this!) how about 18th century samurai armour. Maharishi are selling this bad boy for 17k, I admit thats not chump change but imagine the smile on your lovers face.

]]>
http://www.grantmidwinter.com/2008/02/05/so-valentines-day-coming-up/feed/
Why are dead formats so damn sexy? http://www.grantmidwinter.com/2008/01/31/why-are-dead-formats-so-sexy/ http://www.grantmidwinter.com/2008/01/31/why-are-dead-formats-so-sexy/#comments Thu, 31 Jan 2008 17:28:54 +0000 Grant http://www.grantmidwinter.com/2008/01/31/why-are-dead-formats-so-sexy/

Cassette tape

Here is a great site for the design heads out there. The cassette tape is constant source of inspiration.

]]>
http://www.grantmidwinter.com/2008/01/31/why-are-dead-formats-so-sexy/feed/
Welcome To Grant Midwinter http://www.grantmidwinter.com/2008/01/28/welcome-to-grant-midwinter/ http://www.grantmidwinter.com/2008/01/28/welcome-to-grant-midwinter/#comments Mon, 28 Jan 2008 11:06:39 +0000 Midwinter http://www.grantmidwinter.com/2008/01/28/welcome-to-grant-midwinter/ Firstly, welcome all of you to our new website, we’ve been working hard on it and we hope to give you a better picture of what we’re all about.

Derrick and I took the decision to use a Wordpress based blog as the core of our website for many reasons:

  • We felt it was important to show our clients the skills we possess in working with bloggers and social media - and what better way to be do this than to use Wordpress.
  • It allows us to easily takes beautiful designs (courtesy of Derrick) and allows you to turn them into CSS and HTML that passes W3C validation without a hitch.
  • Derrick and I are extremely keen on not only the quality of our own work but also raising the bar across the industry and showcasing brilliance from all of the excellent companies that we work with to achieve some superb campaigns and sites for our clients.
  • We want to be transparent, so we’ll be blogging reguarly in text and with video to demonstrate some of our core skills in design, SEO and online marketing.

There’s already a wealth of search and SEO content on the site which has come from the old Surrch blog, those domains now redirect here as we’re bringing all of our skills into this one site.

Let me know if you run into any problems - but I hope you enjoy our content and find browsing around the site to be a great user experience.

]]>
http://www.grantmidwinter.com/2008/01/28/welcome-to-grant-midwinter/feed/
SEO Book Ready For Free Download http://www.grantmidwinter.com/2008/01/07/seo-book-ready-for-free-download/ http://www.grantmidwinter.com/2008/01/07/seo-book-ready-for-free-download/#comments Mon, 07 Jan 2008 11:34:48 +0000 Midwinter http://www.grantmidwinter.com/2008/01/07/seo-book-ready-for-free-download/ I’ve uploaded my new book to the interwebs so you can now download and read through it, if the fancy takes you, completely free of charge (SEO Truth - A Bible For The Next Generation Of Search Engine Optimisation).

There may be a few blank pages because it’s been laid out for print; you can get yourself one of these hard copies from this website here.

Let me know what you think and submit some feedback over on lulu by all means! Cheers.

Edit: The download link works correctly now, oops.

]]>
http://www.grantmidwinter.com/2008/01/07/seo-book-ready-for-free-download/feed/
SEO Truth - Apparently I’ve Written A Book On SEO http://www.grantmidwinter.com/2007/12/05/seo-truth-apparently-ive-written-a-book-on-seo/ http://www.grantmidwinter.com/2007/12/05/seo-truth-apparently-ive-written-a-book-on-seo/#comments Wed, 05 Dec 2007 17:31:27 +0000 Midwinter http://www.grantmidwinter.com/2007/12/05/seo-truth-apparently-ive-written-a-book-on-seo/ Hello!

As usual I’ve been spending a horrendously long time without writing anything on my blog - and for that I apologise. However, I have spent some of my time writing an SEO (Search Engine Optimisation) handbook, covering the importance of next generation techniques and practises.

I’m sure there are those of you who are all too familiar with the increasingly backwards approaches used by a few ’special’ SEO agents and individuals out there and perhaps for you this will merely reinforce what you already knew to be true. For those of you who don’t know what I’m talking about -then please read the book and have a good laugh at yourself for being such a silly.

You can order print copies of the book - just not yet… more details on that coming soonly!  I’ll be publishing online chapter by chapter (honestly I have finished writing it, but as an SEO, if I didn’t serialise it then it would look bad).

Enjoy the read and let me know what you think, if the first edition is terrible and you order, of course it’s going to be valuable in 200 years!

Preface

First off, I’d like to introduce myself. I’m a Search Engineer, a developer and programmer. I’ve worked with clients throughout the advertising industry at many different companies. My specialty is developing software that works with the search engines of companies like Google, Yahoo and MSN and attempts to influence the rankings of my client’s websites, as well as report on those ranking changes. I’ve never been to a lecture on computer science, read a book on development methodology and yet I’m in demand. My skills lie in understanding the technology of a search engine and how to capitalise on their ranking algorithms, web crawlers and content filters and it’s the ideas I generate in this area which have kept me in gainful employment.

SEO (Search Engine Optimisation) used to be a fairly simple task where you’d make sure every page on your client’s site had Meta tags, descriptions and content unique to that page. You might then try to analyse the keyword density of your key terms to keep them somewhere between 4 and 7 percent. More often than not most SEO companies wouldn’t even attempt that.

What most SEO companies would never tell you, and this is the industry’s most well kept secret, is that they’re intrinsically lazy. If you had a good client, with good content and a product of interest then their SERs (Search Engine Rankings) would climb entirely naturally to the top spots, you’d have nothing to do but sit back and reap the benefits of your lack of work.

This is of course a sad state of affairs which no real SEO company would allow and part of this book will help you to spot the difference between a professional outfit and rank amateurs and define the widening gap between the two camps.

As the title suggests I’m writing about the next generation of SEO. It’s becoming more difficult to increase the rankings of a particular website and it will only get more difficult to manipulate a website’s ranking without any understanding of how new search engine technology works. Lucky for you, my field is semantics (how to correlate the relationship between one word and another essentially) and you’re in for a whole chapter in manipulating a semantic index similar to those increasingly used by the major search engine players.

 

 

Chapter 1 - The Past

In order to proceed correctly in the future, the most important lesson is for us to understand what happened historically. There’s no shortage of information on the internet and amongst SEOs and webmasters about how Google’s original PageRank system worked. This is in large part thanks to a paper written by Google’s founders, Larry Page and Sergey Brin, whilst they were still studying for their PhDs at Stanford University. Not long after that they received their first investment from a company called Sun Microsystems which enabled them to build upon the hardware they had in their university dorm room and create the international phenomenon we know today.

PageRank was essentially a very simple system. It counted each link from one site to another as a vote for the destination site. By voting for another site the original gave away some of its own PageRank. The idea came from Salton’s Vector Space Model, which is a mathematical principal known to most Computer Science graduates today. This simple method of calculating which websites had the most votes, and therefore deserved higher rankings, is key to all search engine algorithms as it’s extremely fast to calculate. The most important factor in any search engine is its speed in returning and ranking results, especially when you’re dealing with an index of billions of pages.

 Anatomy of a search engine


The Anatomy of a Search Engine, based on the work of Larry Page and Sergey Brin whilst at Stanford.

If you understand that all calculations undertaken by a search engine must be as fast as possible, it allows you to draw logical conclusions:

·       Thinking about a page as a machine would (which struggles to actually understand rather than just read), rather than as a human, is key to analysing your websites content for SEO value.

·       Is every single underlined heading, keyword color, font size, image location, keyword relationship and page title length analysed when a page is crawled? It’s highly doubtful that anything too in depth is going to be indexed, when the crawler has another hundred thousand pages to visit and rank as quickly as possible, use some common sense here. Of course as processor speeds and bandwidth increase more in depth analysis will become possible in a shorter space of time.

·       The search engine needs to maximise two things: the speed of its calculations and its measure of quality relevancy. Occasionally one is going to suffer at the importance of the other, if you were going to choose between indexing a page poorly - or not at all - which would you do?

SEOs in the past were able to capitalise on this speed issue by choosing to concentrate on areas of a page such as the Meta tags, description and page title. The content itself gradually became more important as time went on but still was subject to the speed of indexing. SEOs quickly realised that keyword density (how many times a keyword appears on a page out of the total number of words) was a very quick way to determine some kind of relevancy, and that the search engines were using it too.

Once the search engines got wise they implemented filters that stopped SEOs from flooding a page with keywords. Arguments in the SEO community followed over exactly what was the ideal keyword density for a term, and this usually settled somewhere between 4 and 7 percent.

Of course the PageRank model meant that agencies were keen to build as many links to their client websites as possible. To make matters worse however they were after links that already had high PageRank values to gain the maximum ranking as quickly as possible and this sprang up a cottage industry of people generating high PageRank links, purely to sell on. Google of course were unhappy about this and their anti-spam team began its work. Blacklisting of websites which ‘farmed links’ was becoming fairly common and this moved on to other aspects of ‘black hat’ SEO behavior - where an unfair advantage was being made by some nefarious companies and individuals.

Most SEO agencies at this stage relied heavily on staff who’d be subjected to some extremely tedious and repetitive labour. Going through page after page of a website and adjusting the number of keywords on a page, slightly changing each page title and Meta tag was a boring job and not well paid.

Directors and CEOs didn’t have a whole stack of problems though, if they kept building up link relationships with ranking websites and making sure their Meta tags were in place, their job was done. Often enough they’d have clients who already had an interesting product which did most of the work itself, spreading links around the internet as people registered their interests.

This natural traffic increase was what Google was looking for as they wanted sites which progressed on their own merits rather than trying to beat the system.

 

]]>
http://www.grantmidwinter.com/2007/12/05/seo-truth-apparently-ive-written-a-book-on-seo/feed/
Google Shows How Much You Are Worth To Them http://www.grantmidwinter.com/2007/10/31/google-shows-how-much-youre-worth-to-them/ http://www.grantmidwinter.com/2007/10/31/google-shows-how-much-youre-worth-to-them/#comments Wed, 31 Oct 2007 17:11:01 +0000 Midwinter http://www.grantmidwinter.com/2007/10/31/google-shows-how-much-youre-worth-to-them/ I’ve said many times before that so-called SEOs out there need to stop believing every word that spills from Google’s overactive pen. Google is a business just like any other and they feed information that’s deliberately misleading to stop people from gaining an unfair advantage with their search rankings.

It now appears that, in fact, Google assigns an estimated worth to each ranking on their pages - visible to members of their AdWords sales team they use the information from your PPC campaigns and analytics package in order to figure out whether you’re worth it.

Google’s GG Score Shows How Much You’re Worth To Them In The Search Rankings

I’m sure many of you will draw your own conclusions from this and in time we may see a Google press release, from that department which again knows as much about how their technology actually works as most of the SEOs do. Take it with a pinch of salt is my advice and invest the time to understand how a search engine really works.

This story was broken on the french blog Zorgloob, much credit to them for a brilliant find.

]]>
http://www.grantmidwinter.com/2007/10/31/google-shows-how-much-youre-worth-to-them/feed/
What Is A Search Engine? You Have No Idea Apparently http://www.grantmidwinter.com/2007/07/31/what-is-a-search-engine-you-have-no-idea-apparently/ http://www.grantmidwinter.com/2007/07/31/what-is-a-search-engine-you-have-no-idea-apparently/#comments Tue, 31 Jul 2007 09:15:20 +0000 Midwinter http://www.grantmidwinter.com/2007/07/31/what-is-a-search-engine-you-have-no-idea-apparently/ One of my favorite blogs, that I read just about every day is readwriteweb, a sterling tech, web 2.0 and search blog. Not so long ago their AltSearchEngines regular article was turned into a fully fledged blog in its own right headed by Charles Knight who knows about the existence of more search engines than probably anybody else on the net.

I checked it out this morning and spotted an interesting article:

Today we launch Part I of our 3 Part Series

Part I: What is a Search Engine? by Nitin Karandikar (Mon)

Oh glominy! I thought, glibbily. This is right up my street so I settled in for a powerful, thought provoking read.

Alas, the writer was a complete nitwit and I felt compelled to post this raging comment:

You’re completely wrong, I don’t know why on earth you’d try to reclassify what a search engine is when we’ve known what search engines are for a long time.

A search engine is simply “an information retrieval system designed to help find information stored on a computer system” (Wikipedia).

1. It enhances findability of relevant web content for the user

It doesn’t need to have anything to do with the web. Findability is not a word, even in italics.

2. It searches the entire web or a large subset thereof
(this excludes publisher search engines that search only a single site or group of sites)

No search engine searches the entire web. Don’t listen to the Google PR machine so much, and again, it doesn’t need to touch the web to be a search engine. Plus you’re on AltSearchEngines here… how many verticals do you guys cover?

3. Searches are specified using a keyword, phrase or question, or using input parameters, without the need for undue navigation
(I don’t consider pure directories like dmoz to be Search Engines)

So you’re saying you need an input to get an output? That’s genius.

4. It provides search results on demand, not periodically

I don’t even know what the hell you’re trying to say this for. It’s still wrong. Why does it have to do as a person asks it?

5. It provides some kind of unique or special processing of its own: either in the search algorithm, or in UI improvements, or both
(this excludes pure Rollyo or Google Coop-based search engine subsets)

This is far and away the worst thing you’ve written, you’re clearly grasping at straws. That is until you said:

The criteria described above will not remain static; as technology progresses, Search Engines will need to support increasing levels of functionality to be taken seriously.

No, i’m afraid a search engine, will always be a search engine. No matter how technology progresses it will still be a search engine.

The article you should have written is, “What search engines should have on my holidays”.

Yakov: A search engine doesn’t need to have its own index of the web or build it. A crawler of some description is responsible for building an index - that can take many forms and is often included in the search engine software itself. If you want examples of search engines without their own index, then take a look at the recent Digg API contest for some examples.

I’m hoping Charles gives you a massive kick up the backside and stops you writing what essentially is a load of bollocks.

Yes, it was a little scathing, but I get extremely irate when I see article written by someone who clearly is just trying to write for the sake of saying something. Especially on a source I have a lot of respect for because I don’t want to see them letting it through to the front page, that’s their role as editors - to weed out the rubbish and go with the quality content right?

]]>
http://www.grantmidwinter.com/2007/07/31/what-is-a-search-engine-you-have-no-idea-apparently/feed/
I Bought A New Wig Today http://www.grantmidwinter.com/2007/07/24/i-bought-a-new-wig-today/ http://www.grantmidwinter.com/2007/07/24/i-bought-a-new-wig-today/#comments Tue, 24 Jul 2007 10:51:35 +0000 Midwinter http://www.grantmidwinter.com/2007/07/24/i-bought-a-new-wig-today/ Is the name of my new blog on the slightly odd spam that Akismet catches for me.

We need to relate to the spammers in order to understand their needs, and you can do so right here.

]]>
http://www.grantmidwinter.com/2007/07/24/i-bought-a-new-wig-today/feed/
Sunbeam Is Your Search Engine http://www.grantmidwinter.com/2007/07/23/configuring-sunbeam-previously-known-as-allegro-the-first-user-search-engine/ http://www.grantmidwinter.com/2007/07/23/configuring-sunbeam-previously-known-as-allegro-the-first-user-search-engine/#comments Mon, 23 Jul 2007 16:37:02 +0000 Midwinter http://www.grantmidwinter.com/2007/07/23/configuring-sunbeam-previously-known-as-allegro-the-first-user-search-engine/ Sunbeam The First User Search EngineIn previous versions (for those of you lucky enough to see the Alpha of the world’s first search engine to run directly from the user’s own desktop) Sunbeam would ask you to input your favorite websites as a starting point for its indexing routines. This was a problem for two reasons:

  1. Nobody ever wants to enter anything they don’t have to, especially when that information exists somewhere on their machine.
  2. It limited the ‘profile’ of the user initially available to Sunbeam and how quickly they’d be able to retrieve information actually relevant to them.

It also meant that the semantic engine that appeared in the earliest release was not capable of returning accurate matches for a period whilst the engine cranked up and had indexed at least a few hundred pages.

I’d been musing over these problems for a while, I wanted an experience where the user would be able to just install the program, let it do its work without going through any configuration screens, which they may not understand or that might put them off the install completely.

The solution as it turned out, was fairly simple. Using the browsing history of the user we can track down the urls that are visited most frequently and most recently without damaging privacy. After all these are just starting points to build a profile of interests. Data like this is a goldmine for Sunbeams advanced statistical algorithms and will enable it to deliver the results that mimic the language used in the websites in your browsing history.

It doesn’t stop there though, also added are routines that scan your outlook sent messages, tracking the semantics of your own typed words. These again, are not stored as complete messages anywhere in the system, are not tied to email addresses or even subject lines and privacy here is key. What is most important here is that you as a user will never have to go through a slew of irritating questions when you install Sunbeam, that inadequately attempt to locate and disect your interests.

Seeing as I expect privacy to be such an issue here, let’s turn to another reason to use Sunbeam over Google or Yahoo:

  • Your searches are your own.
  • Your data will never be sent anywhere else (there isn’t the server space for it!).
  • If you choose to share your search database with anyone else (as easy as emailing the one file), then that’s completely up to you and not something you have to ‘opt-in’ to.

This software is entirely your own to play with, these are the things I’m really loving about it:

  • You can play with the open source search algorithm.
  • You can swap, share and amalgamate databases with friends or download one from the web.
  • There are no adverts, no pop ups and no interruptions.
  • If you don’t remember the exact word you’re looking for, just put in a similar one, or a descriptive phrase.
  • If you want to use the same database when you get home, just mail it to yourself.
  • If you don’t like the results you’re getting, run a seperate database for work and for home to match your corporate and downtime moods.
  • If you have to do market research on teenagers, just use the database your nephew compiled.

]]>
http://www.grantmidwinter.com/2007/07/23/configuring-sunbeam-previously-known-as-allegro-the-first-user-search-engine/feed/
Microsoft Are Back On Top With Surface http://www.grantmidwinter.com/2007/05/30/microsoft-are-back-on-top-with-surface/ http://www.grantmidwinter.com/2007/05/30/microsoft-are-back-on-top-with-surface/#comments Wed, 30 May 2007 08:26:05 +0000 Midwinter http://www.grantmidwinter.com/2007/05/30/microsoft-are-back-on-top-with-surface/ I have just watched a video of the most exciting user interface ever seen. It’s not of the forthcoming iPhone nor is it any kind of Apple product. This is Microsoft Surface and it promises a revolution in how we interact with our computers and mobile devices, I’m completely blown away by not the technology behind the system, but how well it’s used to produce a product that will potentially devestate Apple’s market share.

Microsoft Surface Puts The Company Firmly Back On Top Of Apple

If you wondered why Bill Gates was suddenly agreeing to do an interview with Steve Jobs, then I’m pretty sure this is the reason. It doesn’t matter if he does badly in that discussion because as soon as Surface was on show then Steve Jobs had lost out anyway. Will Jobs have a rebuttal product that we haven’t heard about? I doubt it.

Pricing And Availability

You’ll be able to get Surface from winter 2007 for between $5000 and $10000. I know that’s a lot of money right now but they aim to bring the price down to a consumer level quickly and this is the first device I’ve seen that really will fit right in your living room, instead of just attempting to hide in a corner. Designer coffee tables go for far more and I know which I’d rather have.

The New Standard In Interaction

For me, as a search and user interface developer, this fits in extremely nicely with my view of tiling search results as images. An application using Windows Live Search in this way for not just searching but RSS feeds and bookmarks would be highly intuitive and allow the user to see what they want straight off the mark.

Surface Revolutionises Connections To Mobile Devices

One of the most ingenious features they’ve integrated right off the mark is the ability to interact with your mobile devices. We all have phones now; they started with IR then Bluetooth, now some feature WiFi. How many of you actually use these connection abilities reguarly though? I’d guess it’s a low percentage because the hardware and software we have to connect with doesn’t make it simple and easy enough to use frequently in most cases.

What Surface lets you do is put your mobile phone, PDA or digital camera directly on the table top and a ring will appear around it to signify the connection. You can then drag media to and from the device with your finger and a bit of wrist movement, it’s so simple it makes me want to cry. I spend a lot of time shouting about the need for simple and intuitive user interfaces and this is the model we should all start building from.

This is the new standard in user interfaces, keep up.

]]>
http://www.grantmidwinter.com/2007/05/30/microsoft-are-back-on-top-with-surface/feed/
I Can Has Likkle Written Contentz? http://www.grantmidwinter.com/2007/05/29/i-can-has-likkle-written-contentz/ http://www.grantmidwinter.com/2007/05/29/i-can-has-likkle-written-contentz/#comments Tue, 29 May 2007 16:20:59 +0000 Midwinter http://www.grantmidwinter.com/2007/05/29/i-can-has-likkle-written-contentz/ Hi Readers!

The internet is an odd place, as I look at wordpress.com right now I see the top few blogs are I CAN HAS CHEEZBURGER?, passive-aggressive notes from roommates, neighbors, coworkers and strangers and of course Scobleizer.

F or those of you yet to witness the phenomenen of icanhascheezburger then let me summarise for you by saying it’s a blog filled with cute/demonic pictures of animals, mostly feline in nature with captions underneath. The passive-aggressive notes blog is exactly as it says in the title; pictures of amusing passive-aggressive notes.

As a further exercise in demonstrating to you the power of this medium let me give you an example of an icanhascheezburger image (taken of my girlfriend’s cat, yesterday):

f**k bucket, has pub

If you haven’t been to the site, you won’t understand most likely. The ‘bucket’ is an in joke as these websites often produce. Why exactly though is it so popular over the thousands of blogs that produce well written, quality content?

It’s fast

There are many facets to the speed here, firstly it’s very quick for the authors to add a new post. All they need to do is get an image, put it in the wordpress editor, add a couple of lines about the submitter and possibly the humourous content if they can be bothered and they’re done. This means they can generate hundreds of posts in the time it takes the rest of us to put out one or two (sorry wasn’t talking about you Scoble, or you Winer). The other quick thing they can do when they add a wordpress post is to select categories, this is a very fast way of tagging essentially and means as well as quickly refreshed content they also have targeted keywords. Hello good SEO.

It’s also fast for users; if you don’t get the joke in the first pic you see, it’s a 1 click scroll to the next one. You laugh, it’s funny, you whack the link on an email and send it round the office. They even have a lolcats generator that lets you put a caption on your picture of a cat in about 20 seconds AND automatically submit it to the site. Auto generated content essentially, which is just gold.

If the site updates less often the search engines aren’t the only things that return less frequently. The same applies to all your human users as well. They’re far more likely to refresh if they think the content updates often, and even more if they think their cat might appear on the next post.

What next?

I think very soon, you’ll see an abundance of these kinds of websites arriving if people are smart (often they’re not).

All kinds of non text media will benefit from this treatment and a social voting style system for it will allow a much faster turnaround on content. You’ve seen it with Digg and this is one of the reasons they really should add an images section they’re losing out hugely there.

Other websites have also shown the advantage of fast content generation from any source. Twitter allowing updates by mobile phone for example. I can upload pictures to blogspot from my k800i directly, it’s a shame I don’t like the blogging software.

Urrr.

I completely lost my train of thought I went and read some c# documentation and then all my post ideas ran away. I may finish this later when I regain my mind.

]]>
http://www.grantmidwinter.com/2007/05/29/i-can-has-likkle-written-contentz/feed/
Fear Of Google: As Seen On Google Timeline! http://www.grantmidwinter.com/2007/05/25/fear-of-google-as-seen-on-google-timeline/ http://www.grantmidwinter.com/2007/05/25/fear-of-google-as-seen-on-google-timeline/#comments Fri, 25 May 2007 09:30:37 +0000 Midwinter http://www.grantmidwinter.com/2007/05/25/fear-of-google-as-seen-on-google-timeline/ UPDATE:

It would appear Valleywag’s Nick Denton is lacking a sense of irony and unfortunately I seem to have my commenting privileges revoked there now. Shame. He’s thoughtfully left this little nugget seemingly ending the argument with a resounding slap to my pride:

“Hey, Phil, I don’t mind being slagged off. Comes with the job. But you didn’t do it very effectively. One could make the point that mentions of Google itself have become more frequent. But sensationalism? I don’t think you proved your point”

What’s that Nick you can’t hear my answer from all the way over there because you blocked my account? Never mind. Sensationalist articles Nick, seeing as you are unaware, are those that are published without any proof behind them. So I put together my own sensationalist article on your sensationalist article and it appears you lack a sense of humour. Fortunately you’re unable to prove to me you have one because that’d mean you wrote something of substance. Unlike you Nick I won’t delete or remove negative comments even though I rate my blog above a tabloid so feel free to hurl insults from below if you wish.

THE ORIGINAL ARTICLE:

I saw over on Valleywag they’ve written yet another hack piece on the so-called Fear Of Google with the standard sensationalism and lack of humour. They’ve even drawn a pretty graph they collated data on from the Nexis newspaper database showing their spectacular lack of knowledge on current Google events.

Being a bit of a dry and sarcastic git I present to you Fear Of Google: As Seen On Google Timeline! which is a representation of how Google itself sees the phenomenon.

As Seen On Google Timeline!

Don’t bother reading Valleywag’s article, go and read what Scoble says instead if I was you.

Personally I have no fear of Google (though I am typing this in the stationary cupboard but that’s because of my love of pens) and instead feel an increasing need to criticise them rather than run in fear. Then again, people react in the same way with governments and it’s surprising that a company can approach that level.

]]>
http://www.grantmidwinter.com/2007/05/25/fear-of-google-as-seen-on-google-timeline/feed/
I Think I Just Invented The Real Search 2.0 http://www.grantmidwinter.com/2007/05/24/i-think-i-just-invented-the-real-search-20/ http://www.grantmidwinter.com/2007/05/24/i-think-i-just-invented-the-real-search-20/#comments Thu, 24 May 2007 13:12:24 +0000 Midwinter http://www.grantmidwinter.com/2007/05/24/i-think-i-just-invented-the-real-search-20/ Ignore the numbers in the title for the moment if you will and focus on these keywords: social, networking, search, community.

Web 2.0, by many definitions is all about allowing users to network, interact and the read/write web. Search 2.0 in that context does not yet exist. There are in some instances communities that happen to be built around a search engine such as Yahoo and there are new semantic search engines that let the users tag pages and documents to be found (something I’ve talked about before and pointed out as next to useless). None of these let the users actually interact with which results are returned. There is no networking or interaction that takes place with the search engine itself and this is just plain wrong.

Do you know how many people are on the internet at any one time? I sure as hell don’t but it’s a big number :)
Who creates all the content that ends up on the internet anyway? It’s not machines, it’s people, human beings are ultimately responsible for all the content on the internet and that’s never going to change. So why are we asking machines about content created by fellow humans when most of the humans are online anyway and know far more about their subject, and where it’s covered on the internet than any machine is ever likely to?

Enough rhetorical questions, I’m going to tell you what the real Search 2.0 is and you’re going to shout at me and tell me I should have patented and I’m a fool. However, if you don’t hire me to build it for you then you’re a fool because I get these ideas on a daily basis and I will crush you at some point in my life. Just kidding, I’m a fan of open ideas as well as open source especially when they’re for the benefit of us all.

Search 2.0

  • An instant messenger application or website with a live AJAX interface forms the centerpeice of the front end.
  • Users create accounts and select their areas of interest by entering specific key phrases for those topics they feel most knowledgeable about.
  • Users can then also select web pages that match those highly specific key phrases if they choose to.
  • The search box appears as normal, you enter your query and the fun part of search 2.0 begins.
  • Your query is analysed against users on the system, what occurs at this stage is actually a search for users with the best matching key areas against your query.
  • If these users are online they can respond directly to your query, either suggesting a web link or entering a chat with you.
  • If no users are matched online, then the suggested web pages are searched for the best matching content.

That’s the basis of it, but let’s have a look at the immense social power here.

Firstly, you get to rate the responses you receive, meaning that people can gain a reputation score for specific subjects and topics giving them an online credibility for that topic.

Sorry we just had an office fly-hunting session. Don’t ask.

Right, where was I? This system is by its nature, very low spam, it can’t be manipulated to provide results that are less useful because if you try and peddle a corporate product that’s crap, your reputation will drop very quickly and you’ll be banned. If the product is good on the other hand then who’s going to mind being directed to it if it answers their specific need and that’s better advertising than any money’s going to get you.

This concept is all about the users, no massively complicated algorithms need writing here it’s just using the very advanced and articulate knowledge of the very people who create the content you’re looking for, and to get the best answer you’ll ever get from a search engine is it not worth answering a couple of questions every now and again about the subjects you enjoy?

You can also bookmark people just like in any other IM and make friends with people holding the same interests, who you’d never meet on any other social network, and certainly would never think to find from a search engine.

]]>
http://www.grantmidwinter.com/2007/05/24/i-think-i-just-invented-the-real-search-20/feed/
Digg On Your Desktop http://www.grantmidwinter.com/2007/05/23/digg-on-your-desktop/ http://www.grantmidwinter.com/2007/05/23/digg-on-your-desktop/#comments Wed, 23 May 2007 10:45:11 +0000 Midwinter http://www.grantmidwinter.com/2007/05/23/digg-on-your-desktop/ [digg=http://www.digg.com/software/Digg_For_Your_Desktop]I’ve previously posted a video of the new search interface I’ve been experimenting with. In order to get some feedback on how it works I’ve put together a small application that uses the Digg API to display the popular news stories on your desktop background.

The application will update for the latest stories every two minutes or so and refresh the tiles accordingly. If you mouseover a tile the window will ‘fisheye’ slightly, similar to the OSX dock. Click on the tile to expand it to a readable size and then if you decide you want to go to the story then double click the expanded image to open it in your web browser.

You can close the application by right clicking the little rss icon in your system tray and hitting Exit, or if you encounter problems then close the process newsview.exe in your task manager.

You will need the .NET2 framework to run this and the installer should point you in the right direction if you don’t have it. Failing that then go here to download it manually.

I stress this is a little alpha level experiment, but if you do encounter problems then I’m only too happy to help, just leave me a comment here and I’ll try and fix any bugs you may find. What I’m really looking for though is some feedback on the interface: Is it simple enough? Can you see the result clearly? Would you use a search engine that delivered results in this manner?

To download ‘DiggTop’ click here and enjoy.

[dailymotion id=5JuelL8YvPLQgeuYf]

]]>
http://www.grantmidwinter.com/2007/05/23/digg-on-your-desktop/feed/