Adam Singer, The Future Buzz | Jan. 31, 2011, 12:30 PM

Maybe you’ve seen the recent back and forth discussions about Google’s search quality from the A-list. It has created enough buzz Google even responded. Perhaps you are wondering why everyone seems so upset?

Or maybe you have no clue what I’m talking about and have been enjoying a high quality search experience, ignoring the early adopter pontification.

Either way, the truth is most people think search engines do a good job.

A survey reported by Search Engine Land shows this nicely – 89% of users are satisfied with their search results (and a big 0% say poor):

So early adopters aside, the general population thinks search engines do pretty well. I would tend to agree even if there are some edge cases of spam leaking through. I find the results of Google to be very high quality – and in cases I can’t find what I want with a regular search, I’ve got the important operatorsmemorized anyway. That’s why it’s funny to me when tech savvy people complain about lack of quality in Google. If you know how to search you can definitely find what you need.

With that in mind, I wanted to weigh in my opinion about some of the conversations about search going on around the web lately:

Search is not a simple problem to solve

An ever-expanding mix of content, sites and services sprout up daily. All trying to compete with the existing ones. Concurrently, all of them are working to develop content and create signal to their sites that will help show quality ahead of others. All of this while existing sites continue to market themselves to improve both search and social performance. Search engines have been working hard to keep up and maintain relevancy and quality despite the changing landscape of the web. It’s not an easy problem to solve and if you consider how good most of the results are even without using advanced queries I’d say the major engines all do well.

If you go looking for imperfection of course you’re going to find it

Web search is not a problem that is 100% perfectly solved. It may never be. That’s because the one constant of the web is change. In content, in relevant quality signals and how we use it. And that’s OK – dealing with indexing near-infinite amounts of content you will always be able to find a search which isn’t perfect if you look hard enough. Thus is the nature of searching for information. I think a lot of the complaining lately is from users who expect everything to be perfect. We’re trending to a web that’s getting even better at predicating what we want however that’s just not a reality yet. But it’s so good considering how nascent the technology is in the bigger picture that we should all be amazed. Literally frozen in amazement. I think aside from those complaining because they feel slighted by Google, it is a case ofeverything is amazing and nobody’s happy.

Complaints about monopoly are absurd

Our government does some pretty obnoxious things such as allowing businesses to influence them with monetary contributions, rallying against the wrong things to “protect the children” simply to generate public approval (but not actually accomplish anything) and other items which are pretty frustrating, but sadly not out of the norm. So the fact that Google now has to go to Washington to lobby against search engine regulation isn’t a surprise, but is equally as ridiculous. Google is not a monopoly. It’s popular because it’s good. Don’t like it? Use Bing. Or Blekko. Or Duck Duck Go. Or Yahoo. The point is Google is most certainly not the only web search option and it is consumer preference to use them.

Search is still a disruptive business

Web search is hardly new, but it is still a very disruptive business. And so naturally a lot of people don’t like Google. Some media entities see search as a double edge sword – it sends them traffic, but they think Google “takes” advertising revenue from them. I use “take” in quotes because while some media see it as taking, I take a different perspective: that media does not need to be saved from search. They need to evolve how they monetize and build community. The opportunity is there and pivoting their approach could actually make Google their largest ally instead of something to be feared. And yet it’s easier to cling to the past and fight progress. Hard to feel sympathy for anyone who is being disrupted by technology though because that’s sort of how innovation works. Evolve or don’t, but to rally against it makes you look like you can’t compete.

Social does not kill search – they work together

While some people like to think social is “killing SEO” they are being shortsighted: search is a core function of the web and is not going away. My main focus at LEWIS PR is social, not search. But I view the web holistically and for clients we ensure that anything happening in social is helping their search visibility. Social media pros who don’t also understand search need to learn SEO as they are falling short of their craft. Besides, putting social before search is like putting the cart before the horse. The order here matters. Think about it – even if you succeed wildly in social if your main site isn’t optimized you just built a whole lot of signal that didn’t even help you rank or gain traffic. Your competitors will still be found ahead of you by those with an immediate need for their product or service. Social builds community and branding, but organic search produces the cleanest, most relevant traffic.

I think at the end of the day all the ranting about Google falls under one of the following buckets:

• Someone has a product that competes, and thus wants to try to position Google as evil or less relevant.
• An SEO is upset that Google is going a direction they don’t like or want.
• A blogger or reporter writes something negative or anti-Google because it’s good for pageviews and links (and ultimately more traffic from Google – talk about biting the hand that feeds them!).
• Media are upset Google is making ad revenue by indexing their content (yet sending them free traffic – instead of complaining they should take advantage of this to build community and ultimately more revenue).
• A tech geek found a SERP they didn’t like and wants to use it as an example the sky is falling (while a majority of people are finding their searches valuable).

Meanwhile, I remain a fan and supporter of Google – their products are strong, and much of the complaining is either undue or has ulterior motives behind it.

Submitted by John Davidson on Sun, 01/30/2011 – 07:27

In responseto the increased criticism about Google and its low-quality search results, the Internet search giant this week updated its search algorithm in an attempt to bring down the level of webspam in its search results. While traditional webspam comprises chiefly of pages that contained a horde ofkeywords and phrases without context – implying that they intended to “cheat” their way up to higher search ranks -, there are now complaints galore that newer forms of webspam are also emerging.

Defending Google and its reportedly increased spaminess, the company’s principal engineer and search quality lead Matt Cutts said in a blog post: “According to the evaluation metrics that we’ve refined over more than a decade, Google’s search quality is better than it has ever been in terms of relevance, freshness and comprehensiveness.”

Cutts further added that the level of English-language spam in Google’s results presently is less than half of its level five years back; and that the spam level in most of the other languages is notably lower than that in English. Nonetheless, acknowledging a “slight uptick” in spam of late, Cutts said that Google was “evaluating multiple changes that should help drive spam levels even lower.” Later, in another post on his personal blog, Cutts announced that one particular change that affects sites which copy others’ content and sites that feature low levels of original content, has already gone live earlier in the week.

From: The Washington Post

http://www.washingtonpost.com/wp-dyn/content/article/2011/01/28/AR2011012807515.html

Washington Post Staff Writer
Sunday, January 30, 2011

Earlier this month, my friend Rebecca Skloot replaced her hulking big-box TV – I can vouch for its girth, having moved it once – with a flat-screen no thicker than an iPad. She turned it on and, horror of horrors, the picture was lousy.

Displeased, she turned to Google for help. What the search engine delivered was a mess, a collection of spammy sites riddled with ads. So she turned to Twitter, posting: “Old TV died, got newfangled LED TV. Shocked how bad/fake movies look! . . . Others have this prob?”

Solutions to Skloot’s technological melodrama rolled in. Fix this setting, turn this off, shazam! A few hours later, she posted: “Thx 4 fixing my TV today! It’s example of how Google=in trouble. Googled 4 fix, got spam sites. On Twitter answer=asap.”

Skloot’s story seems ever more common these days. Google is facing withering criticism from tech bloggers and search engine experts who say the world’s premier gateway to digital information is increasingly being gamed by spammers. Google, they say, is losing.

One tech blogger, the well-known iPhone app developer Marco Arment, wrote a post about “Google’s decreasingly useful, spam-filled web search.” Another blog offered a piece titled “On the increasing uselessness of Google.” Yet another headline spoke of “Trouble in the House of Google.”

Data seem to back them up. Google’s success rate, as measured by the percentage of users visiting a Web site after executing a search, fell 13 percent last year, according to Experian Hitwise, which monitors Web traffic. Microsoft’s Bing search engine increased its search efficiency by 9 percent over the same period.

Although there could be several reasons for the disparity, one is most certainly spam in Google’s results, analysts said.

“It’s clear that Google is losing some kind of war with the spammers,” said tech guru Tim O’Reilly, who often cheers Google’s technology. “I think Google has in some ways taken their eye off the ball, and I’d be worried about it if I were them.”

Google recently responded with its own blog post, acknowledging some problems and promising to fix them.

“Reading through some of these recent articles, you might ask whether our search quality has gotten worse,” the statement said. “The short answer is that according to the evaluation metrics that we’ve refined over more than a decade, Google’s search quality is better than it has ever been in terms of relevance, freshness and comprehensiveness. . . . However, we have seen a slight uptick of spam in recent months, and while we’ve already made progress, we have new efforts underway to continue to improve our search quality.”

Google’s predicament, analysts say, comes at a critical moment in the life of the Internet. The company generates billions of dollars in revenue from search ads. But social networks such as Twitter and Facebook offer people the ability to gather information online the way we always have offline – by asking people we know. Studies show we often give greater trust to information gathered from sources we know than from those we don’t.

The pool of people who could answer our questions online is growing fast. Internet users spend more than 20 percent of their time online using social networks. Last year, with more than 500 million users, Facebook topped Google as the world’s most visited Web site. (Washington Post Co. Chairman Donald Graham is on Facebook’s board of directors.)

While millions of people still use Google every day with no problems, I now see requests in my Twitter and Facebook feeds for things I once Googled: opinions on new cars, the best home repair person, the best place to eat, how to find a developer for iPhone apps. When I asked a friend on Facebook why she turned to her friends for new-car ratings, she replied: “facebook makes me smile b/c everyone has opinions about everything. i thought that, in this case, the opinions would actually be helpful. i can’t read big google search findings b/c i have no patience.”

While millions of people still use Google every day with no problems, I now see requests in my Twitter and Facebook feeds for things I once Googled: opinions on new cars, the best home repair person, the best place to eat, how to find a developer for iPhone apps. When I asked a friend on Facebook why she turned to her friends for new-car ratings, she replied: “facebook makes me smile b/c everyone has opinions about everything. i thought that, in this case, the opinions would actually be helpful. i can’t read big google search findings b/c i have no patience.”

A good side, a dark side

No doubt, there is a lot to wade through, with more than a million new spam Web pages created every hour, according to Blekko, a new search engine trying to take on Google. Optimizing what search engines find on all those pages is a big business – with a good side and a dark side. On the good side, analysts say, there are legitimate search engine experts such as Jill Whalen, founder of the Boston-area firm High Rankings, who works with big companies to make sure important information appears high up in search results. These search engine optimizers are masters of key words, site design and other techniques.

“It’s very frustrating for decent SEOs right now,” said Whalen, who recently wrote a blog post headlined, “Dear Google, Stop making me look like a fool!” “I would think it would be very hard to get into this business now as a new SEO because there is just so much noise.”

The number of links to a Web page is one measure Google uses to rank Web sites, and that brings us to the dark side of search engine optimization: There are services that sell links to content that spammers are promoting. The more links to a Web page, the better.

In a particularly devious way of goosing a page ranking, some spammers use Amazon’s online marketplace for Web work, Mechanical Turk, to hire people at a nickel a click to follow links that would boost their site’s presence on Google. Panos Ipeirotis, a New York University business school professor, studied Mechanical Turk’s new work offerings late last year and found that more than 40 percent were for spamming work. “The results were disturbing,” he wrote.

Then there are the content farms. Companies such as Demand Media, which raised $151 million in an initial public offering Wednesday, and AOL have created businesses that pay writers – or anyone who can reasonably string sentences together – to post content that answers popular search queries. AOL’s Seed division offers low-paid writing assignments based on hot topics. Demand Media, particularly through its eHow Web site, specializes in how-to information. The company has told investors it relies heavily on revenue generated through Google’s advertising program.

Content farms are masters at surfacing their content high in Google’s search results. The content, critics say, is often less than masterful. A search on Google for “how do I calibrate flat screen tv” turns up two eHow sites in the first three links. One is written by someone named Jenna Johnson, whose other articles include “How to Organize a Small Bedroom” and “How to Use Leftover Meatloaf.” (Make a meatloaf sandwich, she suggests.)

Demand Media declined to comment, citing the quiet period after its IPO, but company executives have in the past rejected the “content farm” label. AOL Senior Vice President David Mason also disputed the “content farm” term, saying the company pushes its authors to write deep, engaging content. Seed recently highlighted an item on its blog about cleaning eyeglasses with toothpaste.

My friend with the lousy TV picture found nothing in the eHow posts – or the other links that Google surfaced – that helped her.

“I was about to take my new TV back to the store because I thought it was just a kind of TV that wouldn’t work the way we wanted it to,” she said. Then Twitter, not Google, saved the day. Asked how her picture is now, Skloot said, “Amazing.”

A step behind

Google, for its part, says that it has long been aware of link buying and other spamming tricks and that its algorithms take all of that into account. But many of Google’s critics wonder how far ahead of the spammers the company really is.

“I have a hard time believing they are losing this badly,” O’Reilly said. “I look at more and more categories of searches that are clearly spam” – medical information, product reviews, technology help, restaurant tips.

As for content farms, Google engineers said the company must come up with better ways to keep that content from showing up so high in search results.

“We are hearing the complaints about this type of shallow content, and we are going to address it,” said Amit Singhal, the most senior engineer on Google’s page-ranking team. “We have to make sure our users are getting the best information, and if they are not, we have failed.”

The company, in its blog post, said it recently launched a new internal technological tool that makes it harder for “spammy” content to rise in results. Other tools and strategies are forthcoming, executives said.

But many Internet analysts wonder whether the new spam-fighting effort, although needed, is beside the point.

“We have kind of stretched the usefulness of search engine algorithms for surfacing the kind of specific content we are looking for,” said William Tancer, general manager of global research for Experian Hitwise.

The future, many believe, is social search. Microsoft, which is a major investor in Facebook, can scour a searcher’s own Facebook account on every query. Facebook, noting an increase in users who ask for information in their status updates, is testing a new function called Facebook Questions.

“If we notice a lot of people using a feature in a particular way, that’s when we consider specializing a product for that need,” said Brett Taylor, Facebook’s chief technology officer.

“Anyone can answer your question, which means you can tap into the collective knowledge of the millions of people on Facebook,” the company says on a Web page explaining the new tool. Answers that users rate as high quality can then be highlighted beyond a user’s own social network, pushing answers from one network of people to many, many others.

A hot start-up called Quora is following a similar question-and-answer model. Meanwhile, Google is also working on social search, with co-founder Sergey Brin reportedly heading up the effort. He recently said the company has scratched just one percent of what social search could be. Google already searches Twitter alongside classic Web page searches. Users signed in on a Google account will also see results such as images or status updates from their contacts. Some information from Facebook is included, but nothing from a user’s account.

Google spent $50 million last year to buy a company called Aardvark that lets users send questions to people in their social networks. If those friends can’t answer the question, it is sent on to friends of friends. Answers are delivered, often immediately, via instant message or e-mail. I gave it a try last week.

I have a nasty case of sciatica and might need to start walking at my desk while I work. I asked Aardvark: “What is the best way to set up a treadmill desk? I want to put a treadmill under my desk sometimes.” One smart aleck quickly answered, “Does your desk need some exercise?” But a few minutes later, I got an instant message from someone named Adam K. in Cranberry, Pa., offering a collection of Web links.

“Was Adam’s answer helpful?” Aardvark asked.

I clicked yes.

POSTED ON JANUARY 29, 2011 – BY WAYNE SUTTON

Yesterday Matt Cutts posted that Google has changed their search algorithm to focus more on original content. Below is an excerpt from his post:

My post mentioned that “we’re evaluating multiple changes that should help drive spam levels even lower, including one change that primarily affects sites that copy others’ content and sites with low levels of original content.” That change was approved at our weekly quality launch meeting last Thursday and launched earlier this week.

This was a pretty targeted launch: slightly over 2% of queries change in some way, but less than half a percent of search results change enough that someone might really notice. The net effect is that searchers are more likely to see the sites that wrote the original content rather than a site that scraped or copied the original site’s content.

Matt’s post summary is the short version from his post on Google’s offical blog entitled “Google search and search engine spam“. This is great news as it seems like the beginning of Google trying to remain the #1 search enginge and fight off what seems like a series of attacks and competition from new search engines and startups such as Wolframalpha, Blekko and the continue traffic increase of Microsoft Bing.

Note, Matt stated that the Google search algorithm change will only affect 2% of search queries but don’t let the small number fool you. Google is under fire about providing relevant content to the answers of search quires and startups like Quora does.

My recommendation for you is to keep providing good, original content and for Google to keep updating their search algorithm or keep buying startups that are answering the questions that people use to search Google for.

Originally posted on SocialWayne.com Read more: http://socialwayne.com/2011/01/29/google-search-algorithm-change/#ixzz1CkfGrvte
Under Creative Commons License: Attribution Share Alike

Searching for file-sharing information via Google is going to take a little bit more effort now, thanks to new steps taken by the search engine to remove all sorts of references to torrents from its instant search and autocomplete features.

http://www.marco.org/2617546197

The lead up to the Christmas and New Year holidays required researching a number of consumer goods to buy, which of course meant using Google to search for them and ratings reviews thereof. But this year it really hit home just how badly Google’s systems have been spammed, as typically anything on Page 1 of the search results was some form of SEO spam – most typically a site that doesn’t actually sell you anything, just points to other sites (often doing the same thing) while slipping you some Ads (no doubt sold as “relevant”). The other main scamsite type is one that copies part of the relevant Wikipedia entry and throws lots of Ads at you. It wasn’t just me who found this – Paul Kedrosky found the same:

Google has become a snake that too readily consumes its own keyword tail. Identify some words that show up in profitable searches — from appliances, to mesothelioma suits, to kayak lessons — churn out content cheaply and regularly, and you’re done. On the web, no-one knows you’re a content-grinder.

The result, however, is awful. Pages and pages of Google results that are just, for practical purposes, advertisements in the loose guise of articles, original or re-purposed. It hearkens back to the dark days of 1999, before Google arrived, when search had become largely useless, with results completely overwhelmed by spam and info-clutter.

And I can’t believe Google doesn’t know this – nor does Paul:

Google has to know this. The problem is too big and too obvious to miss. But it’s hard to know what you can do algorithmically to solve the problem. Content creators are simply using Google against itself, feeding its hungry crawlers the sort of thing that Google loves to consume, to the detriment of search results and utility. For my part it has had a number of side-effects. One, I avoid searching for things that are likely to score high in Google keyword searches. Appliances are an example, but there are many more, most of which I use mechanisms other than broad search. Second, it has made me more willing to pay for things. In this case I ended up paying for a Consumer Reports review of dishwashers — the opportunity cost of continuing to try to sort through the info-crap in Google results was simply too high.

Reading the comment’s on Paul’s blog post was interesting – you can parse the responses into 3 broad groups:

- Yes, we agree with you, and here are some tips on how to deal with it
- Yes, but its not poor Google’s fault, its those evil spammers (ie Google has no way of changing their systems and is at the mercy of SEO)
- No, there is no problem, this is the best of all possible solutions (complete bollocks IMHO, it was definitely better a few years ago)

(Ignoring the ones trying to pimp their own products or agendas of course, and the end posts comparing the economics of online vs library copies of Consumer Reports…..)

Ignoring these comments, I have found my behaviour is exactly the same as Paul’s , i.e. increasingly reaching for paid-for, edited research (Which? in the UK) as Google and some of the “comparison” sights (clearly flooded with Spam, Sock Puppets and Sleazeoids) become less and less credible. (Another aside – I had a gift voucher from Amazon, and searching for a book I wanted I found Page 1 was totally full of results for the book on Kindle, which was very irritating – they need to allow one to select e-book and/or book).

The interesting question to me is what happens if this gets worse, as Google risks attacks on 2 fronts:

(i) Other search engines decide to eschew Ads for accuracy and cut down the spamming, to gain market share. There is an article on Techcrunch today about Blekko, which appears to promise this.

(ii) The market for paid-for search and research grows – how much would you pay per month for a neutral search engine? Which? costs about £7 a month, would you pay that for a neutral engine?

Frankly, I don’t believe that it is not possible to reduce this sort of spam, I think Google’s problem is more that it is trying to navigate a line between income (systemically the more spam there is, the more Ad money it makes) and usefulness (how much spam can you run before the user walks away) and has veered too far to the spamside.

Update – this piece was ReTwt’d by Tim O’Reilly, and had the equivalent of a slashdotting so down went our server – fortunately thosenice people over at Hacker News pointed to the proxies pronto. I liked two of the themes in the comments there:

(i) a lot of these sites must be known, ditto their pattern, so just a few weeks judicious “mechanical turk” work should have a large 80/20 impact

(ii) Google is like a monoculture, and thus parasites have a major impact once they have adapted to it – especially if Google has “lost the war”. If search was more heterogenous, spamsites would find it more costly to scam every site. That is a very interesting argument against the level of Google market dominance

Update 2 - Anil Dash and Coding Horror have picked up on the same issue today.

Update 3 – Bruse Sterling has picked up on the story – his comment:

“we may be approaching a period where the machines will feed you an infinite amount of cunningly-engineered gibberish”

http://paul.kedrosky.com/archives/2009/12/dishwashers_dem.html

Over the weekend I tried to buy a new dishwasher. Being the fine net-friendly fellow that I am, I  began Google-ing for information. And Google-ing. and Google-ing. As I tweeted frustratedly at the tend of the failed exercise, “To a first approximation, the entire web is spam when it comes to appliance reviews”.

This is, of course, merely a personal example of the drive-by damage done by keyword-driven content — material created to be consumed like info-krill by Google’s algorithms. Find some popular keywords that lead to traffic and transactions, wrap some anodyne and regularly-changing content around the keywords so Google doesn’t kick you out of search results, and watch the dollars roll in as Google steers you life-support systems connected to wallets, i.e, idiot humans.

Google has become a snake that too readily consumes its own keyword tail. Identify some words that show up in profitable searches — from appliances, to mesothelioma suits, to kayak lessons — churn out content cheaply and regularly, and you’re done. On the web, no-one knows you’re a content-grinder.

The result, however, is awful. Pages and pages of Google results that are just, for practical purposes, advertisements in the loose guise of articles, original or re-purposed. It hearkens back to the dark days of 1999, before Google arrived, when search had become largely useless, with results completely overwhelmed by spam and info-clutter.

Google has to know this. The problem is too big and too obvious to miss. But it’s hard to know what you can do algorithmically to solve the problem. Content creators are simply using Google against itself, feeding its hungry crawlers the sort of thing that Google loves to consume, to the detriment of search results and utility.

For my part it has had a number of side-effects. One, I avoid searching for things that are likely to score high in Google keyword searches. Appliances are an example, but there are many more, most of which I use mechanisms other than broad search. Second, it has made me more willing to pay for things. In this case I ended up paying for a Consumer Reports review of dishwashers — the opportunity cost of continuing to try to sort through the info-crap in Google results was simply too high.

Something has to give, but I wonder what will — the snake, its tail, or us?

If you are involved in the Google Adsense program, you probably know by now that Google doesn’t disclose the percentage that it pays its publishers. Other programs will declare up front that they take 20%, 30% or 50% of the total revenues, but Google doesn’t declare this, and won’t tell you if you ask.

First off, the question that needs asking is: why doesn’t Google just pick a percentage and stick to it? I believe that they do this to give themselves flexibility in the future. If they announced publicly that they were taking 30% of revenues, and then raised that amount to 35% in the future, there would be a massive outcry. Look at what happened when Ebay raised their rates. By keeping the percentages a secret, Google gives themselves the ability to raise the amount that they take without there being a protest.

Also, this gives Google the ability to “smooth” over their earnings if they ever need to. Let’s say that Google is internally forecasting a rough fourth quarter. Well if they decided to pay out 65% of total earnings to their partners instead of 70%, this extra 5% would go straight towards their bottom line. This would be millions upon millions of dollars going straight towards their net income, which would come in handy in the face of a potentially weak quarter.

So how much does Google pay out to its partners? The answer lies right in their quarterly report. Google said that it paid out $1.06 billion dollars to its Adsense partners. Google Network Revenues, which is what the partner sites (Adsense) generated in revenues, came in at $1.35 billion dollars. So Google paid out about 21.5% to its Adsense publishers in the last quarter. Google Network revenues accounts for 35% of its total revenues in the most recent quarter.

All in all, considering what Google provides you (the network of advertisers, the billing, the customer service, the technology), it’s a great deal for publishers and extremely easy to set up. Yahoo can try all they want to compete with Google with YPN, but Google is lightyears ahead of them and will never be touched, in my opinion. It’s all about the network of advertisers, and Google has Yahoo crushed in that department.

source: http://investor.google.com/releases.html