Bigmouthmedia search engine news: feeds-sitemaps

Google blocks Yahoo's Pipes

Mon, 12 Nov 2007 12:00:00 GMT

The two leading search engines have reached a technical impasse over Google's news service and Yahoo's pipes product.

Pipes is a highly regarded product from Yahoo's Brickhouse team. Users are able to collect data from various sources around the internet, manipulate it, filter it, sort it and then deliver it in another format. For example, with Yahoo Pipes a brand team could monitor news aggregators and blog search engines around the web for mentions of their brand and then deliver this research in one single RSS feed. That's easy to read and easy to monitor.

In the past Google has been used by viruses to query the web looking for forum sites with known vulnerabilities or email addresses. Google's defence to this sort of virus attack is to block automated queries to the search engine and also to Google News.

Google's anti-automated query defense is now blocking the automated queries from Yahoo Pipes. Many of the published pipes in Yahoo's system are now showing as empty and blocked as Google News was an incredibly popular source of information.

Yahoo's Pipes requests for Google News URLs are being forwarded to Google's blocking page. Google is treating Pipes as a virus or other unwanted computer request.

Yahoo Pipes fail gracefully. Pipes constructed from various information sources continue to work and simply exclude the feeds taken from Google. One of the features that sets Yahoo Pipes aside is their unique visualisation of the information stream. Pipes sending data are highlighted in light blue whereas empty pipes, such as those now being blocked by Google, appear in dull grey.

Tim O'Reilly described Yahoo Pipes as a "milestone in the history of the internet" when it launched and Nic Cubrilovic of TechCrunch said, "...my mind is still buzzing thinking about all that can be done with Pipes."

It is within Google's ability to 'whitelist' Yahoo Pipes and allow the internet application to successfully auto-query the search engine, news and blog search. Until the impasse is resolved, however, Pipes may struggle to remain useful.

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia

Yahoo! announces unlimited email storage

Wed, 28 Mar 2007 12:00:00 GMT

Yahoo! has announced that it is to offer its 250 million email clients unlimited storage capacity from May.

In an announcement timed to coincide with the service's tenth birthday, the world's biggest email provider said that the decision had been made in response to the explosive growth in attachment size as people share more photos, music and videos via email.

"We are giving them no reason to ever have to delete old emails," Yahoo co-founder David Filo told Reuters. "You can keep stuff forever."

He added that many people now routinely send attachments larger than Yahoo!'s original storage allocation of four megabytes per user. When the company's free email addresses were launched in 1997 the total memory allocation for the service was 200 gigabytes.

The company currently offers one gigabyte per user, compared to two gigabytes per Hotmail account or 2.8 gigabytes from Gmail.

"We have been closely monitoring average usage. We are comfortable that our users are far under one gigabyte, on average," said Yahoo! vice president John Kremer. "What we see are an increasing number of rich media files as consumers send more photos."

The accounts will be subject to certain non-abuse clauses, such as requirements that the accounts will only be for personal use and will not be offered as commercialised storage space.

Users in Japan and China will not be immediately included in the offer. The decision is also a reflection of rapidly falling storage costs, with modern PCs offering storage of up to a terabyte.

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia

Excite MIX: the latest buzz among UK users

Mon, 15 Jan 2007 12:00:00 GMT

Countless web users are already acquainted with Excite, the European web portal which offers the latest in search on anything from travel and shopping to news and finance. But the site has just taken its offerings a bit further with the launch of its new Excite MIX personalized homepage feature.

The redesigned web portal shows users how to make the most of RSS and to receive real-time content from their favourite sites. For instance, anyone can create a personalised homepage by integrating their favourite content from the web via RSS feeds, as well as receive the latest news from preferred sites in real time. But that's not all: Excite users also have access to a new shopping comparison service, as well as an improved mail system - Excite Mail - which includes podcasting as well as options that allow you to increase your storage space.

Moreover, Excite is making job hunting easier than ever: the web portal has just teamed up with Workcircle to launch its job search engine. The new service gives users access to over 200,000 UK jobs on the Workcircle network. It also allows users to receive the latest job announcements directly onto their Excite MIX personalised pages as they're announced, via RSS feeds.

Bernhard Berger, Business Development Manager, Excite UK, commented:

"Excite is a stylishly designed portal with a host of relevant, useful content and services. A slick, simple service to assist jobseekers is a perfect match for us and Workcircle made it easy and very quick to launch the new channel. Users will be delighted with the new service and can start benefiting from access to the latest jobs for 2007 immediately."

Workcircle CEO Simon Appleton added:

"Partnering with high-profile brands to help them generate revenue from job sites is a key part of our strategy. Excite was an obvious first partner, with an existing under-utilised jobs section, and a team keen to provide the very best services to their users."

Another fruit of the re-design project is the informative Excite MIX blog, Web Twitcher, which offers tips on how to customise MIX and keep up-to-date with the web world.

The site was clearly designed with the needs of UK users in mind, and looks likely to see an increase in its current turnout of five million unique users per month. After all, users can now search the web, view favourite content, check email and construct subject specific pages from a single location. The portal has all the features expected from a high quality site, plus the distinctive trimmings that make Excite one of the leading brands in the UK online consumer market.

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia

Del.icio.us Launch the 'Tagometer'

Wed, 27 Dec 2006 12:00:00 GMT

Del.icio.us, the social bookmarking people have just launched a new function called the 'Tagometer'. The Tagometer is a badge which is similar to the previously launched 'Save To' del.icio.us button but includes more functionality for the user. The 'Save To' button was designed to provide users with an easy way to save a page to del.icio.us and was implemented to websites using a simple piece of javascript code. The 'Save To' button graphic is shown below.

The Tagometershows significant development from its predecessor. Using a JSON data feed from del.icio.us, the Tagometer includes an up-to-date count of others who've already bookmarked the page, as well as a fresh list of the top tags applied. The Tagometer box graphic is shown below.

Visitors can click on the count to see more detail on who bookmarked your page. They are able to discover when it was bookmarked and how they tagged it for. The list of tags shows visitors why your page is interesting, and clicking on a tag takes them to similar sites on del.icio.us. Users are given this useful stream of information which opens information up for them giving another dimension to their user experience!

There are currently two Tagometer box choices for your site. The rectangular box which is shown above is designed for use in sidebars and insets. The other is a single line intended for the end of stories and blog entries.

How to get a Tagometer...

To implement a Tagometer to your site simply take the example code from the del.icio.us website and paste it into your web page or template. The Tagometer will automatically populate itself from the location and title it finds on the page that you have put it on.

For those of you who are more advanced scripters, del.icio.us encourage taking a deeper look into further possibilities with the Tagometer function. People at Feedburner for example have already built a new FeedFlare unit that incorporates the JSON feed data.

The Tagometer is a function that should be received very well by budding online bookmarkers. It certainly has raised the bar over the 'Save To' button and has received some very positive feedback in online forums. Incremental improvements are certainly possible here and we look forward to seeing how this function develops over time.

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia

Search Engine Sitemaps - Sitemaps now supported by top three search engines under a creative commons licence

Thu, 16 Nov 2006 12:00:00 GMT

Some positive news has been revealed in the SEO world today, namely that Google, MSN and Yahoo have all agreed to support a unified system of submitting web pages to their crawlers. Whilst Google launched their sitemap xml programme in June last year, (and Yahoo have taken similar steps with Site Explorer) this move marks the day when the other 'big two' have agreed that they will all now support the Google xml method via the new www.sitemaps.org destination.

Google Sitemaps evolves into Search Engine Sitemaps!

Sitemaps were introduced and supported by Google on the 2nd of June, 2005. Initially released as a beta program, and without prior announcement or previews on Google Labs, Google's sitemaps programme was released under the 'Attribution / Share Alike' Creative Commons license with the prospect that other search engines may adopt the protocol.

As Google states, today:

"Last year we published the Sitemap 0.84 XML protocol as a free and easy way for webmasters to inform search engines about URLs on their web sites so that search engines can more effectively crawl them. We released it under the Attribution/Share Alike Creative Commons license in the hopes that other search engines would adopt the protocol too. And today, we're excited to announce that Yahoo! and Microsoft are joining us in officially supporting the Sitemap protocol."

A recap on Google Sitemaps

The Google Sitemap - no, wait: the Search Engine Sitemap! - consists physically of a hosted XML file, generally named a sitemap.xml file. Think of this sitemap file as the antithesis of the robots.txt file. Robots.txt files are placed on the root folder of a domain to inform search engines of parts of the site which should not be spidered and included in search engines. A search engine sitemap should be similarly placed within a website and should indicated URLs that should be indexed. The sitemap.xml file is essentially a specially formulated set of instructions aimed specifically at search engine spiders with the aim of improving URL inclusion.

This does not mean that only those pages included in the XML file will be indexed, it just help draw the search engine spiders' attention to pages it would not find otherwise, using the traditional method of following links. The original Google sitemap protocol was released as the Sitemap 0.84 XML protocol.

The new Search Engine Sitemap Protocol

"As part of this development, we're moving the protocol to a new namespace, www.sitemaps.org , and raising the version number to 0.9. The sponsoring companies will continue to collaborate on the protocol and publish enhancements on the jointly-maintained site sitemaps.org"

The new Search Engine Sitemap Site: http://www.sitemaps.org /

Are sitemaps always useful for search engine Spidering?

There has been some speculation on the internet that using a sitemap can actually harm your rankings; that a sitemap draws focus and attention to all parts of your site and to that effect a sitemap can harm your standing with Google. The reality is that there is no evidence to support this effect from sitemaps, and documented issues from sites on forums have often been found by the more experience SEO to contain a number of significant issues. The loading of a sitemap was often an easy action to attribute to a negative event after it - Post hoc ergo propter hoc!

What do new search engine sitemap webmaster need to do?

"If you haven't submitted a Sitemap before, check out the documentation on www.sitemaps.org for information on creating one. You can submit your Sitemap file to Google using Google webmaster tools. See the documentation that Yahoo! and Microsoft provide for information about submitting to them."

What about existing Google sitemap users?

Existing sitemaps users can rest easy. There is good news from Google for on-the-ball webmasters:

"If you've already submitted a Sitemap to Google using the previous namespace and version number, we'll continue to accept it."

This is excellent news for those of us regularly involved in attaining submission to each of the search engines, and who have to cater for different standards for each: as life just go a little easier. Of course, we also now have a new benchmark for search engine comparison: indexing wars!

Imagine the situation: a new domain goes live, pages are set up, and a sitemap.xml file is submitted to Google, Yahoo! and MSN through the new sitemap.org standard. Who picks up the pages first?

Get your bets in for the inaugural index race!

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia

RSS: Really Simple Spam?

Fri, 03 Nov 2006 12:00:00 GMT

In today's world, everyone who is anyone has a blog uses RSS to distribute information. But many of us know all too well that a good thing regarding the development of the internet is often accompanied by a bad thing - and yes, you guessed it, it's spam. It is everywhere you go, and there is no escaping it. It appears in your inbox, it pops up in your browser, it gets mailed through your door offering you 'great deals' that are never really great or in fact even deals; what's more, it now even calls you during dinner.

But if you're a blogger or website owner who likes to distribute content to the world via RSS, a new form of spam might strike upon you - Really Simple Spam, otherwise known as RSS Spam.

It has been brewing in the minds of spammers for some time now. It seems they've been letting the blogging scene progress to the biggest thing on the net, just waiting to strike aggressively and reap the benefits of others' misfortunes. And, alas, they've finally done it.

Basically, Really Simple Spam is just our 'hip' way of describing spam in RSS (Really Simple Syndication). You've probably already experienced Really Simple Spam if you are a regular internet user, as it is most often found in blogs which were created solely to manipulate search engine rankings. People have been creating keyword rich spam blogs and then marketing their feeds so that they appear in the RSS search engines, such as Google News. But why? The reason is that RSS spammers can make money when an unsuspecting user clicks through to the website and then is bombarded with advertisement and pop-ups.

But RSS spammers are 'smarter than the average bear'; they've found a bombing source to exploit, and it comes in the form of podcasts. For all who are unaware of what a podcast is, it is basically a multimedia file distributed over the net using syndication feeds such as RSS - what a coincidence.

The RSS spammers create a feed which looks appealing to a user, and when the user clicks on the link thinking they are about to download the latest podcast, what they really get is spam. It can come in many forms, such as a download.exe dialler, malware and the usual 'naughty' stuff that is associated with spam. An experienced internet user would identify this straight away and cancel the transfer; but a novice user may not have the knowledge to understand what the threats are - mainly, spyware or a virus.
At the moment, RSS spam only exists on a small scale, but once the spammers realise that there are ways of manipulating RSS feeds further, they will pounce. Developers will then try to find a way of combating RSS spam just as they did with 'junk e-mail'. But I wouldn't let this put you off creating RSS feeds; they are a great way of distributing information and content if you have control of your feed. Just beware when you are searching the internet for news stories; you could end up a victim of Really Simple Spam.

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia

Product search and Froogle 2.0

Mon, 30 Oct 2006 12:00:00 GMT

During the Professional eBay Sellers Alliance Summit in San Francisco last month, Google officially announced their intention to enhance product search in their web search engine before the end of the year. This big move from Google will be visible in the next few weeks, and will substantially improve its entire product searching experience.

At the moment if you are looking for a specific product you can submit a query to Google.com, Froogle or Google Base. Since most of web searches happen in Google.com, it makes sense for the search engine to integrate the results coming from Froogle and Google Base into the main search results.

Froogle was officially launched back in December 2002 and is Google's retail search engine, providing clear information on a specific product, and is particularly effective for price comparison and product reviews. The service is completely free, and Google does not charge either for product submission or commission from sales. Froogle's index allows the submission of content with associated attributes, such as price or location; Froogle submissions are also accepted into Google Base. As a result, Froogle has become a powerful online price comparison service.

Google Base, essentially a web-based database, was launched in November 2005, and is yet another free service from Google. Similar to Froogle, Google Base allows the free submission of content, although not necessary products, with associated attributes. Furthermore, Google Base is open to submission from individuals and organisations alike.

Froogle is now a subset of Google Base; provided that quality control guidelines are met, any user can post any item to Google Base. Google Base houses everything from jobs to recipes, as well as biographies, property, news, blogs, coupons, tickets, housing, events, reviews, etc

For submissions of information via the web form, there is a maximum limit of up to 15 digital files in the following formats: PDF (.pdf), Microsoft Excel (.xls), Text (.txt), HTML (.html), Rich Text Format (.rtf), Word Perfect (.wpd), ASCII, Unicode and XML.

All the items submitted to Google Base will appear in Google, Froogle and Google Maps if applicable. Google Base is also RSS friendly and your content therefore will benefit from syndication.

But what can we expect from this new version of Froogle?

Google's product search will be integrated into the main Google.com web search. So if you are looking for a specific product, you will get the same results whether you enter your query in Google.com, Froogle or Google Base.

Google will be able to recognize whether or not your query is a product related search, and it will open a second page offering product results based in Google Base listings. Users will be able to refine the search if required via a new web search menu.

The main benefit from this new service is that it provides an integrated framework for product search; users will also be able to refine results without reloading the entire page. Moreover, the information is more clearly presented; but unfortunately there is a heavy AJAX presence, which may create some difficulties in tracking the ROI from taking the time to ensure inclusion in this service.

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia

Dynamic data aids SEO

Mon, 30 Oct 2006 12:00:00 GMT

As the internet constantly expands and develops, so too does the technology behind it. In prehistoric times, when a 56k connection was the height of technological know-how, the words 'dynamic RSS feed' or 'dynamic sitemap' would have invoked only bewilderment and confusion.

But today, an understanding of these terms is crucial for a successful business. RSS (Really Simple Syndication) is now a key part of the major component of a successful search engine marketing campaign: that is, the ready availability of fresh content in all forms.

This is easier if you have a database driven site that uses a CMS (Content Management System). There are various ways that you can generate RSS Feeds dynamically, so that you don't need to manually update them every time you add new content to your website. One popular way of creating dynamic RSS Feeds is with PHP and MySQL, but any database driven site can 'go dynamic' with reasonably little fuss.

Google Sitemaps can also be created dynamically as they are simply another type of feed. This will reduce the time taken to update your sitemap console and speed the pickup of new URLs, as they will automatically update the sitemap.xml when you populate your database. These methods of creating dynamic data are certain to benefit the search engine optimisation strategy of any content producing website.

With dynamic RSS feeds, news that has been added to your site is made readily available to thousands of people if they add your feed to their RSS aggregator. Google Sitemaps, on the other hand, can be made dynamically so that every time you update your database, a new entry will be added to the sitemap. This is extremely beneficial, as Google takes these new URLs into account by downloading your sitemap and looking at the URLs before updating its database information about your site with its subsequent spidering activity.

The benefits of using dynamic data as a key tool in your search engine marketing strategy are evident, and surely show the way forward in the continually developing world of technology and search engine optimisation - the days of 56k connection thinking in website design are, while still beneficial on occasion, increasingly behind us.

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia

Web 2.0 and SEO

Tue, 19 Sep 2006 12:00:00 GMT

Search engine optimization is Web 2.0 - it's official! In fact, even in the original O'Reilly brainstorm cost per click, Google AdSense, blogging and tagging were all "Web 2.0".

It is not uncommon to hear the cry of "Let's do something Web 2.0", but many traditional marketers are yet to fully understand what Web 2.0 is. Web 2.0 is not a new technology. Web 2.0 is not a new set of rules. Web 2.0 is an idea, it is, fundamentally, the concept that the World Wide Web has evolved since the dot-com bubble of 2001.

A core of Web 2.0 is the idea that the Web is a platform. Not an advertising platform, but a platform where users control their own data and from which scalable services are offered. Web 2.0 is about services rather than packaged software and it is about offerings compatible with many devices (mobile phones, portable gaming consoles, different internet browsers, etc).

Tim O'Reilly runs through the original Web 1.0 versus Web 2.0 list in his infamous "What is Web 2.0" article of 2005. If we look at the original list we can see that many of the Web 2.0 ideas have not quite managed the Darwinistic coup d'etat expected. At least: not yet.

BitTorrent was lined up as an evolution of Akamai and if we were to look at the virtual landscape again today that seems unlikely. Akamai provides bandwidth to the likes of Google. And, while the search engine may be trying to become less dependant on that service, it is not taking the BitTorrent route.

Syndication was the Web 2.0 evolution of Stickiness. Stickiness is the attribute web sites have which encourage people to stay on them and not to flee via the first hyperlink or back button. In web marketing and web design stickiness is as important as it ever has been.

Search engine optimization was there to replace domain name speculation. Search engine optimization has evolved significantly further than that. Domain name speculation ebbed away to almost nothing...although at the start of August the domain name wiki.com was sold for nearly $3 million to John Gotts of Searching.com. A trip to Searching.com's beta home page brings up the phrase "Web2.0 beta" in the welcome box. It could just be that as mainstream media perks up their interest in Web 2.0 that domain name speculation rises with it.

Napster was cited as a replacement for MP3.com. Hmm: well!

Many might say that we have moved past the 2001 style of music download site that Napster represented at the time, pre-dating the web 2.0 furore. And yes - some people are now talking about Web 3.0!

When professionals talk about Web 2.0 today they tend to be talking about interactivity on a site (ironically, this tends to make the site more sticky - that Web 1.0 concept).

A prime candidate for interactivity on a site is AJAX. AJAX stands for Asynchronous JavaScript and XML and the search engine Google uses it a lot. Gmail (or Google Mail for those of us in the UK) lets you open or close threads of email conversations without reloading the page. That's an example of AJAX. Pages respond and change to user interaction. Google gives away an AJAX toolkit to make it easier to build pages with it. Here's the catch - AJAX is rarely search engine friendly. Google may use AJAX but Googlebot is not one to actually use JavaScript, especially when it might be submitting data (which AJAX sites tend to do).

User generated content (or "participation" in Web 2.0 terms) is increasingly common. 'Content is King' is a truism in SEO, and it is here that we find the "social" side of the web. MySpace is entirely user generated content: MySpace, YouTube, Digg.com and Wikipedia are all about users being in control of the data.

It is precisely sites such as MySpace, YouTube, Digg.com and Wikipedia which are helping to fuel the rise of the internet and Web 2.0. Rarely a day goes by without one of those sites, or similar, being discussed in mainstream media.

Gartner's "emerging technologies hype cycle" which looks at new technologies and predicts a pattern of hype, disillusionment, and adoption has concluded that Web 2.0 will have a significant impact on businesses over the next ten years.

Here's the prediction - Web 2.0 will come to a site near you. Soon.

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia

Yahoo!'s new site submission process updates their Site Explorer feature

Wed, 09 Aug 2006 12:00:00 GMT

We all know about Google's Sitemap programme, and most of us know about Yahoo!'s version, the urllist.txt submission protocol? Back in September 2005 Yahoo! Search launched Site Explorer, a webmaster linkage data tool used to:

Show all subpages within a URL indexed by Yahoo!
Show the inlinks to a URL indexed by Yahoo!
Submit missing URLs, one at a time or in lists, to Yahoo! free of charge.

An updated version was launched in December last year following requests from webmasters that wanted to be able to show the inlinks to their URLs indexed by Yahoo!, with or without internal links. The update provided a filter to exclude internal links (per domain and sub-domain) amongst other improvements.

At that time Priyank Garg, a Product Manager at Yahoo! Search, asked webmasters to:

"[Continue to] send us feedback, tell us how the product works for you, and let us know what else you'd like to see".

Well it seems that Priyank been taken up on the offer and, as a result, Yahoo! Search has just launched a new, improved, version of Site Explorer (still Beta, though!).

Yahoo! Search details the new features as follows:

More information about your sites, including: Last Crawled Date and Language for your Site URLs; Site Subdomains; and Smoother feed submissions.

You can submit RSS, Atom and URL lists, and manage all of them from one place. For authenticated sites, you can also track when they were submitted and processed. There is also an Update Notification Web Service to notify Yahoo! of feed or site updates.

Once inside Site Explorer (you need to register for a Yahoo! ID) you can now add your sites and authenticate them as you would with Google sitemaps. Further good news is that XML submission is now available, and so the automated updating of your site is much more effective.

One question that's already been raised by a webmaster was "'Submitting a txt file with a URL list' - it sounds as if we have to re-submit when ever there are changes? Or will this be checked up on every now and then?" The quick response from Priyank was:

"Any URLs or feeds we have in the crawler, we will refresh periodically based on our algorithms, but you are welcome to notify us of updates using the UpdateNotification API and we will use that as additional input to direct our crawl".

Another question that's been posed is: "Can you add an alternate mechanism for authentication like 'inserting a meta tag'...because bloggers on Blogspot or MSN Spaces or Yahoo 360, who have no access to their servers, may not be able to upload the file and authenticate the site."

It certainly looks like Priyank and the Yahoo! Site Explorer Team will continue to receive the feedback they've been asking for! This can only be good for Yahoo! Search's market share as webmasters and other interested parties find themselves being listened to and treated like the customers they are.

It seems as though Google's Sitemap Submission process has finally provoked a little competition.

Digg This! | Save to del.icio.us | Reddit | Submit to Netscape | Google Bookmark This | Furl This | Add to Yahoo MyWeb 2 | Contact bigmouthmedia