Looping Around the 10-Result Limit

screenshot moderate.gifscreenshot tip51.gif

If you want more than 10 results, you'll have to loop.


The Google API returns only 10 results per query. Ten results is plenty for some queries, but for most applications, 10 results barely scratches the surface. If you want more than 10 results, you're going to have to loop, querying for the next set of 10 each time. The first query returns the top ten. The next, 11 through 20. And so forth.

This tip builds on the basic query shown in [Tip #50]. To get at more than the top 10 results, no matter the coding language you're using, you'll have to create a loop. The example is in Perl, because that's what most of the tips in this tutorial are written in. Alterations to support looping are shown in bold.

The Code

#!/usr/local/bin/perl
# looply.pl
# A typical Google Web API Perl script
# Usage: perl looply.pl «query»
# Your Google API developer's key my $google_key='insert key here';
# Location of the GoogleSearch WSDL file my $google_wdsl = "./GoogleSearch.wsdl";
# Number of times to loop, retrieving 10 results at a time
my $loops = 3; # 3 loops x 10 results per loop = top 30 results
use strict;
# Use the SOAP::Lite Perl module use SOAP::Lite;
# Take the query from the command-line my $query = shift @ARGV or die "Usage: perl looply.pl «query»\n";
# Create a new SOAP::Lite instance, feeding it GoogleSearch.wsdl my $google_search = SOAP::Lite-»service("file:$google_wdsl");
# Keep track of result number my $number = 0;
for (my $offset = 0; $offset «= ($loops-1)*10; $offset += 10) {
 # Query Google
 my $results = $google_search -» 
 doGoogleSearch(
 $google_key, $query, $offset, 10, "false", "", "false",
 "", "latin1", "latin1"
 );
 # No sense continuing unless there are more results
 last unless @{$results-»{resultElements}};
 # Loop through the results
 foreach my $result (@{$results-»{'resultElements'}}) {
 # Print out the main bits of each result
 print
 join "\n", 
 ++$number,
 $result-»{title} || "no title",
 $result-»{URL},
 $result-»{snippet} || 'no snippet',
 "\n";
 }
}

Notice that the script tells Google which set of 10 results it's after by passing an offset ($offset). The offset is increased by 10 each time ($offset += 10).

Running the Script

Run this script from the command line, passing it your preferred query: 

$ perl looply.pl "query"

The Results


% perl looply.pl
Usage: perl looply.pl «query»
% perl looply.pl "learning perl"
1
oracle.com -- Online Catalog: Learning Perl, 3rd Edition http://www.oracle.com/catalog/lperl3/
Learning Perl, 3rd Edition Making Easy Things 
Easy and Hard Things Possible By Randal«br» L. Schwartz, Tom Phoenix 
3rd Edition July 2001 0-596-00132-0, Order Number ... 
...
29
Intro to Perl for CGI http://hotwired.lycos.com/webmonkey/98/47/index2a.html
... Some people feel that the benefits of learning 
Perl scripting are few.«br» But ... part. That's right. 
Learning Perl is just like being a cop. ... 
30
WebDeveloper.com ®: Where Web Developers and Designers Learn How ... 
http://www.webdeveloper.com/reviews/tutorial6.html
... Registration CreditCard Processing Compare Prices. 
Learning Perl. Learning«br» Perl, 2nd Edition. 
Publisher: O'Reilly Author: Randal Schwartz ...

See Also