Perl Examples

ChilkatHOMEASPVisual BasicVB.NETC#Visual C++CMFCDelphiFoxProJavaPerlPHPPythonRubySQL ServerVBScript



Perl Examples

Quick Start
Unicode
Byte Array
Bz2
Certificates
CSV
Email
Encryption
FTP
HTML-to-XML
HTTP
IMAP
MHT
MIME
POP3
RSA
S/MIME
Signatures
SMTP
Socket / SSL
Spider
SFTP
SSH Key
SSH
SSH Tunnel
Tar
HTTP Upload
XML
XMP
Zip

More Examples...
String
Email Object
DKIM / DomainKey
NTLM
FileAccess
RSS
Atom
Self-Extractor
Service
PPMD
Deflate
DH Key Exchange
DSA
Bzip2
LZW

 

 

 

 

 

 

 

Crawling the Web

If the Chilkat Spider component only crawls a single site, how do you crawl the Web? The answer is simple: as you crawl a site, the spider collects outbound links and makes them accessible to you. You may then instantiate an instance of the Spider object for each site, and crawl it. The task of keeping track of what sites you've already crawled is left to you (for now). This example retrieves the home page of http://www.joelonsoftware.com/ and displays the outbound links.

Download Chilkat Perl Module

use chilkat;

# The Chilkat Spider component/library is free.
$spider = new chilkat::CkSpider();

# The Initialize method may be called with just the domain name,
# such as "www.joelonsoftware.com" or a full URL.  If you pass only
# the domain name, you must add URLs to the unspidered list by calling
# AddUnspidered.  Otherwise, the URL you pass to Initialize is the 1st
# URL in the unspidered list.
$spider->Initialize("www.joelonsoftware.com");

$spider->AddUnspidered("http://www.joelonsoftware.com/");

$success = $spider->CrawlNext();

for ($i = 0; $i <= $spider->get_NumOutboundLinks() - 1; $i++) {
    print $spider->getOutboundLink($i) . "\r\n";
}

 

Need a specific example? Send a request to support@chilkatsoft.com

© 2000-2010 Chilkat Software, Inc. All Rights Reserved.