C# Examples

ChilkatHOMEAndroid™ASPVisual BasicVB.NETC#iOS (IPhone)Objective-CC++CMFCDelphiFoxProJavaPerl
PHP ExtensionPHP ActiveXPythonPowerShellRubySQL ServerVBScript

C# Examples

Bounced Mail
Bz2
Character Encoding
CSV
DKIM / DomainKey
Digital Certificates
Digital Signatures
Email
Email Object
FTP
HTML Conversion
HTTP
IMAP
Encryption
MHT / HTML Email
MIME
POP3
RSA
S/MIME
SMTP
Socket
Spider
SSH
SSH Tunnel
SSH Key
SFTP
Tar Archive
Upload
XML
XMP
Zip Compression


More Examples...
Amazon S3
NTLM
FileAccess
RSS
Atom
String
Byte Array
Self-Extractor
Service
PPMD
Deflate
DH Key Exchange
DSA
Bzip2
LZW

 

 

 

 

 

 

Fetch robots.txt for a Site

Download Chilkat .NET for 4.0 Framework

Download Chilkat .NET for 64-bit 4.0 Framework (x64)

Download Chilkat .NET for 2.0 / 3.5 Framework

Download Chilkat .NET for 64-bit 2.0 / 3.5 Framework (x64)

Download Chilkat .NET for 1.0 / 1.1 Framework

The Chilkat Spider library is robots.txt compliant. It automatically fetches a site's robots.txt file and adheres to it. It will not download pages denied by robots.txt. Pages excluded by robots.txt will not appear in the Spider's "unspidered" list. This example shows how to explicitly download and review the robots.txt for a given site.

// The Chilkat Spider component/library is free.
Chilkat.Spider spider = new Chilkat.Spider();

spider.Initialize("www.chilkatsoft.com");

string robotsText;
robotsText = spider.FetchRobotsText();

textBox1.Text += robotsText + "\r\n";
textBox1.Refresh();
 

© 2000-2012 Chilkat Software, Inc. All Rights Reserved.

Email Component · XML Parser