Chilkat Examples

ChilkatHOMEAndroid™Classic ASPCC++C#C# WinRTDataFlexDelphi ActiveXDelphi DLLVisual FoxProJavaLianjaMFCObjective-CPerlPHP ActiveXPHP ExtensionPowerBuilderPowerShellPureBasicPythonRubySQL ServerSwiftTclUnicode CUnicode C++Visual Basic 6.0VB.NETVB.NET WinRTVBScript

Ruby Examples

Async
Certificates
Email Object
Encryption
FTP
HTML-to-XML/Text
Gzip
HTTP
IMAP
MHT / HTML Email
PEM
PFX/P12
Java KeyStore (JKS)
POP3
RSA Encryption
MIME
SCP
SMTP
Socket/SSL/TLS
SSH Key
SSH
SFTP
Tar Archive
XML
XMP
Zip

More Examples...
ASN.1
PRNG
Amazon S3
Bounced Email
CSV
Diffie-Hellman
DKIM / DomainKey
NTLM

 

 

 

 

 

 

 

(Ruby) Fetch robots.txt for a Site

The Chilkat Spider library is robots.txt compliant. It automatically fetches a site's robots.txt file and adheres to it. It will not download pages denied by robots.txt. Pages excluded by robots.txt will not appear in the Spider's "unspidered" list. This example shows how to explicitly download and review the robots.txt for a given site.

Chilkat Ruby Downloads

Ruby Library for Linux and MAC OS X

require 'chilkat'

CkSpider = Chilkat::CkSpider.new()

spider.Initialize("www.chilkatsoft.com")

robotsText = spider.fetchRobotsText()

print robotsText + "\n";

 

© 2000-2014 Chilkat Software, Inc. All Rights Reserved.