Get Referenced Domains

Demonstrates how to accumulate a list of unique domain names referenced from outbound URLs.

Download Chilkat Spider ActiveX

'  The Chilkat Spider component/library is free.
Dim spider As New Spider

Dim domainList As New CkStringArray

'  Set the Unique property so that duplicates are not added.
domainList.Unique = 1

'  Crawl the home page of joelonsoftware.com and get the outbound URLs
spider.Initialize "www.joelonsoftware.com"
spider.AddUnspidered "http://www.joelonsoftware.com/"

Dim success As Long
success = spider.CrawlNext()

'  Build a list of unique domains.
Dim i As Long
Dim url As String
For i = 0 To spider.NumOutboundLinks - 1
    url = spider.GetOutboundLink(i)
    domainList.Append spider.GetDomain(url)
Next

'  Display the domains.
For i = 0 To domainList.Count - 1
    Text1.Text = Text1.Text & domainList.GetString(i) & vbCrLf
    Text1.Refresh
Next