Setting a Maximum URL Length

The MaxUrlLen property prevents the spider from retrieving URLs that grow too long. The default value of MaxUrlLen is 300.

 Chilkat Java Library Downloads for Windows, Linux, and MAC OS X

import com.chilkatsoft.*;

public class ChilkatExample {

  static {
    try {
        System.loadLibrary("chilkat");
    } catch (UnsatisfiedLinkError e) {
      System.err.println("Native code library failed to load.\n" + e);
      System.exit(1);
    }
  }

  public static void main(String argv[])
  {
    // The Chilkat Spider component/library is free.
    CkSpider spider = new CkSpider();

    spider.Initialize("www.chilkatsoft.com");

    // Add the 1st URL:
    spider.AddUnspidered("http://www.chilkatsoft.com/");

    // This example demonstrates setting the MaxUrlLen property
    // Do not add URLs longer than 250 characters to the "unspidered" queue:
    spider.put_MaxUrlLen(250);

    // ...

  }
}