Examples of WebDownloader

barrysoft.web.WebDownloader
The WebDownloader can be used to download web data using HTTP requests. @author Daniele Rapagnani
net.vidageek.crawler.component.WebDownloader
@author jonasabreu

Examples of barrysoft.web.WebDownloader

  public final String postSite = "http://www.snee.com/xml/crud/posttest.cgi";
  
  @Test
  public void testPost() {
    
    WebDownloader downloader = new WebDownloader();
    downloader.addFormElement("fname", "Masked");
    downloader.addFormElement("lname", "Man");
    downloader.setMethod(WebDownloader.METHOD_POST);
    
    try {
      downloader.setUrl(postSite);
      String source = new String(downloader.download());
      
      if (source.contains("First name: \"Masked\""))
        assertTrue(true);
      else
        fail("Can't find the string:\n\n"+source);

View Full Code Here

Examples of barrysoft.web.WebDownloader

    archiveHandlers = ArchiveHandlerManager.getDefaultHandlers();
    updateValidators = UpdateValidatorsManager.getDefaultValidators();
    
    updatesExecutor = Executors.newSingleThreadExecutor();
    
    downloader = new WebDownloader();
    downloader.addProgressListener(this);
    
    ResourcesManager.setResources(new UpdaterResources());
  }

View Full Code Here

Examples of net.vidageek.crawler.component.WebDownloader

    }
    if (!Pattern.compile("(?s)^http[s]?://.*$").matcher(beginUrl).matches()) {
      throw new IllegalArgumentException("beginUrl must start with http:// or https://");
    }
    this.beginUrl = beginUrl;
    downloader = new WebDownloader();
    normalizer = new DefaultLinkNormalizer(beginUrl);
    minPoolSize = 30;
    maxPoolSize = 30;
    keepAliveMilliseconds = 30000;
    requestDelayMilliseconds = 1000;

View Full Code Here

0 1

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.