Examples of org.archive.crawler.framework.CrawlController$StopCompleteEvent

org.archive.crawler.framework.CrawlController

CrawlController collects all the classes which cooperate to perform a crawl and provides a high-level interface to the running crawl. As the "global context" for a crawl, subcomponents will often reach each other through the CrawlController. @contributor gojomo

    }
    
    @Override
    protected ProcessResult innerProcessResult(CrawlURI curi)
    throws InterruptedException {
        CrawlController controller = getCrawlController();
        StatisticsTracker stats = getStatisticsTracker();
        long allowedRuntimeMs = getRuntimeSeconds() * 1000L;
        long currentRuntimeMs = stats.getCrawlElapsedTime();
        if(currentRuntimeMs > allowedRuntimeMs){
            Operation op = getExpirationOperation();
            if(op != null){
                if (op.equals(Operation.PAUSE)) {
                    controller.requestCrawlPause();
                } else if (op.equals(Operation.TERMINATE)){
                    controller.requestCrawlStop(CrawlStatus.FINISHED_TIME_LIMIT);
                } else if (op.equals(Operation.BLOCK_URIS)) {
                    curi.setFetchStatus(S_BLOCKED_BY_RUNTIME_LIMIT);
                    curi.getAnnotations().add("Runtime exceeded " + allowedRuntimeMs + 
                            "ms");
                    return ProcessResult.FINISH;

Examples of org.archive.crawler.framework.CrawlController$StopCompleteEvent

Related Classes of org.archive.crawler.framework.CrawlController$StopCompleteEvent