Examples of encodedKey()

Examples of net.yacy.kelondro.blob.BEncodedHeap.encodedKey()

100
101
102
103
104
105
106
107
108
109
110
        final String urlHostPort = getHostPort(theURL);
        RobotsTxtEntry robotsTxt4Host = null;
        Map<String, byte[]> record;
        final BEncodedHeap robotsTable = this.tables.getHeap(WorkTables.TABLE_ROBOTS_NAME);
        try {
            record = robotsTable.get(robotsTable.encodedKey(urlHostPort));
        } catch (final RowSpaceExceededException e) {
            log.warn("memory exhausted", e);
            record = null;
        }
        if (record != null) robotsTxt4Host = new RobotsTxtEntry(urlHostPort, record);
View Full Code Here

Examples of net.yacy.kelondro.blob.BEncodedHeap.encodedKey()

128
129
130
131
132
133
134
135
136
137
138
                // if we have not found any data or the data is older than 7 days, we need to load it from the remote server

                // check the robots table again for all threads that come here because they waited for another one
                // to complete a download
                try {
                    record = robotsTable.get(robotsTable.encodedKey(urlHostPort));
                } catch (final RowSpaceExceededException e) {
                    log.warn("memory exhausted", e);
                    record = null;
                }
                if (record != null) robotsTxt4Host = new RobotsTxtEntry(urlHostPort, record);
View Full Code Here

Examples of net.yacy.kelondro.blob.BEncodedHeap.encodedKey()

248
249
250
251
252
253
254
255
256
257
258
    private String addEntry(final RobotsTxtEntry entry) {
        // writes a new page and returns key
        try {
            final BEncodedHeap robotsTable = this.tables.getHeap(WorkTables.TABLE_ROBOTS_NAME);
            robotsTable.insert(robotsTable.encodedKey(entry.getHostName()), entry.getMem());
            return entry.getHostName();
        } catch (final Exception e) {
            log.warn("cannot write robots.txt entry", e);
            return null;
        }
View Full Code Here
TOP
Copyright © 2018 www.massapi.com. All rights reserved.
All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.