Examples of org.apache.stanbol.enhancer.engine.topic.TopicClassificationEngine

org.apache.stanbol.enhancer.engine.topic.TopicClassificationEngine

Enhancement Engine that provides the ability to assign a text document to a set of concepts indexed in a dedicated Solr core. The assignment logic comes from terms frequencies match of the text of the document to categorize with the text indexed for each concept. The data model of the concept tree follows the SKOS model: concepts are organized in a hierarchical "scheme" with a "broader" relation (and the inferred "narrower" inverse relation). Concepts can also optionally be grounded in the real world by the mean of a foaf:primaryTopic link to an external resource such as a DBpedia entry. A document is typically classified with the concept by using the dct:subject property to link the document (subject) to the concept (object). The Solr server is expected to be configured with the MoreLikeThisHandler and the matching fields from the engine configuration. This text classifier method sometimes goes by the name of "Rocchio classification" or "Nearest Centroid classification" in the IR and machine learning literature. It is often slightly less accurate than fitting penalized linear model such as linear kernel Support Vector Machines or penalized Logistic Regression but has the advantage to scale to large number of categories (e.g. more that tens of thousands) without having to load the full statistical model in memory thanks to the use of the inverted index datastructure that also provides the feature extraction and TF-IDF weighting for free. Furthermore it could be refined by using a "Learning to Rank" approach by training a RankSVM, Gradient Boosted Trees or Random Forests on the output on the raw output of the Rocchio classifier so as to re-rank candidate classes more finely. The Learning to Rank refinement is not implemented yet.


        // TODO: use a service track to wait for the registration of the service
        ServiceReference topicEngineReference = context.getServiceReference(TopicClassificationEngine.class
                .getName());
        TestCase.assertNotNull(topicEngineReference);
        TopicClassificationEngine engine = (TopicClassificationEngine) context
                .getService(topicEngineReference);
        TestCase.assertNotNull(engine);
        // TODO: test classification here
    }


        // TODO: use a service track to wait for the registration of the service
        ServiceReference topicEngineReference = context.getServiceReference(TopicClassificationEngine.class
                .getName());
        TestCase.assertNotNull(topicEngineReference);
        TopicClassificationEngine engine = (TopicClassificationEngine) context
                .getService(topicEngineReference);
        TestCase.assertNotNull(engine);
        // TODO: test classification here
    }

Examples of org.apache.stanbol.enhancer.engine.topic.TopicClassificationEngine

Related Classes of org.apache.stanbol.enhancer.engine.topic.TopicClassificationEngine