A B C D E F G H I J L M N O P Q R S T U V W X

A

AbstractScraper - Class in xsmeral.semnet.scraper

A scraper works in co-operation with crawler, extracting data from web pages.

AbstractScraper() - Constructor for class xsmeral.semnet.scraper.AbstractScraper

addEntityDescriptor(EntityDescriptor) - Method in class xsmeral.semnet.crawler.model.HostDescriptor

addEntries(Collection<URLEntry>) - Method in class xsmeral.semnet.crawler.URLManager

Adds given entries to DB.

addEntry(URLEntry) - Method in class xsmeral.semnet.crawler.URLManager

Adds given entry to DB.

addEntry(AssociationRole, String, String) - Method in class xsmeral.semnet.mapper.Mapping

addHost(String) - Method in class xsmeral.semnet.crawler.HostManager

Adds the host with the specified address to the DB.

addSourceURLPattern(Pattern, int) - Method in class xsmeral.semnet.crawler.model.HostDescriptor

allows(String) - Method in class xsmeral.semnet.crawler.util.RobotsPolicy

Checks whether this relative URI is allowed in this host's robots policy

allowsAll() - Method in class xsmeral.semnet.crawler.util.RobotsPolicy

Checks whether this policy allows all URLs for this user-agent

AssociationRole - Enum in xsmeral.semnet.mapper