Uses of Class
org.egothor.core.Token

Packages that use Token
org.egothor.cache Provides classes that help implementing a cache for the egothor project. 
org.egothor.core This package concentrates the core data objects and interfaces. 
org.egothor.core.memory This package contains an implementation of barrels in memory. 
org.egothor.core.query This package contains objects that represent a structure of a query in our inner and binary form, readers of a barrel and a result queue. 
org.egothor.dir This package defines objects of distributed IR layer. 
org.egothor.duplicity.algorithm This package contains top-level classes that implement the duplicity checking algorithm. 
org.egothor.duplicity.visualization This package contains classes implementing the visualization of the duplicities found in a document by the duplicity checking algorithm. 
org.egothor.html This package contains a specific implementation of core objects for HTML with "home" and "content" support. 
org.egothor.parser   
org.egothor.parser.filter This package defines objects that filter tokens. 
org.egothor.parser.plain This package defines JavaCC parser for a plain text. 
org.egothor.query This package contains objects that represent a structure of a query in our inner form. 
org.egothor.query.parser This package defines JavaCC parser of user's query. 
org.egothor.query.runner This package defines a machinery that navigates the rider during query execution in the Vector model
org.egothor.query.runner.enhanced This package defines a machinery that navigates the rider during query execution in any model
org.egothor.text This package contains support of some textual routines/processes. 
 

Uses of Token in org.egothor.cache
 

Methods in org.egothor.cache that return Token
 Token[] CachedQuery.getTokens()
          Gets the tokens that are forming the query
 

Constructors in org.egothor.cache with parameters of type Token
CachedQuery(Query query, Token[] tokens, int offset, int length, long max_hits2scan, double pg_rerank)
          Constructor for the CachedQuery object.
 

Uses of Token in org.egothor.core
 

Fields in org.egothor.core with type parameters of type Token
 Sequence<Token> Filter.prev
          The Tokenizer used by Filter.
 

Methods in org.egothor.core that return Token
 Token Filter.action(Token t)
          Used for changing tokens of the input tokenizer.
 Token Token.newText(java.lang.String name, java.lang.String text)
          Clones this objects with a new name and text.
 Token Filter.next()
          The next token of input tokenizer is modified by Filter.action(org.egothor.core.Token) and the product is also the product of this method.
 Token[] QueryResponse.queryTokens()
          Return an array of all tokens recognized in a query.
 

Methods in org.egothor.core that return types with arguments of type Token
 Sequence<Token> Filter.getPrevTokenizer()
          Return the tokenizer this filter reads from.
 Sequence<Token> DocumentData.words(boolean readlinx, boolean readilinx, boolean lowercase, boolean phonetics, HTMLField.Diacritics diacritics, boolean paragraphs, boolean paragraphsKeepPunctuation, java.lang.String encoding)
           
 

Methods in org.egothor.core with parameters of type Token
 Token Filter.action(Token t)
          Used for changing tokens of the input tokenizer.
 

Method parameters in org.egothor.core with type arguments of type Token
 void Filter.setPrevTokenizer(Sequence<Token> prev)
          Set the tokenizer this filter reads from.
 

Constructors in org.egothor.core with parameters of type Token
QueryResponse(int offset, long wouldBe, long positives, Sequence<Hit> e, Token[] queryTokens, int enum_len, Query adaptedQuery)
          Constructor for the QueryResponse object.
 

Constructor parameters in org.egothor.core with type arguments of type Token
Filter(Sequence<Token> prev)
          Constructor for the Filter object.
 

Uses of Token in org.egothor.core.memory
 

Methods in org.egothor.core.memory that return types with arguments of type Token
 Sequence<Token> FTField.filteredWords()
          Return an enumeration of the terms in the field filtered by filters.
 Sequence<Token> FTField.words()
          Return an enumeration of the terms in the field.
 

Uses of Token in org.egothor.core.query
 

Method parameters in org.egothor.core.query with type arguments of type Token
 void Query.addTerms(java.util.HashSet<Token> to)
          Add the given HashSet of terms to this Query.
 

Uses of Token in org.egothor.dir
 

Methods in org.egothor.dir with parameters of type Token
abstract  DocumentData Group.expandDocMetadata(DocumentData ofBarrel, Token[] interest)
          Retrives the document data, but restrict the data block to the part relevant to some tokens of our interest.
abstract  DocumentData Tanker.expandDocMetadata(DocumentData ofBarrel, Token[] interest)
           
 DocumentData TankerImpl.expandDocMetadata(DocumentData ofBarrel, Token[] interest)
           
 DocumentData TankerImplSecure.expandDocMetadata(DocumentData ofBarrel, Token[] interest)
          Deprecated. 
 DocumentData TankerImplSecure.expandDocMetadataSecure(DocumentData ofBarrel, Token[] interest)
          Standard exapnding of doc metadata, but using multithreaded safe way.
 

Method parameters in org.egothor.dir with type arguments of type Token
 CWI Group.getCWI(java.util.HashSet<Token> terms)
          Return the subset of CWI for the given subset of terms.
 CWI TankerImplSecure.getCWI(java.util.HashSet<Token> terms)
           
 

Uses of Token in org.egothor.duplicity.algorithm
 

Method parameters in org.egothor.duplicity.algorithm with type arguments of type Token
 void PermutatedMinsFiller.computeDocumentMins(DocumentPermutatedMins result, Sequence<Token> terms, long documentUID, int documentDBRevision)
          Computes the permutated mins values for given sequence of tokens of a document and fills it into the result under the identificator documentID.
 

Uses of Token in org.egothor.duplicity.visualization
 

Methods in org.egothor.duplicity.visualization that return types with arguments of type Token
static java.util.List<java.util.List<Token>> DocumentDuplicities.getDocumentUnits(Sequence<Token> words)
          Takes the sequence of document words and depending on the Constants.CHECK_DUPLICITY_LEVEL splits it to the appropriate text units - documents, paragraphs or sentences.
 

Method parameters in org.egothor.duplicity.visualization with type arguments of type Token
static java.util.List<java.util.List<Token>> DocumentDuplicities.getDocumentUnits(Sequence<Token> words)
          Takes the sequence of document words and depending on the Constants.CHECK_DUPLICITY_LEVEL splits it to the appropriate text units - documents, paragraphs or sentences.
 

Uses of Token in org.egothor.html
 

Methods in org.egothor.html that return types with arguments of type Token
 Sequence<Token> HTMLField.words()
           
 

Method parameters in org.egothor.html with type arguments of type Token
 void HTMLField.setAppendix(Sequence<Token> appendix)
           
 

Uses of Token in org.egothor.parser
 

Methods in org.egothor.parser that return Token
 Token Strings2Tokens.next()
           
 

Uses of Token in org.egothor.parser.filter
 

Methods in org.egothor.parser.filter that return Token
 Token DupWithoutDiacritics.action(Token t)
          If the name/type of the token is <WORD> then transform the text of the token to lower case.
 Token LowerCase.action(Token t)
          If the name/type of the token is not <EMAIL/PUNCT/NUM> then transform the text of the token to lower case.
 Token Stemmer.action(Token t)
          A simple stemming algorithm which works as follows:
 Token DupWithoutDiacritics.next()
          Return the next token.
 Token Grammer.next()
          Return the next token.
 Token ParagraphFilter.next()
          Return the next token.
 Token ParagraphPunctFilter.next()
          Return the next token.
 Token Phonetics.next()
          Return the next token.
 Token PunctFilter.next()
          Return the next token.
 Token RemoveDiacritics.next()
          If the name/type of the token is <WORD> then transform the text of the token to lower case.
 Token StopFilter.next()
          Return the next token.
 Token WordNGrammer.next()
          Return the next token.
 

Methods in org.egothor.parser.filter with parameters of type Token
 Token DupWithoutDiacritics.action(Token t)
          If the name/type of the token is <WORD> then transform the text of the token to lower case.
 Token LowerCase.action(Token t)
          If the name/type of the token is not <EMAIL/PUNCT/NUM> then transform the text of the token to lower case.
 Token Stemmer.action(Token t)
          A simple stemming algorithm which works as follows:
static boolean ParagraphPunctFilter.isParagraphDelimiter(Token t)
          Test whether a token is a paragraph delimiter.
static boolean ParagraphPunctFilter.isPunctuation(Token t)
          Test whether a token is a punctuation (mark).
 boolean PunctFilter.isPunctuation(Token t)
          Test whether a token is a punctuation (mark) or it can be ignored.
abstract  boolean StopFilter.isStoppedToken(Token t)
          Test whether a token should be processed or ignored.
 

Constructor parameters in org.egothor.parser.filter with type arguments of type Token
DupWithoutDiacritics(Sequence<Token> prev)
          Constructor for the Diacritics object.
Grammer(Sequence<Token> arg0)
          Constructor for the Grammer object.
LowerCase(Sequence<Token> prev)
          Constructor for the LowerCase object.
LowerCase(Sequence<Token> prev, java.util.Locale locale)
          Construct a LowerCase object using the given localization setting.
ParagraphFilter(Sequence<Token> prev)
          Constructor for the ParagraphPunctFilter object
ParagraphPunctFilter(Sequence<Token> prev)
          Constructor for the ParagraphPunctFilter object
Phonetics(Sequence<Token> arg0)
          Constructor for the Phonetics object.
PunctFilter(Sequence<Token> arg0)
          Constructor for the PunctFilter object
RemoveDiacritics(Sequence<Token> prev)
          Constructor for the Diacritics object.
Stemmer(Sequence<Token> prev, Trie stemmer)
          Construct a Stem object using the given stemmer table.
StopFilter(Sequence<Token> arg0)
          Constructor for the StopFilter object.
WordNGrammer(Sequence<Token> prev)
          Constructor for the WordNGrammer object
 

Uses of Token in org.egothor.parser.plain
 

Methods in org.egothor.parser.plain that return Token
 Token Plain.next()
          Return the next Token.
 Token Simple.next()
          Returns the next token in the stream, or null at EOS.
 

Uses of Token in org.egothor.query
 

Methods in org.egothor.query with parameters of type Token
static int Configuration.defaultBoost(Token tok)
          Description of the Method
static boolean Configuration.isControlToken(Token tok)
          Is this a control token which is not excluded when it has a low idf?
 

Method parameters in org.egothor.query with type arguments of type Token
 void QAnd.addTerms(java.util.HashSet<Token> to)
          Add the given HashSet of terms to this Query.
 void QGroup.addTerms(java.util.HashSet<Token> to)
          Add the given HashSet of terms.
 void QNot.addTerms(java.util.HashSet<Token> to)
          Add the given HashSet of terms to this Query.
 void QOr.addTerms(java.util.HashSet<Token> to)
          Add the given HashSet of terms to this Query.
 void QPhrase.addTerms(java.util.HashSet<Token> to)
          Add the given HashSet of terms to this Query.
 void QProx.addTerms(java.util.HashSet<Token> to)
          Add the given HashSet of terms to this Query.
 void QTerm.addTerms(java.util.HashSet<Token> to)
          Add the given HashSet of terms to this Query.
 

Constructors in org.egothor.query with parameters of type Token
QTerm(Token token, java.lang.String field)
          Constructor for the QTerm object.
QTerm(Token token, java.lang.String field, double idf, boolean req, boolean proh, int boost, int lowerBound, int upperBound)
          Constructor for the QTerm object.
 

Uses of Token in org.egothor.query.parser
 

Methods in org.egothor.query.parser that return Token
 Token Parser.next()
          Returns the next token in the stream, or null at EOS.
 

Uses of Token in org.egothor.query.runner
 

Constructors in org.egothor.query.runner with parameters of type Token
TermRunner(double idf, Rider r, java.lang.String field, Token token, int boost, boolean req, boolean proh)
          Constructor for the TermRunner object
TermRunner(IListWasher washer, double idf, Rider r, java.lang.String field, Token token, int boost, boolean req, boolean proh)
          Constructor for the TermRunner object.
 

Uses of Token in org.egothor.query.runner.enhanced
 

Constructors in org.egothor.query.runner.enhanced with parameters of type Token
TermRunner(int model, double idf, Rider r, java.lang.String field, Token token, int boost, boolean req, boolean proh)
          Constructor for the TermRunner object
TermRunner(int model, IListWasher washer, double idf, Rider r, java.lang.String field, Token token, int boost, boolean req, boolean proh)
          Constructor for the TermRunner object.
 

Uses of Token in org.egothor.text
 

Methods in org.egothor.text that return Token
 Token Generator.next()
           
 

Methods in org.egothor.text with parameters of type Token
abstract  boolean Snipper.matches(Token w)
          Description of the Method
 boolean SnipperOfStrings.matches(Token w)
          Description of the Method
 boolean SnipperOfTokens.matches(Token w)
          Description of the Method
 

Method parameters in org.egothor.text with type arguments of type Token
 java.lang.String Snipper.filter(Sequence<Token> tokens)
          Description of the Method
 

Constructors in org.egothor.text with parameters of type Token
SnipperOfTokens(Token[] word, boolean htmlAware, java.lang.String startHit, java.lang.String endHit)
          Constructor for the Snipper object