org.egothor.core
Class Token

java.lang.Object
  org.egothor.core.Token

public class Token
extends java.lang.Object

This class represents a token of an input document. It contains:

• the name of the token
• part of text of the document that is represented by the token
• location of the token in the document

Author:

Leo Galambos

Field Summary

int	colE Ending column in the document.
int	colS Starting column in the document.
int	lineE Ending line in the document.
int	lineS Starting line in the document.
java.lang.String	name Name of token.
short	paragraph Ordinal number of paragraph, in which the token occured.
int	reloffset
int	sentence Initialized by PunctFilter and ParagraphPunctFilter
short	sentenceInParagraph Ordinal number of sentence whitin the paragraph, in which the token occured.
java.lang.String	text Text that is represented by this token.
int	weight

Constructor Summary

Token(java.lang.String aname, java.lang.String atext, int ls, int le, int cs, int ce)
Constructor for the Token object

Token(java.lang.String aname, java.lang.String atext, int ls, int le, int cs, int ce, int reloff)
Constructor for the Token object

Method Summary

boolean	equals(java.lang.Object obj) Description of the Method
TextUnitID	getTextUnit(long uid, int revision) Returns the identificator of the text unit to which the token falls into.
int	hashCode() Description of the Method
Token	newText(java.lang.String name, java.lang.String text) Clones this objects with a new name and text.
java.lang.String	toString() Convert a Token to a String.Returns the string that represents a token of a given name and text.
static java.lang.String	toString(java.lang.String name, java.lang.String text) Convert a Token to a String.Returns the string that represents a token of a given name and text.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

text

public java.lang.String text

Text that is represented by this token.

name

public java.lang.String name

Name of token. For example it could be: <WORD>, <NUM> .

lineS

public int lineS

Starting line in the document.

lineE

public int lineE

Ending line in the document.

colS

public int colS

Starting column in the document.

colE

public int colE

Ending column in the document.

weight

public int weight

reloffset

public int reloffset

sentence

public int sentence

Initialized by PunctFilter and ParagraphPunctFilter

paragraph

public short paragraph

Ordinal number of paragraph, in which the token occured. Initialized by ParagraphPunctFilter

sentenceInParagraph

public short sentenceInParagraph

Ordinal number of sentence whitin the paragraph, in which the token occured. Initialized by ParagraphPunctFilter

Constructor Detail

Token

public Token(java.lang.String aname,
             java.lang.String atext,
             int ls,
             int le,
             int cs,
             int ce)

Constructor for the Token object

Parameters:

aname - this object's name

atext - the text associated with the token

ls - the starting line in the document

le - the ending line in the document

cs - the starting column in the document.

ce - the ending column in the document.

Token

public Token(java.lang.String aname,
             java.lang.String atext,
             int ls,
             int le,
             int cs,
             int ce,
             int reloff)

Constructor for the Token object

Parameters:

aname - this object's name

atext - the text associated with the token

ls - the starting line in the document

le - the ending line in the document

cs - the starting column in the document

ce - the ending column in the document

reloff - the relative offset of this token

Method Detail

toString

public static java.lang.String toString(java.lang.String name,
                                        java.lang.String text)

Convert a Token to a String.Returns the string that represents a token of a given name and text. This implementation returns a concatenation of the name and text variable values.

Parameters:

name - the name

text - the text

Returns:

name+text

hashCode

public int hashCode()

Description of the Method

Overrides:

hashCode in class java.lang.Object

Returns:

Description of the Return Value

equals

public boolean equals(java.lang.Object obj)

Description of the Method

Overrides:

equals in class java.lang.Object

Parameters:

obj - Description of the Parameter

Returns:

Description of the Return Value

toString

public java.lang.String toString()

Convert a Token to a String.Returns the string that represents a token of a given name and text. This implementation returns a concatenation of the name and text variable values.

Overrides:

toString in class java.lang.Object

Returns:

name+text

newText

public Token newText(java.lang.String name,
                     java.lang.String text)

Clones this objects with a new name and text. Copies all atributes of token, including reloffset.

Parameters:

name - Description of the Parameter

text - Description of the Parameter

Returns:

Description of the Return Value

getTextUnit

public TextUnitID getTextUnit(long uid,
                              int revision)

Returns the identificator of the text unit to which the token falls into. Depends on the duplicity checking level.

Parameters:

uid - id of the document to which the token falls into

revision - revision number of the document to which the token falls into

Returns:

one of TextUnitID child classes - DocumentUnitID, ParagraphUnitID or SentenceUnitID

See Also:

Constants.CHECK_DUPLICITY_LEVEL