org.egothor.dir
Class TankerImplSecure

java.lang.Object
  org.egothor.dir.Group
      org.egothor.dir.Tanker
          org.egothor.dir.TankerImpl
              org.egothor.dir.TankerImplSecure

All Implemented Interfaces:

Resolver<CachedQuery,CachedResult>, Barrel, BarrelWriter

Direct Known Subclasses:

TankerImplDuplicityChecker

public class TankerImplSecure
extends TankerImpl

This object encapsulates standard TankerImpl and adds multithread safe behavior. This class is synchronized with some exceptions:

1. method elementsSecure() - should be synchronized by user as it returns a wrapped iterator.

2. method openSecure() - should be synchronized by user. It is necessary to use its pair method close() in the finally block. Example follows:

    synchronized (tanker) {
       try {
          tanker.openSecure();
          ...
       } finally {
          tanker.close();
       }
    }
    

Author:

Jakub Podhorny

Field Summary

protected DataRepository	localDocDB local repository of documents
protected TankerImpl	localTanker local place where all barrels are appended and other changes are logged.

Fields inherited from class org.egothor.dir.Group

cache

Constructor Summary

TankerImplSecure()

Method Summary

void	append(BarrelReader barrelReader) Appends a reader to this tanker.
void	close() Closes this tanker.
void	commit() Commits all the changes.
protected void	commitEmit(int recordsx4, long[] record4)
java.util.HashMap<java.lang.Long,java.lang.Boolean>	commitWithResults() Performes a commit and returns information about the whole process.
long	deleted() Total number of deleted documents in this group.
void	destroy() Description of the Method
Sequence<Barrel>	elements() Deprecated.
Sequence<Barrel>	elementsSecure() Gets elements of this tanker, done in multithreaded safe way.
Sequence<? extends IListMetadata>	expand(java.lang.String expr) Return the enumeration of all IListMetadata objects which match the expression.
DocumentData	expandDocMetadata(DocumentData ofBarrel, Token[] interest) Deprecated.
DocumentData	expandDocMetadataSecure(DocumentData ofBarrel, Token[] interest) Standard exapnding of doc metadata, but using multithreaded safe way.
void	forEach(Task<Barrel> task) Put the given task to all active (inner) barrels.
Bitmap	getBitmap(java.lang.String label) Return the Bitmap of a given label.
CWI	getCWI(java.util.HashSet<Token> terms) Return the subset of CWI for the given subset of terms.
IListMetadata	getIListMeta(java.lang.String term) Return the inverted list metadata for the given term.
DocumentData	getMeta(long uid) Return the metadata for the document with the given unique identifier.
int	getRootVersionID()
void	initializeTankerSecure(java.lang.String indexDir, DataRepository repo, boolean useBerkeleyGP, boolean MTE, int capacity, int mergeFactor, java.lang.String lockServerConfigFilename, int indexConstPeriodOfTime) Initializations of the tanker.
void	initializeTankerSecure(java.lang.String indexDir, DataRepository repo, boolean useBerkeleyGP, boolean MTE, int capacity, java.lang.String lockServerConfigFilename, int indexConstPeriodOfTime) Initializations of the tanker.
boolean	isDirty() Test whether this Tanker has been modified.
void	iterate(Request<Barrel> task) Iterate through the active (inner) barrels in predefined and constant order and try to give them the request until one of them fulfills it.
java.util.Properties	loadState() Read properties of the Tanker
BarrelReader	open() Deprecated.
IListReader	openIList(java.lang.String term, boolean clean) Deprecated.
IListReader	openIListSecure(java.lang.String term, boolean clean) Opens IList reader using secure multithreading safe way.
BarrelReader	openSecure() Opens a reader for this tanker, using secure multithreading safe way.
void	optimize() Optimize this tanker.
QueryResponse	query(int offset, int length, int p, java.lang.String query, long max_hits2scan, double pg_rerank, java.lang.String... controls) Deprecated.
void	query(Query q, ResultList result) Deprecated.
QueryResponse	querySecure(int offset, int length, int p, java.lang.String query, long max_hits2scan, double pg_rerank, java.lang.String... controls) Standard query, but using multithreaded safe way.
void	querySecure(Query q, ResultList result) Standard query, but using multithreaded safe way.
void	reloadIndex(int indexConstPeriodOfTime) Relaods the whole index.
boolean	removeDoc(long id) Deprecated.
void	removeDoc(long id, int revision)
void	setBitmap(java.lang.String label, Bitmap bitmap) Try to set the Bitmap of a given label.
long	size() Return the number of all documents in this Barrel (including the deleted ones).

Methods inherited from class org.egothor.dir.TankerImpl

commitEmitEnd, initializeTanker, initializeTanker, removeDoc

Methods inherited from class org.egothor.dir.Tanker

finalize, initialize

Methods inherited from class org.egothor.dir.Group

getCache, initialize, initializeCache, initializeCache, initializeCache, initializeCache, resolve

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

localTanker

protected TankerImpl localTanker

local place where all barrels are appended and other changes are logged. It is processed to the global in commit.

localDocDB

protected DataRepository localDocDB

local repository of documents

Constructor Detail

TankerImplSecure

public TankerImplSecure()

Method Detail

initializeTankerSecure

public void initializeTankerSecure(java.lang.String indexDir,
                                   DataRepository repo,
                                   boolean useBerkeleyGP,
                                   boolean MTE,
                                   int capacity,
                                   java.lang.String lockServerConfigFilename,
                                   int indexConstPeriodOfTime)
                            throws java.io.IOException,
                                   com.sleepycat.db.DatabaseException

Initializations of the tanker. Parameter lockServerConfigFilename specifies pathname to a config file for lock server. If it is null, then file locking is used. It must be in java.util.Properties format.

Parameters:

indexDir -

useBerkeleyDocDB -

useBerkeleyGP -

MTE -

capacity -

lockServerConfigFilename -

indexConstPeriodOfTime -

Throws:

java.io.IOException

com.sleepycat.db.DatabaseException

initializeTankerSecure

public void initializeTankerSecure(java.lang.String indexDir,
                                   DataRepository repo,
                                   boolean useBerkeleyGP,
                                   boolean MTE,
                                   int capacity,
                                   int mergeFactor,
                                   java.lang.String lockServerConfigFilename,
                                   int indexConstPeriodOfTime)
                            throws java.io.IOException,
                                   com.sleepycat.db.DatabaseException

Initializations of the tanker. Parameter lockServerConfigFilename specifies pathname to a config file for lock server. If it is null, then file locking is used. It must be in java.util.Properties format.

Parameters:

indexDir -

useBerkeleyDocDB -

useBerkeleyGP -

MTE -

capacity -

mergeFactor -

lockServerConfigFilename -

indexConstPeriodOfTime -

Throws:

java.io.IOException

com.sleepycat.db.DatabaseException

open

@Deprecated
public BarrelReader open()

Deprecated.

Description copied from class: Group

Open this Group.

Specified by:

open in interface Barrel

Overrides:

open in class Group

Returns:

a BarrelReader which can read this Group

openSecure

public BarrelReader openSecure()
                        throws EgothorIndexConstExpiredException

Opens a reader for this tanker, using secure multithreading safe way. If index constancy expired and the index really changed, an EgothorIndexConstExpiredException is thrown. If only the constancy expired, but the index has not changed, it prolongs constancy interval and continues in work.

In case of muliple thread sharing one instance of the tanker this method should be synchronized by user as follows. It is necessary to use its pair method close() in the finally block! :

 synchronized (tanker) {
    try {
       tanker.openSecure();
       ...
    } finally {
       tanker.close();
    }
 }
 

Returns:

Reader of the barrel (the whole tanker)

Throws:

EgothorIndexConstExpiredException - If index constancy expired AND index has changed.

openIList

@Deprecated
public IListReader openIList(java.lang.String term,
                                        boolean clean)

Deprecated.

Description copied from class: Group

Open the inverted list constructed for the given term for reading.

Specified by:

openIList in interface Barrel

Overrides:

openIList in class Group

Parameters:

term - the term for which the inverted list was constructed

clean - whether to remove all the items denoted as deleted

Returns:

an IListReader which can read the inverted list

openIListSecure

public IListReader openIListSecure(java.lang.String term,
                                   boolean clean)
                            throws EgothorIndexConstExpiredException

Opens IList reader using secure multithreading safe way. If index constancy expired and the index really changed, an EgothorIndexConstExpiredException is thrown. If only the constancy expired, but the index has not changed, it prolongs constancy interval and continues in work.

Parameters:

term -

clean -

Returns:

Throws:

EgothorIndexConstExpiredException - If index constancy expired AND index has changed.

query

@Deprecated
public QueryResponse query(int offset,
                                      int length,
                                      int p,
                                      java.lang.String query,
                                      long max_hits2scan,
                                      double pg_rerank,
                                      java.lang.String... controls)
                    throws ParseException

Deprecated.

Description copied from class: Group

Return a QueryResponse to a user query.

Overrides:

query in class Group

Parameters:

offset - where to start reading the result list

length - read the result list up to here

p - power parameter in evaluation formula @{link org.egothor.Constants#DEFAULTMODEL}

query - the user's query

max_hits2scan - maximum positive hits scanned, if achieved then total number of hits is only guessed

pg_rerank - pagerank booster @{link org.egothor.core.query.ResultList}

controls - control BITMAP tokens (String)

Returns:

a QueryResponse

Throws:

ParseException - if there is difficulty parsing the query

querySecure

public QueryResponse querySecure(int offset,
                                 int length,
                                 int p,
                                 java.lang.String query,
                                 long max_hits2scan,
                                 double pg_rerank,
                                 java.lang.String... controls)
                          throws ParseException,
                                 EgothorIndexConstExpiredException

Standard query, but using multithreaded safe way. If index constancy expired and the index really changed, an EgothorIndexConstExpiredException is thrown. If only the constancy expired, but the index has not changed, it prolongs constancy interval and continues in work.

Parameters:

offset -

length -

p -

query -

max_hits2scan -

pg_rerank -

controls -

Returns:

Throws:

ParseException

EgothorIndexConstExpiredException - If index constancy expired AND index has changed.

query

@Deprecated
public void query(Query q,
                             ResultList result)

Deprecated.

Description copied from class: Group

Submit the given query to this Group. If we want to replace local CWI with our global (for all barrels), we can do it in sub-class just before we call super's query.

Specified by:

query in interface Barrel

Overrides:

query in class Group

Parameters:

q - the query

result - the list to search

querySecure

public void querySecure(Query q,
                        ResultList result)
                 throws EgothorIndexConstExpiredException

Standard query, but using multithreaded safe way. If index constancy expired and the index really changed, an EgothorIndexConstExpiredException is thrown. If only the constancy expired, but the index has not changed, it prolongs constancy interval and continues in work.

Parameters:

q -

result -

Throws:

EgothorIndexConstExpiredException - If index constancy expired AND index has changed.

expandDocMetadata

@Deprecated
public DocumentData expandDocMetadata(DocumentData ofBarrel,
                                                 Token[] interest)

Deprecated.

Description copied from class: Group

Retrives the document data, but restrict the data block to the part relevant to some tokens of our interest. Full data block is retrieved if none interest is specified.

Overrides:

expandDocMetadata in class TankerImpl

expandDocMetadataSecure

public DocumentData expandDocMetadataSecure(DocumentData ofBarrel,
                                            Token[] interest)
                                     throws EgothorIndexConstExpiredException

Standard exapnding of doc metadata, but using multithreaded safe way. If index constancy expired and the index really changed, an EgothorIndexConstExpiredException is thrown. If only the constancy expired, but the index has not changed, it prolongs constancy interval and continues in work.

Parameters:

ofBarrel -

interest -

Returns:

Throws:

EgothorIndexConstExpiredException - If index constancy expired AND index has changed.

append

public void append(BarrelReader barrelReader)

Appends a reader to this tanker.

Specified by:

append in interface BarrelWriter

Overrides:

append in class TankerImpl

Parameters:

barrelReader - the Barrel to add

removeDoc

@Deprecated
public boolean removeDoc(long id)

Deprecated.

Description copied from class: Group

Remove the document with the given ID and return whether the operation was successful.

Specified by:

removeDoc in interface Barrel

Overrides:

removeDoc in class TankerImpl

Parameters:

id - the document to remove

Returns:

true if the removal was successful, false otherwise

removeDoc

public void removeDoc(long id,
                      int revision)

Parameters:

id - uid of the document

revision - revision of the document

commitWithResults

public java.util.HashMap<java.lang.Long,java.lang.Boolean> commitWithResults()

Performes a commit and returns information about the whole process.

Returns:

Mapping between document's uid and its commit result.

commit

public void commit()

Commits all the changes.

Specified by:

commit in interface Barrel

Overrides:

commit in class TankerImpl

Throws:

com.sleepycat.db.DatabaseException

java.io.IOException

reloadIndex

public void reloadIndex(int indexConstPeriodOfTime)
                 throws java.io.IOException

Relaods the whole index.

Parameters:

indexConstPeriodOfTime -

Throws:

java.io.IOException

commitEmit

protected void commitEmit(int recordsx4,
                          long[] record4)
                   throws java.io.IOException

Overrides:

commitEmit in class TankerImpl

Throws:

java.io.IOException

destroy

public void destroy()

Description copied from class: TankerImpl

Description of the Method

Specified by:

destroy in interface Barrel

Overrides:

destroy in class TankerImpl

close

public void close()

Closes this tanker.

In case of muliple thread sharing one instance of the tanker this method should be used everytime an openSecure() method is called and synchronized by user as follows:

 synchronized (tanker) {
    try {
       tanker.openSecure();
       ...
    } finally {
       tanker.close();
    }
 }
 

Specified by:

close in interface Barrel

Specified by:

close in interface BarrelWriter

Overrides:

close in class TankerImpl

optimize

public void optimize()

Optimize this tanker. It concerns only commited barrels.

Overrides:

optimize in class Tanker

elements

@Deprecated
public Sequence<Barrel> elements()

Deprecated.

Description copied from class: Tanker

Return the Barrels in this Tanker.

Overrides:

elements in class Tanker

Returns:

an Enumeration of Barrels

elementsSecure

public Sequence<Barrel> elementsSecure()
                                throws EgothorIndexConstExpiredException

Gets elements of this tanker, done in multithreaded safe way. If index constancy expired and the index really changed, an EgothorIndexConstExpiredException is thrown. If only the constancy expired, but the index has not changed, it prolongs constancy interval and continues in work.

In case of muliple thread sharing one instance of the tanker this method should be synchronized by user as follows, as it returns a wrapped iterator:

 synchronized (tanker) {
    Sequence<Barrel> elements = tanker.elementsSecure();
    ...
 }
 

Returns:

Throws:

EgothorIndexConstExpiredException - If index constancy expired AND index has changed.

loadState

public java.util.Properties loadState()
                               throws java.io.IOException

Description copied from class: TankerImpl

Read properties of the Tanker

Overrides:

loadState in class TankerImpl

Returns:

LastModified of the state file, 0L on any error

Throws:

java.io.IOException

getRootVersionID

public int getRootVersionID()

Overrides:

getRootVersionID in class Tanker

isDirty

public boolean isDirty()

Description copied from class: Tanker

Test whether this Tanker has been modified.

Overrides:

isDirty in class Tanker

Returns:

true if it has, false otherwise

deleted

public long deleted()

Description copied from class: Group

Total number of deleted documents in this group.

Specified by:

deleted in interface Barrel

Overrides:

deleted in class Group

Returns:

the number of deleted documents

expand

public Sequence<? extends IListMetadata> expand(java.lang.String expr)

Description copied from class: Group

Return the enumeration of all IListMetadata objects which match the expression.

Specified by:

expand in interface Barrel

Overrides:

expand in class Group

Parameters:

expr - the expression to match

Returns:

null when not supported

forEach

public void forEach(Task<Barrel> task)

Description copied from class: Group

Put the given task to all active (inner) barrels. It may be freely distributed as iterate does or in multithread mode. It is up to the implementation.

Overrides:

forEach in class Group

Parameters:

task - the task to perform

getBitmap

public Bitmap getBitmap(java.lang.String label)

Description copied from interface: Barrel

Return the Bitmap of a given label.

Specified by:

getBitmap in interface Barrel

Overrides:

getBitmap in class Group

Parameters:

label - the label of the requested Bitmap

Returns:

Bitmap of removed documents when label is null

getCWI

public CWI getCWI(java.util.HashSet<Token> terms)

Description copied from class: Group

Return the subset of CWI for the given subset of terms.

Overrides:

getCWI in class Group

Parameters:

terms - a HashSet containing the terms for which CWI should be returned

Returns:

null iff this group has no global CWI and everything is computed as local.

getIListMeta

public IListMetadata getIListMeta(java.lang.String term)

Description copied from class: Group

Return the inverted list metadata for the given term.

Specified by:

getIListMeta in interface Barrel

Overrides:

getIListMeta in class Group

Parameters:

term - the term

Returns:

an IListMetadata containing information about the inverted list constructed for the given term

getMeta

public DocumentData getMeta(long uid)

Description copied from class: Group

Return the metadata for the document with the given unique identifier.

Specified by:

getMeta in interface Barrel

Overrides:

getMeta in class Group

Parameters:

uid - the unique identifier of the document in question

Returns:

a DocMetadDocumentDataing the metadata of the document

iterate

public void iterate(Request<Barrel> task)

Description copied from class: Group

Iterate through the active (inner) barrels in predefined and constant order and try to give them the request until one of them fulfills it.

Overrides:

iterate in class Group

Parameters:

task - the task to perform

setBitmap

public void setBitmap(java.lang.String label,
                      Bitmap bitmap)

Description copied from interface: Barrel

Try to set the Bitmap of a given label.

Specified by:

setBitmap in interface Barrel

Overrides:

setBitmap in class Group

Parameters:

label - the label of the requested Bitmap

bitmap - Bitmap of removed documents when label is null

size

public long size()

Description copied from class: Group

Return the number of all documents in this Barrel (including the deleted ones).

Specified by:

size in interface Barrel

Overrides:

size in class Group

Returns:

the number of documents