Hello,

I' m using compass 2.2.0 and I've faced with several problems...actually the one - indexes got corrupted, there are several different error messages, but I think there is one reason for it:

2010-12-21 23:05:41,240 INFO [STDOUT] 23:05:41,238 ERROR [SpringSyncTransaction] Exception occured when sync with transaction+
org.compass.core.transaction.TransactionException: Failed to execute callable; nested exception is org.compass.core.engine.SearchEngineException: Failed to add transaction index to sub index [step1]; nested exception is java.io.FileNotFoundException: /var/local/services/project/luceneindex/cache/index/step1/_eolp_1.del (No such file or directory)

2010-12-23 03:26:37,091 INFO [STDOUT] 03:26:37,090 ERROR [SpringSyncTransaction] Exception occured when sync with transaction+
org.compass.core.transaction.TransactionException: Failed to execute callable; nested exception is org.compass.core.engine.SearchEngineException: Failed to add transaction index to sub index [step1]; nested exception is java.io.IOException: read past EOF
org.compass.core.engine.SearchEngineException: Failed to add transaction index to sub index [step1]; nested exception is java.io.IOException: read past EOF
java.io.IOException: read past EOF
at org.apache.lucene.store.BufferedIndexInput.refill( BufferedIndexInput.java:151)

2010-12-23 17:58:12,253 INFO [STDOUT] 17:58:12,252 ERROR [ErrorMapper] Request failed
org.compass.core.engine.SearchEngineException: Failed to open writer for sub index [step2]; nested exception is org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock@/var/local/services/pro...ea4-write.lock

I can provide log files if needed.

How indexing is done in application:


private static final Object compassLock = new Object();

@Autowired
private Compass compass;

private CompassTemplate compassTemplate;

....
public void index(LogXmlParser parser, Common common,
Command command) {
.....
synchronized (compassLock) {
updateStates(logCommon);
compassTemplate.save(xmlObject);
}

}

public void updateStates(final Common common) {
updateProperty(common, "state", common.getState());
}

public void updateProperty(final Common common, final String propertyName, final Object value){
compassTemplate.execute(new CompassCallback<Void>() {+
@Override
public Void doInCompass(CompassSession session)
throws CompassException {

CompassQueryBuilder queryBuilder = session.queryBuilder();
CompassHits hits = queryBuilder.queryString(
"id:" + common.getId()).toQuery().hits();
for (CompassHit hit : hits) {
Resource resource = hit.getResource();
resource.setProperty(propertyName, value);
session.save(resource);
}
return null;
}
});
}

The index() method itself is called from other object's method, marked with Spring @Transactional annotation. This method is called rother often, several times per minute.

*My configuration is:*
compass.transaction.lockTimeout=30,
compass.engine.cacheIntervalInvalidation=10000, compass.engine.store.jdbc.connection.provider.clas s=org.compass.core.lucene.engine.store.jdbc.Extern alDataSourceProvider, compass.transaction.clearCacheOnCommit=false,
compass.transaction.factory=org.compass.spring.tra nsaction.SpringSyncTransactionFactory
compass.transaction.lockPollInterval=100,
compass.engine.store.jdbc.lockType=org.apache.luce ne.store.jdbc.lock.SelectForUpdateLock,
compass.engine.localCache.__default__.connection=p rojectindex/cache


My suggestion was that indexes become corrupted because of uncoordinated multithreaded access to index files. I don't have access to server myself, so I asked to stop server, delete all files and folders (as indexes are already broken) from directory /var/local/services/project/luceneindex/, then I made the whole index() and updateProperty() methods synchronized, this new version was deployed, and server started again. I was told there were FileNotFoundExceptions anyway :(

I measured how long index method runs, it starts from 60-70 ms, then grows up to 2-3 sec.


Does somebody faced with similiar problems? What is the recomended strategy now - experimenting with configuration (setting different timeout variables to higher values, disabling schedulled optimizer ?), something else?


Thanks in advance.