parallel indexing?
ok i have a large dataset to index and its taking hours to do so on a dual opteron /4GB server
so im after wasting few hours making scripts that partition the data and send datasets of 1000 rows at a time to my other servers (9servers in a cluster) > a script at the remote server make a lucene index out of the received data > and each server sends back the index it made
but im after finding out i have no way of merging these separate indexes into one large one + optimising it !
so yea how can separate lucene indexes be joined into one?
|