Unable to find Documents by numeric values

kolbi · #1 (**permalink**) 09-21-2007, 01:49 PM

Hello there,

I just set up the Zend Lucene Search Engine (ZF 1.0.1) and want to use it for indexing a huge number of userdata. I already have a unique numeric ID for each user, so I try to reuse that for retrieving the document for a certain user (in order to delete and re-add it).

The problem is now: Lucene does not let me search for numeric stuff.
I tried all this:

1. Using the Zend_Search_Lucene_Field::Text function when adding fields, instead of Zend_Search_Lucene_Field::Keyword,
2. Instead of adding ...Keyword('id', $userId) I used a custom fieldname

But - in all cases, what ever I do , I am facing the Issues that
1. Lucene overwrites the id with it's own if I write ...::Keyword('id', $userId)
2. Disregarding if I search in the predefined id column or in my custom 'userid' column, - when I use numeric values for the ID, Lucene does not find any hit.

So, it seems - in this Version of Lucene - it overwrites the Document-id ALWAYS with it's own id, and searching for numeric values is simply not possible.

My Workaround was now, to convert the numeric ID into some String (e.g. a hash) - then it works perfectly - but it's not a very clean solution in my opinion.

So, who can tell my what I'm doin wrong?

Here's the code snippet how I'm indexing:

Code:

$index = new Zend_Search_Lucene($location);
$doc = new Zend_Search_Lucene_Document();
$doc->addField(Zend_Search_Lucene_Field::Text('id' , $userId, 'utf-8'));
$doc->addField(Zend_Search_Lucene_Field::Keyword('userid' ,$userId, 'utf-8'));
$index->addDocument($doc);
$index->commit();

And this is how I try to search:

Code:

// Search for lucene internal ID which has been overwritten but does not work anyway
$hits = $index->find('id:113');

// Search for my Custom user id which also does not deliver any hit
$hits = $index->find('userid:113');

Btw: I already approved, all the data is stored correctly in the index - at least I can verify all the data when I display the content of all documents in the index.

Thank you in advance!

kolbi · #2 (**permalink**) 09-26-2007, 01:29 PM

Okay, I got the answer myself: Just do not use the default analyzer.

dhughes · #3 (**permalink**) 12-20-2007, 09:04 PM

Could you post what analyzer you found that works for numeric values?

abatardi · #4 (**permalink**) 01-04-2008, 12:28 AM

I am having problems with this myself.. I have tried a few different analyzers including UTF8 one and the "TextNum" ones with no luck. I have tried doing the numeric field as a Text/Keyword/UnStored also.

Any help is appreciated...

- aba

captainbeef · #5 (**permalink**) 01-18-2008, 02:25 AM

I would also be interested to see what analyzer allows integer searches.

FYI, I used md5 to convert my database key to a unique searchable entry for each index record:

PHP Code:


			
$index = Zend_Search_Lucene::open($index_path);

$uid = md5($database_id);

$term = new Zend_Search_Lucene_Index_Term($uid, 'uid');

$uid = new Zend_Search_Lucene_Search_Query_Term($term);

$hits = $index->find($uid);

...

ohsmitt · #6 (**permalink**) 08-15-2008, 03:49 AM

Ok, after messing with this for a long time I figured out that other people can't search for numeric data. Here is a solution:

PHP Code:


			
$index = Zend_Search_Lucene::open();

$pathTerm  = new Zend_Search_Lucene_Index_Term('$numericData', 'fieldname');

$pathQuery = new Zend_Search_Lucene_Search_Query_Term($pathTerm);

$query = new Zend_Search_Lucene_Search_Query_Boolean();

$query->addSubquery($pathQuery, true /* required */);

$hits = $index->find($query);

Note: fieldname can be omitted but obviously if you are looking to delete a Lucene Doc to update it you want to search for the exact field.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode