Zend_Search_Lucene is a general purpose text search engine written entirely in PHP 5. Since it stores its index on the filesystem and does not require a database server, it can add search capabilities to almost any PHP-driven website. Zend_Search_Lucene supports the following features:

Zend_Search_Lucene was derived from the Apache Lucene project. For more information on Lucene, visit http://lucene.apache.org/java/docs/.

Zend_Search_Lucene operates with documents as atomic subjects for indexing. A document is divided into named fields, and fields have content that can be searched.

A document is represented by the Zend_Search_Lucene_Document object, and this object contains Zend_Search_Lucene_Field objects that represent the fields.

It is important to note that any kind of information can be added to the index. Application-specific information or metadata can be stored in the document fields, and later retrieved with the document during search.

It is the responsibility of your application to control the indexer. This means that data can be indexed from any source that is accessible by your application. For example, this could be the filesystem, a database, an HTML form, etc.

Zend_Search_Lucene_Field class provides several static methods to create fields with different characteristics:

<?php
$doc = new Zend_Search_Lucene_Document();

// Field is not tokenized, but is indexed and stored within the index.
// Stored fields can be retrived from the index.
$doc->addField(Zend_Search_Lucene_Field::Keyword('doctype',
                                                 'autogenerated'));

// Field is not tokenized nor indexed, but is stored in the index.
$doc->addField(Zend_Search_Lucene_Field::UnIndexed('created',
                                                   time()));

// Binary String valued Field that is not tokenized nor indexed,
// but is stored in the index.
$doc->addField(Zend_Search_Lucene_Field::Binary('icon',
                                                $iconData));

// Field is tokenized and indexed, and is stored in the index.
$doc->addField(Zend_Search_Lucene_Field::Text('annotation',
                                              'Document annotation text'));

// Field is tokenized and indexed, but that is not stored in the index.
$doc->addField(Zend_Search_Lucene_Field::UnStored('contents',
                                                  'My document content'));

?>

Each of these methods (excluding Zend_Search_Lucene_Field::Binary() method) has optional $encoding parameter. It specifies input data encoding.

Encoding may differ for different documents as well as for different fields within one document:

<?php
$doc = new Zend_Search_Lucene_Document();
$doc->addField(Zend_Search_Lucene_Field::Text('title', $title, 'iso-8859-1'));
$doc->addField(Zend_Search_Lucene_Field::UnStored('contents', $contents, 'utf-8'));
?>

If encoding parameter is omitted, then current locale is used at processing time. For example:

<?php
setlocale(LC_ALL, 'de_DE.iso-8859-1');
...
$doc->addField(Zend_Search_Lucene_Field::UnStored('contents', $contents));
?>

Fields are always stored and returned from index in UTF-8 encoding. Conversion to UTF-8 proceeds automatically.

Text analyzers (see below) may also convert text to some other encodings. Actually, default analyzer converts text to 'ASCII//TRANSLIT' encoding. Be care with this, such translation may depend on current locale.

Fields' names are defined only by your own choice.

Java Lucene uses "contents" field as a default field to search. Zend_Search_Lucene searches through all fiels by default, but it's also possible to change this behavior. See "Default search field" chapter for details.

Zend_Search_Lucene offers HTML parsing feature. Documents can be created directly from HTML file or string:

<?php
$doc = Zend_Search_Lucene_Document_Html::loadHTMLFile($filename);
$index->addDocument($doc);
...
$doc = Zend_Search_Lucene_Document_Html::loadHTML($htmlString);
$index->addDocument($doc);
?>

Zend_Search_Lucene_Document_Html class uses DOMDocument::loadHTML() and DOMDocument::loadHTMLFile() methods to parse source HTML, so it doesn't need HTML to be well formed or to be XHTML. From the other side it's sensitive to encoding mentioned in "meta http-equiv" header tag.

Zend_Search_Lucene_Document_Html class recognizes document title, body and document header meta tags.

'title' field is actually /html/head/title value. It's stored within index, tokenized and available for search through.

'body' field is actually body content. It doesn't include scripts, comments and tags' attributes.

loadHTML() and loadHTMLFile() methods of Zend_Search_Lucene_Document_Html class also have second optional argument. If it's set to true, then body content is also stored within index and can be retrieved from index. Body is only tokenized and indexed, but not stored by default.

Document header meta tags produce additional document fields. Field name is taken from 'name' attribute, 'content' attribute gives field value, which is tokenized, indexed and stored, so documents may be searched by their meta tags (for example, by keywords).

Parsed documents may be extended by user with any other field:

<?php
$doc = Zend_Search_Lucene_Document_Html::loadHTML($htmlString);
$doc->addField(Zend_Search_Lucene_Field::UnIndexed('created',
                                                   time()));
$doc->addField(Zend_Search_Lucene_Field::UnIndexed('created',
                                                   time()));
$doc->addField(Zend_Search_Lucene_Field::Text('annotation',
                                              'Document annotation text'));
$index->addDocument($doc);
?>

Document links are not included into generated document, but may be retrieved with Zend_Search_Lucene_Document_Html::getLinks() and Zend_Search_Lucene_Document_Html::getHeaderLinks() methods:

<?php
$doc = Zend_Search_Lucene_Document_Html::loadHTML($htmlString);
$linksArray = $doc->getLinks();
$headerLinksArray = $doc->getHeaderLinks();
?>