Lucene.Net Tutorial with Lucene 2.9.2



Yet another quickstart tutorial; this time on Lucene.Net 2.9.2. If you're interested I've also now put together lucene.net video tutorial

You're going to need:

  1. VS2010 - Express edition will do. (we're going to be writing a Console Application)
  2. SQL Server 2008 - Express edition should do
  3. Lucene.Net 2.9.2 binary download
  4. Luke - See the insides of your Lucene Index
  5. Full Tutorial Source Download

Step 1 - Create a new Console Application

Then extract the Lucene.Net.dll from the Apache-Lucene.Net-2.9.2-incubating.bin.zip file into your lib folder.

You'll notice lots of other bits in  this zip file. Especially of interest to you later might be the stuff in the contrib folder. I might get to that in a later tutorial, but for now lets keep it simple.

Step 2 - Add a Reference to the Lucene.net.dll

Your references should look like this

Step 3 - Create a Document

Ok the next step is to create a simple Document with the appropriate fields which you'll want to search on. In our case we're going to create 3 cars. a Ford Fiesta, a Ford Focus and a Vauxhall Astra.

using System;
using System.IO;
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.Documents;
using Lucene.Net.Index;
using Lucene.Net.Store;
using Directory = Lucene.Net.Store.Directory;
using Version = Lucene.Net.Util.Version;

namespace LuceneNet.App
{
    class Program
    {
        static void Main(string[] args)
        {
            var fordFiesta = new Document();
            fordFiesta.Add(new Field("Id", "1", Field.Store.YES, Field.Index.NOT_ANALYZED));
            fordFiesta.Add(new Field("Make", "Ford", Field.Store.YES, Field.Index.ANALYZED));
            fordFiesta.Add(new Field("Model", "Fiesta", Field.Store.YES, Field.Index.ANALYZED));

            var fordFocus = new Document();
            fordFocus.Add(new Field("Id", "2", Field.Store.YES, Field.Index.NOT_ANALYZED));
            fordFocus.Add(new Field("Make", "Ford", Field.Store.YES, Field.Index.ANALYZED));
            fordFocus.Add(new Field("Model", "Focus", Field.Store.YES, Field.Index.ANALYZED));

            var vauxhallAstra = new Document();
            vauxhallAstra.Add(new Field("Id", "3", Field.Store.YES, Field.Index.NOT_ANALYZED));
            vauxhallAstra.Add(new Field("Make", "Vauxhall", Field.Store.YES, Field.Index.ANALYZED));
            vauxhallAstra.Add(new Field("Model", "Astra", Field.Store.YES, Field.Index.ANALYZED));



            Directory directory = FSDirectory.Open(new DirectoryInfo(Environment.CurrentDirectory + "\\LuceneIndex"));
            Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_29);


            var writer = new IndexWriter(directory, analyzer, true, IndexWriter.MaxFieldLength.LIMITED);
            writer.AddDocument(fordFiesta);
            writer.AddDocument(fordFocus);
            writer.AddDocument(vauxhallAstra);

            writer.Optimize();                        
            writer.Close();
            

            
        }
    }
}

In the code above you can see we firstly create a series of Document types. For each document we then define a series of Fields. You might think this looks similar to SQL where you create the table schema and each Document is like a row in the table! There's an important difference here, documents do not have a schema. If we'd wanted to I could of given the 2 Ford documents an extra field called SpecialFordField which wouldn't have existed at all on the Vauxhall documents.

Step 4 - Create a Directory

Now we need to sort out where we're going to store our index:

Directory directory = FSDirectory.Open(new DirectoryInfo(Environment.CurrentDirectory + "\\LuceneIndex"));

The above code creates a Directory. this is the place in which we store or write our Index to. In this case we use the FSDirectory.Open() factory method to create an Lucene Index on the FileSystem in a folder/directory called LuceneIndex. We could of equally created new RamDirectory() and just stored our index in RAM for super high performance but Lucene is so fast, that for the most part this is unecessary.

Step 5 - The Analyzer

Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_29);

We next create a new Analyzer which essentially turns our text into Tokens which are stored in our Index.

Step 6 - Writing the Documents to the Index

Finally we actually have to write the Documents to our Index

var writer = new IndexWriter(directory, analyzer, true, IndexWriter.MaxFieldLength.LIMITED);
            writer.AddDocument(fordFiesta);
            writer.AddDocument(fordFocus);
            writer.AddDocument(vauxhallAstra);

            writer.Optimize();                        
            writer.Close();

We define an IndexWriter telling it to use our Directory and Analyzer defined above. We then add all the documents to the IndexWriter call the writer.Optimize() which essentally does the semi-equivilent of a defrag on the index and then finally writer.Close(). At this point we have a perfectly good index to search.

Step 7 - Opening the Index for Searching

Ok now let's actually search our newly created index.

 

IndexReader indexReader = IndexReader.Open(directory, true);
            Searcher indexSearch = new IndexSearcher(indexReader);

 

We firstly need to open an IndexReader here we're passing in the Directory we created above and wrote our Documents into. An IndexReader simply reads the index it doesn't do the magic search part. For that we need a Searcher in our case a IndexSearcher which obviously needs to read our index internally.

Step 8 - Creating our Search Query

Prepare to search with a simple search query. The reality is that it's not quite as simple as typing in a Google query, though it's not far off.

 

var queryParser = new QueryParser(Version.LUCENE_29, "Make", analyzer);
            var query = queryParser.Parse("Ford");

 

Here we're creating a QueryParser and specifying that we want to search the Make field of our Index. It is totally possible to search more than one field but for simplicity I'm not going to do that here. We also use the same StandardAnalyzer we use above to apply the same tokenization to our query.

Finally we parse our raw query Ford. So hopefully we'll be able to find some Ford cars in our Index specifically in the Make field.

Step 9 - Performing the Search

 

Console.WriteLine("Searching for: " + query.ToString());
            TopDocs resultDocs = indexSearch.Search(query, indexReader.MaxDoc());
            
            Console.WriteLine("Results Found: " + resultDocs.totalHits);

 

Above we perform our search and return the TopDocs that match. Hopefully this should be 2 results if yours is anything like mine. There are indeed 2 Fords in our index.

Step 10 - Displaying our Search Results

Last part. we just quickly loop through and print out the cars that match.

 

var hits = resultDocs.scoreDocs;
            foreach (var hit in hits)
            {                
                var documentFromSearcher = indexSearch.Doc(hit.doc);
                Console.WriteLine(documentFromSearcher.Get("Make") + " " + documentFromSearcher.Get("Model"));
            }

            indexSearch.Close();
            directory.Close();

 

That's it, good luck, hope you find what you're looking for.

Full Download of the Lucene.Net tutorial

SimpleLuceneNetTutorial.zip (1.07 mb)

Recommended Reading

Although this book is aimed at the Java verion Lucene in Action is a really good in-depth book, that'll teach you possibly more than you ever wanted to know about Lucene.


blog comments powered by Disqus

Search

Disclaimer

The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2014