Working with Solr Using Solrnet
Tech And Development

Working with Solr Using SolrNet

Dec 06, 2017
Prasanth Nittala

There may be use cases where you want to integrate with Solr using the SolrNet library instead of going through Sitecore Content Search API.  This post describes how to configure and work with Solr using SolrNet library with code snippets to perform different queries such as FilterQuery Parameter, Facet Parameters, Filter Query Parameters with Values, Facet Pivot Parameters, Grouping Parameters, Date Range Parameters and their corresponding solr Urls. My next post will describe the concepts of analyzers/filters and how to debug the scoring mechanism of Solr.

Configuration

Solr Configuration

Basic request handler configuraton for Search can be done as follows in Solrconfig.xml:

SolrConfig configuration
< requestHandler name = "/search" class = "solr.SearchHandler" >
   <!-- default values for query parameters can be specified, these
         will be overridden by parameters in the request
   -->
   < lst name = "defaults" >
      < str name = "echoParams" >explicit</ str >
      < int name = "rows" >10</ int >
      < str name = "defType" >edismax</ str >
      < str name = "q.op" >OR</ str >
      < str name = "df" >description_t</ str >
 
       <!-- highlight -->
      < str name = "hl" >true</ str >
      < str name = "hl.simple.pre" >&lt;b&gt;</ str >
      < str name = "hl.simple.post" >&lt;/b&gt;</ str >
      < str name = "hl.snippets" >1</ str >
      < str name = "hl.fragsize" >200</ str >
 
 
      <!-- spellcheck -->
      < str name = "spellcheck" >true</ str >
      < str name = "spellcheck.collate" >true</ str >
      < str name = "spellcheck.dictionary" >default</ str >
      < str name = "spellcheck.onlyMorePopular" >true</ str >
      < str name = "spellcheck.count" >1</ str >
      < str name = "spellcheck.alternativeTermCount" >1</ str >
      < str name = "spellcheck.collateExtendedResults" >false</ str >
      < str name = "spellcheck.maxResultsForSuggest" >1</ str >
 
 
      <!-- facet min count-->
      < str name = "facet.mincount" >1</ str >
   </ lst >
 
    <!-- add spell check component to this handler -->
   < arr name = "last-components" >
      < str >spellcheck</ str >
   </ arr >
</ requestHandler >
 
<!--define Spellcheck component -->
< searchComponent name = "spellcheck" class = "solr.SpellCheckComponent" >
   < lst name = "spellchecker" >
      < str name = "name" >default</ str >
      < str name = "field" >spellcheck_field</ str >
      < str name = "classname" >solr.DirectSolrSpellChecker</ str >
      < str name = "distanceMeasure" >internal</ str >
      < float name = "accuracy" >0.7</ float >
      < int name = "maxEdits" >2</ int >
      < int name = "minPrefix" >1</ int >
      < int name = "maxInspections" >5</ int >
      < int name = "minQueryLength" >4</ int >
      < float name = "maxQueryFrequency" >0.01</ float >
      < float name = "thresholdTokenFrequency" >.01</ float >
   </ lst >
</ searchComponent >
 

Schema Configuration

In the fields section, the following fields were configured and corresponding field type "text_general" is configured in the schema.xml.

Schema configuration
< fields >
   < field name = "car_make" type = "string" indexed = "true" stored = "true" />
   < field name = "car_model" type = "string" indexed = "true" stored = "true" />
   < field name = "description_t" type = "text_general" indexed = "true" stored = "true" />
   < field name = "car_updated_date" type = "tdate" indexed = "true" stored = "true" />
   < field name = "spellcheck_field" stored = "false" type = "text_general" multiValued = "true" indexed = "true" termVectors = "true" />
   < copyField source = "car_model" dest = "spellcheck_field" />
</ fields >
 
< types >
   < fieldType name = "text_general" class = "solr.TextField" positionIncrementGap = "100" >
      < analyzer type = "index" >
         < charFilter class = "solr.HTMLStripCharFilterFactory" />
         < tokenizer class = "solr.WhitespaceTokenizerFactory" />
         < filter class = "solr.LowerCaseFilterFactory" />
         < filter class = "solr.WordDelimiterFilterFactory" splitOnNumerics = "1" splitOnCaseChange = "1" generateWordParts = "1" generateNumberParts = "1" catenateWords = "1" catenateNumbers = "1" catenateAll = "1" preserveOriginal = "1" />
      </ analyzer >
      < analyzer type = "query" >
         < charFilter class = "solr.HTMLStripCharFilterFactory" />
         < tokenizer class = "solr.WhitespaceTokenizerFactory" />
         < filter class = "solr.LowerCaseFilterFactory" />
         < filter class = "solr.WordDelimiterFilterFactory" splitOnNumerics = "0" splitOnCaseChange = "1" generateWordParts = "1" generateNumberParts = "1" catenateWords = "1" catenateNumbers = "1" catenateAll = "1" preserveOriginal = "1" />
      </ analyzer >
   </ fieldType >
</ types >

 

Queries with SolrNet

Base Search Parameters

BasicSolrResultsRetrieval
QueryOptions searchParameters = new QueryOptions();
AbstractSolrQuery solrQuery = BuildBasicSearchResultsSolrQuery(searchRequest, ref searchParameters);
SolrQueryResults< SearchResult > solrResults = GetSolrOperations< SearchResult >().Query(solrQuery, searchParameters);
string spellCheckString = string.Empty;
if (solrResults != null && solrResults.SpellChecking != null))
   spellCheckString = GetSpellCheck(solrResults).FirstOrDefault();
SearchResult Class
public class SearchRequest
{
   public string SearchText { get; set; }
 
   public string Language { get; set; }
 
   public int PageNo { get; set; }
 
   public int PageSize { get; set; }
 
   public List< string > FacetFields { get; set; }
 
   public Dictionary< string ,List<string>> Filters { get; set; }
 
   //asc or desc
   public string Sort { get; set; }
 
   public DateTime? RecordUpdatedFromDate { get; set; }
 
   public DateTime? RecordUpdatedToDate { get; set; }
 
   public SearchType SearchType { get; set; }
}
 
public class SearchResult
{
   [DataMember]
   [SolrField("description_t")]
   public string Description { get; set; }
 
   [DataMember]
   [SolrField("car_make")]
   public string Make { get; set; }
 
   [DataMember]
   [SolrField("car_model")]
   public string Model { get; set; }
 
   [DataMember]
   [SolrField("car_updated_date")]
   public string UpdateDate { get; set; }
}
BuildBasicSearchSolrQuery
private AbstractSolrQuery BuildBasicSearchResultsSolrQuery(SearchRequest searchRequest, ref QueryOptions searchParameters)
{
   AbstractSolrQuery solrQuery = new SolrQuery(searchRequest.SearchText);
 
   searchParameters.ExtraParams = new Dictionary< string , string > {
         { "qt" , "/search" }, //querying the "search" request handler configured in solrconfig
         { "qf" , "car_make car_model description_t" }, //query fields are car_name and description_t Solr fields
         { "hl" , "true" }, //enable highlighting
         { "hl.fl" , "description_t" }, //highlighted fields are car_name and description_t
         { "spellcheck.q" ,searchRequest.SearchText } //to perform spellchecking for the searchtext entered
      };
 
   //working with pagination
   searchParameters.Rows = searchRequest.PageSize; //pagesize is set to 10
   searchParameters.Start = (searchRequest.PageNo - 1) * searchRequest.PageSize; //start will be 0 initially
 
   //sort by ascending or descending on Solr "car_name" field
   if (! string .IsNullOrEmpty(searchRequest.Sort))
   {
      SolrNet.SortOrder[] order = new [] { new SolrNet.SortOrder( "car_name" , searchRequest.Sort.Equals( "asc" )?Order.ASC:Order.DESC}; //sort by car_name solr field
      searchParameters.OrderBy = order;
   }
 
   searchParameters.SpellCheck = new SpellCheckingParameters() { };
   return solrQuery;
}
SpellCheck results
private IList< string > GetSpellCheck(SolrQueryResults<ContentSearchResult> results)
{
   IList< string > spell = new List< string >();
   if (results != null )
   {
      SpellCheckResults spellCheckResults = results.SpellChecking;
      foreach (SpellCheckResult res in spellCheckResults)
      {
         foreach ( string suggestion in res.Suggestions)
         {
            spell.Add(suggestion);
         }
      }
   }
   return spell;
}

This results in a query similar to what's shown here: http://localhost:8983/solr/car_search_index/search?q=corolla&start=0&rows=10&qf=car_make+car_model+description_t&wt=json &indent=true&qt=/search&hl=true&hl.fl=description_t&spellcheck.q=corolla&spellcheck=true

FilterQuery Parameters

Adding a filter query for querying solr is as follows:

FilterQueries
private void AddLanguageFilterQuery(SearchRequest searchRequest, ref QueryOptions searchParameters)
{
   searchParameters.AddFilterQueries( new ISolrQuery[] { new SolrQueryByField( "car_make" , "Toyota" ) }); //to get all vehicles with make Toyota
}

This results in appending fq parameter to the request as shown in bold here: http://localhost:8983/solr/car_search_index/search?q=*&start=0&rows=10&qf=car_make+car_model+description_t&wt=json &indent=true&fq=(car_make:(Toyota))

Facet Parameters

Adding Facet Field parameters are as follow:

Facet Parameters
private void CreateFacetParameters(SearchRequest searchRequest, ref QueryOptions searchParameters)
{
   FacetParameters fp = new FacetParameters();
   //facet fields for query
   // example searchRequest.FacetFields = {"car_make", "car_model"} Solr Fields in the Solr Document
   foreach ( string facetFieldName in searchRequest.FacetFields)
   {
      fp.Queries.Add( new SolrFacetFieldQuery(facetFieldName));
   }
   searchParameters.Facet = fp;
}

This results in appending facet parameters to the request as shown in bold here: http://localhost:8983/solr/car_search_index/search?q=*&start=0&rows=10&qf=car_make+car_model+description_t&wt=json &indent=true&facet=true&facet.field=car_make&facet.field=car_model

FilterQuery to filter based on facet values:

This is useful in cases where you are filtering the results based on facets. For example, 

FacetValue FilterQuery Parameters
private void CreateFilterQueryBasedonFacetValue( ref QueryOptions option, string filterFieldName, IList< string > filterFieldValues)
{
   List<ISolrQuery> sqList = new List<ISolrQuery>();
   foreach ( string filterFieldValue in filterFieldValues)
   {
      if (! string .IsNullOrWhiteSpace(filterFieldValue))
      {
         var fieldQuery = new SolrQueryByField(filterFieldName, filterFieldValue);
         sqList.Add(fieldQuery);
      }
   }
 
   if (sqList.Count > 0)
   {
      //using OR criteria among the filter values
      option.FilterQueries.Add( new SolrMultipleCriteriaQuery(sqList, SolrMultipleCriteriaQuery.Operator.OR));
   }
}

Assuming we are filtering by "Corolla" OR "Camry" models, then the filterquery request is as shown in bold here: http://localhost:8983/solr/car_search_index/search?q=*&start=0&rows=10&qf=car_make+car_model+description_t&wt=json&indent=true&fq=(car_model:(Corolla)+OR+car_model:(Camry))

Facet Pivot Parameters

FacetPivot Parameters
private void CreateFacetPivotParameters(SearchRequest searchRequest, ref QueryOptions searchParameters)
{
   FacetParameters fp = new FacetParameters();
 
 
   //assuming car_make, car_model are two solrfields in the solr document on which we are pivoting
   List< string > pivots = new List< string >
   {
      "car_make" ,
      "car_model"
   };
 
   var facetPivotQuery = new SolrFacetPivotQuery()
   {
      Fields = pivots,
      MinCount = 1
   };
 
 
   fp.Queries.Add(facetPivotQuery);
   fp.Sort = false ;
   searchParameters.Facet = fp;
}

This results in appending facet parameters to the request as shown in bold here: http://localhost:8983/solr/car_search_index/search?q=*&start=0&rows=10&qf=car_make+car_model+description_t&wt=json &indent=true&facet=true&facet.pivot=car_make,car_model &facet.pivot.mincount=1&facet.sort=false

Grouping Parameters

Adding grouping parameters to Solr Query is as follows:

Grouping Parameters
private void AddGroupingParametersByContentTypeId(SearchRequest searchRequest, ref QueryOptions searchParameters)
{
   var gp = new GroupingParameters{
         Fields = new string [] { "car_model" }, //grouping by solr "car_model" field,
         Limit = 3, //limit 3 results per group
      };
 
   searchParameters.Rows = 10; //return at max 10 groups, as this can limit the number of groups returned
   searchParameters.Grouping = gp;
}

This results in appending grouping parameters to the request as shown in bold here: http://localhost:8983/solr/car_search_index/search?q=*&start=0&rows=10&qf=car_make+car_model+description_t &wt=json&indent=true&group=true&group.field=car_model&group.limit=3

Date Range Query Parameters

Date Range Parameters
private void AddDateRangeFilters( ref QueryOptions option, SearchRequest searchRequest)
{
   var dateQuery = new SolrQueryByRange<DateTime?>( "car_updated_date" , searchRequest.RecordUpdatedFromDate, searchRequest.RecordUpdatedToDate);
   option.FilterQueries.Add(dateQuery);
}

Assuming you are searching between RecordUpdatedFromDate as 06/13/2017 and RecordUpdatedToDate as 07/13/2017 (1 month), this results in appending filter query parameters to the request as shown in bold here: http://localhost:8983/solr/car_search_index/search?q=*&start=0&rows=10&qf=car_make+car_model+description_t&wt=json &indent=true&fq=car_updated_date:[2017-06-13T00:00:00Z+TO+2017-07-13T00:00:00Z]


Related insights