Skip to main content

Sitecore integration with Solr

This post is very helpful for following activity:  
1.    Generate Solr Schema.xml file using Sitecore
2.    Apache Tikka Configuration
3.    Sitecore integration with Solr

Solr ? 
Solr is highly reliable, scalable and fault tolerant, providing distributed indexing, replication and load-balanced querying, automated failover and recovery, centralized configuration and more. Solr powers the search and navigation features of many of the world's largest internet sites.

1.   Prerequisite Software 
·         JAVA ( install it from https://java.com/en/download/)
·         NSSM 2.24 ( Non Sucking Service manager) https://nssm.cc/release/nssm-2.24.zip
·         Sitecore 8.1 

2.   Setup Solr Instance

2.1  Download Solr
·         Download the Solr and extract it in one folder.
         Ex : D:\SolrPOC\Solr\Solr-4.5.1\
·         Setup the Solr , Follow below link  

https://www.sitecorebites.net/2019/12/setup-solr-as-window-service.html


2.2  Generate an XML Schema for Solr
You can modify an existing schema with the Generate the Solr Schema.xml file tool. This tool automatically generates a basic schema and ensures all the fields that Sitecore needs are present. You can add your own fields to this schema, as long as you do not change the system index fields.
Follow below steps to generate the schema.xml file.
·         Login into Sitecore CMS instance
·         Click on Control Panel
·         In the Control Panel, INDEXING section ,click Generate the Solr Schema.xml file:

·         In the Source file field, enter the path to the Solr schema that you want to use (schema.xml file present inside D:\SolrPOC\Solr\Solr-4.5.1\\server\solr\configsets\basic_configs\conf).

·         In the Target file field, enter the destination for the new schema file.
·         Click on Generate button
·         Sitecore updates the schema.xml file and adds all the necessary fields that it needs.
·         Keep newly generated schema file handy. We will use in step 2.3
Note : Create path folder on server where Sitecore is hosted. 

2.3  Core folders Setup
·         Navigate to folder D:\SolrPOC\Solr\Solr-4.5.1\server\solr\configsets and copy folder “basic_configs”
·         Paste this folder under D:\SolrPOC\Solr\Solr-4.5.1\server\solr\
·         Rename the folder as “sitecore_master_index”
·         Copy the generated file (schema.xml ) and paste to replace them in D:\ \Solr-4.5.1\server\solr\sitecore_master_index\conf
·         Duplicate the “sitecore_master_index” folder and rename it as “Sitecore_web_index”
·         Similarly duplicate the folder “Sitecore_master_index” and rename it as “Sitecore_core_index”
Note: You must you use a separate core for each index. Solr introduces the multiple core concept. When you configure Solr, use the 64-bit version of the Java Virtual Machine (JVM) for production servers. Solr allocates 512 MB of memory by default. If you require Solr to have access to more memory you can adjust these limits when Solr starts up. 
                                                           

2.   Apache Tika configuration
Apache Tika detects and extracts metadata and text from different file types (such as PPT, XLS, and PDF). It can take URL of documents also to extract content.

Please follow below steps to enable Apache Tika in Sitecore_Master_Index core.
It is required to be done only in CMS.
·         Copy dist and contrib folders from path D:\SolrPOC\Solr\Solr-4.5.1\
·         Paste these folders in D:\SolrPOC\Solr\solr-5.4.1\server\solr\sitecore_master_index\
·         Paste below lines after <luceneMatchVersion>5.4.1</luceneMatchVersion> in D:\SolrPOC\Solr\Solr-4.5.1\server\solr\sitecore_master_index\conf\solrconfig.xml
<lib dir="dist/" regex=".*\.jar" />
 <lib dir="contrib/extraction/lib" regex=".*\.jar" />
·         Copy below request handler configuration in D:\SolrPOC\Solr\solr-5.4.1\server\solr\sitecore_master_index\conf\solrconfig.xml
 <requestHandler name="/update/extract"
                 startup="lazy"
                 class="solr.extraction.ExtractingRequestHandler" >
    <lst name="defaults">
      <str name="lowernames">true</str> 
      <str name="fmap.content">mediacontent_t</str>
      <str name="defaultField">text</str>
      <str name="_datasource">sitecore</str>     
      <bool name="ignoreTikaException">true</bool>
    </lst>
  </requestHandler>
·         Restart the Solr service from Window service.


3.   Integrating Solr with Sitecore
3.1  Disable / Delete Lucene and enable Solr configuration files
Your website Include folder contains several configuration files. Lucene search is enabled by default. If you want to use Solr, you normally disable the Lucene search config files and enable the Solr config file .This enables Solr integration and gives you access to all the Solr specific configuration settings.
To switch configuration files so that Solr is enabled and Lucene is disabled:
·         Navigate to the website Include folder: \website\App_Config\Include\ and search for Lucene
·         Disable all file by adding .example to the file name extension.
·         Search for Solr and enable all files by removing .example from file name extension.



·         By default below index files present in include folder :
Sitecore.ContentSearch.Solr.Index.Master.config
Sitecore.ContentSearch.Solr.Index.Web.config
Sitecore.ContentSearch.Solr.Index.Core.config
·         If you want to create another index then copy and paste file and rename as : Sitecore.ContentSearch.Solr.Index.NA.Master.config
Sitecore.ContentSearch.Solr.Index.NA.Web.config
Sitecore.ContentSearch.Solr.Index.EU.Master.config
Sitecore.ContentSearch.Solr.Index.EU.Web.config
·         Enter the index name accordingly : ex : <index id="eu_master_index"
·         Set Crawler root and database accordingly.
<crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
      <Database>master</Database>
      <Root>/sitecore/content/2_EU</Root>
</crawler>
Note: These config files only require in CMS server.

3.2  Solr Specific Settings
The following Solr specific settings can be found in the Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config file
·         Specifying a Solr Service Address :
This setting tells Sitecore where the Solr server is located. Sitecore appends the core name so only the base address needs to be supplied.
<setting name="ContentSearch.Solr.ServiceBaseAddress" value="http://localhost:8983/solr" />
·         Enabling a Search Provider :
This setting tells Sitecore that Solr is enabled and so attempts to connect to the Solr server the next time the index is accessed.
<setting name="ContentSearch.Provider" value="Solr" /> (Default: “Lucene”)
·         Maximum Number of Search Result
This is a global setting found in the Sitecore.ContentSearch.config file.
This setting contains the maximum number of documents to retrieve on a single request if a limit has not been specified in the query, for example, Take(10).
<setting name="ContentSearch.SearchMaxResults" value="500" />
·         Enabling Batch Mode
When an item is indexed, the composed document is saved to the search index. When the default Lucene provider is enabled then each write is being flushed to a file on the local disk. When a document is written using the Solr provider the update has to travel over a network.
When an index is rebuilt a large number of document updates are created, this could result in a lot of network traffic which is not very efficient. Therefore using batch can help to optimize the update process as your indexes grow in size.
        <setting name="ContentSearch.Update.BatchModeEnabled" value="true" />
        <setting name="ContentSearch.Update.BatchSize" value="500" />


4.   Sitecore Indexing
·         Login to Sitecore and go to Control Panel
·         Click on Indexing Manager
·         Select index name to rebuild
·         Click on Rebuild and wait until the indexing is completed.
·         After indexing completed, browse below url and check records are there are not http://localhost:8983/solr/sitecore_master_index/select?q=*
·         Similarly repeat above steps to rebuild sitecore_web_index and sitecore_core_index.

                        



Source :
https://doc.sitecore.net/sitecore_experience_platform/81/setting_up_and_maintaining/search_and_indexing/walkthrough_setting_up_solr

Feel free if you face any issue or want to configure CD server in distributed environment. Solr rocks :) 

Comments

Popular posts from this blog

Sitecore Publishing target and Solr Configuration

Creating a new publishing target is very helpful if your Sitecore environment is setup in multiple content delivery servers. We can publish data from Master database to one or more publishing target databases. Main thing to notice here is Master database contains all versions of any content or assets while publishing target database contains single latest version for each language.      This post aims to provide systematic instructions to set up publishing target. It comprises of a target database (where the content will be published) and publishing target definition item in master database that points to that target database.             Post creation of publishing target need to take care of Solr configuration. This post cove master slave Solr configuration and onPublishEndAsync strategy. We are using 1 Solr Master instance and 6 Slave instance (2-EU, 2-Ap, 2- NA).     1   So...

Setup Sitecore Habitat v9.0 project in simple way

In this article , I would describe how to setup the Sitecore habitat V9.0 project with Sitecore Experience Platform 9.0 rev in very simple way. I used below system configuration to setup this project: 1. Windows 10 (32/64-bit) 2. IIS 10.0 3. .NET Framework 4.6.2 4. Microsoft Visual Studio 2017 5. Microsoft SQL Server 2017 6. Solr 6.6.2 7. Microsoft PowerShell® version 5.0 8. Web Platform Installer 5.0. 9. Web Deploy 3.6 for Hosting Servers 10. Node-v8.9.4-x64 Follow below steps to setup the project. A).   Habitat require Sitecore instance. Setup the clean Sitecore 9.0 instance. Ex : http://demo.local.com , Directory location : C:\inetpub\wwwroot\demo.local.com   For more info visit -   https://dev.sitecore.net/ B). Download the Habitat V9.0 project from Github site :  https://github.com/Sitecore/Habitat/     Note  :  When you download the packages, it is possible that the .ZIP packages are marked as blocke...

Web Deploy setup and configuration with Sitecore

Web Deploy empowers Visual Studio and WebMatrix to help developers streamline the deployment of Web applications to Microsoft IIS Web servers or to Microsoft Azure Websites. In addition, integration with Web Platform Installer allows developers to simply and easily install community webapplications.   This post describe the installation of Web Deploy and Configuration with Sitecore instance.  This post will be very useful if you want to sync your Sitecore media items with all of the CD's environment (Distributed environment) . In one of my previous project, I was keeping all the media items in file system and synchronizing all the files from CM to Cd's server. This solution is tested with 1 CMS and 6 CD's server (2-EU, 2- NA, 2- AP) with more than 5 Lakh files.  1.       Software Pre-requisites           a.       OS – Window 7/ Windows Server 2008 R2 Enterprise       ...