Wednesday, March 13, 2013

SharePoint 2013 Search: Moving the Query Component to a Web Front End

Introduction
When you first create the Search Service Application, the search topology by default has all components running on one server. I performed the SSA creation on our "Index" server and therefore all components are there accordingly. However, I want to scale out the SSA a bit and have at least the Query Components running on the web front ends. (This does require more resources as the query processing is much more robust than in previous versions). Therefore I used PowerShell commands as outlined in the Microsoft TechNet documentation. This post documents the script I created to perform the task. The full script is located at the end of the post.

Initial state of the Search Topology:




You may use the SharePoint 2013 Management Console, or use the PowerShell console issuing the SharePoint add-in:
Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue


Step #1 - Clone the Active Search Topology

# Clone the active search topology
$ssa = Get-SPEnterpriseSearchServiceApplication
$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
$clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone –SearchTopology $active


Step #2 - Add the New Search Component on the WFE
# Add New Search Component
$wfe = Get-SPEnterpriseSearchServiceInstance -Identity "<<web front end server name here>>"
New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $clone -SearchServiceInstance $wfe


Step #3 - Start the Search Instance on the WFE# Start Instance On Different Server
Start-SPEnterpriseSearchServiceInstance -Identity $wfe


Step #4 - Activate the Cloned Search Topology
# Activate Search Topology
Set-SPEnterpriseSearchTopology -Identity $clone

 

At this point the Query Component should be running on the original location as well as the WFE:




So we need to remove it from the original server.

Step #5 - Clone the Search Topology Again
# Clone Again
$ssa = Get-SPEnterpriseSearchServiceApplication
$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
$clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone –SearchTopology $active


At this point, you either need the ComponentID or the Name of the component you wish to remove. By default the first query component is named QueryProcessingComponent1 so the full script will work. However, if you issue the cmdlet statement  Get-SPEnterpriseSearchComponent -SearchTopology $clone you can identify both:


Step #6 - Get the ComponentID to Move
# Get the Search Component ID To Move
$queryComponentID = (Get-SPEnterpriseSearchComponent -SearchTopology $clone -Identity QueryProcessingComponent1).componentID


Step #7 - Remove the Search Component from the Original Server
# Remove Search Component
Remove-SPEnterpriseSearchComponent -Identity $queryComponentID.GUID -SearchTopology $clone -confirm:$false


Removing the search component from the original server may show a red x icon on the topology in Central Admin:



Step #8 - Activate the Search Topology Again
# Activate Search Topology Again
Set-SPEnterpriseSearchTopology -Identity $clone



Once the topology is reactivated, all is good:

 
The Query Processing component has now been moved to the web front end!!

It is recommended to also have the Index Partition (ala the Index Components) reside on the same server as the Query Components. Here is a post about also moving the Index Partition to the web front end.


Full Script


Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue

# Clone the active search topology
$ssa = Get-SPEnterpriseSearchServiceApplication
$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
$clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone –SearchTopology $active


# Add New Search Component
$wfe = Get-SPEnterpriseSearchServiceInstance -Identity "<<web front end server name here>>"
New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $clone -SearchServiceInstance $wfe


# Start Instance On Different Server
Start-SPEnterpriseSearchServiceInstance -Identity $wfe


# Activate  Search Topology
Set-SPEnterpriseSearchTopology -Identity $clone


# Clone Again
$ssa = Get-SPEnterpriseSearchServiceApplication
$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
$clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone –SearchTopology $active


# Get the Search Component ID To Move
$queryComponentID = (Get-SPEnterpriseSearchComponent -SearchTopology $clone -Identity QueryProcessingComponent1).componentID


# Remove Search Component
Remove-SPEnterpriseSearchComponent -Identity $queryComponentID.GUID  -SearchTopology $clone -confirm:$false


# Activate  Search Topology Again
Set-SPEnterpriseSearchTopology -Identity $clone


45 comments:

  1. Hi Steve,
    How can we move the indexPartion0 from the App Server to WFE 1 where the Query Processing resides now as you have shown.

    if i have WFE2 also , i need the replica of Index Partition0 in WFE 2.

    I am trying to mimic this architecture as shown in Microsoft case study of Mavention http://technet.microsoft.com/en-us/library/jj822912.aspx

    Thanks

    ReplyDelete
    Replies
    1. I too will be adding additional WFEs. I will be looking at the Index Partitions soon as well. I think I am going to keep the Index Partition on our "Index" Server but I will look at moving/replicating for you. From what I understand right now, if you have two query components, the index gets split into "a" and "b" versions. I'll keep you posted.

      Delete
    2. I will be building out a new index box very soon. I will post the steps I take although I'll be following this as a guideline: http://technet.microsoft.com/en-us/library/jj862355.aspx

      Delete
    3. Looks like there is a script to move the index partition here: http://gallery.technet.microsoft.com/office/Move-SharePoint-2013-242869e2

      Delete
    4. I scaled out my index servers and I have several posts about that but I just moved the Index Parition to the WFE for performance purposes. Here is my post on that:

      http://stevemannspath.blogspot.com/2013/05/sharepoint-2013-search-moving-index.html

      Delete
  2. Replies
    1. Great! Thanks for the comment!

      -=Steve

      Delete
  3. Hi Steve,

    Is it possible to perform
    Remove-SPEnterpriseSearchComponent -Identity $queryComponentID.GUID -SearchTopology $clone -confirm:$false
    If my farm is contains only two server, 1. App Server, 2 DB server.
    My aim to stop the Query processing component. I want to execute specific test case.

    regards,
    sachin

    ReplyDelete
    Replies
    1. I haven't tried to completely remove a component from the topology. I am sure it is possible. Not sure of any adverse effects. I would backup the Search Service Application first just in case of any issues.

      Delete
  4. Hi, I tried to follow this on a completly new wfe.
    I had en error at #2
    New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $clone -SearchServiceInstance $wfe

    Error:
    New-SPEnterpriseSearchQueryProcessingComponent : The search service instance on this server is not online

    Perhaps I should have prepared the wfe with some settings first? Like start services or something?

    ReplyDelete
    Replies
    1. Perform step #3 first and then try #2 again.

      Delete
    2. I too facing same error. I performed step#3 and tried step# 2, but same error.

      Delete
    3. Try setting it online manually:
      $wfe.Status=0
      $wfe.Update()

      Delete
  5. Hi,

    Have you tried using a security trimmer in this scenario? I set one up (in WFE), but the trimmer never gets called.

    ReplyDelete
    Replies
    1. No I haven't implement a custom security trimmer...

      Delete
  6. Hi Steve,

    I have tried your script and works well, but I want to move all the Search components to a dedicated Hard Drive. How I can do this?

    Thanks

    ReplyDelete
    Replies
    1. The search components are processes that run in memory. The only thing that resides on disk is the search index. Please expand on what you are trying to accomplish.

      Delete
  7. Hi Steve,


    We have SharePoint 2010 Search is configured as :

    Index Partition 0 - with Query component 1 on WFE1, Query component 2 on WFE2
    Index Partition 1 - with Query component 3 on WFE1, Query component 4 on WFE2

    Crawl component 1 - with Search Server1
    Crawl Component 2 - with search server2

    Administration component - with admin server

    We wanted to move query components from WFEs to search servers to avoid disk space issue on WFEs. Can you please suggest approach and recommendations?

    Thanks.

    ReplyDelete
    Replies
    1. The query components shouldn't be the problem. It is the index that takes up space. The query and index should be on the same server. Are you running out of space?

      Delete
    2. yes, running out of space on WFEs while crawling and index file size keeps increasing. At present the query components as being saved in the C drive where OS and other applications are installed. will it support to save the query components(where the index file gets saved) path to other drives in same WFEs, that will be easy to increase the disk space? Please advise.

      Delete
    3. We use a separate disk on the WFEs that operations can increase in size if needed. You would need to recreate the index components using the new location. One thing to mention is that in the index folders you'll see a Journal folder. Inside the journal folder there will be all kinds of backupsection_xxx_xxx.* files. These are like transaction logs for the crawls. I believe you or SharePoint can use these to restore an index but in our case, we can easily recrawl our sites and data. Therefore, we do not need all of the backup files. I was running out of space as well and even when I increased the disk, it just kept taking up more space. Since we don't technically need those files, I wrote a PowerShell script that deletes the files over 24 hours old. This is probably not a conventional thing to do and people will probably yell about doing this but a) it works, b) it doesn't affect anything, and c) we are not using up disk space.

      -=Steve

      Delete
  8. Hi,
    I’m having a difficulties to put a search Application to work.
    I have a 3 layer solution, BD server, App server and wfe server. My access to the sites is done by https://mysharepointsite/site.
    Despite this all communication inside the server occurred on port 80.
    If I set my site form crawling as http://mysharepointsite the query run smoothly and I get searchable results, but if I put https://mysharepointsite I get a error, cannot connect to the repository.
    When the users do a search in https://mysharepoint /site I doesn’t get any result.
    What must I do to get users accessing https://mysharepoint /site returning results?
    The solution is heavily firewalled.
    Thanks in advanced.

    King regards,
    João Madeira

    ReplyDelete
  9. Did you configure the alternate access mappings to use https? There could also be a certificate issue. See here: http://technet.microsoft.com/en-us/library/ee792873(v=office.15).aspx

    ReplyDelete
  10. I have the AAM configured and have already change the option highlighted in the link you posted. Any other ideas?

    ReplyDelete
  11. Hi Steve,

    Thank you for writing this and saving many hours of SP Admins.

    I have two Crawler, Content Processing and Analytics Processing servers in my farm. App01 and App02 which are running in both roles. App01 is hosting my CA and was the default server for all search roles initially.

    Now i want to remove these above three roles for that server and want App02 to do all the stuff. However i am unable to remove these roles from server. I have tried the below script for crawler removal but didn't work.

    $searchapp = Get-SPEnterpriseSearchServiceApplication "SearchApp1)"
    $topo = Get-SPEnterpriseSearchCrawlTopology -SearchApplication $searchapp
    New-SPEnterpriseSearchCrawlTopology -SearchApplication $searchapp -clone -CrawlTopology $topo
    $newtopo = Get-SPEnterpriseSearchCrawlTopology -SearchApplication $searchapp -Identity 11111111-1111-1111-1111-111111111111
    $cc = Get-SPEnterpriseSearchCrawlComponent -CrawlTopology $newtopo -Identity 22222222-2222-2222-2222-222222222222
    $cc | Remove-SPEnterpriseSearchCrawlComponent
    $newtopo | Set-SPEnterpriseSearchCrawlTopology -Active

    Source: https://technet.microsoft.com/en-us/library/ff607845(v=office.14).aspx

    Any idea?

    Best Regards,
    Muhammad Zeeshan Tahir

    ReplyDelete
    Replies
    1. Run each statement one at a time. Inspect each element to make sure you are using the correct Identity values (GUIDs).

      Also, from my code listing, I have been starting the instance on the server before adding or removing any components.

      If you need direct assistance you can hit me up for a session:
      https://www.wyzant.com/Tutors/stevetheman

      Delete
  12. Awesome! how do you move the index file to another drive other than C:\?

    ReplyDelete
    Replies
    1. I've done this by creating a new index component using the new location and removing the old index component.

      Delete
    2. I've done this by creating a new index component using the new location and removing the old index component.

      Delete
  13. Hey Steve,

    Could you describe the step by step process for SharePoint 2010 Farm. I want to know move Crawl and Query component from CQ01 & CQO2 to APP01 & APP02. after that I will remove CQ01 & CQ02 from the farm.

    Please help me out.

    Br,
    Manoj Kumar

    ReplyDelete
    Replies
    1. I haven't used SharePoint 2010 in years. Sorry.

      Delete
  14. Hi Steve,

    We are installing SharePoint 2013 and our farm will have Two WFE servers, one Search Server dedicated to search and one App Server that will host all other service application.
    Is it a good idea to run all six search components on Search Server or should I move Query and Index components to WFE's. Our search index will have 1 million items.
    Please suggest

    Thanks

    ReplyDelete
    Replies
    1. No it is not a good idea. Search needs to be scaled out. The WFE approach is not an official topology but will indeed help your situation.

      Delete
    2. No it is not a good idea. Search needs to be scaled out. The WFE approach is not an official topology but will indeed help your situation.

      Delete
  15. Thanks Steve. But WFE's will also run the distributed cache service, so if I move the query and Index components to WFE's will it affect the distributed cache performance?

    ReplyDelete
    Replies
    1. You need to monitor the load and resource usage on the WFEs regardless.

      Delete
  16. I’ve been trying to follow the directions, but I’m getting error on the second line of code –
    PS C:\Users\spadmin> $ssa = Get-SPEnterpriseSearchServiceApplication
    PS C:\Users\spadmin> $active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
    Get-SPEnterpriseSearchTopology : Cannot convert ‘System.Object[]’ to the type ‘Microsoft.Office.Server.Search.Cmdlet.SearchServiceApplicationPipeBind’ required by parameter
    ‘SearchApplication’. Specified method is not supported.
    At line:1 char:61
    + $active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
    + ~~~~
    + CategoryInfo : InvalidArgument: (:) [Get-SPEnterpriseSearchTopology], ParameterBindingException
    + FullyQualifiedErrorId : CannotConvertArgument,Microsoft.Office.Server.Search.Cmdlet.GetSearchTopology

    ReplyDelete
    Replies
    1. Check the value of $ssa after you run the first line. Looks like it is not getting the search service application.

      Delete
  17. Can't you leave the Query Component on both servers? Meaning you would stop at step #4 and then repeat if you had a second WFE to add?

    ReplyDelete
    Replies
    1. Yes. You can have components all over the place. There are, however, certain search topologies that are the recognized or accepted methods.

      http://zoom.it/PKfA#full

      Delete
  18. I have ran into an little bit of an interesting problem that I need to solve. Currently my topology looks like the end product in your post where all but one of the components are on one server and the QPC(Query Processing Component) is on another. After having many long and drawn out discussions with microsoft, I need to add another QPC to the server that has all the other components already running on it. My fear is, if I clone the QPC and try to move it to the index server where all the other components are that it will somehow remove the other components and just leave the two QPC's. Do you mind if we have a discussion about how to accomplish copying the QPC back to where it was originally without disturbing/deleting the other components? I look forward to hearing from you.

    Thank you,
    Marc

    ReplyDelete
    Replies
    1. If you need to add a new QPC, then just add a new one. Don't go through the move process.

      Delete
    2. If you need to add a new QPC, then just add a new one. Don't go through the move process.

      Delete
  19. Thanks Steve. I took you Index Move and Query Proc Script and merged it into one for our purposes.
    Do all servers need the same partitions? When i tried to move Index first it said "F:" not found. It was present on the WFE but not on the AS where the index was (on drive C).
    I needed to add a new partition on the AS named F: to get things going

    ReplyDelete