SharePoint 2013/2016: Opening the Memory Gates for Service Application Web Services

I ran into an issue this morning attempting to use some PowerShell cmdlets for the BCS service application. I received a "The requested service could not be activated" as shown here:

The requested service,
'http://sp2013:32843/4f39326869244b19af72d89efe182a87/BdcService.svc/http'
could not be activated. See the server's diagnostic trace logs for more
information.

The error message shows the URL to the BDCService so I navigated to the service in a new browser to see if I could glean anymore information:

Memory gates checking failed because the free memory (128397312 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element. ]

Since I am working on a local virtual machine that is limited in memory, apparently there is a threshold in order for the service application web services to run. The SharePoint 2013 service applications use WCF web services (by the way).

The error message here actually explains how to resolve it. All you need to do is adjust the minFreeMemoryPercentageToActivateService parameter of the serviceHostingEnvironment configuration element.

The parameter and the configuration elements do not actually exist so defaults are used. Therefore, you need to add these to the web.config of the web service you want to adjust. For me it was the BCS (or BDC) web service.

You can easily locate the web.config by going to IIS Manager and finding the web service application under SharePoint web services (the guid value was shown to me in the command console):

Right click and select Explore:

This opens up a Windows Explorer window at the location of the web service:

There's the web.config you need to modify. Open it up in Visual Studio or Notepad.

The serviceHostingEnvironment config element must be within the <system.serviceModel> section. Therefore I decided to scroll to the end of the <system.serviceModel> section and place the serviceHostingEnvironment element there, setting the minimum percentage to 1:

<serviceHostingEnvironment minFreeMemoryPercentageToActivateService="1" />

Save the changes and then refresh the web service browser instance:

Cha-Ching! We are back in business!

Bang Two Pound Four: Remote Blob Storage (RBS) with SharePoint 2013 using SQL Server 2012

Check out the whole SharePoint 2013 Solution Series
New Titles Added Weekly!

I am in the process of evaluating SharePoint 2013 using the Preview (Beta) edition. Since we are moving from MOSS 2007 directly to SharePoint 2013, any new features that were introduced with SharePoint 2010 are fair game.

One of the advantages with the latest SharePoint versions and SQL Server versions is the ability to implement Remote Blob Storage (RBS) using the FileStream Provider. This allows for documents and files that are larger than a specified amount of bytes (default 100KB) to be stored on a connected file system of the SQL Server box instead of inside the database itself. The overall advantage is keeping your content databases smaller and more manageable.

The steps for implementing RBS with SharePoint 2013 and SQL Server 2012 are very similar of that with SharePoint 2010 and SQL Server 2008. Since I found several blog posts and articles that explain the exact same steps (who copied who???) with the exact same code, I decided to change it around a bit and not use the exact names and passwords.

Whoops! When setting up the MASTER KEY in SQL Server on my main content database, I decided to use a different password. The blog posts and articles all used
N'Admin Key Password !2#4' - so I decided I wouldn't copy their (I thought made up) password and use N'Admin Key Password SharePoint1'.

That sent me through some SharePoint hell since I did not see any files being stored in the blob storage folder. However, once I altered and regenerated the master key with the password using the correct !2#4 (bang two pound four), everything magically started working. You don't learn unless you make mistakes so I am actually glad I went through some trouble.

Anyway, here are the steps I took to convert an existing SharePoint 2013 (Preview) content database to use RBS within SQL Server 2012:


1. Enable File Stream on the SQL Server
On the SQL Server 2012 box, open SQL Server Configurations Manager.

Select SQL Server Services from the left pane. Right-click on the SQL Server process in the right window and select Properties:

The SQL Server Properties dialog appears. Select the FILESTREAM tab and check all of the check boxes:

Click OK.

Start SQL Management Studio and open a new query window.

Execute the following two SQL Statements (on any database):

EXEC sp_configure filestream_access_level, 2

RECONFIGURE

Using your content database (the one I used was WSS_CONTENT_<guid> because I was playing around and was too lazy to remove the GUID or create a proper name in the first place for that matter), open a new query:

Create the master key using the following SQL:

if not exists (select * from sys.symmetric_keys where name = N'##MS_DatabaseMasterKey##')
     create master key encryption by password = N'Admin Key Password !2#4'

Create a new filegroup for the RBS:

if not exists (select groupname from sysfilegroups where groupname=N'RBSFilestreamProvider')
      alter database [<your content database>]
      add filegroup RBSFilestreamProvider contains filestream

Add the "file" using the following SQL:

alter database [<your content database>]
add file (name = RBSFilestreamFile, filename= 'c:\SPBlobStorage')
      to filegroup RBSFilestreamProvider

The "file" is a folder on a connected drive. It can be a local drive on the SQL Server or an attached iSCSI drive. The folder cannot exist already. Executing the SQL statement above automatically creates the folder specified on the drive specified.


2. Install RBS on the DB and Web/Application Servers
The RBS bits need to be installed on the database server and on each Web server and Application server that exist in the SharePoint farm. The RBS bits are a separate installation as part of the SQL Server 2012 Feature Pack. You can access the feature pack for SQL Server 2012 here.

Scroll down on the feature pack download page and locate the RBS download:

Download the appropriate package (hopefully x64) and save it in a central location.

For the first web server (or database server), create a new batch file, in the same location as you downloaded RBS.msi, using the following code (all on one line - no line breaks):

 msiexec /qn /lvx* rbs_install_log.txt /i RBS.msi

TRUSTSERVERCERTIFICATE=true FILEGROUP=PRIMARY
DBNAME="<ContentDbName>"
DBINSTANCE="<DBInstanceName>"
FILESTREAMFILEGROUP=RBSFilestreamProvider
FILESTREAMSTORENAME=FileStreamStore  

Open a Command Prompt and execute the first batch file:

Verify success by reviewing the rbs_install_log.txt file that was generated in the same folder. The completion message is not the very last thing but it is towards the end of the log:

On all of the other web servers and application servers, place the following code into a batch file (again no line breaks):

 msiexec /qn /lvx* rbs_install_log.txt /i  RBS.msi DBNAME="ContentDbName" DBINSTANCE="DBInstanceName" ADDLOCAL="Client,Docs,Maintainer,ServerScript,FilestreamClient,FilestreamServer"

Open a Command Prompt and execute the second batch file on each web server and application server:

3. Enable RBS on Content Database
To enable RBS on the content database, you must use PowerShell. Therefore, open up PowerShell or Notepad and create the following PowerShell Script:

$cdb = Get-SPContentDatabase -WebApplication "<Web Application Name>"
$rbss = $cdb.RemoteBlobStorageSettings
$rbss.Installed()
$rbss.Enable()
$rbss.SetActiveProviderName($rbss.GetProviderNames()[0])
$rbss

You may also add $rbss.MinimumBlobStorageSize=1048576 to increase the minimum file size that will be considered for RBS. The example number shows 1MB.

Save the script as a .ps1 file and then open the SharePoint 2013 Management Shell:

Execute the script:

 The first output is"True" which means RBS has been installed correctly. The second output displays the RemoteBlobStorageSettings object and shows that RBS is enabled and displays the name of the Active Provider (which should be the same name you used as the FILESTREAMSTORENAME in the RBS installation batch file.


4. Test RBS

Now it's time to see this working in action! Navigate to your SharePoint site related to the content database you just configured. Open a document library:

Upload a bunch of documents into the document library:

Investigate the folders within the local file system blob storage location:

Files appear in small chunks.

SharePoint 2013: We Don't Get Fooled Again

I finally got SharePoint 2013 Preview up and running and started poking around. Overall I am not impressed. I think The Who's song "We Don't Get Fooled Again" describes my thoughts:

Note: This is just my first impression and I am sure once things start getting baked in and the actual release comes out my feelings will be more positive.

UPDATE (8/21/2012): My first impressions were solely based on installing the preview and causally  poking around. After a few weeks of digging into other areas, I realized that although the core is similar to 2010, there are many new features and functionality that require further exploration. There are some real paradigm shifts and improvements. It is going to take a lot of time to get your head around everything . It's still cool to correlate thoughts to actual lyrics of a song though.

"...The change, it had to come
We knew it all along "
     We knew there had to be a new product since there is a 3-year cycle. SP 2010 broke out the SSP and provided some great architecture modifications while keeping the core system in place. Some great UI modifications as well (e.g. InfoPath form to modify New, Edit, View, etc.). We anxiously awaited for SharePoint 15.

"We were liberated from the fold, that's all "
     I think we are all just happy there is something out there and we can play with it. Not sure what the SharePoint MVPs had to play with in February but it must have been awful.

"And the world looks just the same
And history ain't changed "
    A few more service applications; calling certain web parts "Apps" now; The more things change the more they stay the same. Wrap it up in a different package and call it something different.    

"Cause the banners, they are flown in the next war "
     The Office desktop products move along with SharePoint(or vice-versa) and they continue to be there to help drive some of the SharePoint evolution (e.g. Office Web Apps, Office 365 Home Editions)

"I'll tip my hat to the new constitution
Take a bow for the new revolution "
     I still need to acknowledge and cheer the new version. I need to learn everything new and understand the overall impact. Still need to embrace the latest and greatest even though my first impressions are not great.

"Smile and grin at the change all around "
     More of a cynical grin since many things may appear different but they are very similiar to SharePoint 2010.

"Pick up my guitar and play
Just like yesterday "
    I'll keep coding my customizations, coming up with solutions, writing books and guides - makes no difference to me. Gotta keep learning and earning!

"Then I'll get on my knees and pray
We don't get fooled again
No, no! "
   Like I said, I hope I see things in a different light, but to wait two years for this crap?? WTF

"Meet the new boss - Same as the old boss"