Russ' SharePoint Blog

SharePoint Saturday Denver and RBS Updates

Russ Houberg — Mon, 02 Aug 2010 13:34:09 GMT

Wow. Long break from blog posting! Sorry about that. Once again, duty calls. I was booked solid from March until mid-summer. Then it was vacation time. So there was little time to generate meaningful blog post content.

What happened in the meantime? Well, Microsoft released this tiny little server product called SharePoint Server 2010 and a tiny little client application called Office 2010! Yeah, yeah... I know. I'm a bit behind. Well, I have been playing with SP2010 and over the coming months, I'll be posting on some interesting things around Enterprise Search in SP2010. Our scalable architecture options have grown considerably and I'd like to cover some of that.

I will be presenting on a couple of topics at SharePoint Saturday in Denver this Saturday (August 7th). "Architecting for Scale in SP2010" will cover some of those new scale out topology options we have in SP2010. "Remote BLOB Storage Deep Dive" will get into some of the nuts and bolts of RBS. Both of these were queued up for the SharePoint Evolution / Portugal TechDays conferences earlier this year. Unfortunately, the Iceland volcano had other plans for me. Fortunately, SharePoint Saturday has provided a new forum for me to deliver this content.

In preparing for SharePoint Saturday this weekend, I realized that the RTM version of SharePoint 2010 had a few minor changes to the PowerShell commands for enabling RBS. I've updated my earlier post to reflect the proper commands.

Hope to see all of you SharePoint types in Denver this weekend!

Otherwise, I'm back in the saddle with a laundry list of research topics. So stay tuned...

A Guide to Enabling the CodePlex RBS Provider on SharePoint 2010

Russ Houberg — Mon, 18 Jan 2010 23:33:29 GMT

Over the holiday break I spent quite a bit of time trying to get a real good handle on exactly how SharePoint 2010 and RBS interact in order to externalize content to a BLOB store. As part of that effort, I implemented the CodePlex RBS Provider as an exercise to learn more about the roles of SQL Server 2008, the RBS Framework, and SharePoint 2010.

So why would you want to try out this exercise? Well, there are some limitations to the FILESTREAM provider. First, it only works with local volumes on the SQL Server. So if you want to store your binaries out on a file share that consists of cheap storage, you’re out of luck. Also, by default, FILESTREAM provider seems to put all binaries in a single folder. NTFS used to have problems when you had too many files in a single folder. That may have been mitigated in newer platforms, so I can’t testify to this specifically. But in general, I’m not a big fan hundreds of thousands of files in single folder.

I will be doing an RBS Deep Dive session at the SharePoint 2010 Evolution conference on April 21st and there are going to be some real juicy tidbits in that session. But in the meantime, I thought I would provide a guide to installing the CodePlex RBS Provider in SharePoint 2010 (beta). Note that some of these steps may be subject to change as I am currently working with the beta bits.

In order for the CodePlex provider to work, it has to be tweaked a bit because SharePoint 2010 requires the provider to properly dispose the BlobStore object in the framework. So I’ve recompiled the provider appropriately. You’ll find the link in the steps below.

Also, before we get started, the steps below assume you have a single WFE in your farm. If you have more than one WFE, steps 2 - 9 will need to be executed on each WFE.

So here we go:

Prepare your content database. This script must be executed in SQL Management Studio in the context of the content database that will have RBS installed:
IF NOT EXISTS (SELECT * FROM sys.symmetric_keys WHERE NAME = N'##MS_DatabaseMasterKey##') CREATE MASTER KEY ENCRYPTION BY PASSWORD = N'Admin Key Password !2#4'
Download rbs_x64.msi from here and save it to your WFE.
Execute rbs_x64.msi on your WFE to begin installation of the RBS Framework.
- Buzz through the standard agreement and registration forms.
- All features should be installed except FILESTREAM Provider is not necessary. So your feature options should look like this:
- Configure the connection to your content database. Use “default” for the Filegroup name. Test the connection to ensure your good before proceeding. Should look something like this:
- On the database configuration form, no need to check the “Request connecting clients…” box.
- On the Maintainer Task form, scheduling the Maintainer is optional. I typically do because it’s easier to let the installer help you set up the task. It’s important to configure the “Run As” account as well as the task schedule which is disabled by default.
- On the Client Configuration form, I typically set all log settings to “Warning”.
- You should be ready to “Install” at this point. Once you kick it off, the installer will go to town setting up the framework binaries and running RBS SQL scripts on your content database. If you told the installer to configure a scheduled task for the Maintainer, the Schedule Task configuration box will pop up for you.
- Once the installer has completed, your content database should have some new tables in it that look like this:
Download the updated CodePlex RBS Provider that I updated here.
For the sake of this guide, create a directory to permanently contain the RBS Provider binaries at c:\CodePlex_RBS_Provider
Copy the contents of the “Binaries” directory in the rbs.zip file to c:\CodePlex_RBS_Provider
Now you need to use Notepad to edit c:\CodePlex_RBS_Provider\InstallProvider.cmd
- The “RootDir” parameter needs to be c:\CodePlex_RBS_Provider
  - If you choose a different RBS Provider location for the RBS binaries, this value needs to reflect that location
- The “ProviderName” parameter needs to be FileStoreProvider_1 for the purposes of this example.
- The “DataLocation” parameter contains the location of your BLOB store. This will typically be a network share somewhere but it could be a local directory.
  - Note that the RBS provider will be invoked in the context of the SharePoint content access (Application Pool) account specified for the Web Application that the content database is associated with. So that service account needs to have FULL Permissions to this folder.
  - I typically create a subdirectory in the file share for each BLOB Store which I create 1 to 1 for each Content Database that I enable for RBS.
- The “ServerName” parameter contains the name of your SQL Server instance.
- The “DatabaseName” parameter contains the name of the content database that has had the RBS resources already installed (new RBS tables, etc).
- Save the InstallProvider.cmd which should now looks something like this:
  
  NOTE!!! The application pool account for the SharePoint web application that contains the content database that will have RBS enabled, must have full permission to the final BLOB store folder as well as read access to the location of the RBS provider DLL (if it's not installed to the GAC).
Launch a command prompt with Run As Administrator:
Run the InstallProvider command:

C:\CodePlex_RBS_Provider\InstallProvider.cmd
- When it runs correctly, the result looks like this:
Run the SharePoint 2010 Management Shell as Administrator.
Execute the following commands to enable the CodePlex RBS Provider for the content database:

$site = get-spsite http://siteurl
$rbss = $site.ContentDatabase.RemoteBlobStorageSettings
$rbss.Enable()
$rbss.SetActiveProviderName($rbss.GetProviderNames()[0])
$rbss
- When you execute “$rbss” you should see a result that looks like this:
- Congratulations, you just enabled RBS! If all goes well, all future binaries added to your content database will be sent to the BLOB Store.
- In order to externalize existing BLOB content, you need to run one more PowerShell command.
  
  $rbss.Migrate()
  - This command will send all binaries currently in the database to the active RBS Provider BLOB Store.
- What if I want to pull all my BLOBS back into the database? Simple. Just execute:
  
  $rbss.SetActiveProviderName("")
  $rbss.Migrate()
  - This will pull all the BLOBs back in-line in the content database. This is handy when you need to move your BLOB store or if you need to change your RBS Provider all together.

Ok, so that was all pretty cool, so how do I know that it worked? Well, before you enable the RBS Provider, the AllDocStreams table in your content database will show values in the “Content” varbinary(max) column like this:

After you enable the RBS Provider, the AllDocStreams table in your content database will show NULL values in the “Content” varbinary(max) column and instead show values in the “RbsId” varbinary(64) column like this:

Also, if all goes well, then the BLOB Store directory that you specified in the InstallProvider.cmd configuration file will have BLOB files showing up like this:

Finally, I’ll say that while this is a great exercise to show you what’s possible with an RBS Provider other than FILESTREAM, the CodePlex provider should probably be used at your own risk in production. In my mind, this is an extremely useful but elaborate teaching example. It should be thoroughly tested before being deployed in mission critical systems.

It's also important to know that Microsoft ISV Partners are working on several different RBS Providers that will provide feature support for file shares, BLOB encryption, BLOB compression, REST, and Cloud Storage as well as support for specific storage platforms such as Fujitsu Eternus, EMC Centera, and Hitatchi HCAP.

I’m convinced the flexible RBS Provider options in the works will squash any remaining FUD out there regarding SharePoint storing binaries in the database. The scalability story was already great, and it is so much better for SP2010 that it makes my job as a SharePoint evangelist that much easier!

Cheers.

SharePoint 2010 Evolution Conference - London

Russ Houberg — Tue, 05 Jan 2010 20:38:30 GMT

So I've been spending the last month or so disecting the RBS framework in SharePoint 2010. I've already succeeded in enabling the CodePlex RBS example provider in SP2010. I'm sure I'll be posting a "how to" on that in the coming weeks.

But I'm also cooking something else. The CodePlex RBS SP2010 example is the foundation for an RBS Deep Dive session that I'm assembling for the SharePoint 2010 Evolution Conference in London. I'm currently scheduled to speak on Wednesday (4/21/2010).

The session will have just a few overview slides on the concepts of RBS and how RBS affects scalable architecture and then I'll dive right in to demos and show you how all the pieces and parts of the RBS subsystems in SharePoint 2010 and SQL Server 2008 R2 work together!

Come check it out!

Slides from SharePoint Saturday KC

Russ Houberg — Tue, 22 Dec 2009 16:15:13 GMT

I'm finally getting around to posting the slide deck that presented at SharePoint Saturday KC. The topic was "Architecting for Scale in SharePoint 2010"

It's a pretty juicy deck with lots of detail that will help you with storage architecture and SQL tuning. It also provides good background on Remote BLOB Storage (RBS) and the improvements to the search subsystem in SP2010.

You'll find the slides here:

Architecting for Scale in SharePoint 2010

SharePoint Saturday Kansas City, MO - 12.12.2009

Russ Houberg — Wed, 09 Dec 2009 18:24:24 GMT

So this weekend I'm off to present at SharePoint Saturday in Kansas City. It is shaping up to be a stellar event and it looks like there are still a few (free) seats left!

I'll be presenting a session on "Architecting for Scale in SharePoint 2010". Here's the session description:

"Scaling SharePoint has presented many challenges in the past. Those days are gone. Learn how new features like Remote BLOB Storage (RBS), a dramatically improved search subsystem, improved database efficiency, and resource throttling in SharePoint 2010 can be leveraged to scale SharePoint to heights we’ve not seen before."

I'll be covering topics such as Storage Architecture, SQL Tuning, Scalable Taxonomy, Throttling, RBS, and Search Topology in SP2010! I know, it's a lot right! Well, I'm going to try to squeeze it all in along with 2 demos. The slide deck is jam packed with juicy tidbits!

Is it possible? Could we potentially see 1 BILLION documents in SharePoint??? Well, it's pretty much my at the top of my professional bucket list!

Hope to see you there!

SP2010 Single Server Complete Install - An Extra Note

Russ Houberg — Thu, 19 Nov 2009 23:59:12 GMT

So I set out to fire up a fresh SP2010 install with the new BETA bits today...

Since I wanted to do a "Complete" install using SQL 2008 (not Express), I promptly referenced Neil Hodgkinson's "From the Field" blog top on "Single Server Complete Install of SharePoint 2010 using local accounts." Excellent post Neil! It was VERY helpful.

But being the control freak that I am, I wasn't content going back to the world of GUID laden Administration databases. So I did a little digging in the New-SPConfigurationDatabase powershell cmdlet and I found a handy extra parameter.

Instead of using just "New-SPConfigurationDatabase", try this...

PS C:\Users\spadmin> New-SPConfigurationDatabase -AdministrationContentDatabaseName SP2010_CentralAdmin_DB

It works as advertised:

Yeah, OK. It's a pretty minor little feature. But that's what blog posts are for, right? Obscure little features hidden in the corners of the world?

Now... If I could only figure out how to get the GUIDs out of all those crazy SP2010 Services databases! There's always tomorrow.

SP2010 Coming Out Party!

Russ Houberg — Wed, 14 Oct 2009 05:00:00 GMT

I'll talk about the SharePoint 2009 conference in a sec, but a few have been asking where I been the last couple months? Well, I've been grinding on a huge solution implementation for a gigantic customer that unfortunately must remain nameless. It will be the largest SharePoint based document imaging system that I've had the pleasure to work on. Think 100 to 200 MILLION documents in SharePoint! Our CERT environment alredy has 15 million. Cool stuff! So that's where I've been. Where am I going? Well...

LAS VEGAS BABY!

The time is nearly here for the SharePoint 2010 feature set to be unveiled to the world. Folks, there is a lot to be excited about here. There are some clutch new features that absolutely crush the rediculous FUD out there that try to play SharePoint as an limited ECM / Document Imaging repository.

So, on that note, I'll mention that I'm co-presenting a session called "ECM on SharePoint 2010 - Features that Rock!". Ok, the title could have been a little more creative, I'll admit! But the content is solid. I'll be touching on a few key SP2010 features that allow us to to handle virtually ANY size structured Imaging Repository right in SharePoint. We've really never hit a document limit that we couldn't architect around with enough gear (and storage) in MOSS. But Microsoft really hits it out of the park with SP2010 and I can't wait to tell you about it!

For those of you about to head out to Vegas, come see Chris Caplinger and myself at 4:30pm on Monday in the South Pacific C room.

Then on Tuesday night, KnowledgeLake and our Fujitsu friends are hosting a sweet beach party from 7pm to 11pm with a live concert from Huey Lewis and the News! Probably even more important is the OPEN BAR!

Also, come stop by the KL booth if you're remotely interested in Document Imaging. KL will be unveiling some sweet new tech that will absolutely AMAZE! Don't forget to bring your KL insert sheet from your swag bag! You've got a shot at a whole lot of free $$$ to go blow at the poker table!

</ShamelessPlug>

Sorry about that whole plug thing. I'm just jazzed about this conference and I wanted to let you know where we're going to be! Needless to say, if you're not among the lucky 7,000 or so folks at this sold out conference, then dang... sorry about that! If it makes you feel any better, I'll be starting a new series of blog posts based on my SPC2009 presentation on ECM for SP2010. I'm going to show you how SharePoint 2010 can be architected to scale efficiently from tens of thousands of docs to hundreds of of millions!

Stay tuned!!!

Microsoft Certified Master - SharePoint 2007

Russ Houberg — Tue, 28 Jul 2009 16:11:42 GMT

So back in March, I blogged on the SharePoint Master (MCM) program. It was an amazing experience. Unfortunately, even though I did pass the grueling qualification lab, I only passed 2 of 3 written tests on the first pass.

Well, I just knocked out my retake and the results are in.... PASS!

It really has been an interesting experience. There was a lot of pressure to pass on that 2nd attempt. But I made it! I have to thank God for that!

I know that it's not always "cool" to talk about your failures but I wanted to go ahead and write this post anyway to encourage those who might not skate straight through on the first attempt at the written tests or the qualification lab. You have to ask yourself, "Why did I attend MCM training?". Was it for the information? If that's all then maybe not having the cert is good enough. Was it for the networking? Well, you probably already scored that? But is it enough?

My thought is this. Why spend all that time and pain on the journey and not travel that last (but very difficult) mile? By the time the training and the initial tests are over, you've already invested so much.

So I just wanted to encourage anyone who came up a bit short during class. Put in the time... Work hard... and go get that MCM!

Remember this... The harder you work to achieve a goal, the better it feels when you get there! Because, in the end, it's not about the result... It's about the process along the way.

Cheers,

Russ Houberg
SharePoint MCM

Microsoft is Forcing Me to use VMWare!

Russ Houberg — Wed, 01 Jul 2009 16:10:11 GMT

Ok, so I have to get this off my chest if for no other reason to add my voice to what I hope will be a loud shout from the SharePoint development community.

So follow me through a logic exercise here...

REQUIREMENT - Multiple VM Based SharePoint Development Environments: Due to the nature of being a consultant, I work on a wide variety of projects in a relatively short period of time. I develop on or otherwise support a large number of SharePoint instances. It is not practical for me to rebuild my laptop everytime I need to develop for a different customer. Virtual Machines are an absolute requirement for my job. In fact, I've gotten to the point where my Host OS contains very little software. If I need to rebuild my machine, I can do so in just a couple hours and all my dev environments are safe.

Ok, so we have the requirement. Now let me take you through a few facts:

FACT: SharePoint 2010 will require a 64-bit OS
FACT: Virtual PC 2007 and Windows 7 Virtual PC DO NOT support a 64-bit Guest OS

Houston... we have a problem. That means that very soon, I am going to be forced into 1 of 2 paths. Either run Windows Server 2008 (or R2) with Hyper-V as my host OS or run VMWare Workstation in order to support the 64-bit guest!

Yes, I know that Hyper-V is an option. I've used it. In fact, I love it.... for what it was intended for which much more "production" in nature than what I want for a dev platform. There are many drawbacks to using this type of rig.

When you enable Hyper-V, graphic acceleration seems to really suck. Flash video grinds your machine to oblivion and 3D gaming has some real challenges (I don't game often, but occasionally it's very refreshing to blow some stuff up when you're bored in a hotel room.)
Windows Server 2008, while it shares the kernel with Vista/ (or Windows 7 for R2), is not a desktop operating system. Sure you can fiddle with it and get it close, but at the core, it's been optimized to run as a server. You just don't get all that warm fuzzy that you get with Windows 7 where everything is geared towards being a workstation.
The virtual integration components cause incompatibility problems with Virtual PC. I can't just build up VMs and share them with team members because none of them are running Hyper-V. (Yes, I know they can be converted, but it's not a quick process.)
The whole concept of easy desktop integration (drag and drop, etc) is not there with a Hyper-V VM.
A lot of the software I use (virus scanning, backup, etc) doesn't work on a server platform unless I pay "server prices".
Uhhhh.... Licensing anyone? Can you imaging the cost for licenses if everyone in your consulting organization had to run Windows Server 2008 as a workstation? This is perhaps the most profound reason.

So yes.. I get that Hyper-V is technically an option and in fact I used this for a while (didn't like the performance hit for workstation applications). But until MS provides "cheap" licensing for Windows Server 2008 with Hyper-V as a "Workstation" (and includes it in our MSDN Universal Subscription), it is not a practical option.

So that leaves me with VMWare. Man, I am not into bashing on the MotherShip. But I think someone dropped the ball here!

Thankfully, I can disable that little VMWare icon in the system tray, so when I RDP into my virtual server during demos and presentations, I can hide the ugly truth from session attendees.

The fix is doable. VMWare has already figured it out. So I'll go ahead and (un)officially ask...

Hey Microsoft. Can we have x64 Guest OS support for Virtual PC?

TIA,

Russ

Consider me a Tweeter

Russ Houberg — Tue, 30 Jun 2009 18:53:14 GMT

Well, I finally took the plunge. I jumped on the bandwagon. I finally buckled under the peer pressure. I am now on Twitter.

http://twitter.com/rhouberg

I don't yet understand the magic of Twitter, but not unlike how beer was an acquired taste, I'm going to dive in and see if I can plug into Twitter like the rest of the free world has.

So we'll see. I'll give it the college try. I'll try to tweet at least daily so that everyone with more time on their hands than I have can check out what I'm doing!

tweet...tweet