Wow.  Long break from blog posting!  Sorry about that.  Once again, duty calls.  I was booked solid from March until mid-summer.  Then it was vacation time.  So there was little time to generate meaningful blog post content. What happened in the meantime?  Well, Microsoft released this tiny little server product called SharePoint Server 2010 and a tiny little client application called Office 2010!  Yeah, yeah... I know.  I'm a bit behind.  Well, I have been playing with SP2010 and over the coming months, I'll be posting on some interesting things around Enterprise Search in SP2010.  Our scalable architecture options have grown...

Over the holiday break I spent quite a bit of time trying to get a real good handle on exactly how SharePoint 2010 and RBS interact in order to externalize content to a BLOB store.  As part of that effort, I implemented the CodePlex RBS Provider as an exercise to learn more about the roles of SQL Server 2008, the RBS Framework, and SharePoint 2010. So why would you want to try out this exercise?  Well, there are some limitations to the FILESTREAM provider.  First, it only works with local volumes on the SQL Server.  So if you want to store...

So I've been spending the last month or so disecting the RBS framework in SharePoint 2010.  I've already succeeded in enabling the CodePlex RBS example provider in SP2010.  I'm sure I'll be posting a "how to" on that in the coming weeks. But I'm also cooking something else.  The CodePlex RBS SP2010 example is the foundation for an RBS Deep Dive session that I'm assembling for the SharePoint 2010 Evolution Conference in London.  I'm currently scheduled to speak on Wednesday (4/21/2010). The session will have just a few overview slides on the concepts of RBS and how RBS affects scalable architecture and then I'll...

I'm finally getting around to posting the slide deck that presented at SharePoint Saturday KC.  The topic was "Architecting for Scale in SharePoint 2010" It's a pretty juicy deck with lots of detail that will help you with storage architecture and SQL tuning.  It also provides good background on Remote BLOB Storage (RBS) and the improvements to the search subsystem in SP2010. You'll find the slides here: Architecting for Scale in SharePoint 2010

So this weekend I'm off to present at SharePoint Saturday in Kansas City.  It is shaping up to be a stellar event and it looks like there are still a few (free) seats left! I'll be presenting a session on "Architecting for Scale in SharePoint 2010".  Here's the session description: "Scaling SharePoint has presented many challenges in the past.  Those days are gone.  Learn how new features like Remote  BLOB Storage (RBS), a dramatically improved search subsystem, improved database efficiency, and resource throttling in SharePoint 2010 can be leveraged to scale SharePoint to heights we’ve not seen before." ...

Ok, so lets do a little math here. In SharePoint 2007, you can define only 1 Records Center at a time for official files.  A single site collection (including a Records Center) in its own content database might contain somewhere around 1 million documents at a meager average of 55KB per document.  That keeps us under the 100GB content database recommended size limit. 1 million documents?  Seriously?  I suppose you could try to manage multiple records centers but this would get interesting if you actually needed to put something in legal hold in multiple records centers.  So for all practical purposes, we were severey...

Binary Large Objects, or BLOBs as the SQL types like to call them, are the byte arrays that represent documents and other files in SharePoint.  Typicaly, they are stored in the SharePoint content database.  The reality is, the ECM industry has known for decades that RDBMS is not the best place to store BLOBs.  SQL database storage needs to be high IOPS and low latency... translated... EXPENSIVE storage.  It's much more efficient if we are able to store the BLOBs on lower cost, possibly even archival-class storage while we continue to invest in high performance storage for the structured content metadata. As...

For the last several years, I've worked on several projects that stretch the recommended limits regarding the amount of content that SharePoint can handle.  Back in December of 2007 I started on an interesting scalability journey with a couple of awesome guys at Microsoft.  The first, Paul Learning, is a quality MCS SharePoint guy out of Detroit.  The second, Andy Hopkins, served as our red-tape bulldozer.  The three of us worked to put a small server room full of Fujitsu blades and storage arrays to good use in order to prove that SharePoint could do 50 million documents. The result of our efforts was a very...

I have been very fortunate over the last several years in that I've had many opprtunities to architect many extremely high scale SharePoint systems.  Everything from your standard 3 million document Imaging Repository to systems with 10's and even more than 100 million documents (thanks to FAST ESP!) As I look back on SharePoint 2003 and even to existing SharePoint 2007 solutions, there have definately been several challenges as we design systems that can handle the millions of documents we throw at them.  So it is with great pleasure that I am able to present my 4 favorite improvements in SP2010...

Well, here I am, 24 hours past the SPC2009 ride.  Now I feel like I'm on a long slow decompression ascension (lame scuba reference) to the public launch of SP2010 next year some time. Now that the SP2010 veil is lifted, I'm finally free to talk about some incredible new SP2010 features that are abslutely CLUTCH for scaling SharePoint to sizes we may only have dreamed of before! I have all these ideas floating around for what I want to say.  So I started writing this post and it just plain grew too large.  So I decided to break it up into seperate posts in a SP2010 Scalability...

I'll talk about the SharePoint 2009 conference in a sec, but a few have been asking where I been the last couple months?  Well, I've been grinding on a huge solution implementation for a gigantic customer that unfortunately must remain nameless.  It will be the largest SharePoint based document imaging system that I've had the pleasure to work on.  Think 100 to 200 MILLION documents in SharePoint!  Our CERT environment alredy has 15 million.  Cool stuff!  So that's where I've been.  Where am I going?  Well... LAS VEGAS BABY! The time is nearly here for the SharePoint 2010 feature set to be unveiled...

So this has been a long time coming.  It's been over a year since I originally wrote the Storage Architecture whitepaper and it was in need of refreshing. While the original content was certainly accurate, there were a few things that I wanted to clarify, particularly after having attended the SharePoint Master training.  There were also a couple things for which conventional wisdom has shifted a bit like the fact that we are now allowed to split the SearchDB into 2 file groups. Also, I wanted to add in some additional guidance on a few more topics that I receive a LOT...

Ok. So I've been waiting for this day for about 3 months now! The culmination of the efforts of several incredible individuals has officially materialized. ANNOUNCING: Using Microsoft® Office SharePoint® Server to implement a large-scale content storage scenario with rapid search availability This document is a scalability whitepaper that has been reviewd by Microsoft and posted on TechNet. This white paper presents the key decision points, architecture design and definition, test cr

Ok. So after writing that last post on the 100GB database limitation I got some nice feedback from several people. Also, I'm finding that this is information that really needs to get "out there". Far to often, KL is brought in to implement a document imaging or file share conversion solution in SharePoint. The problem lies in the fact that often times the sandbox that everyone learns to play in becomes production! This ends up presenting a lot of challenges for us as consultants when tryi

Right...he's a Star Wars geek. Check. I wanted to discuss something else I heard a lot about at the SharePoint 2008 conference. The 100GB database limitation. Organizations are now looking at SharePoint as a legitimate large scale application. They want to believe. They want to engage. Then they all hit their heads on the same thing. 100GB database size recommendation. Folks... it's a recommendation. The answer to the question of can we go bigger is the same as what I heard several

So the Microsoft SharePoint Conference 2008 wrapped up on Thursday. What an amazing ride! It blows me away to see how SharePoint is absolutely exploding. It fires up my passion for SharePoint technology even more! Paul Learning, Andy Hopkins, and I did finally present during the last session slot of the conference. There weren't as many attendees as I would have like to have seen, but what the heck, I usually bail early on the last day too! Anyway, with the smaller group of only about 30

Well, the last couple weeks have been a whirlwind. In the past few months, I've been working hard with Paul Learning (Microsoft Consulting Services), Andy Hopkins (Technical Development Manager, Microsoft) as well as a few guys on the MS SharePoint Product team. Paul and I have been busy loading up a massive Fujitsu server farm while Andy burns through logistical hurdles all in an effort to develop a SharePoint Scalability whitepaper. It looks like we're going to have this whitepaper done i