Brian on Collaboration

How Search Products Work

It seems everybody has an enterprise search solution nowadays...vendors from all walks (taxonomy & ontology, content management, portals, document management etc.) are purporting the benefits of their product over others. Search is a huge component of effective collaboration, because, after all...if you can't find the document/ content/ thread/ issue/ task /email that you were collaborating on again, how can you collaborate on it? Findability is king.

So how can you leverage search to collaborate better and find more relevant content? In a multi-part blog, let's review the search basics about products in this space:

Every search product has at least three major components: The Gatherer (Indexer), The Ranker & The User Experience (vendors may use different words to represent their search product). There's a plethora of administrative or higher level stuff that I can't go into to such as reporting (who is searching for what), or index syndication or auto-categorization & taxonomy development, which are also important topics when choosing a search engine, but we've got to start somewhere. I should note that whether we're talking web search, enterprise search or desktop search...all of these concepts exist.

The Gatherer / Indexer is responsible for going out and finding content and creating one or more indexes. All gatherers are limited in the types of repositories they can index and the type of files they can properly index...which really is based on their ability to parse the content of the files and bring the content into their index. In the Microsoft world, the ability to filter new file types is facilitated through an ifilter. There are many more available than what's out of the box with Microsoft's search solution: SharePoint. These ifilters are also used by other search products that leverage the Microsoft platform. A gatherer is also limited by the types of repositories it can search. Some gatherers can only browse file shares and folders, while others can crawl links on websites. Yet others can crawl Line of Business systems such as document management products. As a final note about gatherers: robust products in this space have adaptive indexers, which become more and more intelligent over time to updating their index with the most critical, used or important information quicker than old or less frequently used information.

The Ranker is responsible finding the best results based on the searched terms. Simple raking looks only for "free text" matches to the searched words and returns some sort of simple result list (i.e. organized by date). Other more complex raking products use special, complex algorithms to sort results and determine the relevancy of a result to the query. Good systems usually use meta data (such as document description) to help drive up the relevancy of an item search results. Sometimes this is assisted with things such as thesauri, which look for similar words or word rephrasing (RCMP could also be "Royal Canadian Mounted Police, or R.C.M.P.). Word stemming is another more complex feature which looks variations of parts of a word (i.e. ran, running, run, etc.) There's many more features that assist in raking in various products. In SharePoint, you can define "best bets" which essentially will move a search result up to the top of the list...other products have similar features.

The User Experience is what most people are familiar with when they search. The UE has two parts the search interface (i.e. simple search, advanced search) and the results interface. For Google, most people use the simple search box instead of moving to the advanced search page. Search results contain a web page title and snippet of text from the page (hopefully relevant). They also contain a link to searching for more relevant items. Search results pages differ from product to product (and really should...what a user sees in the results is highly relevant to what they are searching for)...although Google seems to be defining the standards on this one. Yahoo still includes what they call "Category" with (some of) their results (i.e. Companies >Technology >Habanero). While this type of classification (including some connection with a planned taxonomy or ontology) is probably very relevant in internal enterprise searches...it seems to be less and less relevant on the Internet (see also msn search).

Last year, we reviewed some key search products for websites. We're about to update the report, but here are the 2004 results.

The Portal Interface

There is no doubt a Portal brings more value to a business than Intranets of old. No longer a a simple venue for disseminating key corporate news and knowledge in a branded internal website, Portals have become the focal point for many knowledge workers. Key to this success is the orientation devices given to users: Does the User Interface assist users in getting what they need? Does the Information Design make sense to its users? Generally this is called the User Experience. In Portal product implementations, such as SharePoint (but this could apply to any product), one of the key challenges is implementing a great user experience (high usability) and ensuring corporate culture and branding needs are met, while not deviating too far from the base features of the product you're implementing.

Which brings us to our next big SharePoint question:

2) We don't like the out of the box SharePoint interface...can we change it?

The answer is: of course. SharePoint is a web based product and is built upon standards such as cascading style sheets. There are also a core set of templates responsible for driving the majority of the SharePoint User Experience: templates which can be modified to meet any design needs. The REAL question is: how far should you go? You would never think to completely change the user interface of a desktop product such as Word or Excel. Should you really change a Portal product?

As usual, there's no one correct answer. Every company's branding requirements differ. The ability to train employees versus the ability of employees learning things on their own also makes a difference. Deviate too far from the original intent of the product, and you could not only be working against the purpose of the solution, but also undertaking a costly and risky venture.

The key questions to ask are: What are your branding requirements? What is your budget for this type of undertaking? What resources do you have to maintain and enhance the User Experience over time? Do you have the ability to increase training to deal with less than ideal usability issues with the product you're dealing with, or do you have the time it would take to make the product easier to use?

Once you've answered these questions, you're ready to start working. Microsoft has a great white paper on this subject: Branding a SharePoint Portal Site.

The difference between SharePoint Portal and WSS

As a Microsoft Gold partner, the company I'm with (Habanero) has an affinity to Microsoft solutions. The key offering in the Collaboration space is Microsoft's Office System and specifically SharePoint. There are probably no less than 127,234 different advice sites on SharePoint, so I'm pretty keen not to reinvent the wheel, but over the next few days, I'll answer the most common questions we get.

1) What's the difference between SharePoint Portal Server and Windows SharePoint Services?

This is by far the biggest point of confusion for most people.

Windows SharePoint Services (WSS) is a technology built into Microsoft's Windows Server 2003. It's a service that allows you to create team sites to assist with smaller group collaboration with features such as document and form libraries (somewhat akin to a shared file system folder, but with check in/out, version control, integration with office, etc.), custom lists such as issue, task, contact and any other types of list you can think up (think of a list as a set of columns and rows), simple search-ability, discussion boards, image libraries (for storing photos and other images) and surveys. That all said, there are valid reasons to create team sites that the whole company might visit. It takes a good process to determine when to put something in a team site and when to put it in the portal. That's where we usually help out with our SharePoint Information Design process. The good news: WSS is free. The bad news...it's so good you'll quickly need...

SharePoint Portal Server (SPS) is server product offering (read: not free) that helps you organize all of those great team sites you built using WSS. It also has many more enterprise focused features such as enterprise search which can search your company file shares, external website or other repositories and user profiles: allowing you to create a very rich enterprise employee directory (including great integration with Active Directory). It also includes features such as "My Sites" which allow each employee in an organization to create their own personal private and public web-spaces and a great tool for publishing corporate news. Even though it's called a portal, that doesn't mean a company would only have one of them. In some cases companies need more than one for various reasons. The Microsoft Solution Accelerator for Intranets briefly talks about multi-portal scenarios.

There is a Microsoft document on this subject, but it's not the best in the world...but for reference it's available on Microsoft's site here.

Microsoft Acquires Groove

Collaboration for the offline worker has always been a challenge. Personally, I was a big fan of the Lotus Notes model of syndicated databases to ensure data synchronization for offline use. Unfortunately, Lotus had a terrible user interface and a poor marketing message. Microsoft's recent acquisition of Groove and the inclusion of Groove's founder Ray Ozzie (the original brain behind Lotus Notes) into the Microsoft leadership team is a great move on their part...I'd expect to see Groove tightly integrated into the next version of Microsoft Office (Office 12).