Making Search Better Within Microsoft

[Guest Bill Baer]

The Challenge

 

 

Five years ago employee satisfaction with finding information within the company was very low. it was the lowest rated it service among all those we surveyed about. Related surveys done by other teams supported this, for instance that our software engineers “finding information” as one of the most wasteful frustrating activities in their job, costing the company thousands of man years of productivity.

 

 

 

A project team was formed to improve this. In the years since we have pursued:

 

• Improving search result relevance
  
• Improving search content completeness
  
• Address content quality issues
  

 

 

 

The Microsoft Search Environment

 

 

Microsoft has >300,000 employees working around the globe, and collectively, our employees use or access many petabytes of content as they move through their workday. within our employee base, there are many different personas who have widely varying search interests and use hundreds of content sources. Those content sources can be file shares, Microsoft sharepoint sites, documents and other files, and internal websites. our employees also frequently access external websites, such as hr partners’ websites.

 

 

 

 

 

 

 

 

We began with user satisfaction survey net score at 87 (scale of 1-200, with 200 being perfect). We have reached satisfaction of 117. Our goal is 130+.

 

 

 

What We’ve Done

 

 

Core to our progress has been:

 

1. Understanding the needs of the different personas around the company. At Microsoft, personas are commonly clustered based on three factors: their organization within the company, their profession, and their geographic location. For example, a Microsoft seller working in Latin America has different search interests than an engineer working in China.
   
   1. 
      
   2. Has resulted in targeting bookmarks to certain security groups.
      
   3. Has led to outreach to certain groups and connecting with efforts they had underway to build a custom search portal or improve content discoverability.
      

 

 

 

1. Understanding typical search behavior. For instance, the diagram below shows that a relatively small number of search terms account for a large portion of the search activity.
   

 

 

1. We ensure bookmarks exist for most of the high frequency searches.
   
   
2. We look for commonalities in low frequency searches for potential content to connect in.
   
   

 

 

 

1. Improving content quality. This has ranged from deleting old content to educating content owners on most effective ways to adding metadata to their content so it ranks better in search results. As part of our partnership with this community, we provide reporting on measurable aspects of content quality. We are in early stages of pursuing quality improvement, with much to do in building a community across the company, measuring, and enabling metadata.
   

 

 

 

 

 

 

 

1. For those site owners actively using this reporting, we have seen a decrease of up to 70% in content with no recent updates.
   
   

1. Utilizing improvements delivered in product, from improved relevance ranking to UX options like custom filters.
   
   1. Making our search coverage more complete. Whether it’s via bookmarks or connectors, there are many ways of making the search experience feel like it covers the entire company.
      
      1. Analytics. Using SharePoint extensions, we capture all search terms and click actions on our corporate portal’s search page. We’ve used these extensively in deciding what actions to take. The sample below is a report on bookmarks and their usage. This chart alone enabled us to remove 30% of our bookmarks due to lack of use.
         

       

      

       

       

       

       

       

      In analyzing the click activity on our corporate portal, the most impactful elements are:

       

      Bookmarks	Are clicked on in 45% of all searches and significantly shortens the duration of a search session. We currently have ~1200 bookmarks making for quick discovery of the most commonly searched for content and tools around the company.
      Topics	Are clicked on in 5-7% of all searches.
      Connectors	Are clicked on in 4-5% of all searches.
      Metadata	Good metadata typically moves an item from the bottom of the first page to the top half and from page 2 or later onto the bottom of page 1.

       

       

       

      Additional details will be published in later blog posts. If of interest, details as to exactly what Microsoft search admin does in its regular administrative activities are described here.

       

       

       

      Business Impact of Search

       

       

       

       

      As shown in the preceding table, roughly half of all enterprise-level searches benefit from one of the search admin capabilities. Employees who receive such benefits average a one-minute faster search completion time than those whose searches don’t use those capabilities. Across 1.2 million monthly enterprise-level searches at Microsoft, that time savings amounts to more than 8,000 hours a month of direct employee-productivity benefit.

       

       

       

      We achieve these results with an admin team of part-time individuals, investing a total of <300 hours per month doing direct search administration, responding to user requests to help find individual items, and maintaining a self-help site which advises employees on where and how to search best. We also have a larger improvement program striving to improve information discoverability across the company.

       

       

       

      So 5 years into our improvement efforts, we have significantly improved user satisfaction, can now measure the productivity impact search is having, and built numerous partnerships across the company that are expected to continue yielding improvements in the years to come.

       

       

       

      Lessons from this work is actively improving search has significant payback. The first step is to actively administer search, doing whatever helps the most popular searches to deliver the right results.

       

      Continue reading...

   
   
2. We currently have 7 connections, one of which is custom built and brings in 10 different content sources. This content is clicked on in 5% of searches on our corporate SharePoint portal.
   
3. About half of our bookmarks (~600) point to URLs outside of the corporate boundary, such as third-party hosted services.
   

We have seen steady improvement in result ranking.
We also take advantage of custom filters and custom result KQL.
We use Viva Topics. Topics now receive the most clicks after Bookmarks.