Designing search systems

In document- and content-oriented applications and websites, the quality of the user experience often depends on the user being able to find what she is looking for, and so effective search functionality becomes critical when there is a large repository of content.

Let’s examine search systems and look at what factors you have to consider when designing one.

Searching

In a search scenario, the user enters a search query, and in response, the system retrieves matching items from a repository.

Alternatively, some people prefer to think of searching as a filtering mechanism: The user chooses filter criteria, and the system filters out any items that do not match those criteria.

When designing a search system, you need to think about and decide on the following:

• What types of items are in the repository to be searched?  Files, documents, images, videos?  Can the search return multiple types of items?

• What is the form of the search query?

• What is the scope of the search?  For example, when searching documents, are matches for the search term sought only in the text of the document, or are metadata such as the filename, document title, document properties, and any tags searched as well?

• Does the search attempt to find appropriate variations of the search term?  Will the technique of stemming be employed so that a search for “eat” also finds instances of inflected forms like “eats”, “eating”, and “ate”?

• How are search results presented? When the user submits a search query, are the results presented on a separate page? Or are the results presented on the same screen and filtered in real-time as the user modifies the search criteria?

If there are many results, are the results broken up across multiple pages?

If the search results are textual documents, is a small snippet of the text surrounding the match presented to provide context? What if there are multiple matches within the same document?

• Can users save and retrieve queries that they expect to use frequently?

Search quality

The perceived quality of search results has a big impact on the user experience. Search quality is a function of the following aspects:

• Accuracy of recall: Search results must include all of the items that match the search criteria, and exclude any items that do not match the search criteria. If the search does not locate all of the matching items, or if irrelevant items are presented, then users can become frustrated when their expectations are not met. In some cases, users may also be misled by inaccurate search results.

• Relevance: It’s desirable to sort the results to show the most relevant items first, although the definition of “relevance” depends on the application. Recent news articles would usually be presented ahead of older articles, for instance, and an article featuring the phrase “financial crisis” in the headline and containing multiple references to that phrase in the body would be considered more relevant than an article that mentioned that phrase only once. Users can become frustrated when they have to dig through “noisy” results to find the items that they perceive to be relevant.

• Performance: Search results should be delivered promptly. But there is often a trade-off involved here; building performant search systems can sometimes be a very tricky and costly technical challenge.

Lookups

A specialized form of search functionality is the lookup function associated with some data-entry fields. Some fields have constraints on what values are valid, but there are too many valid values to make a drop-down list practical.

For example, a customer number field might allow the user to enter a customer number directly, but it is rare that users will have memorized the number for a particular customer, and there may be tens of thousands of customers on file. In this case, the field should provide a lookup button (or a shortcut keystroke) that allows the user to search for a customer by name. Upon selection of a customer, the customer number field is then populated with the corresponding customer number.

Finding text within a document

In document-based applications like word processors and web browsers, users will expect to be able to find all of the instances of a search term within the current document. (The generally-accepted terminology in English-language software is to “search” to locate instances of a term within a repository of documents, and to “find” to locate instances of a term within an individual document.)

For editable documents, the ability to replace instances of the search term with another term will also be expected.

Some applications will highlight all instances of a search term within a document:

Alternatives to searching

While searching is convenient, in most applications, it should not be the only means of navigation.

In many systems, there will be some items that are accessed frequently. You might offer shortcut links, for example, to provide rapid access to the most popular items, the most recently-added items, or the user’s most recently-accessed items. Allowing the user to bookmark locations or search results may be useful as well.

We can also imagine cases where the user might prefer to browse the contents of the repository. For example:

• the user doesn’t know what is in the repository; or,
• the user is looking for something specific, but doesn’t know the right words to describe it; or,
• the user has a vague idea of what they want, but it may not be easily describable in words — it’s a case of “I’ll know it when I’ll see it” (for example, the user wants to find an action photo of an athlete playing baseball, but the specific baseball player doesn’t matter as much as the general appearance and composition of the photo).

Hierarchical menus, keyword indices, and sitemaps can be useful strategies for allowing users to browse the repository and discover content.

Some systems might benefit from allowing users to tag items with keywords. Browsing the list of keywords then becomes another way of getting an overview of the repository contents and accessing items.

Some applications can take advantages of metaphors that simulate real-world situations. For example, a website for a bookstore or library might allow users to view the covers of books in various categories, providing an experience similar to browsing titles on a physical bookshelf.