Cheryl Lemmens - Indexing and Editorial Services

Web Site Indexing: Feature Article

Getting to the Heart of the Matter: The Art of Indexing Web Sites
By Cheryl A. Lemmens

AUTHOR'S NOTE: Since this article was first written, many of the Web sites to which reference was made have been completely redesigned, resulting in the loss of some indexes and maps. As well, some online articles about site indexing have disappeared, although the references have been retained in the bibliographic section, with explanatory notes. This all goes to show that the Web is, by its very nature, an ever-changing, ever-evolving entity — but it also illustrates Jakob Nielsen's point about the value of keeping Web pages alive forever!

Introduction
When we think of an index, we think of the traditional back-of-the-book index found in works of non-fiction. We also know how frustrating it is to try to find information in a book with a poorly-constructed index (or, worse, no index at all). But do we need indexes on the Internet? Indeed, it is perhaps not as obvious that most Web sites, particularly large ones, should also contain their own indexes — and, if not an alphabetical index, at least a section-by-section site map.

The art of helping users find their way around a Web site, unfortunately, is still one which some companies and organizations have yet to learn. Not all Web sites contain an index or map. Some provide indexes and site maps that are poorly designed or incomplete. And some offer search utilities that don't exactly measure up to expectations.

Back to Top

The Index: From Print to Cyberspace
The index — defined, according to British indexing standard BS 3700:1988, as "a systematic arrangement of entries designed to enable users to locate information in a document" — is a traditional and integral component of print publishing, but has not yet made a complete transition to the Internet world. Web sites sometimes appear to be created without any thought to helping users find things quickly and easily. A search function may fill the void, but — as Jared Spool and his team at User Interface Engineering found — is often frustrating to use.

The solution is an index or map designed by someone who has gone through the site from start to finish, one who knows all the sections and subsections. This type of tool is invaluable to a user, whether it be a site map arranged by section or a traditional index arranged in alphabetical order.

Of course, as the American Society of Indexers (ASI) notes, "Indexing the Web is not a simple task." But there is hope, as the ASI's article Indexing the Web goes on to say:

Some organizations are seeing that including indexes on their web sites is just as important as including indexes in books and online manuals. We've seen some good and some bad, some computer-generated, some obviously not constructed by professional indexers, and some professionally prepared. In any case, all site owners should be commended for recognizing the need for an index.

Back to Top

The Search Utility as an Alternative to Indexing
Before going on to discuss site indexes and maps, a brief look at the search utility as an indexing alternative is in order. Many Web sites provide this type of tool in place of, or even in addition to, their indexes or maps. The usefulness of the search utility, however, is debatable, and while this paper does not attempt to argue its pros and cons, it is worth noting briefly what has been said about search tools as compared to Web site indexing.

The ASI acknowledges that search utilities are "certainly better than nothing," but adds that users run into the same problems as they do in conducting other types of full-text database search:

The major problem is, of course, relevancy of items found via the search. For example, on a software publisher's site a search for a product called Home Office, ends up retrieving all documents with the word "office" in them, because at the end of every page is the word "home". If there is a site index, you can go directly to the "H" section, and find the one relevant page, thus saving time for other projects.

It has been suggested that the problem presented by search utilities — "a high level of retrieval and a low rate of relevancy" — could be solved by some fine-tuning. As the ASI goes on to note:

Most search engines actually search an index, a list of terms that robots return from their voyages. Indexes could be manipulated or constructed for these engines to use, especially on an Intranet, by careful use of the META tag. This is an area that indexers should be researching and understanding, so that we can index for these engines.

Most indexers, however, and undoubtedly many users, feel that "the precision rate most search engines provide is just not as good as true indexing." This sentiment is supported emphatically by usability expert Jared Spool, whose article Why On-Site Searching Stinks seems to encapsulate in its title the problems with search utilities. In marked contrast to the ASI, which comments that search utilities are "better than nothing," Spool calls them "worse than nothing — significantly worse."

Spool and his team at User Interface Engineering found that "using an on-site search engine actually reduced the chances of success, and the difference was significant."

First, users often did not know how to use search utilities, or what terms to enter. Typos also presented a problem, because misspelled keywords would not be recognized by the search tool, and the user would not necessarily realize that he or she had entered a misspelled word. Moreover, some search results made no sense: "Users often had trouble determining why a search returned a particular item."

On the old Fidelity Investments site, for example, a search for "money market" returned nine hits, eight of them links to specific mutual funds. Users were still not sure which links to follow, however, since most of the mutual funds returned in the search were not explicitly money market funds — for example, the first hit was a link to the Fidelity Global Bond Fund. In fact, only three of the hits were links to money market funds; one hit was a link to a table of contents.

An even more unsuccessful result occurred when the word "dinosaur" was entered into the search tool on the Smithsonian Magazine site. The very first hit was a link to an article about the steel industry, in which the following text appeared:

As Spool concludes:

A full-text search is a blunt instrument for chipping away at a large block of information in order to sculpt the desired result. An index is a more precise tool. No self-respecting human indexer would have referenced the steel industry article under "dinosaur." Good indexing is a skill; humans do it better than machines. We anticipate that professional indexers may become more involved in web site design in the future. This may be difficult and expensive to implement, but the resulting user satisfaction might make it worthwhile.

Back to Top

Site Index Versus Site Map
Although the difference between the two entities may seem obvious, there is a need to differentiate between a site index and a site map. Many sites use the terms "index" and "map" interchangeably — for example, a link will be entitled "Site Index," but what you go to is actually a site map. Here's the basic difference:

A site map is a graphical representation of the site contents, section by section. Usually this may be a straightforward listing in column format, but some site maps are also beautifully designed, combining function with aesthetics.

A site index has the same elements in cyberspace as it does in print: it is a systematically ordered (usually alphabetical) list of information to be found on the site.

Back to Top

Traditional Elements in a Site Index
A Web site index is structured in the same way as a print index, using elements that give the user more than one point of entry for looking up an item. For example:

• Cross-references to other entries (using "see" or "see also"). For example, the ASI index includes a entry for its Strategic Planning Committee, but it also includes the entry "long-range planning. see Strategic Planning Committee (ASI)." (Another good example of cross-references in the Seattle Public Library site index is shown below in the next section.)

• Synonymous entries such as those using the terms "Jobs" and/or "Employment" to direct users to a section called "Careers."

• Inverted entries to bring specific terms to the fore. For example, a section entitled "Personal Lines of Credit" would also be indexed as "Lines of Credit, Personal," as users might check under "lines of credit" first rather than under the word "personal."

• Separate entries for full names (e.g., of organizations) and their abbreviations or acronyms - an example of what indexers call "double posting." The ASI index includes the following "double-posted" entry:
  
  AusSI (Australian Society of Indexers)
  Australian Society of Indexers (AusSI)
  The first entry allows the user familiar with the organization's acronym to find the reference; the second uses the organization's full name for someone unfamiliar with the acronym. These two entries happen to fall next to each other alphabetically, but both are retained because a good index always provides more than one point of entry if possible.

Back to Top

The Use of Hypertext "Locators"
Although a Web site index utilizes many of the traditional elements of a print index, there is one obvious difference in the way in which entries are accessed. A back-of-the-book index uses "locators," or "reference locators" (usually page numbers), to show the user where to find information on an entry. A site index, on the other hand, uses entries that are themselves locators — they are hypertext links that, when clicked, take the user directly to the page represented by the entry.

Not all site index entries are links, however. Non-hypertext entries are included when more than one item falls under a category that is not a linkable entry. The old Seattle Public Library Web site, for example, provided clear and easy-to-navigate access to all sections, supplemented by a navigational bar on the right-hand side that served as a branch locator. The old site index followed the same simple design of hypertext links on a white background.

An example of non-hypertext entries from the old index, pertaining to the library's services for the blind, follows below:

"Services for the blind" was not represented on a specific Web site page per se, but was nevertheless a subject covered on the site. If there had been a link for this subject, however, the entry might look like this:

In summary, then, a Web site index uses all of the elements that would be used in a traditional back-of-the-book index, with hypertext links as locators, and as many entry points for access to information as possible.

Back to Top

What Makes a Good Site Index?
In an article entitled Organizing Your Site from A-Z, information architect Lou Rosenfeld lists the four steps to creating a site index followed at his firm, Argus Associates:

• Before you do anything, refresh yourself on who your site's users are and what their general information needs are.

• Review your site and compile a list of content components that you believe your site's users will find important.

• Whack your list down to size. Keeping the list to a manageable size — say, from 30 to 60 entries — allows users to scan it quickly.

• Term rotate. This is the concept discussed earlier as "inverted entries." In other words, transpose words from phrases so that the entry is easier to find — such as "Lines of Credit, Personal" for "Personal Lines of Credit," to use the example given earlier.

This list forms a practical base on which to build not only site indexes, but site maps (in which case inverted entries would not be needed). Rosenfeld identifies the main features of a good index as (i) reflecting the site and its users, (ii) including all of the important content on the site, and (iii) being as concise as possible — and this would certainly be true of a good site map as well.

Of course, once a site index is created, it must be maintained on a regular basis, staying current as the site evolves. We would therefore add another step to Rosenfeld's list:

• Update your index as content on the site is added or removed, or whenever links change.

The indexer and the site creator (or Webmaster, if applicable) must keep in touch on a regular basis to ensure that both site and index are "in sync." Broken or dead links (a continuing problem on the Web) should not turn up in a site index.

When a Site Index or Map is not Needed
Sometimes, of course, a site may not need an index or map. This can be true of both very large sites and small, easily navigable sites. Still, even online retailing giant Amazon.com at one time offered users a high-level site map to give them a hand — so it is possible.

Another way of helping users find their way around a site without using either an index or a map is to create what I call "site assistance" of some kind — a page that shows users how to navigate, through, for example, the use of a screen shot (or shots) and explanatory copy.

Although this article will not explore site assistance in detail, the earlier version of the Wendy's Restaurants site took a proactive stance in this regard. The home page featured a picture of Wendy's founder Dave Thomas with the words, "How may I help you?" When users clicked on this picture, they were taken to a Site Assistant page that helped them navigate to each section. After Dave Thomas passed away early in 2002, the Wendy's Web site was redesigned, and the Site Assistant page has since been removed.

Back to Top

Site Indexes of Note
Please note: Some of the examples below are no longer extant, and are identified as "archived."

American Society of Indexers (ASI) — A – Z index

European Parliament— alphabetical subject index and thematic index [archived]

Harvard University — A to Z index

Ryerson University — A – Z index, with links to actual campus locations; the locations link to a campus map [archived]

United States Navy — site index [archived]

University of Toronto — alphabetical Website list [archived]

U.S. State Department (website of former Secretary of State Madeleine Albright) — site index and site map [archived]

Back to Top

Final Thoughts
As one can see from the preceding examples, creating a good site index or map is well worth the effort. The end result is a user-friendly site that visitors, even first-time visitors, can find their way around with relative ease. Even more importantly, site indexing creates goodwill on the part of Web users, who — as usability expert Jakob Nielsen observed in his column How Users Read on the Web — often scan rather than read online, and don't always have the time to search for what they want.

In this light, it's disappointing to find that examples of excellence in site indexing and map creation sometimes disappear in the course of site "renovation." Casualties of the reconstruction process include the Apple Computer site index (which at one time complemented the still-extant site map); the elaborate but easy-to-use Internal Revenue Service Site Tree, and (of particular interest in the era of reconciliation) the CIBC's beautifully designed Aboriginal Banking site map.

Both the traditional alphabetical site index and the section-by-section site map have important roles to play in Web site development. To reiterate the words of the ASI, the creators of sites that provide these services "should be commended for recognizing the need for an index" - indeed, some go above and beyond the call of duty. It is to be hoped that as new sites are built or existing sites refurbished, indexes and maps will become standard elements, helping users find their way around the Internet.

Back to Top

References
• American Society of Indexers (ASI). Indexing the Web. [Note: This article © the American Society of Indexers. All rights reserved.]

• Broccoli, Kevin. Indexes: An Old Tool for a New Medium. Contentious Magazine, November 17, 1998.

• Indexes : A Chapter from the Chicago Manual of Style. 14th ed. Chicago: University of Chicago Press, 1993.

• Mulvany, Nancy C. Indexing Books. Chicago: University of Chicago Press, 1994.

This classic text is indispensable to anyone wishing to start a career in indexing. Interestingly, Mulvany uses the terminology of the Internet to describe the print index, pointing out that the text of a print index "can be described as a hypertext" [emphasis mine]. As she explains on page 69:

Although printed book indexes are presented in a linear format, they are not used in a linear fashion. Indexes are not meant to be read in a linear order, from beginning to end. Index users jump around in an index seeking the location of the information they want. Internal guideposts in the index may send readers to another part of the index. Readers go directly to that other portion of the index; they do not read the material in between the two points. [Emphasis by the author.]

Mulvany also discusses British standard BS 3700 (Recommendations for preparing indexes to books, periodicals and other documents), which has since been replaced by BS ISO 999: 1996 Information and documentation - guidelines for the content, organization and presentation of indexes). See pages 14-15.

• Nielsen, Jakob. Be Succinct! (Writing for the Web). Alertbox, March 15, 1997.

• ____________. Designing Web Usability: The Practice of Simplicity. Indianapolis, Indiana: New Riders, 1999.

• ____________. How Users Read on the Web. Alertbox, October 1, 1997.

• ____________. Site Map Usability. Alertbox, January 6, 2002.

• Outing, Steve. "Does Your Web Site Need an Index?" Editor & Publisher Interactive, October 30, 1998.
[Note: The link to this article is now defunct and has been removed.]

• Rosenfeld, Lou. Organizing Your Site from A-Z. Web Review, October 3, 1997.

• Spool, Jared M. Why On-Site Searching Stinks.

• Spool, Jared M., et al. Web Site Usability: A Designer's Guide. San Francisco: Morgan Kaufmann, 1998.

Back to Top