Kelley Blue Book

Marketing Research, 1/e, Cooper/Schindler NetConversions Influences Kelley Blue Book cases> Kelley Blue Book (KBB) is one of the most visited automotive sites on the Web. Visitors flock there to estimate the price of a car they might buy or sell. KBB needed to enhance its site’s performance for advertisers, who had become a major source of revenue as sales of the printed Kelley Blue Book had declined. NetConversions is one of the new Web analytic services to evaluate Web site performance. This case reveals how Web sites are evaluated so that new design elements can be developed and tested. www.netconversions.com; www.kelleybluebook.com >Abstract >The Scenario 1 When you buy a car, new or used, you want to be sure that you do not overpay. Kelley Blue Book (KBB), a publisher of detailed cost and other data on cars, is in the business of assisting buyers determine fair prices. For decades, KBB annually published a book that flew off bookstore shelves. But like many publish- ers in the dot-com age, KBB needed to find a new business model. Today, it makes the same information available via the Internet, deriving its revenue from advertising exposures (measured by page view impressions) and activity on partner links on its Web site (measured by visits generated by mouse clicks on the link). According to Nielsen//NetRatings, the KBB Web site is the number-one Web site for automotive research. In October 2002, KBB served up its 1 bil- lionth customer automotive pricing report, the backbone of its Web site. In late 2001, KBB saw a demonstration of the information-rich actionable data that NetConversions (NetC) could provide with its TrueUsability™ assessment. Companies spend hundreds of thousands of dollars developing revenue-produc- ing Web sites and need to make them as effective as possible. Each time they change an element on a page, add a page, or delete a page, they take the risk of endangering their customer satisfaction and loyalty as much as enhancing it. KBB needs car buyers to return to its site repeatedly; such visits generate a major source of its revenue. NetConversions’ Adrian Chiu describes TrueUsability™ as a detailed research methodology divided into two primary phases (1) site analysis and (2) Hill Climbing™. Site analysis is further divided into three subphases: (a) visitor tracking, (b) data analysis, and (c) reporting. Site Analysis. In the first subphase, NetC tracks site visitor behavior. >TrueUsabilityTM Marketing Research, 1/e, Cooper/Schindler 2 NetConversions Influences Kelley Blue Book This process can take between 48 hours and 7 days, depending on the number of visitors to a site and the level of analysis the client desires on each page. Much like retail observation studies, during tracking NetC follows Web site visitors from the moment they enter the client’s Web site and as they drill down through the site’s numerous pages. This part of the process provides the data that are used during the detailed analysis that follows, and later is compared to the Hill Climb- ing™ data. Data Analysis. The data analysis subphase is really a series of different analytical processes. Most Web site research suppliers provide the big picture of customer visits with the equivalent of NetC’s path analysis. In this multistep process, NetC not only reveals which pages are visited but also the actual paths that users follow when drilling down through the client’s Web site. “Essentially, we’re looking for ‘low hanging fruit’,” shared Chiu. NetC wants to know weaknesses along the path, the opportunities for partner exposure (adver- tisers and other alliance partners from whom the revenue is generated), and where visitors abandon the site. “Path analysis by itself isn’t enough to provide true actionable data,” explains Chiu. “If you recommend redesigns based on path analysis alone, you would be guessing about a great deal of visitor behavior.” Luckily for KBB, NetConversions has other tools for providing that actionable data. Exhibit NetC-1 provides a picture of one such tool, click density analysis. Every visitor to a given page leaves his or her mark, represented as a red dot for every mouse-click. “One client seeing a color reproduction of click density analysis for the first time, thought the printer’s quality control measures weren’t working,” remembered Chiu. The Web page is seen as a canvas, with the visitor depositing paint with every mouse click. “From the click density analysis, we can tell physically were the visitor clicks, showing items that need to be hot-linked, which content on the page is perceived as most valuable, and which physical space is allocated the greatest visitor value.” In a retail study the space nearest the door is leased at a higher cost per square foot than the space farthest removed from the escalator on the second floor. For a Web site each space on a visitor’s screen is valuable real estate, but some real estate is far more valuable that others. Click density revealed that the KBB logo appearing as a graphic on each page was perceived by many visitors as a logical link to return to the site’s home page. “It was an obvious recommendation that the logo should be more than a design graphic; it should be hot-linked,” concluded Chiu. Exhibit NetC-2 shows a page before its NetC analysis. Another tool used on the KBB project was scrolling analysis. NetC divides a Web page into 10 zones. Depending on the detail of a given page, only some of those zones are initially visible when the page downloads to the visitor’s monitor. To reach deeper zones, the visitor must scroll down using the screen scroll bar or the wheel on their mouse. “If their computer offers fast download, visitors usually prefer a page design that requires limited scrolling,” revealed Chiu, describing one of many insights NetC has gained from years of meta-analysis of Marketing Research, 1/e, Cooper/Schindler 3 NetConversions Influences Kelley Blue Book Web sites. “Scrolling analysis revealed that visitors were hunting for information via mouse-click that was only available by scrolling. They just weren’t scrolling down to reach the data.” Thus scrolling analysis revealed where a long page on KBB’s site needed to be shorter, offering links to various levels of detail on subsequent pages. It also revealed where within the KBB site pop-up windows should be used. The image of a car was one element that the KBB visitor would scroll to, but advertisers and other partners weren’t as inclined to pay for what they perceived as less valuable real estate. “Converting the car visual to a pop-up window increased the revenue opportunities with partners,” shared Chiu. “While with our archive of page design research data this might seem intuitive, the client values actual data from their Web site before changing anything.” Reporting. The last subphase of data analysis is reporting. While this is sometimes done with personal presentations, to reduce research costs and travel time for their employees many clients prefer a written report that can be discussed with the NetC team over the phone. Each report, filled with detailed images of site visitor data, is accompanied by recommendations. “For some clients this is enough. They are willing to rely on our extensive research based knowledge of Web page design and visitor behavior, along with their own knowl- edge of their product. But for others, those who want to be sure that the indicated changes will enhance a visitor’s experience, we offer Hill Climbing. KBB chose to participate at this next level. Hill Climbing™ is a process for testing individual changes to a Web site. It uses a script overlay technique that eliminates the need to create a whole page with a new design to compare any change on that page with the existing design. This is a major cost savings for the client. For a sample of Web site visitors, when they download a given page, the script overlay changes what they see. NetC uses an algorithm to determine which visitors see the change; it generates a random sample of visitors who participate unknowingly in the test of the change. Roughly one of every 1,000 visitors to the KBB site was exposed to a Web site design change during Hill Climbing™. For example, if a visual of the KBB logo was not a hot-link on the original design, the script overlay could hot- link the logo for the test. NetC tracks the test group’s behavior and compares it to the behavior of those seeing the page as it is currently designed. “Depending on the number of design recommendations that the client wants to test, this process can take months,” estimates Chiu. KBB chose to use Hill Climbing on many of the recommended changes. As a result, it added drop-down boxes, shortened several of its pages, and created pop-up windows for some data. You can see a sample of the orignial page and >Hill ClimbingTM >Results Marketing Research, 1/e, Cooper/Schindler 4 >Sources >Discussion NetConversions Influences Kelley Blue Book the changed design in Exhibit NetC-2. You may see more original pages by visiting WayBackMachine.org, a digital Web page archive (go to www.waybackmachine.org then type www.kbb.com in the search window). Compare these older pages to the KBB current site. The changes confirmed by research and implemented on the KBB Web site increased page views and partner advertisement impressions by 62 percent for the key New Car Pricing Report page. Other research, shared with advertis- ers, enhanced the ad click-through rate. Revenue increased more than enough to offset the cost of the research. www.kbb.com; www.netconversions.com 1 Develop the management-research question hierarchy through investiga- tive questions for this project. 2 Using the research process model (Exhibit 4-1), describe and evaluate the research design of this project. 3 Describe and evaluate the sampling design for this project. >URLs Adrian Chiu, NetConversions, interviewed March 25, 2003, and March 28, 2003. “Content Site Case Study: Kelley Blue Book,” NetConversions, downloaded March 14, 2003 (http://www.metconversions/case_content.htm). “Kelley Blue Book Serves 1 Billion Consumer Automotive Pricing Reports: Record Number of Consumers Flock to Trusted Resource kbb.com for Used and New Car Pricing,” Kelley Blue Book, downloaded October 28, 2002 (http://www.kbb.com/media/). “What Is True Usability™?” NetConversions.com, downloaded March 14, 2003 (http://www.netconversions.com/true_usability.htm). NetConversions Influences Kelley Blue Book Marketing Research, 1/e, Cooper/Schindler 5 Exhibit NetC-1 Sample ClickDensity™ Map Completions Abandonments Exhibit NetC-2 After and Before Conversion After Before