Usability of Online Services: The Role of Technology Readiness and Context*
By Massey, Anne P Khatri, Vijay; Montoya-Weiss, Mitzi M
ABSTRACT An important prerequisite for the success of any online service is ensuring that customers’ experience-via the interface- satisfies both sensory and functional needs. Developing interfaces that are responsive to customers’ needs requires a perspective on interface design as well as a deep understanding of the customers themselves. Drawing upon research in consumer behavior concerning consumer beliefs about technology, we deploy an alternative way to describe customers based on psychographic characteristics. Technology readiness (TR), a multidimensional psychographic construct, offers a way to segment online customers based upon underlying positive and negative technology beliefs. The core premise of this study is that the beliefs form the foundation for expectations of how things should work and how specific online service interfaces are evaluated by customers. At the same time, usability evaluations of specific online services might be contingent on contextual factors, specifically the type of site (hedonic vs. utilitarian) and access method (Web vs. wireless Web). The aspects of usability examined here are those incorporated into the usability metric and instrument based on the Microsoft Usability Guidelines (MUG). The results of an empirical study with 160 participants indicate that (i) TR customer segments vary in usability requirements and (ii) usability evaluations of specific online service interfaces are influenced by complex interactions among site type, access method, and TR segment membership. As organizations continue to expand their online service offerings, managers must recognize that the interface exists to serve the customers, so their design must be matched to market needs and TR.
Subject Areas: Customer Interface, e-services, Laboratory Experiments, Latent Class Analysis, Psychographics, Technology Readiness, and Usability.
Services dominate the modem economy, accounting for more than 50% of the labor force in countries such as Japan and Germany and 75% of the labor force in the United States and United Kingdom (U.S. Census Bureau, 2006). Of particular note is that the services sector represents 80% of the $2.1 trillion annual worldwide information technology spending (IBM Research, 2006). This trend explains the emergence of the Web as a significant means for commerce, communication, and content access. Technology has become prominent within the firm-customer relationship, dramatically changing how services are conceived and delivered (Bhner & Brown, 2006; Rust & Miu, 2006). For example, more than 75% of online customers deploy the Web to research a product or service online before buying, more than one-half of airline reservations are made through the Web, and online sales reached $30 billion during the 2005 holiday season, up 30% from 2004 (Pew Internet, 2005; Nielsen//NetRatings, 2005). Recently, customers expect to be able to access online services through multiple methods. For example, mobile devices (e.g., PDAs, cell phones) are increasingly being used to access wireless Web sites where customers can conduct transactions, access news and other information, interact with database services (e.g., map guides), and download entertainment (e.g., games) (Urbaczewski, Valacich, & Jessup, 2003; Lin & Wang, 2006).
In an online context, delivery entails service implementation via the user interface, which may differ depending on the type of site (e.g., pleasure oriented or productivity oriented) and access method (e.g., mobile vs. traditional Web access devices). Thus, organizations must actively manage and measure service delivery across all site types and access methods to ensure the quality of the entire customer experience (Sasser, Olsen, & Wyckoff, 1978; lievens, Moenaert, & S’Jegers, 1999; Rust & Miu, 2006). An important prerequisite for the success of any online service is ensuring that the customer experience via the interface satisfies both sensory and functional needs (Rice, 1997; Straub & Watson, 2001; Palmer, 2002).
For online services, the user interface is the mechanism through which customers conduct transactions, communicate, and search for/ access content (Chau, Cole, Massey, Montoya-Weiss, & O’Keefe, 2002; Palmer, 2002). Usability-the ease with which a customer can employ a site to achieve a specific goal-has been shown to be a key driver of site use (Rice, 1997; Lohse & Spiller, 1998; Nielsen, 1999). By implication, site usability and online service success are inextricably linked. Prescriptions regarding the link between online service interface design and site usability can be found in both the academic literature and trade press (Lam & Lee, 1999; Keeker, 1997; Cunliffe, 2000; Nielsen, 2000; Karkkainen & Laarni, 2002; Palmer, 2002; Ballard & Miller, 2001). Although the importance of usability has been acknowledged, there has been limited research examining how customer characteristics influence usability requirements or usability perceptions. Also, there, has been limited research about how requirements and perceptions might vary across different types of online sites or for different access methods. Arguably, these relationships are of interest to researchers and practitioners, as a deeper understanding could facilitate improved decision making regarding online service interface design and marketing strategies.
Because of technology’s expanding role in service delivery, we draw upon research in consumer behavior that has investigated customers’ beliefs about technology to examine customer usability perceptions for online services (Cowles, 1989; Cowles & Crosby, 1990; Dabholkar, 1996; Mick & Fournier, 1998; Parasuraman, 2000; Parasuraman & Colby, 2001). We focus specifically on technology readiness (TR), a multidimensional psychographic construct that describes the propensity of customers to embrace and use new technologies for accomplishing goals (Parasuraman, 2000; Parasuraman & Colby, 2001). The purpose of this study is to examine the relationship between TR and usability in an online service context. We contend that TR and usability are related because: (i) both involve inherently subjective phenomena such as attitudes and opinions, (ii) both are concerned with how specific customers interact with a technology-based service, and (iii) both focus on the accomplishment of goals via technology.
In the following sections, we describe the theoretical background and offer our exploratory research hypotheses. We then present and discuss the results of a study with 160 subjects and conclude with a discussion of implications for research and practice.
The usability and design of online interfaces have received attention in the human computer interface (HCI) literature and Web- specific usability research (Palmer, 2002 provides a review). Past research suggests that usability should reflect design from the customer’s perspective of interaction requirements (Norman & Draper, 1986; Hartson, 1998). In this study, we adopt the ISO 9241 definition of usability: “the extent to which a product [/service] can be used by specific customers to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” (Karat, 1997, p. 34).
Online services can be accessed through Web site interfaces via traditional devices (e.g., PCs, laptops). Increasingly, online services can also be accessed through wireless Web site interfaces via mobile devices (e.g., cell phones, PDAs). The growth of the Web in the 1990s led to several suggestions for improving the usability of Web sites (Nielsen, 1999; Nielsen, 2000). Early on, many service providers attempted to design Web sites as if they were laying out traditional print media such as advertisements or brochures, without recognizing that the new medium provided unique opportunities to engage customers (Settles, 2002). Over time, usability guidelines focused on improving Web page components such as page design, content design, and site design. Today, the small screen sizes of mobile access devices preclude organizations from simply shrinking Web pages to fit these smaller devices. Guidelines developed for traditional PC-based interface design might not directly translate to smaller access devices and wireless site interfaces.
While the bulk of user interface design guidelines arose from practice (Hartson, 1998; Nielsen, 2000; Palmer, 2002), much of the underlying extant theory comes from cognitive psychology (Hammond, Gardiner, Christie, & Marshall, 1987; Barnard, 1993). As an example, Norman’s theory of action explains the path from user goals to actions, back to perception and evaluation (i.e., were goals met) (Norman, 1986). Similarly, cognitively based theories of work activity and of task analysis underlie techniques such as contextual inquiry (Carroll & Rosson, 1992). Another important perspective is the model of humans as cognitive information processors (Lindsay & Norman, 1977; Card, Moran, & Newell, 1983). The information processing (IP) model explains the cognitive process that occurs between stimulus (e.g., sensory perception of an interface) and response (e.g., action, evaluation) and provides a way to categorize user behavior. The IP model involves both short-term memory (information under current consideration) and long-term memory (knowledge stored for future use). According to the IP model, perception of an interface is based on current information and previously stored knowledge is based on prior experiences (Card et al., 1983). While providing alternative frameworks for conceptualizing usability, these theories and models do not take individual differences into account (Hartson, 1998). Specifically, a literature review suggests that past research has not considered the relationship between customer characteristics and usability requirements or evaluations. “Know thy user” (Hansen, 1971, p. 524) is often an underemphasized aspect in the interface design process (Hartson, 1998). In addition, perceptions of usability may also be contingent on the type of service the interface is designed to support (Lecerof & Paterno, 1998; Agarwal & Venkatesh, 2002) and the method through which the online service is accessed (i.e., Web vs. wireless Web) (BeIk, Wallendorf, & Sherry, 1989; Olson & Olson, 2003; Venkatesh, Ramesh, & Massey, 2003b; Venkatesh & Ramesh, 2006). Overall, usability design guidelines (derived from application of theory or practice) do not prescribe which specific aspects of usability are more (less) important for different types of customers or more (less) critical for different types of online services or access methods.
Prior research in information systems (IS) and marketing theorizes about the relationships between individual characteristics and demographic variables (e.g., age, gender, income) as predictors of technology-related behaviors (Venkatesh, Morris, Davis, & Davis, 2003a) and consumer behaviors (Morton, Zettlemeyer, & Silva-Risso, 2001) as outcomes. Demographic variables are often used to segment customers (Dawar, Ratneshwar, & Sawyer, 1992) and have been shown to influence technology perceptions (Gefen & Straub, 1997; Venkatesh & Morris, 2000). Various demographic characteristics have been related to online shopping behaviors (e.g., Internet use, education) (Burke, 2002; Burroughs & Sabherwal, 2002; Moe, 2003). Observable characteristics such as demographics are generally useful in characterizing market segments, but they do not explain distinctly unobservable differences such as underlying beliefs, attitudes, and motivations.
For online services, there is a strong and growing interest in understanding what individuals believe about technology and how these beliefs can (or should) influence design requirements and ultimately evaluations of success or failure (Barley, 1986; DeSanctis & Poole, 1994; Orlikowski & Gash, 1994; Tan & Hunter, 2002). Underlying this interest is the recognition that beliefs form the foundation for expectations of how things should work (i.e., needs or requirements) and establish the basic foundation for evaluation (Kelly, 1969; Orlikowski & Gash, 1994; Tan & Hunter, 2002). Given this, we now turn our attention to an alternative way to understand customers based on their underlying psychographic technology beliefs and readiness to use technology-based systems.
Because of technology’s expanding role in service delivery, it is necessary to understand customers’ readiness to use technology- based systems such as the Web and/or wireless Web (Parasuraman, 2000; Burke, 2002). Prior research suggests distinct differences in the psychographic profiles of technology adopters (Cowles, 1989; Cowles & Crosby, 1990; Greco & Fields, 1991; Dabholkar, 1996; Eastlick, 1996; Mick & Fournier, 1998). Psychographic typologies are intended to characterize customers along psychological dimensions (Wells & Tigert, 1971; Wells, 1975). Based on the attitudes, interests, and opinions of individuals (Ziff, 1971), psychographic typologies provide insights into the lifestyles, values, and motivations that encompass consumer behavior. By addressing why individuals react or behave in certain ways, psychographics offer a more complete profile of a potential (or current) market. Psychographics are often combined with demographic data to design new products, create product positioning strategies, and develop marketing communications (Mitchell, 1994; Gilbert & Warren, 1995; Heath, 1996; Ailawadi, Neslin, & Gedenk, 2001).
The extant literature strongly suggests that people simultaneously hold positive and negative beliefs about technology- based products and services (Cowles, 1989; Davis, Bagozzi, & Warsaw, 1989; Cowles & Crosby, 1990; Dabholkar, 1994; Mick & Fournier, 1998). Thus, the meaning of technology for any person arises out of a process of contrast and similarity (Kelly, 1955; Duck, 1994; Marsden & Littler, 1998). Although paradoxical beliefs about technology may coexist, people can be arrayed along a continuum anchored by strongly positive (highly technology ready) at one end to strongly negative (highly technology resistant) at the other (Mick & Fournier, 1998; Parasuraman, 2000). Past research suggests that an individual’s position on the continuum will be correlated with their need for, evaluations of, and ultimately acceptance (or rejection) of a technology-based product or service (Cowles, 1989; Cowles & Crosby, 1990; Dabholkar, 1996; Eastlick, 1996).
TR is a multidimensional psychographic construct that refers to the propensity of customers to embrace and use new technologies for accomplishing goals (Parasuraman, 2000; Parasuraman & Colby, 2001). TR has been applied in a variety of contexts including consumer markets (Parasuraman & Grewal, 2000), online services (Taylor, Celuch, & Goodwin, 2002; Rust & Kannan, 2003), educational choice (Hendry, 2000), and health care (Rosen, Mittal, Mulsant, Degenholtz, Castle, & Fox, 2003).
TR consists of various technology beliefs that are categorized into four components (Parasuraman, 2000). Two of the components- optimism and innovativeness-are contributors that increase a customer’s TR, while the other two components-discomfort and insecurity-are inhibitors that suppress TR. Briefly, optimism is a positive view of technology and a belief that it offers people increased control, flexibility, and efficiency in their lives. Innovativeness reflects the extent to which an individual believes s/ he is at the leading edge of trying out new technology-based products or services. Discomfort is a perceived lack of control over technology and a feeling of being overwhelmed by it. Here, individuals believe technology is not really designed for use by ordinary people and is simply too complicated. Lastly, insecurity reflects an inherent distrust of technology and doubt about its ability to work properly. It is important to note that, while TR includes innovativeness, TR encompasses more than the degree to which an individual is relatively earlier in adopting an innovation (Midgley, 1976; Midgley & Dowling, 1978; Agarwal & Prasad, 1999).
Parasuraman and Colby (2001 ) offer a typology of customers based on the various combinations of both positive (contributing) and negative (inhibiting) beliefs regarding technology. The typology provides a useful means to segment customers into distinct groups and is based on data collected from a nationwide annual survey of American adults, called the National Technology Readiness Survey (NTRS). Data collected from the NTRS provides an in-depth view of customer technology beliefs and readiness to adopt technology; for details refer to Parasuraman and Colby (2001) and the Robert H. Smith School of Business’ Center for Excellence in Service.
In describing TR, Parasuraman and Colby (2001 ) identify five distinct groups: Explorers, Pioneers, Skeptics, Paranoids, and Laggards. Explorers score higher on the contributors (optimism, innovativeness) and lower on inhibitors (discomfort, insecurity) than the other segments. Explorers are a relatively easy group to attract when a new technology-based product or service is introduced and represent the first wave of customers. Laggards are the opposite of Explorers, ranking lower on the contributor factors and higher on the inhibitor factors than all groups as a whole. Laggards are also typically the last group to adopt a new technology-based product or service. The middle three segments (Pioneers, Skeptics, and Paranoids) have more complicated beliefs about technology. Pioneers share the optimism and innovative beliefs of Explorers, but they simultaneously feel some discomfort and insecurity. They desire the benefits of technology, but are more practical about the difficulties and challenges. Skeptics tend to be dispassionate about technology, but also have few inhibitions; thus, they need to be convinced of benefits. Paranoids may find technology interesting, but they are also concerned about risks, exhibiting high degrees of discomfort and insecurity.
Prior research on the adoption of various new technologies suggests that each TR segment enters the market at different times (i.e., Explorers before Pioneers, Pioneers before Skeptics, etc.) (Parasuraman & Colby, 2001; Tsikriktsis, 2004). For example, research concerning the penetration of U.S. Internet access (NTRS, 2001) suggests that Explorers reached a 10% penetration rate in mid- 1995, while it took about 3 years for the remaining TR segments to achieve a similar level of penetration. Thus, as markets evolve, so do the customer segments that are in the market. The TR typology describes market penetration in terms of a progression in the types of customers the market attracts.
The underlying belief structures of each TR segment may provide meaningful explanations of why specific customers place more (or less) emphasis on particular usability features. Such insight is critical for the design of online service interfaces because it helps isolate usability features for specific TR segments and thereby drives market acceptance. Understanding the relationship between usability and TR segments should facilitate matching design evolution to market evolution for technology-enabled services. It also provides further information to marketers on what groups to target and how to communicate effectively with them (Caillat & Mueller, 1996). The notion of TR is consistent with IS research that considers how individual beliefs about technology relate to system requirements, evaluations, and, ultimately, system success (Orlikowski & Gash, 1994; Tan & Hunter, 2002). According to Parasuraman (2000, p. 308), the TR construct can be viewed as an “overall state of mind resulting from a gestalt of mental enablers and inhibitors.” Likewise, usability requirements can be viewed from a gestalt perspective of what the features of a usable system might be (Byrd, Cossick, & Zmud, 1992; Agarwal & Venkatesh, 2002; Davidson, 2002). Both TR research and IS research about people- technology interactions suggest that different customers will have different technology needs due to their underlying beliefs (Darian, 1987; Zeithaml & Gilly, 1987; Greco & Fields, 1991; Parasuraman & Colby, 2001; Burke, 2002; Taylor, Celuch, & Goodwin, 2002; Rust & Kannan, 2003). Thus, we expect that TR segmentsExplorers, Pioneers, Skeptics, Paranoids, and Laggards-will differ in usability requirements. Stated formally:
Hl: TR segments differ in usability requirements for online service interfaces.
Generally speaking, usability requirements do not represent customer evaluations of specific online interfaces. When faced with a specific online service interface, the underlying system of beliefs or meaning that one attaches to the broader domain of technology provides the interpretive lens through which an individual evaluates the interface (Kelly, 1969; Checkland, 1981; Orlikowski & Gash, 1994). This suggests that customers with similar (dissimilar) belief systems will have similar (dissimilar) evaluative perceptions of any given online service interface. Therefore, we hypothesize:
H2: Usability evaluations of specific online service interfaces will differ across TR segments.
We have argued that the system of beliefs lies behind every specific act of evaluation that a customer makes. At the same time, usability evaluations may also be contingent on other contextual factors (BeIk, 1975; Dickson, 1982; Burke, 2002; Yang, Allenby, & Fennell, 2002). Specifically, we expect usability evaluations to also be contingent on site type (Lecerof & Paterno, 1998; Agarwal & Venkatesh, 2002; van der Heijden, 2004) and the method through which an online interface is delivered (i.e., Web vs. wireless Web) (BeUc et al., 1989; Oison & Oison, 2003; Venkatesh et al., 2003b; Venkatesh & Ramesh, 2006).
Online sites may be broadly characterized into two types: pleasure-oriented (hedonic) or productivity-oriented (utilitarian) (Babin, Darden, & Griffin, 1994; van der Heijden, 2004). This characterization is consistent with consumer behavior literature that distinguishes between utilitarian and hedonic products (Hirschman & Holbrook, 1982; Holbrook & Hirschman, 1982). Hedonic sites aim to provide self-fulfilling value and enjoyment to a customer. In its purest form, interacting with a hedonic site is designed to be an end in itself (e.g., playing a game, watching a video). In contrast, utilitarian sites aim to provide instrumental value. Instrumentality implies there is a goal external to the interaction between the customer and the site (e.g., getting directions, buying a product) (Brief & Aldag, 1977; Babin et al., 1994). The dominant design objective of a hedonic-oriented site is to encourage prolonged use via pleasurable experiences, while the dominant design objective of a utilitarian-oriented site is productive use (van der Heijden, 2004). Prior research has shown that user evaluations are dependent on whether a system is intended to provide hedonic or utilitarian value (Teo, Lim, & Lai, 1999; Cheung, Chang, & Lai, 2000; Moon & Kim, 2001; van der Heijden, 2004).
In addition to site type, we also consider two primary access methods for online services: traditional access via PCs or laptops and access via wireless handhelds such as cellular phones or PDAs. Recent studies identified a key stumbling block for wireless sites as being the lack of usability (Ramsay & Nielsen, 2000; Jackson- Sanborn, Odess-Harnish, & Warren, 2002; Yankee Group, 2002). Understanding how wireless Web sites should be designed differently from traditional Web sites is still at an emerging stage (Venkatesh & Ramesh, 2006). Prior research suggests that usability evaluations of an online interface is dependent on whether the interface is accessed via the Web or the wireless Web (Ballard & Miller, 2001; Barnes, Liu, & Vidgen, 2001 ; Venkatesh et al., 2003b; Venkatesh & Ramesh, 2006). Wireless handhelds impose limitations such as small screen size, limited screen resolution, small keyboard, and so on. Therefore, we hypothesize the following:
H3: The relationship between usability evaluations of online service interfaces and TR segments will be moderated by site type (hedonic vs. utilitarian) and access method (Web vs. wireless Web).
We designed a between-subjects study to explore the relationship between TR and usability and the effect(s) of site type and access method contextual factors. We recruited 169 subjects from a campus of a large Midwestern university. Of our 169 subjects, nine were excluded in our data analysis due to incomplete surveys, thus our final sample size is 160. In the final sample, 63.1% were 18-25 years old, 31.3% were 26-35 years old, and 5.6% were over 36 years old. Also, 42.5% of respondents were female, 83.1% were graduate (masters, doctoral) students, 6.9% were undergraduate students, and 4.4% were nonstudent university employees. A total of 5.6% did not report. We collected data on individuals’ experience using the Web for hedonic-oriented and utilitarian-oriented purposes on a four- point scale (with 1 = never and 4 = frequently) (Kraut, Mukhopadhyay, Szczypula, Kiesler, & Scherlis, 1999; O’Keefe, Cole, Chau, Massey, Montoya-Weiss, & Perry, 2000). The average experience with hedonic-oriented sites was 3.46 (implying frequent use) and utilitarian-oriented sites, 1.60. We also evaluated our respondents’ experience with both wireless voice and wireless data-related services on a fivepoint scale (1 = no plans to use and 5 = frequently used in the past 12 months). While the average exposure to voice-related services was 2.94 (implying occasional use in the last 12 months), the average exposure to data-related services was 2.23 (implying a proclivity to use in the next 12 months, although there were no actual uses). Participation was voluntary and subjects received a $20 award for their participation.
For the first part of the experiment, participants completed a 36- item survey (five-point scale, with 1 = strongly disagree to 5 = strongly agree) to measure technology-related beliefs based on scales developed by Parasuraman (2000) and Parasuraman and Colby (2001). Next, participants responded to questions about online site usability requirements drawn from scales based on Microsoft’s Usability Guidelines (MUG), a comprehensive collection of usability criteria consistent with the ISO definition of usability (Agarwal & Venkatesh, 2002). The usability scale addresses five main categories of site design intended to increase the usability of online service sites: content, ease of use, made-for-the-medium, promotion, and emotion. Briefly, content is the extent to which a site offers informational and transactional capability. ease of use is the extent to which a site is free of effort. Made-for-the-medium is the extent to which a site can be tailored to fit specific needs of customers. Emotion is the extent to which a site evokes emotional reactions. Finally, promotion is the extent to which a site is well promoted on the Internet or through other media.
To capture usability requirements, we followed the procedures developed by Agarwal and Venkatesh (2002). Specifically, subjects were instructed to distribute 100 points to indicate the relative importance of the five main categories of usability. At this point, participants were not evaluating the usability of any particular site, but, rather, how important each dimension of usability was to a subject. The results of this part of the experiment were used to test the hypothesis that TR segments differ in usability requirements (Hl).
In the second part of the experiment, participants evaluated specific online service sites. We selected online services that were delivered via both Web and wireless methods and varied the types of sites to examine the differential effects of access method and site type on usability evaluations. We selected two hedonic (HED) sites (Yahoo and ESPN) and two utilitarian (UTL) sites (Mapquest and Fodors) for this study. In the Web delivery condition, subjects accessed sites using a personal computer. In the wireless delivery condition, each subject was provided with and trained on the use of an HP Jornada 568 Pocket PC, one of HP’s personal digital assistant (PDA) offerings. The PDA included a Sierra Wireless PCMCIA card that enabled live access to Verizon Wireless’ CDMA IxRTT mobile data network. The subjects were randomly assigned to one of the four experimental conditions. In the first and second conditions, respectively, 36 subjects assessed the usability of the two HED Web sites and 42 subjects assessed the two HED wireless sites. In the third and fourth conditions, respectively, 39 subjects assessed the usability of the two UTL Web sites and 43 subjects assessed the two UTL wireless sites.
In all conditions, subjects explored the sites and completed site evaluation forms providing their perceptions of the usability of each site along the five main usability categories on a EI-point scale (1 = extremely poor to 10 = extremely good). On average, subjects took 20 minutes to explore the sites and complete the evaluations. We also asked the 85 subjects in the wireless conditions three questions regarding PDA device ease-of-use (on a seven-point scale, with 7 = strongly agree) (Venkatesh & Davis, 1996). The average response for these three questions was 6.11, suggesting no difficulty in using the device. The results of the second part of the experiment were used to test the hypotheses concerning usability evaluations, TR, and contextual factors (H2 and H3). ANALYSIS AND RESULTS
To identify TR segment membership we used a classification function derived from latent class cluster analysis applied to the 2004-2005 NTRS database (described earlier). The 2004-2005 NTRS was based on a random sample of 1,000 U.S. adults (18 years or older) and was administered in December 2004 by telephone by the Robert H. Smith School of Business’ Center for Excellence in Service. The latent structure of TR beliefs was examined by applying latent class cluster analysis (LCA) using Latent Gold(R) software to the NTRS 2004-2005 data, which is the same year in which our sample was collected. Because the NTRS study is designed to be a nationally representative survey, it was most appropriate to develop the latent class cluster model using the broader sample and then apply a classification function from the resulting cluster structure to our smaller, localized sample.
LCA postulates that the association among TR beliefs is due to an underlying class structure. Information about the underlying class structure is conveyed through (i) the latent class probabilities, class prevalence estimates, which indicate the proposition of the sample assigned to each class, and (ii) the response probabilities, which are the percentages of class member reporting each TR belief. In latent class cluster models, the observed data is assumed to arise from a number of a priori unknown segments or latent classes that are mixed in unknown proportions. The goal of LCA is to identify the smallest number of latent classes that adequately describe the associations among the TR beliefs, starting with the most parsimonious 1 -class null model and fitting successive models with increasing numbers of classes.
Like any clustering technique, the appropriate latent class cluster model is determined by theory and the ability to find meaningful and statistical differences. There is no single criterion for determining the number of segments. One set of criteria, known as information criteria, is based on assessing the degree of improvement in explanatory power adjusted for the number of degrees of freedom required for estimation of additional parameters. The most common information criteria include BIC (Bayes information criterion), AIC (Akaike information criterion), and CAIC (Consistent AIC, which penalizes for sample size as well as model complexity). These are goodness-of-fit measures that take into account model parsimony, penalizing for the number of parameters in relation to the maximum possible number of parameters. The lower the BIC, AIC, or CAIC values, the better the model in comparison with another (Dayton, 1998). Other methods for evaluating LC cluster models are based on the separation of the clusters and measure how well the segmentation variables are able to predict class membership. The entropy statistic is such a statistic. The entropy criterion accounts for separation in the estimated posterior probabilities and adjusts for overparameterization compared to a single segment model. Ultimately, no single criterion determines the “right” number of segments; rather multiple criteria must be used in conjunction with guiding theory to select the “best” latent class structure.
Goodness-of-fit statistics and theory guided our evaluation of alternative segmentation class structures, resulting in a five segment structure solution. The fiveclass solution had the lowest BIC (77788.9), AIC (81229.9), and CAIC (77086.9) values, respectively. The entropy R2 (.88) and standard R2 (.86) both suggest an acceptable fit of the latent class cluster structure to the data. Using the five-segment latent class cluster solution developed from the NTRS 2004-05 data set, we derived a classification function to apply to our smaller (n = 160) data set to classify respondents into TR segments. The resulting segments exhibit high withinsegment homogeneity (i.e., similar underlying technology belief structures) and high between-segment heterogeneity (i.e., different belief structures).
Five distinct segments were found within our data set: 52 Explorers (32.7%), 12 Pioneers (7.5%), 85 Skeptics (53.5%), 10 Paranoids (6.3%), and 1 Laggard (
Table 1: Segment variable profiles.
Figure 1: Belief structures of technology readiness segments.
Table 2: Significant segment mean (between-subject) differences on technology beliefs.
Table 3 characterizes the four segments in terms of prior Web experiences (hedonic vs. utilitarian activities) and traditional demographics (age, gender). The results indicate significant differences between and within the TR segments in terms of prior Web experiences and between the segments with regard to gender, but no significant difference in age.
Table 3: Segment demographics and significant differences
Table 4: Technology readiness segments and relative importance of usability requirements.
Table 5: Significant segment mean (between-subject) differences on usability requirements.
TR and Usability Requirements: Testing Hl
To test the hypothesis (Hl) that TR segments differ in usability requirements, we conducted analysis of variance (ANOVA). Tables 4 and 5 present our ANOVA results: means, standard deviations, and statistical comparisons among the four TR segments and the relative importance of the five usability categories (content, ease of use, made-for-medium, emotion, and promotion). For example, Table 4 shows that the relative importance of the usability category content by Explorers, Pioneers, Skeptics, and Paranoids is 39.36%, 30.75%, 38.22%, and 30.00%, respectively. Additionally, the within-subjects f-test analysis (see superscripts in Table 4) that compares the mean differences across segments shows, as an example, that content and ease of use are the two most important usability requirements for Pioneers. Emotion and promotion are two least important usability requirements across all segments. The between-subjects analysis shown in Table 4 illustrates that TR customer segments do, in fact, vary in the relative importance placed on the usability dimensions. Thus, Hl is supported.
Evaluations of Online Service Interfaces: Testing H2 and H3
Given the four experimental conditions-site type (HED vs. UTL) ? access method (Web vs. wireless Web)-and the limited numbers of respondents in certain cells, we focus our analysis for H2 and H3 on two key segments, Explorers (n = 52) and Skeptics (n = 85). It is important to note that, in general, Explorers and Skeptics represent two key customer groups. Often revolutionary in their thinking, Explorers are driven by a desire for cutting-edge functionality and lead all entrants to a technology market. Skeptics, on the other hand, represent the early majority of customers moving into the mainstream market. Evolutionary in their thinking, Skeptics are driven by a sense of practicality and a desire for functionality that is in widespread use (Rogers, 1983; Moore, 1991; Parasuraman & Colby, 2001, p. 115; Mohr, Sengupta, & Slater, 2005, p. 178).
To test the hypotheses related to usability evaluations (H2-H3), we conducted a multiple analysis of covariance (MANCOVA) with TR (Explorers/Skeptics), site type (HED/UTL), and access method (Web/ wireless Web) as the main factors and the five usability categories as dependent variables. We also included covariates shown to be important in past research: age, gender, and Internet experience with specific site types. Table 6 reports the main effects as well as all two- and three-way interaction effects.
The results in Table 6 indicate TR segment membership does not have a significant main effect on usability evaluations, but site type (Hotelling’s T2 = .30, F = 7.25, n value
DISCUSSION AND MANAGERIAL IMPLICATIONS
The goal of this article was to empirically investigate the relationship between customers’ beliefs about technology and usability requirements/evaluations of online services delivered through the customer interface. Drawing on HCI, IS, and marketing research, we argued that beliefs form the basis for expectations of how online service interfaces should work (i.e, usability requirements), and these beliefs establish the lens through which customers evaluate the usability of specific sites. We also argued that usability evaluations are contingent on other contextual factors such as site type (hedonic vs. utilitarian) and the method through which an online interface is accessed (Web vs. wireless Web). In our investigation, we used a psychographic construct-TR-to understand customer’s underlying beliefs about technology. Empirical results provide support for the proposed relationship between customer technology beliefs and usability requirements, and the role of contextual factors on usability evaluations of online services. In this section, we discuss our results and offer managerial implications. Table 6: Usability evaluations of online services- multivariate analysis of covariance results.
TR and Usability Requirements
Because TR segments hold different beliefs about technology, we argued that this will influence the importance customers place on various usability requirements (Hl). While researchers have proposed different usability dimensions (Eighmey & McCord, 1998; Gehrke & Turban, 1999), in our study we examined specific aspects of usability using the MUG instrument (Agarwal & Venkatesh, 2002). With regard to our hypothesis, our empirical results provide evidence that the TR segments differ in usability requirements as evidenced by the importance placed on various usability categories (Tables 4 and 5).
Usability requirements address what is important to customers. At a more granular level, TR provides a foundation for understanding why various aspects of usability may be more (or less) important to different customer segments. As shown in Table 5, both Explorers and Skeptics placed a significantly greater emphasis on content than either Pioneers or Paranoids. Content (e.g., relevance, depth and breath, and currency) addresses the benefits a customer derives from site use, and is similar to constructs such as perceived usefulness (Davis et al., 1989) and relative advantage (Moore & Benbasat, 1991). With lower inhibitions (discomfort and insecurity) than Pioneers or Paranoids, Explorers and Skeptics will tend to focus on the benefits or usefulness of online services-explaining the relative importance placed on this requirement.
Simultaneously, Explorers placed significantly less emphasis on ease of use (i.e., the extent to which an online service site is free of effort) relative to the other three segments (Table 5). Explorers differ from the other segments in that they possess higher levels of contributing beliefs (optimism and innovativeness), while concurrently possessing lower levels of inhibiting beliefs (discomfort and insecurity). Compared to the other TR segments, Explorers are more inclined to believe that technology improves their lives, and they are typically at the leading edge of trying out new technology-based products or services. Explorers likely prefer capability over simplicity or easy to use technology and tend to want to learn new technology on their own-and in fact, they prefer to figure it out themselves. Conversely, the other segments with varied and often countervailing combinations of contributing and inhibiting beliefs may want and need more guidance, advice, and support.
The varied emphasis the TR segments place on the usability requirements of content and ease-of-use do pose a challenge for designers of online service interfaces. Across all TR segments, content and ease of use were the two most important usability requirements (Table 4). Here, it is worth commenting on the within- subjects f-test analysis (see superscripts in Table 4). Both Explorers and Skeptics tend to place a greater emphasis on content (or usefulness) than on ease of use. When attempting to attract new customers to an online service, emphasizing content may help persuade both Explorers and Skeptics to try it, albeit for different reasons. While Explorers like to try new things, particularly with advanced functionality, Skeptics must be fundamentally convinced that a new service will benefit them. For example, airlines are grasping the importance of having Web sites that offer extra value- not just fares, but airline-hotel packages and easier ways to book award travel (Sharkey, 2007). These redesigned sites will address the cutting-edge functionality desired by Explorers and provide added usefulness required by Skeptics.
Conversely, for Pioneers and Paranoids, the usefulness of an online service site (content) and user-friendly operation (ease of use) are equally important. When targeting these customers, ease of use should be given more importance. For example, while search has become a prominent part of the customer online experience, sites often prevent customers from getting the content they want due to confusing navigation. The design of an online service interface should include what customers increasingly expect in terms of search: a text input box, a clickable search button, and a list of prioritized results. Similarly, if content is too much or too complex, Pioneers and Paranoids may be driven away. Here, design aspects should consider the provision of summaries and guides to provide support and reassurance to address the shared inhibitions of these TR segments.
Thus, online service design should take both customer TR and the service offering into consideration. Today, site content is delivered based on information such as customer-provided data, past purchase or search behaviors, geographic location, or access method, with no consideration for the underlying technology beliefs of customers. For example, a common approach is to require customers to register on a site or, in a business-to-business context, complete lead-generation forms in order to access certain content. Because of their low inhibitions, Explorers and Skeptics may be willing to provide this kind of information, but other TR segments may be reluctant. To establish a certain level of credibility for more technology-resistant customers before providing personal data, the designer of an online service should consider moving as much content as possible outside access barriers.
Finally, Explorers and Paranoids placed a greater emphasis on made-forthe-medium as a usability requirement than Skeptics, with Paranoids greater than Explorers (Tables 4 and 5). Our finding concerning made-for-the-medium offers a different perspective than past research, which has found this requirement to be of varying (un)importance. Our study may explain the discrepant and nonsignificant findings. It may be that differences across segments actually masked important results when customers were treated as an undifferentiated mass. Made-for-themedium refers to the extent to which a site can be tailored or personalized to fit specific needs of customers. Importantly, the reason why this usability requirement is important to both Explorers and Paranoids may differ. Explorers are the most optimistic and innovative and the least uncomfortable and insecure segment. For Explorers, the ability to personalize an online service site supports their belief in the ability of technology to provide independence and efficiency. Conversely, while somewhat optimistic about technology, Paranoids are held back by strong inhibitions and a lack of innovative tendencies. For Paranoids, the ability to tailor an online service site likely provides a degree of control and assurance that the online service site is working as desired, thus addressing their high levels of discomfort and insecurity.
From a pragmatic standpoint, psychographic profiles are generally more difficult to observe in a population than demographic data. Thus, it is useful to correlate psychographics with demographic information. Specifically, if there are differences between TR segments on demographics as well as beliefs, then demographic profiles may be used as surrogates (Gilbert & Warren, 1995). As Table 3 indicates, we found significant differences among our TR segments with regard to prior Web experiences and gender. As an example, Explorers in our study were predominately male (71 %) and generally more experienced than all other segments in using the Web for both utilitarian and hedonic activities. Conversely, Skeptics represented the largest percentage of females (53%) and possessed significantly fewer prior experiences. Ultimately, demographic characterizations of TR segments may be surrogates for psychographic constructs, and they may prove useful in usability design and marketing strategies.
Overall, our work complements research concerned with serving and marketing to customers through online interfaces (Burke, 2002; Parasuraman & Zinkhan, 2002; Zeithaml, Parasuraman, & Malhotra, 2002). Ultimately, customer adoption behavior will be affected by satisfaction with the online service that is often indistinguishable from the online interface. For marketers, the success (or failure) of an online service may hinge on whether the interface meets the usability requirements of specific customer segments. Our findings extend current thinking in IS regarding the role that technology beliefs play in influencing user requirements (Orlikowski & Gash, 1994; Darke & Shanks, 1997; Tan & Hunter, 2002). For designers of online service sites, requirements specifications may be incomplete or inaccurate without consideration of underlying customer beliefs, thus leading to difficulties during development and implementation. For example, many sites use some form of segmentation to organize content (e.g., age, company size, familiarity with a product or service), in which customers must click through to the appropriate segment content. However, the segments may not match the underlying technology beliefs of customers and their associated usability requirements.
Ultimately, a key element of online service success will be the integration of marketing and interface design strategies, i.e., the definition of the target customer market, the understanding of usability requirements, and the design of an appropriate interface design. Otherwise, the online service interface may not be sufficiently advanced or too advanced, depending on TR customer segment. Given that different segments tend to enter a market in an evolutionary way, the design aspects of sites should change over time with designers focusing on the usability requirements of emerging market segments. For example, Amazon.com started as a virtual, searchable bookstore that presented a single interface to every customer. Technological capabilities have created the opportunity to move away from the “one face to the world” approach to interface design that characterized early Web sites. Now, Amazon is a collection of tailored services. Returning customers are known by name, the site recommends products and services, customers can rate and review items, and it can show a customer what’s new since s/ he last visited. For advanced customers with different needs, the site even offers ways to make money by partnering with Amazon. Amazon changed the design and functionality of its Web site as technology evolved and in an effort to interpret preferences based on past buying behavior. Ideally, interface design would also take into account the readiness and appetite of customers for customized interfaces. Relating interface design to TR segments would allow a company like Amazon to match emerging design capabilities to market TR characteristics so new functionality is more rapidly and widely adopted. Usability Evaluations: TR and Contextual Factors
When faced with specific online service interfaces, we argued that usability evaluations of specific interfaces will differ across TR segments (//2). At the same time, we also argued that evaluations will be contingent on contextual factors, specifically site type (hedonic vs. utilitarian) and access method (Web vs. wireless Web) (H3). While our MANCOVA results (Table 6) do not support H2, they do offer support for H3, including a three-way interaction effect that relates TR segment membership to the two contextual factors.
Overall, we found that usability evaluations of online service interfaces are driven by a complex interaction between access method, site type, and TR. Table 7 presents the post-hoc univariate results. As shown, there are several statistically significant differences. In addition, Table 7 reports Cohen’s d, an effect size measure based on standardized group-mean difference, pertinent where comparison of group means is of primary interest (Fan, 2001). While statistical significance estimates the probability of obtaining the sample outcome by chance (Kirk, 1996), practical significance (effect size) provides a sample-size independent measure of treatment effect (Fan, 2001 ; Thompson, 2002). Specifically, the effect size would be zero if the mean usability evaluations for hedonic versus utilitarian interfaces were the same.
Table 7: Usability evaluations-Univariance results.
When comparing hedonic and utilitarian Web service interfaces, the results in Table 7 indicate that, in general, the Explorers are more sensitive than Skeptics to differences in usability. Explorers evaluated the usability of the hedonic Web interfaces significantly higher than the utilitarian ones along all usability categories, except for ease of use where they perceived no difference. As discussed earlier, recall that Explorers place significantly less emphasis on ease of use as a usability requirement than other segments. Conversely, Skeptics evaluated the utilitarian Web interfaces significantly higher than the hedonic ones in terms of ease of use and made-for-the-medium, and lower with regard to emotion. The effect sizes were large for Explorers and medium for Skeptics.
When comparing hedonic and utilitarian wireless Web interfaces, both Explorers and Skeptics evaluated the utilitarian interfaces as significantly easier to use than the hedonic interfaces, with the effect sizes both large. Explorers also evaluated the hedonic wireless interfaces as significantly better (with a medium effect size) in terms of the category made-for-the-medium, while Skeptics reported no difference. As discussed earlier, as a usability requirement, the ability to personalize an interface is more important to Explorers than Skeptics. As such, Explorers are likely to consider this category more closely, particularly when faced with an access method (wireless Web) that is relatively novel to them. Finally, with regard to the category emotion, the results indicate that both segments found significant differences (with medium effect sizes) between the hedonic and utilitarian wireless Web interfaces, although in opposite directions.
Overall, our results complement and extend current thinking in HCI and IS on the role of context as a critical predictor of usability evaluations (Venkatesh & Agarwal, 2006; Venkatesh & Ramesh, 2006). Usability evaluations stem not only from contextual factors, but also via a complex interaction with the technology readiness of customer segments characterized by different belief structures. This is particularly important with respect to the introduction of wireless Web as a means to deliver hedonic and utilitarian-oriented services. In nascent markets, when a new technology like the wireless Web is first introduced, most customers are novice users. When exposed to the new technology, it is likely that prior knowledge from a similar, related context (e.g., Web) is likely to play a key role in reactions to the new artifact (Card et al., 1983).
RESEARCH OPPORTUNITIES AND LIMITATIONS
The sample in this study provided both strengths and weaknesses. For example, nearly 93% of the participants were between 18-34 years old, commonly described as a key target market for technology- enabled products and services. While our sample is appropriate given our research focus on online services, as the market for mobile services evolves, future research should include a more diverse sample in terms of both psychographic (i.e., TR segments) and demographic (e.g., age, culture etc.) characteristics. As we expand our population of inquiry, additional research is needed regarding the relationship between psychographic characteristics and demographics. Here, the objective becomes one of relating general psychographic segments to previously identified target subgroups of a diverse overall population (Gilbert & Warren, 1995; Ailawadi et al., 2001).
The relative demographic homogeneity of our sample highlights the value of psychographics as a deeper basis for understanding attitudes and motivations. While organizations may have a demographic profile of a target market, understanding the underlying psychographic characteristics, such as TR, should become the next goal to develop deeper insights as related to interface design. With these insights, online service interface designers may improve usability by focusing on aspects of design pertinent to specific TR segments. Similarly, because customers can vary greatly in their technology beliefs, they may react very differently to marketing communications (Ailawadi et al., 2001). Knowing the usability requirements of specific TR segments provides further information on which customer groups to target with a particular interface design and marketing program.
In our study, we focused our attention on the key differences between the belief structures of the TR segments and how these differences were related to varying usability requirements. While participants were segmented based on a confluence of positive and negative beliefs, it may be useful to examine more closely the relative influence of contributors (i.e., optimism vs. innovativeness) and inhibitors (i.e., discomfort vs. insecurity) on usability requirements. For example, while Skeptics were significantly less optimistic and innovative than Explorers and Pioneers, within the Skeptics segment, they were significantly more optimistic (3.67) than innovative (3.08) (see superscripts in Table 1 segment profiles). A closer examination of the relative influence of each belief on usability requirements may be a useful avenue for future research.
Prior research (Parasuraman & Colby, 2001) has shown that each customer segment enters a technology market at different points in time. Based on the current TR segments of the target population, we speculate that the interface design aspects of an online service would also evolve with time. Future longitudinal study must explore how the changing usability requirements could potentially be predicted based on the TR composition of the target customer population.
With regard to evaluations of the usability of specific online services, our sample had limited experience with mobile data services (and consequently wireless Web sites). However, this is not surprising, given the current state of mobile data services adoption in the United States. While our results illustrated the complex interplay between contextual factors (site type, access method) and TR segments, further research with more experienced user populations should lend additional insights. Furthermore, we focused here on online service sites that may be characterized as hedonic- or utilitarian-oriented in purpose. A logical extension would be to include sites that are designed to support both purposes as well as those that enable direct buying. Based on our current results, this will likely influence usability evaluations. Moreover, as technology experience is acquired, longitudinal research should examine whether segment membership changes and whether the four individual components of TR are relatively stable over time.
In this study, we examined aspects of usability incorporated into the usability metric and instrument based on the Microsoft Usability Guidelines (Agarwal & Venkatesh, 2002). While MUG reflects a fairly comprehensive collection of usability criteria, prior research has proposed additional usability factors (Eighmey & McCord, 1998; Nielsen, 2000). Further research should examine the role of TR and context using alternative conceptualizations of usability found in HCI. Also, the segmentation structure was created using data from a single source which creates a potential common method bias. Future research should include additional sources of data to overcome this potential bias. Finally, we believe that our results are directly related to research concerned with system success where usability would be one driver of satisfaction. For example, the ServQual literature (Parasuraman, Zeithaml, & Berry, 1994) examines the antecedents and outcomes of perceived service quality. In an online service context, it may be argued that usability of the online interface is a key element of the tangibles dimension. An examination of this relationship might be an interesting avenue for future research.
In summary, further empirical testing via both controlled experiments and field studies would lend to developing a rich understanding of the TR-usability link in the online service context.
Over the last decade, we have witnessed the emergence of online services, both hedonic and utilitarian in purpose, delivered via the Web. Many observers predict the next wave of technological revolution will be related to the wireless Web (Lu, Yao, & Yu, 2005; Lin & Wang, 2006). Both methods offer several opportunities for service providers to attract new customers and/or strengthen relationships with existing customers. In an online service context, exposure to a product or service is mediated by technology via the customer interface. Thus, designing online service interfaces that are responsive to customers’ needs is a critical prerequisite for success. In 2005 alone, companies earmarked $100 billion (10% of marketing expenditures) to develop better sites to reach and service customers (Blackfriars Communications, 2005). Usability requires not only a perspective on interface design aspects, but also on the customers themselves.
The objective of this research was to investigate the relationship between customer TR and usability requirements/ evaluations of online service interfaces. Our core premise was that the beliefs that underlie TR form the foundation for expectations of how things should work and how specific service interfaces are evaluated. Our approach acknowledges that not all customers are the same or motivated by the same beliefs. The results of our exploratory analysis provide evidence that (i) TR customer segments vary in usability requirements and (ii) usability evaluations of specific online service interfaces are influenced by a complex interaction between site type, access method, and TR customer segment. As organizations continue to expand their online service offerings, managers must recognize that the interface exists to serve the customers, so their design must be matched to market needs and TR. [Received: March 2006. Accepted: February 2007.]
*We thank C. L. Colby and J. Taliuaga, Rockbridge Associates Inc., and A. Parasuraman for their assistance. We also thank V. Venkatesh and E. Karahana for their comments on earlier versions of this article. We would also like to thank Verizon Wireless, Sierra Wireless, and Hewlett-Packard for supporting this study. In particular, we acknowledge the support of Benjamin Ortiz, Verizon Wireless’ Regional Director of Data Services.
Agarwal, R., & Prasad, J. (1999). Are individual differences germane to the acceptance of new information technologies? Decision Sciences, 30(2), 361-391.
Agarwal, R., & Venkatesh, V. (2002). Assessing a firm’s web presence: A Heuristic evaluation procedure for the measurement of usability. Information Systems Research, 13(2), 168-186.
Ailawadi, K., Neslin, S., & Gedenk, K. (2001). Pursuing the value- conscious consumer: Store brands versus national brand promotions. Journal of Marketing, 65(1), 71-89.
Babin, B. J., Darden, W. R., & Griffin, M. (1994). Work and/or fun: Measuring hedonic and utilitarian shopping. Journal of Consumer Research, 20(4), 644656.
Ballard, B., & Miller, B. (2001). WML style guide for the phone.com 4.X Browser(TM): Sprint PCS white paper, accessed November 5, 2006, available at http://developer.openwave.com/docs/carrier_sg/ WMLStyleGuide 1 .pdf.
Barley, S. R. (1986). Technology as an occasion for structuring: Evidence from observation of CT scanners and the social order of radiology departments. Administrative Science Quarterly, 31(1), 78- 108.
Barnard, P. (1993). The contributions of applied cognitive psychology to the study of human-computer interaction. In Baechker, Grudin, Buxton, Greenberg (Eds.), Readings in human computer interaction: Toward the year 2000. San Francisco: Morgan Kaufmann.
Barnes, S., Liu, K., & Vidgen, R. (2001). Evaluating WAP news sites: The WebQual/m approach. Proceedings of the 9th European Conference on Information Systems. Bled, Slovenia, 344-355.
BeIk, R. W. (1975). Situational variables and consumer behavior. Journal of Consumer Research, 2(3), 157-167.
BeIk, R. W., Wallendorf, M., & Sherry, J. E, Jr. ( 1989). The sacred and the profane in consumer research: Theodicy on the odyssey. Journal of Consumer Research, 16(1), 1-38.
Bitner, M. J., & Brown, S. W. (2006). The evolution and discovery of services science in business schools. Communications of the ACM, 49(7), 73-78.
Blackfriars Communications. (2005). One tenth of trillion dollar marketing spend, accessed February 6, 2006, available at http:// www.clickz.com/showPage. html?page=3510121.
Brief, A. P., & Aldag, R. J. (1977). The intrinsic-extrinsic dichotomy: Toward conceptual clarity. The Academy of Management Review, 2(3), 496-500.
Burke, R. (2002). Technology and the customer interface: What consumers want in the physical and virtual store. Journal of Academy of Marketing Science, 30(4), 411^32.
Burroughs, R. E., & Sabherwal, R. (2002). Determinants of retail electronic purchasing: A multi-peri