August 27, 2008

Learning Indicators and Collaborative Capacity

By Getha-Taylor, Heather

ABSTRACT Learning has been identified as important component of collaborative capacity (Bardach, 1998, 2001). Without the capacity to learn on the organizational, team, and individual levels, public organizations may also lag in their ability to collaborate successfully across organizational boundaries. Nowhere is the need to collaborate more apparent than in the U.S. Department of Homeland Security (DHS), which was created in part to foster collaboration among 22 distinctly different member organizations. This manuscript examines the connection between organizational learning and collaborative capacity, with special emphasis on DHS. By matching questions from the Dimensions of the Learning Organization Questionnaire (DLOQ) with questions from the 2004 Federal Human Capital Survey (FHCS), this study assesses key learning indicators on the federal level. Findings suggest that DHS employees lag behind their counterparts on one key measure of organizational learning: the ability to assess performance gaps. This lag may affect the organization's ability to collaborate effectively, but progress may be made if potential organizational learning disorders (OLDs) are examined and addressed. Special attention is given to action learning as a tool for overcoming potential learning disorders and improving organizational performance.


The Homeland Security Act of 2002 mandated collaboration as the necessary mechanism for accomplishing the complex and critical mission of protecting the American people. However, in the years since the U.S. Department of Homeland Security (DHS) was created as part of this Act, collaborative success still seems elusive. Part of this problem may be attributed to a potential lack of collaborative capacity among DHS organizations, as identified by such scholars as Goodman et al. (1998) and Bardach (1998, 2001). The capacity to collaborate, as described by these authors, is built upon such components as trust, communication, intellectual capital, creative opportunity, acceptance of leadership and learning.

The ability to learn is of primary importance as the public management environment is in a constant state of change. Specifically, the need to work across organizational boundaries to solve complex public problems requires adaptation. For instance, scholars such as Ackoff (1974), Clarke and Stewart (1997), Huxham and Vangen (2000), O'Toole (1997), and Trist (1983) have suggested that while bureaucratic organizations were once ideal for solving easily defined problems, today's problems call for new knowledge and organizational forms. "There seems little doubt that the public sector management in the 21st century will need to be sophisticated in its understanding of the skills, processes, structures, tools, and technology needed for working across organizational boundaries" (Huxham & Vangen, 2000).

While new organizational forms, including networks and collaborative bodies are promising in terms of their ability to address the complex problems of modern governance, they also present a challenge. Because their function is different from that of hierarchies, traditional management thought does not apply cleanly to these new organizational structures (Kettl, 1996; O'Toole, 1997).

As noted by Senge (1990), organizations must learn in order to adapt to "emerging conditions." The "emerging condition" of interest for this research is the need to collaborate across organizational boundaries to achieve public goals. This research investigates the connection between collaboration and organizational learning by examining learning indicators on the federal level, with special emphasis on the U.S. Department of Homeland Security, where collaboration is central to achieving the critical organizational mission.

Learning and innovation, says West (2001), are the "fundamental reason[s] for humans to work in groups" (p. 270). And while the end goal of collaborative capacity building is ongoing learning, Argyris and Schon (1996) contend that organizational learning is especially important in increasingly volatile environments, which homeland security would certainly qualify. Lipshitz et al. (2002) also note that organizational learning is most critical in organizations that are under crisis or when the organizational activities involve "relatively high costs of error" (p. 91). Given these findings, DHS is an ideal organization for focused examination.

Learning, Performance, and the U.S. Department of Homeland Security

Following the creation of the U.S. Department of Homeland Security in 2002, the U.S. Government Accountability Office (GAO) issued a report (2003) that hinged the success of the newly-formed organization on its ability to build upon on the lessons learned from the response to September 11 and enhance planning for the future. Regrettably, it seems that GAO's admonition was largely ignored.

Two years following the report's publication, DHS's widely criticized response to Hurricane Katrina illustrated that important lessons had already been forgotten. In 2006, the House of Representatives concluded that the effort illustrated "the U.S. government's failure" to appropriately apply lessons learned from the September 11th terrorist attacks.

According to Irons (2005), DHS "did not approach the challenges posed by the disaster with the point of view of a learning organization but, rather, as a top-down bureaucracy." Donahue and Tuohy (2006) suggest that failure to learn is not specific to DHS, but is endemic to the public sector generally speaking: "we identify the same lessons again and again, incident after incident" (p. 1).

According to these authors, "experience suggests that purported lessons learned are not really learned; many problems and mistakes are repeated in subsequent events" (p. 3, emphasis in original). Given this finding, the mandate is clear: understanding how DHS can improve its capacity to learn and thus, improve future performance, is i. An important first step, however, is to define the concept of organizational learning.

Organizational and Individual Learning

Scholars have debated the proper level of aggregation to focus on learning within groups and organizations. At which level, asks Argyris (1999), does is make sense to speak of "productive organizational learning?" That is, should we examine learning on the individual, group, inter-group, or whole-organization level? To examine this question, it is important to first differentiate individual learning from organizational learning.

According to Yeung et al. (1999), "individual learning occurs as members within the organization acquire knowledge through education, experience, or experimentation. Organizational learning occurs as its systems and culture retain this knowledge and transfer it from individuals" (p. 9). Snyder and Cummings (1998) further differentiate these two approaches: learning is organizational to the extent that: 1) it is done to achieve organization purposes; 2) it is shared or distributed among members of the organization; and 3) learning outcomes are embedded in the organization's systems, structures and culture.

Despite these attempts at division, the fact is organizational and individual learning are intertwined. As noted by Turniansky and Hare (1998), organizations only learn through the individuals employed within them: "it is actually the individuals in the organizations who learn about the structure and processes in their organization and how their organization relates to other organizations, to the larger society and the environment" (p. 112). The authors clarify this point by saying, "organizations themselves do not learn, but the individuals in the organizations learn as they communicate, learn and change, and this in turn influences organizational learning and change" (p. 113).

While scholars seem to agree that organizational learning occurs through individuals, organizational learning is not simply the cumulative result of employee learning. "Organizations do not have brains, but they have cognitive systems and memories. As individuals develop their personalities, personal habits, and beliefs over time, organizations develop worldviews and ideologies. Members come and go, and leadership changes, but organizations' memories preserve certain behaviors, mental maps, norms, and values over time" (Hedberg, 1981, p.6).

While a considerable amount of work has been done to differentiate organizational from individual learning, it is the work of Argyris and Schon (1974, 1978), including their definition of learning ("the detection and correction of error") that is considered to be the foundational work in this area of study. As noted by Argyris and Schon (1978), "individual members are continually engaged in attempting to know the organization, and to know themselves in context of the organization" (p. 16).

According to Lipshitz et al. (2002), there are three organizational policies that facilitate learning on multiple levels: commitment to learning, tolerance for error, and commitment to workforce development (p. 88). Of these three policies, tolerance for error is of particular interest in this research. When organizations adopt an approach in which "errors in the service of learning will not be punished but indeed valued as opportunities for learning" (Lipshitz et al., 2002, p. 89), a culture of learning can begin to take root. One potential way to create a culture in which learning is valued and error is tolerated is through the application of action learning principles in the workplace. Action Learning

Organizational learning, says Lipshitz and colleagues (2002), "is crucially dependent on people's willingness to care for and share knowledge with others" (p. 88). The application of action learning principles in the workplace offers an opportunity to develop trust and communication among a set of diverse individuals who have together identified learning as their primary outcome of interest.

According to Marquardt (1999), "action learning is both a process and a powerful program that involves a small group of people solving real problems while at the same time focusing on what they are learning and how their learning can benefit each group member and the organization as a whole" (p. 4, emphasis in original).

The key component of action learning is the "set" of individuals, whose shared concerns center on "acting on the learning gained from reflection" (McGill & Beaty, 1992, p. 22). From this perspective, action learning can be a powerful tool in 1) recalling lessons learned, and 2) improving future organizational performance through reflection on past experiences.

The value of this approach was emphasized by West (2001): "innovation is not a solitary activity that results from the vigorous championing of an idea by one individual. It is more usually the result of concerted activities of groups of people developing and implementing their ideas over a period of time, and then diffusing successful innovations throughout organizations or societies" (p. 276).

The application of action learning strategies is especially appropriate for turbulent contexts such as homeland security. "Action learning in effect creates a working context in which people are encouraged to ask, learn and take action in a constantly changing environment where no one person knows exactly what to do next" (Inglis, 1994, p. 25).

A commitment to action learning contributes to the creation of a culture that supports learning on the individual, team, and organizational levels (Revans, 1982; Marquardt, 1999). "As teams learn, they can become a microcosm for learning throughout the organization: insights gained by the team are put into action, and skills developed by the team can be transferred to other individuals and to other teams. The team's accomplishments can set the tone and establish a standard for group learning for the larger organization" (Marquardt, 1999, p. 103).

Additionally, a commitment to action learning helps to create an environment of "psychological safety." Organizational learning is rare, says Lipshitz et al. (2002), because it requires "psychological safety," which is necessary for people to feel that they can take the necessary risks for learning. According to work by Edmondson (1997, 1999), there is a strong relationship between psychological safety and learning in organizations.

Schein's (1993b) point is especially salient in this context: "For habit and skill learning to take hold, we need opportunities to practice and make errors. We need consistent rewards not only for correct responses but also for detecting errors so that they can be corrected?this kind of learning, therefore, is constrained not only by the difficulty of getting the response in the first place, but by lack of a safe environment in which to practice and make lots of errors" (p. 87). Action learning, which emphasizes a safe arena for diverse perspectives and individual reflection, offers an opportunity to create the necessary conditions for organizational learning.

The Practical Challenge: Learning and Collaboration

Especially when faced with a world of "turbulent change," organizations must learn faster, placing even greater value on developing a culture of learning (Schein, 1992). Schein claims that "the primary task of a leader in contemporary organizations is to create and sustain such a culture" (p. 372).

The Office of Personnel Management (2001) agrees that a culture of learning relies on good leaders. "Leaders foster a learning culture that provides opportunities for continuous development and encourages employees to participate. Leaders invest in education, training, and other developmental opportunities to help themselves and their employees build mission-critical competencies" (p. 2).

Scholars such as Argyris and Schon (1996) and Schein (1992) contend that leaders also help foster a learning culture by engaging in behaviors that communicate the vision and value of learning and then rewarding the associated behavior when it is displayed within the organization. The importance of rewarding desired cultural behaviors is repeated in the literature (see Amabile, 1983; Kanter, 1983; Schein, 1993b; West, 2001).

A focus on learning could be seen as just one more priority to add to the unwieldy list of demands that public managers face on a daily basis. However, the result of neglecting learning cannot be overstated. In the wake of Hurricane Katrina, a number of publications were released that identified the failures, from communication to leadership. One failure that was also highlighted by Kettl (2005) was our inability to learn from the past: "We've been taught the lessons we need to learn. The question now is whether we will learn from them. If not, the worst is surely yet to come" (p. 16).

If learning is a critical component of collaborative capacity and if DHS has exhibited problems in collaborating, the question of interest becomes: how does DHS compare to other federal organizations on organizational learning indicators?

Data and Methods

While there are currently no databases that track indicators of learning in the federal government, the Federal Human Capital Survey (FHCS) offers an opportunity to investigate such indicators indirectly. The FHCS is the largest federal employee survey to date. Nearly 150,000 employees responded to the 2004 survey (51% response rate). Using the Dimensions of the Learning Organization Questionnaire (DLOQ) as a guide, questions were matched from the 2004 FHCS to track indicators of learning on the individual, team, and organizational levels. The DLOQ has been recommended for use in organizational studies based on its construct validity and reliability (Yang et al., 2004).

Although validity is a concern when fitting new constructs to existing survey data, the match between learning indicators and survey questions was supported through the use of multiple coders (five in total). Inter-coder reliability rates of 80% or better were used to select the best matches. A total of 12 questions from the Federal Human Capital Survey (FHCS) were matched with the Dimensions of the Learning Organization Questionnaire (DLOQ) to assess learning on the individual, team, and organizational levels (Table 1). Factor analysis confirmed that these 12 questions identified a single construct: learning.


Survey responses to the above questions were analyzed, controlling for organizational affiliation. As a result, mean responses were compared for employees in DHS organizations and all others. Based on this analysis, one dimension of organizational learning was found to be significant. Organizational Indicator #1 (described in Table 1 and highlighted in Table 2), speaks to an organization's ability to measure gaps between present and expected performance. It was on this dimension of organizational learning where a significant difference existed between DHS responses and responses from employees in other organizations. On this dimension, DHS responses are significantly lower. Results are presented in Table 2.

These results suggest that DHS employees may be less able to track progress in meeting goals and thus, performing effectively. This finding has significant consequences for overall organizational performance in addition to collaborative performance. According to a 2005 U.S. Government Accountability Office (GAO) report, eight key practices improve collaborative outcomes, including individual and organizational accountability for results. If this ability is lacking, it negatively affects overall collaborative capacity.

Explanations for Differences in Learning

Given that a significant difference is observed on an organizational-level learning indicator, the work of Snyder and Cummings (1998) is relevant. According to Snyder and Cummings, there may be a number of organizational "learning disorders" to blame. Because learning on the organizational level translates to learning within collaborative bodies, it is important to understand the causes of organizational learning deficiency. Snyder and Cummings note that these deficiencies have been referred to as "learning disabilities" (Senge, 1990), "learning errors" (Marsick & Watkins, 1990), "learning barriers" (Shaw & Perkins, 1992) and "learning obstacles," (McGill & Slocum, 1994). Snyder and Cummings (1998) group these concepts together as "organization learning disorders (OLDs)" which "prevent organizations from achieving their learning potential, and consequently can have an adverse effect on organizations' performance capability" (p. 874).

According to Snyder and Cummings, organizational learning disorders (OLDs) may occur at any stage of the learning cycle from discovery to invention to production to generalization. The authors provide a list of various OLDs, which may occur at each stage of the learning process. Based on Snyder and Cummings' summary, the previously presented empirical findings, and the review of the literature as it pertains to organizational learning, four potential OLDs were selected which seem most fitting to DHS. Table 3 explains each OLD along with the potential effects, if not corrected. It is not sufficient to describe the potential learning disorders. It is necessary to understand how each applies to the factor of interest: inability to measure gaps between current and expected performance. Further, it is important to identify options for intervention in order to make corrections. These applications and potential interventions, as they apply to DHS, are addressed in Table 4 (below).

Improving Performance Assessment through Action Learning

An additional way to improve learning capacity and overcome potential learning disorders in the U.S. Department of Homeland Security is through the application of action learning principles. Given that DHS's specific difficulty appears to center on assessing gaps between current and expected performance, action learning can offer an opportunity to engage in ongoing reflection, which may contribute to future performance.

McGill and Beaty (1992) contend that the connection between reflection and performance is the very foundation of action learning. "We all learn through experience by thinking through past events, seeking ideas that make sense of the event and help us to find new ways of behaving in similar situations in the future. This thinking through or reflection is the essential link between past action and more effective future action" (p. 17).

In order for action learning to be effective, however, Inglis (1994) cautions that members of the action learning set must "feel relaxed and comfortable in confronting internal issues, critically discussing each others' ideas and proposals and in sharing confidential information" (p. 42). Given the difficulty faced among DHS component agencies in the past, special attention should be given to establishing and improving inter-organizational relationships in tandem with implementing action learning principles. Perhaps, Marquardt (1999) says, action learning strategies may be exactly what are needed to overcome previously strained relationships: the sharing that takes place in action learning settings can help establish the trust and cohesiveness that contributes to improved organizational performance.

Perhaps the chief threat to the efficacy of action learning principles in the context of homeland security is the increasing federal focus on achieving results rather than focusing on developmental experiences (Springer, 2006). This emphasis will likely limit the degree to which learning is valued on both the individual and organizational levels, given Fiol and Lyles's (1985) finding that if the organizational culture does not value learning and the associated processes, including action learning, then broader organizational learning is unlikely to take place.


Based on this analysis, it is clear that DHS lags behind other federal organizations on one important component of collaborative capacity: organizational-level learning. More specifically, DHS employees lack capacity to assess gaps between current and expected performance. A number of suggested intervention strategies may mediate this problem. Perhaps the most promising approach involves a commitment to action learning strategies. Action learning promotes ongoing reflection and is uniquely suited to environments that are characterized by perpetual change. Further, action learning promotes relationship-building and supports the formation of a culture that values learning on the individual, team, and organizational levels.

The careful examination of learning as a component of collaborative capacity speaks directly to the larger issue of organizational performance. According to Kettl (2005), failures in the wake of Hurricane Katrina did not simply illustrate administrative gaps. Rather, they illustrated the inability to learn. "When faced with Katrina, government, at all levels, failed. In fact, the bungled response ranks as perhaps the biggest administrative failure in American history. September 11 was thus a major lost opportunity. Government could have - and should have - learned from that awful day about how to make homeland security work. When put to the test, it failed. The reason? We failed to learn."

Learning is no longer an option; it is a necessity if organizations are to meet changing performance expectations with increasingly limited resources. However, identifying learning deficiencies that may negatively affect collaborative performance is just one piece of the puzzle. Scholars and practitioners should together identify additional ways to track progress over time and improve on metrics of interest. Supporting learning on the individual, team, and organizational levels is an important step in the right direction. By addressing potential organizational learning disorders on all three levels, public management scholars and practitioners alike can help build the collaborative capacity that is necessary to collectively solve our most pressing public problems.


Ackoff, R. L. (1974). Redesigning the future: A systems approach to societal problems. New York: Wiley.

Amabile, T. M. (1983). The social psychology of creativity: A componential conceptualization. Journal of Personality and Social Psychology, 4(2), 357-376.

Argyris, C. (1999). On organizational learning (2nd ed.). Oxford: Blackwell.

Argyris, C., & Schon, D. A. (1974). Theory in practice: Increasing professional effectiveness. San Francisco: Jossey-Bass.

Argyris, C., & Schon, D. A. (1978). Organizational learning: A theory of action perspective. Reading, MA: Addison-Wesley.

Argyris, C., & Schon, D. A. (1996). Organizational learning II: Theory, methods and practice. Reading, MA: Addison-Wesley.

Bardach, E. (1998). Getting agencies to work together: The practice and theory of managerial craftsmanship. Washington, DC: Brookings Institution Press.

Bardach, E. (2001). Developmental dynamics: Interagency collaboration as an emergent Phenomenon. Journal of Public Administration Research and Theory, 11(2), 149-164.

Clarke, M., & Stewart, J. (1997). Handling the wicked issues: A challenge for government. Birmingham, UK: University of Birmingham, Institute of Local Government Studies.

Donahue, A. K., & Tuohy, R. V. (2006). Lessons we don't learn: A study of the lessons of disasters, why we repeat them, and how we can learn them. Homeland Security Affairs, 2(2), 1-28.

Edmondson, A. (1997). Learning when it's safe: A group-level investigation of antecedents and consequences of learning behavior (Working Paper No. 07-069). Harvard Business School. Cambridge, MA.

Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44, 350-383.

Fiol, C. M., & Lyles, M. A. (1985). Organizational learning. Academy of Management Review, 10, 803-13.

Goodman, R. M., Speers, M. A., McLeroy, K., Fawcett, S., Kegler, M., Parker, E., Smith, S. R., Sterling, T. D., & Wallerstein, N. (1998). Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Education & Behavior, 25(3), 258-278.

Hedberg, B. (1981). How organizations learn and unlearn. In Nystrom & Starbuck (Eds.), Handbook of Organizational Design. Oxford: Oxford University Press.

Huxham, C., & Vangen, S. (2000). Leadership in the shaping and implementation of collaboration agendas: How things happen in a (not quite) joined-up world. Academy of Management Journal, 43(6), 1159- 1166.

Inglis, S. (1994). Making the most of action learning. Brookfield, VT: Gower.

Irons, L. (2005). Hurricane Katrina as a predictable surprise. Homeland Security Affairs, 1(2), 1-18.

Kanter, R. M. (1983). The change masters: Corporate entrepreneurs at work. New York: Simon & Schuster.

Kettl, D. F. (1996). Governing at the millennium. In J. L. Perry (Ed.), Handbook of Public Administration (2nd ed.). San Francisco: Jossey-Bass.

Kettl, D. F. (2005). The Worst is yet to come: Lessons from September 11 and hurricane Katrina. (Report No. 05-01). Fels Institute of Government.

Lipshitz, R., Popper, M., & Friedman, V. J. (2002). A multifaceted model of organizational learning. The Journal of Applied Behavioral Science, 38(1), 78-98.

Marquardt, M. J. (1999). Action learning in action: Transforming problems and people for world-class organizational learning. Palo Alto, CA: Davies-Black.

Marsick, V. J., & Watkins, K. E. (1990). Informal and incidental learning in the workplace. London: Routledge.

McGill, I., & Beaty, L. (1992). Action learning: A practitioner's guide. London: Kogan Page.

McGill, M., & Slocum, J. (1994). The smarter organization: How to build a company that learns, unlearns, and adapts to capitalize on marketplace needs. New York: Wiley.

O'Toole, L. J. (1997). Treating networks seriously: Practical and research-based agendas in public administration. Public Administration Review, 57(1), 45-52.

Revans, R. W. (1982). The origins and growth of action learning. Bromley, UK: Chartwell-Brat.

Shaw, R. B., & Perkins, D. N. T. (1992). Teaching organizations to learn. In D. A. Nadler, M. Gerstein, & R. B. Shaw (Eds.), Organizational architecture. San Francisco: Jossey-Bass.

Schein, E. H. (1992). Organizational culture and leadership. San Francisco: Jossey-Bass.

Schein, E. H. (1993). On dialogue, culture, and organizational learning. Organizational Dynamics, 22(2), 40-51.

Schein, E. H. (1993b). How can organizations learn faster? The challenge of entering the green room. Sloan Management Review, 34(2), 85-92.

Senge, P. (1990). The fifth discipline. New York: Doubleday.

Snyder, W. M. & Cummings, T. G. (1998). Organization learning disorders: Conceptual model and intervention hypotheses. Human Relations, 51(7), 873-895.

Springer, L. M. (2006, January 30). Memorandum for chief human capital officers: Building a results-oriented performance culture. Trist, E. (1983). Referent organizations and the development of inter-organizational domains. Human Relations, 36, 269-284.

Turniansky, B., & Hare, A. P. (1998). Individuals and groups in organizations. London: Sage.

U.S. Government Accountability Office. (2003). Major management challenges and program risks: Department of Homeland Security (Issue Brief No. 03-102).

U.S. House of Representatives (2006). Failure of Initiative: The Final Report of the Select Bipartisan Committee to Investigate the Preparation for and Response to Hurricane Katrina. February 15. Accessed online at:

United States Office of Personnel Management. (2001). Guide to the Senior Executive Service. Author.

West, M. A. (2001). The human team: Basic motivations and innovations. Handbook of industrial, work, and organizational psychology: Vol. 2. London: Sage.

Yang, B., Watkins, K.E., & Marsick, V.J. (2004). The construct of the learning organization: dimensions, measurement, and validation. Human Resource Development Quarterly. 15(1), 31-55.

Yeung, A. K., Ulrich, D. O., Nason, S. W., & Von Glinow, M. A. (1999). Organizational learning capability. New York: Oxford University Press.


University of South Carolina

Copyright Southern Public Administration Education Foundation Summer 2008

(c) 2008 Public Administration Quarterly. Provided by ProQuest LLC. All rights Reserved.