September 30, 2007
Predicting Performance of One-Year Mba Students
By Fish, Lynn A Wilson, F Scott
Although several studies have been performed, Graduate Admissions programs are still encountering difficulties uncovering criteria that will predict academic success in their programs. Researchers have analyzed Executive, full and part-time MBA programs and can only conclude that undergraduate grade point average and the GMAT are significant factors to predicting success; however, predictability with these factors is less than 19%. Similar to other studies, regression analysis is used to analyze potential factors to predict success in a highly-controlled One-Year MBA program at an AACSB-accredited American college on the United States-Canadian border. Model predictability increases over previous studies as the Canadian-factor, GMAT-Verbal and undergraduate grade point average are significant factors. These results raise questions regarding the significance of the GMAT-Verbal versus the GMAT-Quantitative and differences between American and Canadian school systems. LITERATURE REVIEW
Since admissions decisions are critical at educational institutions, various studies have reviewed the incoming factors that may assist in predicting MBA student performance. Researchers point to the necessity for each MBA program to individually determine the relationship among predictor variables and graduate level performance in its program [Wright and Palmer, 1997]. Various programs have different admissions processes ranging from review of undergraduate record (grade point average), type of courses taken, trends and progress over time, level of analytical and quantitative skill required in current and past professions, recommendations, and the Graduate Management Aptitude Test (GMAT). Noteworthy points to this study include analysis of prediction factors for a One-Year, one-classroom MBA cohort program; the Canadian, GMAT- Verbal and undergraduate grade point average (GPA) are significant factors; and an improvement in predictability over similar studies.
Previous studies to predict MBA per- formance focus on predicting overall MBA quality point average (QPA). Factors tested to predict performance include, but are not limited to: total GMAT, GMAT- Verbal score, GMAT-Quantitative score, undergraduate grade point average (GPA), junior/senior GPA, length of time out of school, sex, age, undergraduate major, undergraduate institution, undergraduate major, gender, and work experience [Braunstein, 2002; Carver, Jr. and King, 1994; Deckro and Woudenberg, 1973, Fisher and Resnick, 1990; Graham, 1991; Hecht et al., 1989; McClure, 1986; Paolillo, 1982; Remus and Wong, 1982; Sobol, 1984; Wilson and Hardgrave, 1995; Wright and Palmer, 1997]. Researchers vary in their handling of students dismissed or who left the program, and current students versus graduates.
Over twenty-years of similar studies, results demonstrate total GMAT and undergraduate GPA are always significant factors [Braunstein, 2002; Hecht et al., 1989; McClure, 1986; Paolillo, 1982; Wright and Palmer, 1997] with prediction equations explaining 19% or less of the graduate GPA [Wilson and Hardgrave, 1995]. Total GMAT has been shown to be statistically significant in differentiating high performers versus other students [Wright and Palmer, 1997; Braunstein, 2002]. Only one exception to this predictability has been uncovered - an Executive MBA program at Tulane in New Orleans, Louisiana, where the coefficient of determination was .36 [Arnold, Chakravarty and Balakrishnan, 1996]. In this Executive MBA program, GMAT remains the best single indicator, but qualitative factors, such as work experience, motivation and business success, enhance the predictive ability of the model [Arnold, Chakravarty and Balakrishnan, 1996]. In another study, the GMAT-Verbal score, but not the GMAT-Quantitative score, is a significant factor to differentiate between high performers and other students [Wright and Palmer, 1997]. The authors acknowledge that the GMAT-Verbal may be a factor of curriculum content and may not be significant for every program. In yet another study, stepwise regression results reveal junior/senior GPA as the first variable to enter into the equation [Paolillo, 1982]. The accuracy of using undergraduate GPA as an indicator declines with time as undergraduate GPA becomes less significant in the prediction of MBA performance for older applicants [Hecht, Manning, Swinton, and Broun, 1989].
Other variables included in previous studies, but vary in their impact as to whether they are significant or not, include length of time out of school [Arnold, Chakravarty and Balakrishnan, 1996], sex [Wilson and Hardgrave, 1995], undergraduate major [McClure, 1986], undergraduate institution [McClure, 1986], age, gender, and work experience [Carver, Jr. and King, 1994; Deckro and Woudenberg, 1973, Everett and Armstrong, 1990; Fisher and Resnick, 1990; Graham, 1991, Paolillo, 1982; Remus and Wong, 1982; Sobol, 1984; Wright and Palmer, 1997]. Analysis of work experience as a predictive factor is inconclusive as one study shows significance through ANOVA and correlation analysis [Adams and Hancock, 2000], while it is insignificant in another study [Everett and Armstrong, 1990].
Regression analysis has been the chosen method for most researchers; however, due to its low predictability, nonparametric methods, such as neural nets and ANOVA have also been used. Neural nets can effectively classify students into groups and hold promise for the future [Naik and Ragothaman, 2004]. In fact, a neural network model performs as well as statistical models and can be a useful tool in predicting MBA student performance [Naik and Ragothaman, 2004; Wilson and Hardgrave, 1995]. For an Executive MBA program, neural nets appear to perform slightly better than regression techniques [Arnold, Chakravarty and Balakrishnan, 1996]. ANOVA has been used to demonstrate differences between no risk (> 3.3 graduate GPA), questionable (3.0 - 3.3 graduate GPA), and risky (
Our objective with this research is to investigate potentially relevant factors to predicting One- Year MBA student performance, and based upon the results, potentially modify the graduate admissions process. Our objective is not to compare different modeling methodologies.
Six years of data, from fall 1999 through spring 2005, for a One- Year MBA program at an AACSB-accredited college on the United States- Canadian border in the Northeast are analyzed in this study. The One- Year MBA program is small, consisting of a single cohort each year, starting and ending in late August. The data set contains 143 students including 12 Canadians and 12 International students (non- North Americans). Students admitted to the program are required to obtain a minimum score on the GMAT and a minimum formula score, calculated based upon their GMAT score and undergraduate GPA.
A list of factors, including some factors which other researchers have investigated, are tested as predictors of academic performance, including GMATVerbal, GMAT-Quantitative, undergraduate GPA, Canadian student, International student (non-North American), years since graduation with bachelors, business undergraduate and undergraduate of the particular school. On average, students obtain their undergraduate degree 2.54 years prior to entering the program, carry a 3.04 under- graduate GPA and score 492 on the GMAT. Forty-nine (34%) students are business students as undergraduates, and 42 (29%) are undergraduates of the particular school (not necessarily business students).
In this study, the dependent variable is the academic success of the student as represented by the final graduate QPA.
Correlation analysis for the independent variables demonstrates only slight relationships between the variables as shown in Table 1 . A high positive correlation occurs between undergraduate GPA (UG- GPA) and the respective school (.278). Negative correlations between the years out of school and the respective school (-.233), Canadian students and the respective school (-.195), undergraduate GPA and Canadian students (-.328), and the GMAT- Verbal and international students (-.316) exist.
Regression analysis, as shown in Table 2, indicates that significant factors include the Canadian factor (p=.000), undergraduate GPA (p=. 000), and GMAT- Verbal (p=.002). The best- fit model for predicting MBA performance for data gathered from 1999- 2004 is:
Graduate QPA = 2.249 + .378 Canadian + .271 UG-GPA + .004 GMAT- Verbal
with predictability of R^sup 2^ = .230. Business undergraduate students (p=.941), undergraduates from the respective school (p=.251), the number of years since graduation (p=.969), the International factor (p=. 629) and the GMAT-Quantitative (p= 0.917) are not statistically significant. DISCUSSION
The One- Year MBA program, where students are instructed as a one- classroom cohort, is a very controlled situation, and therefore, may shed light on the true relationship between input and output factors. Analysis of incoming factors to predict success in a One- Year MBA program reveal a significant Canadian factor, improvement in predictability over similar studies, and verification of the undergraduate GPA and GMAT- Verbal as significant predictors for the program.
The One- Year MBA program model improved predictability (R^sup 2^=0.230) over other published results (2.8%) [Naik and Ragothaman, 2004; Wilson and Hardgrave, 1995]. As expected, the undergraduate GPA is a significant fac tor to predicting graduate performance, (as demonstrated through the graduate QPA). This study verifies the significance of the GMAT- Verbal (over the GMAT-Quantitative) again [Wright and Palmer, 1997]. Therefore, in keeping with other studies [Wright and Palmer, 1997; Wilson and Hardgrave, 1995], while obviously relevant, use of GMAT- Verbal and undergraduate GPA by themselves cannot predict student success. To Graduate Admissions departments, this result by itself indicates that MBA admission tothis One- Year program should not be solely based upon GMAT- Verbal and undergraduate GPA, similar to results in an Executive MBA program [Arnold, Chakravarty, and Balakrishnan, 1996]. The GMAT score, a standardized test, although significant, does not represent qualitative factors, such as individual motivation or business success, which may account for the other eighty-percent of student predictability. The undergraduate GPA, although significant, may be a poor predictor of graduate performance due to grade inflation and differences between institutions in determining grades.
The important factor to note with respect to this study is the statistical relevance of the Canadian factor. This result raises questions regarding differences in school systems to properly prepare students for business school education. Do Canadian business students do as well at Canadian schools? Do American students do well at Canadian business schools? Why is the Canadian system able to better prepare future MBA students at an American college? Should Admissions Directors be encouraging other Canadians to pursue an American MBA? Does the length of pre-college study make a significant difference ? Prior to two years ago, the Canadian school system required 13 years of preparation prior to pursuing an undergraduate degree, while the American system requires just 12. Canadian students in this study graduated from the 13-year program. Will Canadian students from the 12-year program perform as well? Should Canada consider returning to a 13-year program of study? Also noteworthy, is the negative correlation between undergraduate GPA and Canadian students (-.328), implying that applying Canadian students' GPA' s are lower than other applying students.
Of particular importance to this specific study is the nature of the program under study - a one-year, single class per year, MBA program. The key to the study is the fact that all students in the program are educated by the same faculty, with little variation between years, and therefore are measured in the same system within a particular year and across years. Essentially, the differences in graduate QPA in other studies may be due to differences in instructors and grading methodologies. The nature of the oneyear MBA program allows for a highly controlled system.
Also of note were the independent variables that were insignificant, specifically, the international factor, GMAT- Quantitative, business undergraduate, respective school undergraduate, and years (time) since undergraduate graduation. The international factor was insignificant, while the Canadian factor was significant for this program. Due to the various international student backgrounds the effect of different school systems around the world may not have been as apparent as the Canadian factor. Interestingly, the negative correlation between the GMAT- Verbal and the international fac tor implies that international students will not perform as well in the program. Surprisingly, but similar to other studies [Wright and Palmer, 1997], GMAT-Quantitative is not significant. This result may be a function of the particular curriculum and course offerings. The insignificance of the respective school background indicates that students from the respective school do not have an unfair advantage over other students. The negative correlation between the respective school and Canadians indicates that the graduates are not undergraduate Canadian students who continue on at the respective school. Similarly, business students do not have an advantage over their non- bachelor's background peers. As expected, based upon other studies [Arnold, Chakravarty and Balakrishnan, 1996], the time since bachelor's graduation is insignificant, again reinforcing the concept that qualitative factors have asignificant impact in predicting student success in an MBA program.
With respect to the modeling methodology, as noted by other researchers, regression analysis used to predict MBA QPA will naturally encounter issues due to the skewed distribution of graduate grades [Naik and Ragothaman, 2004]. Neural networks and ANOVA analysis may assist creating more relevant models. However, our intent is not to investigate the other relevant models, but rather to analyze the factors that may be relevant to predicting MBA performance in the One- Year program and acknowledge that other methodologies that perform equally-well, exist.
Based upon these results, the Graduate Admissions process, which includes a minimum formula based solely on GMAT score and undergraduate GPA, should be revised to include other factors. The methodology will be modified to include potential measures of individual motivation or business success [Arnold, Chakravarty, and Balakrishnan, 1996], or work experience [Adams and Hancock, 2000]. Future statistical analysis will analyze these factors and their impact upon predicting MBA success. Also, as the Canadian students enrolled in the One- Year MBA program changes from those educated in the 1 3-year program to the 12-year program, further analysis of the difference in performance of Canadians in these programs can be undertaken.
Adams, A. J. and Hancock, T. (2000). Work experience as Predictor of MBA Performance. College Student Journal, 34, 211-216.
Arnold, L.R., Chakravarty, A.K., and Balakrishnan, N. (1996). Applicant Evaluation in an Executive MBA Program. Journal of Education for Business, 71, 277-283.
Braunstein, A. W. (2002). Factors Determining Success in a Graduate Business Program. College Student Journal, 36, 471-7.
Carver, Jr. M. and King, T. (1994). An empirical investigation of the MBA admission criteria for nontraditional programs. Journal of Education for Business, 69, 95-98.
Deckro, R. and Woudenberg, H. (1973). M.B.A. admission criteria and academic success. Decision Sciences, 8, 765-769.
Ekpenyong, D. (2000). Empirical analysis of the relationship between students' attributes and performance: Case study of the University of Ibadan (Nigeria) MBA Programme. Journal of Financial Management and Analysis, Mumbai, Jul-Dec. 2000, Vol. 13, Issue 2, pp. 54-63.
Everett, J. and Armstrong, R. (1990) Segmenting the MBA Market: An Australian Strategy. Journal of Marketing for Higher Education, Vol. 3(1) 1990, pp. 151-163.
Fisher, J. and Resnick, D. (1990). Standardized testing and graduate business school admission: A review of issues and an analysis of a Baruch College MBA cohort. College and University, 65, 137-148.
Graham, L. (1991). Predicting academic success of students in a master of business administration program. Educational and Psychological Measurement, 51, 721-727.
Hecht, L.W., Manning, W.H. Swinton, S.S., and Braun, H.I. (1989). Assessing older applicants for admission to graduate study in management: The role of GMAT scores and undergraduate grades. Graduate Management Admission Council Occasional Papers. Los Angeles: Author.
McClure, R., Wells, CE., and Bowerman, B.L. (1986). A Model of MBA Student Performance. Research in Higher Education, v25, n2, p 182-93.
Naik, B. and Ragothaman, S. (2004). Using Neural Networks to Predict MBA Student Success. College Student Journal; Mar2004, Vol. 38, Issue 1, pp 143-9.
Paolillo, J. (1982). The predictive validity of selected admissions variables relative to grade point average earned in a Master of Business Administration program. Educational and Psychological Measurement. 42, 1163-1167.
Remus, W. and Wong, C. (1982). An evaluation of five models for the admission decision. College Student Journal, 16,53-59.
Sobol, M. (1984). GPA, GMAT and SCALE: A method for quantification of admissions criteria. Research in Higher Education, 20, 77-88.
Wilson, R. and Hardgrave, B. (1995). Predicting graduate student success in an MBA program: Regression versus classification. Educational and Psychological Measurement, 55, 186-195.
Wright, R. and Palmer, J. (1997). Examining performance predictors for differentially successful MBA students. College Student Journal: Jun 97, Vol. 31 Issue 2, p 276-82.
LYNN A. FISH, PH.D.
Department of Management & Marketing
F. SCOTT WILSON, PH.D.
Department of Economics & Finance
Copyright Project Innovation, Inc. Sep 2007
(c) 2007 College Student Journal. Provided by ProQuest Information and Learning. All rights Reserved.