Hierarchical Linear Modeling

A practical guide to calculating Cohen's f2, a measure of local effect size, from PROC MIXED

by Arielle (Schmidt) Selya

Co-authored with Jennifer Rose, Lisa Dierker, Donald Hedeker, and Robin Mermelstein, and published in Frontiers of Quantitative Psychology and Measurement

Reporting effect sizes in scientific articles is increasingly widespread and encouraged by journals; however, choosing... more Reporting effect sizes in scientific articles is increasingly widespread and encouraged by journals; however, choosing an effect size for analyses such as mixed-effects regression modeling and hierarchical linear modeling can be difficult. One relatively uncommon, but very informative, standardized measure of effect size is Cohen’s f2, which allows an evaluation of local effect size, i.e., one variable’s effect size within the context of a multivariate regression model. Unfortunately, this measure is often not readily accessible from commonly used software for repeated-measures or hierarchical data analysis. In this guide, we illustrate how to extract Cohen’s f2 for two variables within a mixed-effects regression model using PROC MIXED in SAS® software. Two examples of calculating Cohen’s f2 for different research questions are shown, using data from a longitudinal cohort study of smoking development in adolescents. This tutorial is designed to facilitate the calculation and reporting of effect sizes for single variables within mixed-effects multiple regression models, and is relevant for analyses of repeated-measures or hierarchical/multilevel data that are common in experimental psychology, observational research, and clinical or intervention studies.

Success in online high school Biology: factors influencing student academic performance

by Feng Liu

Liu, F., & Cavanaugh, C. (2011).
The Quarterly Review of Distance Education, 12(1)

This paper describes the effect of teacher comments, students’ demographic information and utilization of learning... more This paper describes the effect of teacher comments, students’ demographic information and utilization of learning management system (LMS) on student academic performance in a K-12 virtual learning environment. Students who completed Biology courses in a state virtual school in the Midwestern U.S region during 2007-08 and took the end of course test participated in this study. The hierarchical linear modeling (HLM) technique was employed for data analysis. The results show these factors can influence student academic performance in Biology courses in different ways. The implications for teaching were addressed. Further research is proposed based on the results and limitations.

Ethical Preferences for Influencing Superiors: A 41-Society Study

by Olivier Furrer

David A Ralston, Carolyn P Egri, María Teresa de la Garza Carranza, Prem Ramburuth, Jane Terpstra-Tong, Andre A Pekerti, Ilya Girson, Harald Herrig, Marina Dabic, Moureen Tang, Paulina Wan, Philip Hallinger, Ian Palmer, Detelin S Elenkov, Olivier Furrer, Vojko V Potocan, Florian V Wangenheim, Isabelle Maignan, Pamela L Perrewé, Ana Maria Rossi, Tomasz Lenartowicz, Donna E Ledgerwood, Ruth C May, Mark J Weber, Jorge C Jesuino, Ping Ping Fu, Irina Naoumova, Tania Casado, Liesl Riddle, Malika Richards, Arif N Butt, Wade M Danis, Francisco B Castro, Jaime Ruiz-Gutiérrez, Laurie P Milton, Mahfooz A Ansari, David M Brock, Narasimhan Srinivasan, Arunas Starkus, Tevfik Dalgic, Fidel León-Darder, Hung Vu Thanh, Yong-lin Moon, Ho Beng Chia, Min-Hsun Christine Kuo, Mario Molteni, Maria Kangasniemi, Kamel Mellahi, & Alan Wallace
Journal of International Business Studies
Vol. 40, 2009, pp. 1022–1045. doi:10.1057/jibs.2008.109

With a 41-society sample of 9990 managers and professionals, we used hierarchical linear modeling to investigate the... more With a 41-society sample of 9990 managers and professionals, we used hierarchical linear modeling to investigate the impact of both macro-level and micro-level predictors on subordinate influence ethics. While we found that both macro-level and micro-level predictors contributed to the model definition, we also found global agreement for a subordinate influence ethics hierarchy. Thus our findings provide evidence that developing a global model of subordinate ethics is possible, and should be based upon multiple criteria and multilevel variables.

Effectiveness of a supplemental early reading intervention scaled up in multiple schools

by Kim Nimon

Hierarchical linear modeling applications in the context of giftedness research.

by Kim Nimon

A national evaluation of the impact of the secondary Social and Emotional Aspects of Learning (SEAL) programme

by Michael Wigelsworth

Educational Psychology

How Do Habit and Prior Outcomes Affect Online Gambling Use? A Multilevel Dynamic Panel Data Analysis

by Xiao (Sean) Ma

Invited talk at informs 2010 annual meeting

A Cross-National Comparison of Intra-Class Correlation Coefficient in Educational Achievement Outcomes

by Cengiz Zopluoglu

this paper is in submission

The plausible range of intra-class correlation coefficient (ICC) is essential for both a priori sample size... more The plausible range of intra-class correlation coefficient (ICC) is essential for both a priori sample size calculations in planning cluster-randomized trials and statistical adjustments of misaligned analysis of clustered data in meta-analytic studies. Recent efforts to create databases for ICC in educational achievement outcomes are based on the studies published only in the US and Europe. The current study aims to extend the existing information for the plausible range of ICC values in educational achievement outcomes to a global scale by examining the distributional characteristics of two-level unconditional ICC estimates across countries participating in two international studies, TIMSS and PIRLS. The findings suggest large variability in the unconditional ICC estimates across countries, and current standards do not apply to every country. Researchers should look for country-specific ICC estimates in planning cluster-randomized trials and in comparing studies across countries.

Social and Emotional Aspects of Learning (SEAL) programme in secondary schools: National evaluation

by Michael Wigelsworth

WHite Enrollment in Nonpublic Schools, Public School Racial Composition, and Student Performance

by Carl L Bankston

Modeling Wetland Blackbird Populations as a Function of Waterfowl Abundance in the Prairie Pothole Region of the United States and Canada.

by Wayne Thogmartin

Forcey, G. M., G. M. Linz, W. E. Thogmartin, and W. J. Bleier. 2008. Modeling wetland blackbird populations as a function of waterfowl abundance in the Prairie Pothole region of the United States and Canada. Environmental Bioindicators 3:124–135.

Blackbirds share wetland habitat with many waterfowl species in Bird Conservation Region 11 (BCR 11), the prairie... more Blackbirds share wetland habitat with many waterfowl species in Bird Conservation Region 11 (BCR 11), the prairie potholes. Because of similar habitat preferences, there may be associations between blackbird populations and populations of one or more species of waterfowl in BCR11. This study models populations of red-winged blackbirds and yellow-headed blackbirds as a function of multiple waterfowl species using data from the North American Breeding Bird Survey within BCR11. For each blackbird species, we created a global model with blackbird abundance modeled as a function of 11 waterfowl species; nuisance effects (year, route, and observer) also were included in the model. Hierarchical Poisson regression models were fit using Markov chain Monte Carlo methods in WinBUGS 1.4.1. Waterfowl abundances were weakly associated with blackbird numbers, and no single waterfowl species showed a strong correlation with any blackbird species. These findings suggest waterfowl abundance from a single species is not likely a good bioindicator of blackbird abundance; however, a global model provided good fit for predicting red-winged blackbird abundance. Increased model complexity may be required for accurate predictions of blackbird abundance; the amount of data required to construct appropriate modelsmay limit this approach for predicting blackbird abundance in the prairie potholes.

The problem of serial order in behavior: Lashley's legacy

by Rajal G. Cohen

Rosenbaum, D. A., Cohen, R. G., Jax, S. A., Van Der Wel, R., & Weiss, D. J. (2007). Human Movement Science, 26, 525–554.

In a prescient paper Karl Lashley (1951) rejected reflex chaining accounts of the sequencing of behavior and argued... more In a prescient paper Karl Lashley (1951) rejected reflex chaining accounts of the sequencing of behavior and argued instead for a more cognitive account in which behavioral sequences are typically controlled with central plans. An important feature of such plans, according to Lashley, is that they are hierarchical. Lashley offered several sources of evidence for the hierarchical organization for behavioral plans, and others afterward provided more evidence for this hypothesis. We briefly review that evidence here and then shift from a focus on the structure of plans (Lashley’s point of concentration) to the processes by which plans are formed in real time. Two principles emerge from the studies we review. One is that plans are not formed from scratch for each successive movement sequence but instead are formed by making whatever changes are needed to distinguish the movement sequence to be performed next from the movement sequence that has just been performed. This
plan-modification view is supported by two phenomena discovered in our laboratory: the parameter remapping effect, and the handpath priming effect. The other principle we review is that even single movements appear to be controlled with hierarchically organized plans. At the top level are the starting and goal postures. At the lower level are the intermediate states comprising the transition from the starting posture to the goal posture. The latter principle is supported by another phenomenon discovered in our lab, the end-state comfort effect, and by a computational model of motor planning which accounts for a large number of motor phenomena. Interestingly, the computational model hearkens back to a classical method of generating cartoon animations that relies on the production of keyframes first and the production of interframes (intermediate frames) second.

International cooperation in psychology is on the rise.

by Reinhold Kliegl

Kliegl, R., & Bates, D.M. (2011). International cooperation in psychology is on the rise. Scientometrics, 87, 149-158.
DOI 10.1007/s11192-010-0299-0

Data and R script at Potsdam Mind Research Repository:
( http://read.psych.uni-potsdam.de/pmr2 )
Filed under "Other Experiments and Analyses"

There has been a substantial increase in the percentage for publications with co-authors located in departments from... more There has been a substantial increase in the percentage for publications with co-authors located in departments from different countries in 12 major journals of psychology. The results are evidence for a remarkable internationalization of psychological research, starting in the mid 1970s and increasing in rate at the beginning of the 1990s. This growth occurs against a constant number of articles with authors from the same country; it is not due to a concomitant increase in the number of co-authors per article. Thus, international
collaboration in psychology is obviously on the rise.

Maternal Adaptation to Pediatric Neurosurgical Diagnosis: A Growth Curve Analysis

by Sigal Tifferet

OBJECTIVE: To describe trajectories of change in maternal adaptation to chronic pediatric neurosurgical diagnosis and... more OBJECTIVE: To describe trajectories of change in maternal adaptation to chronic pediatric neurosurgical diagnosis and to identify variables predicting the level and rate of adaptation.

METHODS: One hundred and thirty seven mothers of children diagnosed with neurosurgical illness participated. Mothers reported socio-demographic variables, neuroticism, optimism, spouse support, mental health, and personal growth. The coordinating nurse assessed illness variables. Data were collected from diagnosis to 32+months post-diagnosis.

RESULTS: Results showed poor maternal mental health at diagnosis, followed by a gradual improvement over the next 18 months, and then a slight regression. Optimism, spouse support and religiosity were identified as resistance factors, while neuroticism, treatment needs, maternal age, and maternal education were identified as risk factors. The rate of change in mental health was moderated by neuroticism.

CONCLUSIONS: Maternal adaptation to pediatric illness changes over time. The trajectories of adaptation vary in relation to personal, social and medical variables.

Modeling Levels and Time in Entrepreneurship Research: An Illustration With Growth Strategies and Post-IPO Performance

by Tim R. Holcomb

Published in 'Organizational Research Methods', 2010

New ventures lack resources, are buffeted by environmental factors, and often experience rapid growth and... more New ventures lack resources, are buffeted by environmental factors, and often experience rapid growth and organizational transformations that can have profound effects on performance and survival. This indicates that factors at multiple levels and across time affect new venture outcomes. Research examining these outcomes often address relationships that cross levels or time, but rarely both. Because scholars potentially can make rich theoretical contributions by simultaneously investigating temporal relationships that cross levels, the authors illustrate multiyear, multilevel model building with random coefficient modeling (RCM) using language that is accessible to entrepreneurship scholars. Specifically, they model the effects of strategic growth actions on new venture performance using a longitudinal data set of young, IPO-stage firms. Their illustration demonstrates the statistical advantages of modeling levels and time simultaneously and offers a roadmap for entrepreneurship scholars interested in examining these effects, including a step-by-step guide with SAS code for working with these data. They also describe some specific research questions to help advance theory development using RCM.

Multi-Campus Studies of College Impact: Which Statistical Method is Appropriate?

by Nida Denson

Astin, A. W., & Denson, N. (2009). Multi-Campus Studies of College Impact: Which Statistical Method is Appropriate? Research in Higher Education, 50, 354-367.

In most multi-campus studies of college impact that have been conducted over the past four decades, investigators have... more In most multi-campus studies of college impact that have been conducted over the past four decades, investigators have relied on ordinary least squares (OLS) regression as the analytic method of choice. Recently, however, some investigators have advocated the use of Hierarchical Linear Modeling (HLM), a method specifically designed for analyses that involve both individual (student) and aggregate (institutional) level measures.
Cross-validation analyses using a national database show that the two methods yield an equally good ‘‘fit’’ with empirical data. Existing OLS software has the advantage of enabling one to perform path analytical causal modeling; HLM has the advantage of yielding a more conservative estimate of the significance of institution-level effects.

Land use and climate influences on waterbirds in the Prairie Potholes

by Wayne Thogmartin

Forcey, G. M., W. E. Thogmartin, G. M. Linz, W. J. Bleier, and P. C. McKann.  2011.  Land use and climate influences on waterbirds in the Prairie Potholes.  Journal of Biogeography 38:1694–1707.

Aim We examined the influences of regional climate and land-use variables on mallard (Anas platyrhynchos), blue-winged... more Aim We examined the influences of regional climate and land-use variables on mallard (Anas platyrhynchos), blue-winged teal (Anas discors), ruddy duck (Oxyura jamaicensis) and pied-billed grebe (Podilymbus podiceps) abundances to inform conservation planning in the Prairie Pothole Region of the United States.

Location The US portion of Bird Conservation Region 11 (US-BCR11, the Prairie Potholes), which encompasses six states within the United States: Montana, North Dakota, South Dakota, Nebraska, Minnesota and Iowa.

Methods We used data from the North American Breeding Bird Survey (NABBS), the National Land Cover Data Set, and the National Climatic Data Center to model the effects of environmental variables on waterbird abundance. We evaluated land-use covariates at three logarithmically related spatial scales (1000, 10,000 and 100,000 ha), and constructed hierarchical spatial count models a priori using information from published habitat associations. Model fitting was performed using a hierarchical modelling approach within a Bayesian framework.

Results Models with the same variables expressed at different scales were often in the best model subset, indicating that the influence of spatial scale was small. Both land-use and climate variables contributed strongly to predicting waterbird
abundance in US-BCR11. The strongest positive influences on waterbird abundance were the percentage of wetland area across all three spatial scales, herbaceous vegetation and precipitation variables. Other variables that we included in our models did not appear to influence waterbirds in this study.

Main conclusions Understanding the relationships of waterbird abundance to climate and land use may allow us to make predictions of future distribution and abundance as environmental factors change. Additionally, results from this study can suggest locations where conservation and management efforts should befocused.

Hierarchical Bayesian estimation of the population viability of an epixylic moss. Journal of Ecology.

by Alejandro Ruete

Co-authored with Wiklund K & Snäll T.
Journal of Ecology 2012

SUMMARY
1. Understanding the variation in population abundances requires accounting for the environmental... more SUMMARY
1. Understanding the variation in population abundances requires accounting for the environmental variability and uncertainty on different scales. We developed and evaluated a Bayesian hierarchical model for the interannual variation in population abundance of the epixylic bryophyte Buxbaumia viridis. The model accounts for spatio-temporal variability on two spatial scales. We used data on population abundance and on the weather variables at regional level collected 1996-2003, and data on dead wood amount collected 1996 and 2008. We also provide a Bayesian estimate of the population viability, specifically the population stochastic growth rate (log λS), which accounts for natural variability and uncertainty.
2. Previous estimates of population viability did not account for uncertainties in a satisfactory way. First, point estimates of log λS cannot, by definition, express variation. Second, the commonly used approach to estimate log λS and its confidence interval underestimates uncertainties. The approach aims to estimate the mean of log λS, with the confidence interval representing the uncertainty in the estimate of this mean. The interval does not reflect the natural variation and uncertainty.
3. We estimated a probability distribution of log λS, where the probability distributions of the year-specific growth rates (log λy) are accounted for. The species is likely to decline under current environmental conditions. Based on the probability distribution of log λS, we estimated this risk to be 81%.
4. We found support for the hypotheses that the population dynamics are driven by autumn frosts, by spring precipitation and temperature (regional variables), and by the preceding year’s population abundance (local variable).
5. Synthesis. Statements about the viability of populations should not be based on point estimates of log λS. Instead, the full probability distribution of log λS should be used, which explicitly accounts for the hierarchically structured natural variability and uncertainty. This distribution allows estimating the risk for a population decline, or providing an estimate of the confidence in a statement about a decline. This quantitative information can be weighed against other interests. We expect this Bayesian approach to be especially useful in the viability analysis of natural populations experiencing environmental variability.

Meta-Analysis in Higher Education: An Illustrative Example Using Hierarchical Linear Modeling

by Nida Denson

Denson, N. & Seltzer, M. (2011). Meta-Analysis in Higher Education: An Illustrative Example Using Hierarchical Linear Modeling. Research in Higher Education, 52, 215-244.

The purpose of this article is to provide higher education researchers with an illustrative example of meta-analysis... more The purpose of this article is to provide higher education researchers with an illustrative example of meta-analysis utilizing hierarchical linear modeling (HLM). This article demonstrates the step-by-step process of meta-analysis using a recently-published
study examining the effects of curricular and co-curricular diversity activities on racial bias in college students as an example (Denson, Rev Educ Res 79:805–838, 2009). The authors
present an overview of the meta-analytic approach and describe a meta-analysis from beginning to end. The example includes: problem specification; research questions; study retrieval and selection; coding procedure; calculating effect sizes; visual displays and summary statistics; conducting HLM analyses; and sensitivity analyses. The authors also offer guidelines and recommendations for improving the conduct and reporting of research which in turn can provide the information necessary for future and more comprehensive meta-analytic reviews.

Attachment orientations and depression: A longitudinal study of new parents.

by Jamie Kohn

Rholes, W. S., Simpson, J. A., Kohn, J. L., Wilson, C. L., Martin, A. M., III, Tran, S., & Kashy, D. A. Journal of Personality and Social Psychology, 100(4), Apr 2011, 567-586. doi: 10.1037/a0022802

In this longitudinal study, we followed a large sample of first-time parents (both partners) across the first 2 years... more In this longitudinal study, we followed a large sample of first-time parents (both partners) across the first 2 years of the transition to parenthood. Guided by attachment theory (Bowlby, 1969), we tested several predictions about how attachment anxiety and avoidance are related to the incidence, maintenance, increase, and decline of depressive symptoms in both sexes across the first 2 years of the transition. We found that (a) the association between attachment anxiety and depressive symptoms was moderated by factors related to the marital and/or romantic relationship; (b) the association between avoidance and depressive symptoms was moderated by factors related to family responsibilities; (c) styles of caregiving provided by romantic partners affected depressive symptoms differently among anxious and avoidant persons; and (d) in certain predictable situations, depressive symptoms persisted at higher levels or increased to higher levels in anxious or avoidant persons across the 2-year transition period. Important implications of these results are discussed.