19 Jun NDUS-IR Empowers Data-Driven Decision-Making: A Nursing Program Example
Institutional researchers do some of their best work when they collaborate with stakeholders who are committed to data-driven decision-making. Such collaborations may be initiated when someone realizes they need more information or evidence before making an important decision. This was precisely the case when Julie Traynor, the award-winning Dakota Nursing Program Director, reached out to our office for some help in determining how well scores on the TEAS (Test of Essential Academic Skills) predict nursing students’ success.
When prospective students apply to the multi-campus Dakota Nursing Program, TEAS scores can be used in admissions decisions. It is helpful to know an applicant’s likelihood of success before accepting or rejecting the applicant, and TEAS scores are considered predictive of nursing student success. Julie Traynor realized, however, that it was not clear what might be an ideal “cut score” where students below the score are rejected and students above the score are accepted. She figured (correctly) that we could provide the kind of evidence needed to support such a determination.
Ms. Traynor sent us detailed data on a few cohorts of Dakota Nursing Program students from around the state. The data set contained demographic variables, various outcome variables (such as Practical Nursing Certificate and/or Associate in Applied Science completion), and various test scores. The analysis was assigned to me, and I dived in by preparing the data for analysis and deciding that using binary logistic regression (a statistical method) was an ideal approach to the analysis.
I found that the power of TEAS scores in predicting Dakota Nursing Program outcomes is robust. In my analyses, when I controlled for demographics known to influence academic outcomes as well as existing GPAs, doing so did not weaken the predictive power TEAS scores. There was also only weak (and inconsistent) evidence that the predictive power of TEAS score varies across campuses. Thus, a given TEAS score mostly means the same thing at each campus. Altogether, this indicates that the TEAS is indeed a good measure to use.
Here is the potency of TEAS scores’ predictive power across outcomes:
- TEAS score is a good predictor of completion of the Practical Nursing Certificate. It is also a statistically reliable (and decent) predictor of completion of the program’s Associate in Applied Science. See Figure 1.
- TEAS score is a weak predictor of passing the Practical Nursing National Council Licensure Examination on the first try.
- TEAS score is a somewhat weak (but scientifically acceptable) predictor of taking the Registered Nursing National Council Licensure Examination. It is an even better predictor of passing that exam, especially on the first try. See Figure 2.
The students in the sample had TEAS scores ranging from 41 to 94, and the admissions cut scores under consideration roughly range from 40 to 60. The following table demonstrates the probability of success at various potential cut scores:
From here, Ms. Traynor and her colleagues can choose a cut score based upon the minimum likelihood of success that they desire. For instance, if they want at least two out of every three students (roughly 67%) to complete the Practical Nursing Certificate, a good cut score for the TEAS would be 52 or 53.
An institutional researcher’s role is not to decide what decision will be made, but rather to empower stakeholders with high-quality evidence so that they can make an informed decision. The Dakota Nursing Program is now empowered to consider these findings and choose a cut score that best fits with their priorities.
============================================================================
 Dr. Ellie Shockley is an Educational Data Warehouse Specialist on the NDUS institutional researcher team. In this capacity, she also works closely with the Department of Public Instruction (PK-12) and the Information Technology Department. She often responds to education data requests that come from state agencies or from outside of state government. Her work ranges from pulling raw data and sharing it according to our best practices, to conducting complex statistical analyses in order to answer research questions, to assisting with inter-agency collaborations related to education data, and more. |