Neal Martin Kingston, Ph.D.
School of Education - Educational Psychology, Center for Educ Testing and EvaluationRoom 605A
Neal Kingston, Ph.D., came to the University of Kansas in 2006 and is a University Distinguished Professor in the Research, Evaluation, Measurement, and Statistics track of the Educational Psychology and Research Program and Director of the Achievement and Assessment Institute. His research focuses on large-scale assessment, with particular emphasis on how it can better support student learning through the use of learning maps and diagnostic classification models. He has been the principal investigator or co-principal investigator for over 200 research projects, including two of his current projects, Enhanced Learning Maps (https://enhancedlearningmaps.org/) and Project Lead The Way.
Dr. Kingston is known internationally for his work on large-scale assessment, formative assessment, and learning maps. He has served as a consultant or advisor for organizations such as the AT&T, Department of Defense Council on Military Personnel Testing, Edvantia, General Equivalency Diploma (GED), Kaplan, King Fahd University of Petroleum and Minerals, Merrill Lynch, National Council on Disability, Qeyas (Saudi Arabian National Center for Assessment in Higher Education), State of New Hampshire, State of Utah, and the U.S. Department of Education.
A hallmark of Dr. Kingston’s career has been a focus on research management. His first such responsibilities were as Director of Selection Research for the Los Angeles County Department of Personnel where he and his staff were responsible for job analysis, construction of new testing material, revision of old testing material, review of appropriateness of old testing material for specific job classes, content validation, criterion related validation, construct validation, adverse impact analysis, setting cut scores, item and test bias studies, employee retention studies, and development and utilization of new technology (microcomputers, minicomputers, intelligent terminals, scanners, etc.).
As Director of Research and Test Development for the Graduate Records Examination (GRE) Program, Dr. Kingston was responsible for all administrative aspects of GRE research. He worked with the GRE Board and its Research Committee to develop and help execute long-term research plans. Among numerous committee and task force memberships at ETS, served on the Research Planning Committee on Technology and Testing and the Strategic Planning Committee on Alternative Test Delivery.
As Director of Research and New Testing Initiatives for Educational Testing Services’ Graduate Records Examinations (GRE) Program, Dr. Kingston designed and administered an extensive research program that led to the development of models for future versions of the GRE including computerized adaptive testing and addition of a writing measure. He supported the GRE Board Research Committee to allow it to efficiently guide GRE Program research and development, and proper use of GRE tests.
As Associate Commissioner, Office of Curriculum, Assessment, and Accountability, Kentucky Department of Education, Dr. Kingston developed a research agenda to support Kentucky’s assessment and curriculum goals. He chaired a review group of nationally prominent researchers from within and outside the state.
At Measured Progress, Dr. Kingston served as Senior Vice President and Chief Operating Officer. During this time he systematized and named what is now known as the Body of Work standard setting method and was among the first to implement cross-grade performance standards articulation.
Since 2009, as Director of the Center of Educational Testing and Evaluation and now Director of the Achievement and Assessment Institute, Dr. Kingston has served as principal investigator or co-principal investigator for more than 120 grants totaling about $200 million. Of particular note was the Dynamic Learning Maps Alternate Assessment, which was the largest grant in KU history and which currently serves 19 state departments of education. Other testing projects include the Kansas Assessment Program, Career Pathways Collaborative, and Adaptive Reading Motivation Measures. In his capacity as Director of the Achievement and Assessment Institute Dr. Kingston is responsible for five research centers with a staff of about 260 year-round staff and about 140 temporary employees.
Education
M.Phil., Educational Measurement, Teachers College, Columbia University
Ph.D., Educational Measurement, Teachers College, Columbia University
M.Ed., Educational Measurement, Teachers College, Columbia University
M.A., Psychology in Education, Teachers College, Columbia University
B.A., Liberal Studies (concentrations in Biology & Education), State University of New York
Teaching
Starting in 2019, Dr. Kingston's teaching will focus on advanced test theory and practice using an integrated framework that covers classical and modern approaches.
Prospective Graduate Students:
I am seeking one or two new doctoral students (previous masters degree not required) to start in fall 2019. Students will receive a graduate research assistant position that covers their tuition and fees and provides a salary (https://aai.ku.edu/sites/aai.ku.edu/files/docs/pdfs_general/GRA_Handbook...). Only candidates with very strong verbal and quantitative reasoning and communication skills will be considered. An interest in learning and applying test theory to better support student learning is of critical importance.
Teaching Interests
- Educational Measurement theory and practice
- Instructionally embedded assessment
- Learning maps
- Meta-analysis
Research
The field of education moves slowly. Theoretical improvements often take 30-50 years before wide-spread implementation. Often research-based practices are crowded out by the fad of the day. Making the challenge of improving education even greater, sub-disciplines within education far too often work in isolation. Simple solutions that do not address the complexity of individual students or the dynamics of a classroom at best have little impact and too often have a negative impact. Such has been the case in large-scale assessment where the use of assessment to drive curriculum and instruction has had numerous negative consequences.
Coming to the University of Kansas gave me the opportunity to consider the fundamental issues of education from a broader perspective. As have many others, I had long realized that thinking about curriculum, instruction, and assessment needs to be integrated. However, few researchers attempted to develop models or theories to do this. I was impressed by the efforts of some, particularly the research trajectories of Susan Embretson and Kikumi Tatsuoka, but I remained frustrated regarding how incomplete this work was and how little impact it was having on federally mandated state assessment programs. This led me to develop three conference presentations in 2009 that served to focus my thinking. The first was presented at the National Council on Measurement in Education and was entitled, “What Have We Learned about the Structure of Learning from 30 Years of Research on Integrated Cognitive-Psychometric Models? Not Much.” The second was presented at the American Educational Research Association Conference – The Efficacy of Formative Assessment: A Meta-Analysis. The third, presented at the National Conference on Student Assessment, was entitled, “Large-Scale Formative Assessment: Panacea, Transitional Tool, or Oxymoron.”
In 2010 an opportunity presented itself and allowed me to solidify my thinking. The US Department of Education issued a request for proposals to develop a large-scale assessment system for students with significant cognitive disabilities – the approximately one percent of students with the greatest learning challenges. It was clear to me that such an assessment system needed to do far more than measure learning – it needed to facilitate learning. I identified six features that needed to be present to do this. They are as follows.
1. Comprehensive fine-grained learning maps that guide instruction and assessment
2. A subset of particularly important nodes that serve as content standards to provide an organizational structure for teachers
3. Instructionally embedded assessments that reinforce the primacy of instruction
4. Instructionally relevant testlets that model good instruction and reinforce learning
5. Accessibility by design
6. Status and growth reporting that is readily actionable
No one had ever tried to develop a learning environment in this way. Comprehensive fine-grained learning maps did not exist. The concept of instructionally relevant assessment previously was unnamed and in its infancy. Clearly much research – both basic and applied – was necessary and this has become the focus of my research.
A related secondary research foci has remained important to me and is worthy of note. Test development and universal design have close ties to features 3-5 in the list above. I separate it as a research focus because it is also applicable to traditional testing programs.
Research Interests
- Large-scale assessment
- Computer-based testing
- Diagnostic classification modeling
- Learning maps
- Test development
- Score reporting
