The Utilization of the ETS
Mathematics Major Field Test at
Pittsburg State University
Tim Flood, Chair
[email protected]
Math Department
Pittsburg State University
Pittsburg, Kansas
History of the ETS Major Field Test at PSU
• We have been administering the paper-
and-pencil based MFT to all of our
seniors in our capstone course each
semester since the spring of 2005
• We have data on 126 students in 15
cohorts ranging in size from 5 to 14
• All of our students (BS and BSE) are
aggregated together
• We do not provided incentive for
performance on the exam
History of the ETS Major Field Test at PSU
• We utilize the results for
– Program assessment and improvement
– Internal and external assessment report
data
– Institutional Program Review data
• To this point, we have only used the
data sparingly to guide change, we
really felt that we needed a large sample
to be meaningful
Academic Areas for the MFT (from ets.org)
• Exams are offered in the following
disciplines [year instituted by ETS]
• And for the following programs
Biology [1989]
Mathematics [1989]
Chemistry [1989]
Music [1989]
Computer Science [1989]
Physics [1989]
Criminal Justice [2001]
Political Science [1989]
Economics [1989]
Psychology [1989]
Literature in English [1989]
Sociology [1989]
Associate Degree in Business [2007]
Bachelor’s Degree in Business [1990]
Master of Business Administration [2002]
Format for the MFT (from ets.org)
• Major Field Tests offers flexible options
– There are no preset test administration
dates
– You can customize the Major Field Tests
content by adding up to 50 locally authored
test questions for tests administered on
campus
– You can choose between paper-and-pencil
and online testing
Format for the MFT (from ets.org)
• The Major Field Test consists of multiple
choice questions ranging from 50 for
Math to 150 each for Biology, Criminal
Justice , and English Literature
• The questions are drawn from the
courses of study most commonly offered
as part of an undergraduate curricula or
the MBA
• Programs can choose when and where
to administer the tests
Format for the MFT (from ets.org)
• The tests are designed to take two hours
and may be split into two sessions
• The tests must be given by a proctor
• Most areas (including math) state that
mathematical operations do not require
the use of a calculator
• There is only one version of each test
and it is revised every 4-5 years
Costs (from ets.org)
Number of Tests Ordered
Price per Each Subject Test
Undergraduate Online Tests
1–99
$25
100 or more (single orders)
$24
Undergraduate Paper-based Tests
1–99
$27
100 or more (single orders)
$26
MBA Tests – Online Tests
1–99
$30
100 or more (single orders)
$25
MBA Paper-based Tests
1–99
$31
100 or more
$26
Pricing for Special Reports (from ets.org)
• ETS offers special data reports through a
subscription service
Special Reports
Duration
Pricing
Premium Report Package
One Year Subscription
$700
Item Information Report
One Year Subscription
$350 (a la carte)
Design Your Own Analysis
One Year Subscription
$150 (a la carte)
Subgroup Report
One Year Subscription
$200 (a la carte)
Custom Comparative Data
Report
One Year Subscription
$300 (a la carte)
Data from the MFT (from ets.org)
• Total Score
– Reported for each student and summarized
• Assessment Indicators
– Average percent of correct answers, in a
particular subject area
– Reported only for groups of students
– A minimum of 5 per group is required
• Subscores
– Reported for each student and summarized
– Subscores are not available for all tests
Data from the MFT (from ets.org)
• Areas with subscores
• Areas without subscores
Math Topics Covered (from ets.org)
• Calculus (~30%)
– Single-variable calculus
– Multivariable calculus
– Separable differential equations
• Algebra (~30%)
– Linear Algebra
– Abstract Algebra
Math Topics Covered (from ets.org)
• Additional Topics (~40%)
– Complex analysis
– Differential equations
– Discrete mathematics (including graph
theory and combinatorics)
– Foundations (including logic, proofs, sets,
functions and relations)
– Geometry
– Point-set topology
– Probability and statistics
Literature in English Topics Covered
• British Literature Pre-1660 (20–25%)
• British Literature 1660–1900 (20–25%)
• American Literature to 1900 (20–25%)
• British and American Literature 1901–
1945 (15–20%)
• Literature in English since 1945 (15–
20%)
• Literary Sources and Influences
(Classical, Biblical, Continental and
Comparative) (5–10%)
Criminal Justice Topics Covered
• The Law (~20%)
• Law Enforcement (~20%)
• Corrections (~20%)
• The Court System in the United States
(~20%)
• Theories of Criminal Behavior (~20%)
Criminal Justice also has two other
overlapping categories
Criminal Justice Topics Covered
• Critical Thinking (overlapping and drawn from
all content areas) requires students to:
– Draw inferences from theories and data
– Recognize unstated assumptions
– Deduce conclusions from information presented
– Interpret and weigh evidence
– Evaluate the strengths of comparative arguments
– Apply knowledge to new problems
– Read and interpret tables of data and graphs
– Recognize the strengths and limitations of both
quantitative and qualitative data
– Compare and contrast subjects and ideas
Criminal Justice Topics Covered
• Research Methodology and Statistics
(overlapping and drawn from all content
areas) requires students to understand:
– Quantitative and qualitative methods
– Research design (including basic and applied
approaches and ethics in research)
– Statistics with application to criminal justice
subject matter
– Sampling
– Statistical software
– Hypothesis testing
Math Assessment Indicators (from ets.org)
• Reported for the group* only
– Calculus (15)
– Algebra (15)
– Routine (27–28)
– Nonroutine (12–13)
– Applied (10)
Numbers in parentheses are the approximate
number of questions in each category
*A minimum of five (5) students is required
for assessment indicators to be reported
Psychology Assessment Indicators
• Reported for the group* only
– Memory and Cognition (19)
– Perception/Sensation/Physiology (22)
– Developmental (15)
– Clinical/Abnormal (15)
– Social (16)
– Measurement and Methodology (29)
Numbers in parentheses are the approximate number
of questions in each category
*A minimum of five (5) students is required for
assessment indicators to be reported
Sociology Assessment Indicators
• Reported for the group* only
– General Theory (21)
– Methodology and Statistics (21)
– Criminology and Deviance (15)
– Social Stratification (15)
– Race, Ethnicity, Gender (15)
– Social Institutions (15)
– Social Psychology (11)
– Gender (15)
– Global (13)
Subscores (from ets.org)
• Some are just an aggregation of Assessment
Indicators as with Physics
Assessment Indicators
Subscores
Classical Mechanics and
Relativity (13)
Introductory Physics (38)
Electromagnetism (12)
Optics and Waves;
Thermodynamics and Statistical
Mechanics (13)
Quantum Mechanics and Atomic
Physics (16)
Advanced Physics (32)
Special Topics (12)
Subscores (from ets.org)
• Others are unrelated as with Political Science
Assessment Indicators
Subscores
Comparative Government and
Politics (22–30)
Analytical and Critical
Thinking (20–26)
International Relations (22–30)
Methodology (7–14)
United States Government and
Politics (48–56)
Political Thought (11–20)
Math Sample Questions (from ets.org)
Math Sample Questions (from ets.org)
Math Sample Questions (from ets.org)
Biology Sample Question (from ets.org)
Business Sample Question (from ets.org)
Music Sample Question (from ets.org)
Individual Student Report
Mathematics
Form Code 4AMF
STUDENT SCORE REPORT
Name :
ISAAC NEWTON
Student Id :
0000001
Test Date :
4/12/1657
Authorization
Number :
0001
Scaled score 143
Overall SEM= 8.8*
The Scale range for the total score is 120-200 and for subscores is 20-100.
Individual Student Report
*SEM is the Standard Error of Measurement that reflects the inherent error that is a part of any testing process. Each
SEM provided is based upon a large sample of test takers. Subtracting the SEM from your score twice to obtain a
minimum and then adding the SEM to your score twice to obtain a maximum will provide a range of scores. If you were
to take any number of tests equivalent to the one you have just completed, your score would fall within this range with a
statistical confidence level of 95%.
For example, if the given SEM for a score was 2.5, and the achieved score was 150, then 150 minus 5 equals 145 and 150
plus 5 equals 155 (where 5 is equal to 2.5 times 2). This means that 95% of the time if you took a test that was equivalent
to the test that you just took you would score within the range of 145 to 155.
To compare your score with the current national comparative data, go to the MFT Comparative Data Guide on the web
at www.ets.org/hea/mft/compare.html.
Copyright © 2011 Educational Testing Service. All rights reserved.
ETS protects the confidentiality of student data.
Assessment Indicator Report
DEPARTMENTAL SUMMARY OF ASSESSMENT INDICATORS
Test: Mathematics
Form Code: 4AMF
Institution: Pittsburg State University
Cohort: Combined
Closed on: Combined
Assessment Indicator Number
Assessment Indicator Title
Mean Percent Correct
1
Calculus
27
2
Algebra
34
3
Routine
30
4
Nonroutine
21
5
Applied
34
Students responding to less than 50% of the questions: 0
Students in frequency distribution: 7
Students tested: 7
How We Track and Report the Data
161.1
154.3
140.0
152.0
140.3
160.8
148.5
149.9
148.7
145.3
147.6
159
152.43
151.25
150.6
162.4
154.1
145.8
125
130
135
140
145
150
155
160
165
n
8
9
5
10
9
4
8
7
10
10
9
8
14
8
119
Date
Apr-05
Oct-05
Apr-06
Dec-06
Dec-07
Apr-08
Dec-08
Apr-09
Nov-09
May-10
Nov-10
May-11
Nov-11
Apr-12
Overall
Overall MFT Score
PSU Math
Mean +1sd
National Mean
Mean -1sd
How We Track and Report the Data
38.4
31.1
24.0
29.0
24.0
40.0
26.0
30.0
32.0
27.0
38.0
30.0
25.0
29.8
41.2
35.0
28.8
0
5
10
15
20
25
30
35
40
45
n
8
9
5
10
9
4
8
7
10
10
9
8
14
8
119
Date
Apr-05
Oct-05
Apr-06
Dec-06
Dec-07
Apr-08
Dec-08
Apr-09
Nov-09
May-10
Nov-10
May-11
Nov-11
Apr-12
Overall
Area 1, Calculus
PSU Math
Mean +1sd
National Mean
Mean -1sd
Thank You!
• Questions
• Contact information
–
Tim Flood
–
Math Department
–
Pittsburg State University
–
[email protected]du