9.4 Assessments and Reports
Much of the power of ALEKS comes from its capacity for accurately
and efficiently assessing the current state of a learner's knowledge.
What is an ALEKS assessment (knowledge check)?
An assessment by the ALEKS system consists of a sequence of
problems posed to the student. The answers are in the form
of mathematical or statistical expressions and constructions produced by the system's
input tools (no multiple choice). The student can answer "I don't know" where necessary.
During an ALEKS assessment, the student is not told whether answers
are correct or incorrect. The assessment is adaptive. Each question
after the first is chosen on the basis of answers previously submitted.
Assessment problems (like practice problems) are algorithmically
generated, with random
numerical values. The length of the assessment is variable, between 15
and 35 questions. There are no time constraints, but some assessments
can take less than a half-hour and a few more than an hour and a half.
Students taking an assessment need to have paper and pencil. The ALEKS calculator
button will become active when use of a calculator is permitted.
No help whatsoever should be given to students taking a knowledge check,
not even rephrasing problems. Outside help can easily lead to false
assessment results and hinder subsequent work in the ALEKS Learning
Students may be assessed when they first register with ALEKS.
It is advisable that all assessments from which the instructor uses
data for grading or a similar purpose take place under the instructor's
supervision. At a minimum, the Initial Assessment should be supervised.
How does the ALEKS assessment work?
In assessing a student's knowledge, the system is in fact determining
which of the feasible knowledge states for that subject correspond
to the student's current knowledge. The assessment is probabilistic,
so it is not fooled by odd careless errors. (Lucky guesses are very
rare, because multiple
choice answers are not used.) Likelihood values (values for the likelihood that
the student is in a particular knowledge state) are spread out over
the states belonging to the structure. With each correct answer, the
likelihood of states containing the item for which a correct answer
was given is raised and that of states not containing the item lowered.
The reverse occurs for incorrect answers or "I don't know." At each
step of the assessment, the system attempts to choose an item for which
it estimates, based on current likelihood values, that the student has
about a fifty-fifty chance of success; such questions are maximally
informative. When the likelihood values of a few states are extremely
high and those of all the rest are extremely low---in technical terms,
when the entropy of the structure is lower than a
certain threshold value---the assessment ends and results are produced.
If a student makes a careless error or lucky guess, this will appear
inconsistent with the general tendency of the student's responses,
and the system will "probe" that area of knowledge until it is sure.
For this reason, inconsistent assessments may require more questions.
How should I interpret the assessment (knowledge check) report?
The results of an ALEKS assessment are shown in the form of a
color-keyed pie chart. A pie chart corresponds to a subject matter
(domain) or to the curriculum of a particular course. Each slice
of the pie corresponds to a general topic. The degree to which the
slice is filled in with solid color shows how close the student is to
mastering that area.
An extremely important aspect of the pie chart is its indication of what
a student is currently most "ready to learn" (that is, the "outer
fringe" of the student's current knowledge state). These items are
listed beneath the pie chart in an Assessment Report and are also given
through the pie chart itself. When the mouse pointer is placed over a
slice of the pie, a list expands out of the pie, showing the concepts
that the student is most "ready to learn" in that part of the
curriculum. Clicking on any of these concepts takes the student into
the Learning Mode.
The pie chart is displayed following assessments, after a concept
has been worked on in the Learning Mode, or when a student clicks on
"MyPie" to change topics. At any given time, a student can only
choose to work on concepts that the student is currently "ready
to learn." This number may vary between two and a few dozen, depending
on what part of the structure is involved.