9.4 Assessments and Reports

Much of the power of ALEKS comes from its capacity for accurately and efficiently assessing the current state of a learner's knowledge.

What is an ALEKS assessment (knowledge check)?

[Chapter 4] An assessment by the ALEKS system consists of a sequence of problems posed to the student. The answers are in the form of mathematical or statistical expressions and constructions produced by the system's input tools (no multiple choice). The student can answer "I don't know" where necessary. During an ALEKS assessment, the student is not told whether answers are correct or incorrect. The assessment is adaptive. Each question after the first is chosen on the basis of answers previously submitted. Assessment problems (like practice problems) are algorithmically generated, with random numerical values. The length of the assessment is variable, between 15 and 35 questions. There are no time constraints, but some assessments can take less than a half-hour and a few more than an hour and a half. Students taking an assessment need to have paper and pencil. The ALEKS calculator button will become active when use of a calculator is permitted.

No help whatsoever should be given to students taking a knowledge check, not even rephrasing problems. Outside help can easily lead to false assessment results and hinder subsequent work in the ALEKS Learning Mode.

Students may be assessed when they first register with ALEKS. It is advisable that all assessments from which the instructor uses data for grading or a similar purpose take place under the instructor's supervision. At a minimum, the Initial Assessment should be supervised.

How does the ALEKS assessment work?

[Sec. 8.2.5] In assessing a student's knowledge, the system is in fact determining which of the feasible knowledge states for that subject correspond to the student's current knowledge. The assessment is probabilistic, so it is not fooled by odd careless errors. (Lucky guesses are very rare, because multiple choice answers are not used.) Likelihood values (values for the likelihood that the student is in a particular knowledge state) are spread out over the states belonging to the structure. With each correct answer, the likelihood of states containing the item for which a correct answer was given is raised and that of states not containing the item lowered. The reverse occurs for incorrect answers or "I don't know." At each step of the assessment, the system attempts to choose an item for which it estimates, based on current likelihood values, that the student has about a fifty-fifty chance of success; such questions are maximally informative. When the likelihood values of a few states are extremely high and those of all the rest are extremely low---in technical terms, when the entropy of the structure is lower than a certain threshold value---the assessment ends and results are produced.

If a student makes a careless error or lucky guess, this will appear inconsistent with the general tendency of the student's responses, and the system will "probe" that area of knowledge until it is sure. For this reason, inconsistent assessments may require more questions.

How should I interpret the assessment (knowledge check) report?

[Sec. 4.12] The results of an ALEKS assessment are shown in the form of a color-keyed pie chart. A pie chart corresponds to a subject matter (domain) or to the curriculum of a particular course. Each slice of the pie corresponds to a general topic. The degree to which the slice is filled in with solid color shows how close the student is to mastering that area.

An extremely important aspect of the pie chart is its indication of what a student is currently most "ready to learn" (that is, the "outer fringe" of the student's current knowledge state). These items are listed beneath the pie chart in an Assessment Report and are also given through the pie chart itself. When the mouse pointer is placed over a slice of the pie, a list expands out of the pie, showing the concepts that the student is most "ready to learn" in that part of the curriculum. Clicking on any of these concepts takes the student into the Learning Mode.

The pie chart is displayed following assessments, after a concept has been worked on in the Learning Mode, or when a student clicks on "MyPie" to change topics. At any given time, a student can only choose to work on concepts that the student is currently "ready to learn." This number may vary between two and a few dozen, depending on what part of the structure is involved.