tablet-based math assessment: what can we learn from math apps? - best tablet for graphic design

Introduction of tablet as a wide era-
With the release of Apple's iPad, consumer devices have spread to 2010.
In 3 years, tablet shipments have exceeded desktop or laptop shipments and exceeded the sum of all PCs in 2013 (IDC, 2013).
Demand for tablets in education is stronger (
Interactive Education System Design Co. , Ltd. , 2013).
In the field of educational assessment, tablet devices have been recognized as an alternative test delivery platform and various validity issues have been studied (
See, for example, Laurin Davis, Strain-
Seymour, and gay, 2013; Strain-
Seymour, craft, Davis, and Elbom, 2013).
However, how to design educational assessments specifically for tablets remains largely an unknown area.
However, change is imminent.
Decision Support touch for well-known test consortia such as PARCC and Smarter Balanced
Screen devices for tablets
Push basic assessment into the spotlight (
SeeSmarter balance assessment alliance, 2012).
High-risk testing is not the only driving force when educators, students and parents order tablets (See Blume 2013);
Instead, they are attracted to the many possibilities that tablets have, including the idea that thattablets will personalize learning and improve student learning outcomes.
The key to achieving this is to track students' learning and provide timely feedback through formative assessment. In sum, tablet-
Based on evaluation-
Summative and formative-
Plan to launch soon. Are we ready?
The answer depends on what people expect.
If the question is whether we are ready for tablet delivery of assessments designed specifically for paper and pencils or PCs, answers and debates can be found in various technical analysis and usability studies (e. g. , Isabwe,2012;
Covalski & Gardner, 2009;
Kim, Lim, Choi, Hahn, 2012;
Toby, MA, Lai, Lin, jacwa, 2012;
Laughlin Davis and others. , 2013;
Cho ei, Cho & Chan. , 2013; Strain-Seymour et al. , 2013).
There seems to be a consensus that the technology can work.
But the goal is tablets.
According to the assessment, like many apps that students already enjoy on tablets, we still have a long way to go.
This report is part of a research project under cognitive-based learning assessment (CBAL)
ETS's initiative, designed for innovative K-
Using 12 evaluation systems for learning science and other related studies: recording student achievement (of learning);
Help determine how to plan (for learning);
As a valuable educational experience
Learn, see Bennett, 2010).
The survey was inspired by the work of the first author to develop atablet
The basic evaluation prototype of primary school mathematics, that is, students' understanding of scores and decimals.
When we started the project, we found that most published studies on the use of tabletsin's mathematical assessment were either narrowly focused on specific features of tablet technology, such as the availability of digital ink. e. g.
, Ren & Moriya, 2000;
McKnight and ferton, 2010; Kim et al. , 2012)
Or, more broadly, about the relationship between the use of classroom tablets and terminalsof-
Student learning outcomes for the semester (
Hieb & Ralston, 2010; Isabwe, 2012; Kowalski et al. , 2009).
Given the apparent lack of systematic review of how to design a math assessment for a tablet, we decided to start with a survey of the math education application, hoping to understand their design principles and techniques.
We also hope to avoid potential pitfalls in our work.
This report is a summary of this qualitative study.
In the following, we will outline the dimensions of mathematical content and interaction that serve as the basis for the review.
We then outline the methodology we use to sample applications and evaluate criteria.
In the results section, we summarize the main results of our survey.
The discussion section focuses on lessons learned and key lessons learned from the survey
Key points of mathematics evaluation and teaching application design (i. e.
How do we apply these technologies on tablets?
Design based on evaluation).
The dimension of mathematics review we focus on the four dimensions of review :(1)
The quality of mathematical content ,(2)
Feedback and scaffolding ,(3)
Rich interaction, and (4)
Adaptability of application.
These four areas have been cultivated from previous research on mathematical digital tools (e. g.
Digital tools for educational standards in algebra;
Bokhove & Drijvers, 2011)
Judging from the evaluation of mobile applications in algebra (Harrison, 2013)
Design principles from Learning Objects (e. g.
Evaluation Indicators of learning objects;
Kai and narak, 2008)
And the quality of mathematics teaching (e. g. , Hill et al. ,2008).
These dimensions are described below.
Mathematics content we specialize in the two main dimensions of mathematics content, which are derived from the quality of mathematics in teaching (MQI; seeHill et al. , 2008)
: The accuracy of mathematics and the richness of mathematics.
The mathematical precision is adjusted according to The MQI size of "error" and "inexact. " Moyer-
Packenham, Salkind, Bolyard (2008)
Define mathematical accuracy as "the degree to which a mathematical object is faithful to the basic mathematical properties of the object in a virtual environment "(p. 204).
In this case, we are looking at whether the mathematical content in the application conforms to its pure mathematical form, if not, where is the inaccuracy.
The richness of mathematics consists of two elements: focusing on the mathematical facts and the meaning of the program, and the contact with the mathematical practice (Hill et al. , 2008).
A wealth of mathematical experience captures the meaning of mathematical practice and provides links, explanations, and generalization.
A basic condition to support students in learning and improving their chances of success is to provide feedback and scaffolding (
Gibbs Simpson, 2004;
Hattie & Timperley 2007
Nicole and McFarlaneDick 2006).
We pay special attention to three dimensions: feedback, scaffolding, and opportunities for reflection.
Feedback dimensions from digital tools for algebra Education Standards (
Bochove & Drijvers (2010)
, Including whether to give any feedback, whether feedback is related to input and content, timeliness of feedback, type of feedback (e. g.
, Concept, procedure, correction).
Scaffolding and opportunities for reflective dimensions come from the learning object evaluation instruction (
Howie and muyhead, 2005)
"The opportunity to extend the learning activity beyond the scope of the object itself ".
Scaffolding dimensions include whether the application provides any scaffolding that helps students learn, and, if provided, the form of scaffolding (e. g.
, Hintsand guidance questions).
Opportunities for reflective dimensions focus on whether students have the opportunity to think or explain their thinking processes.
This reflection may occur in the form of oral or typing explanations, transfer to other questions or answering questions related to the process of resolution.
Rich interaction between students and applications meaningful interaction not only helps students learn outcomes (
Chan & Black (2006)
It will also affect the quality and interpretation of students' behavior.
We focus on two elements of interaction derived from evaluation criteria in algebraic education digital tools (
Bokhove & Drijvers, 2010)
: The Interaction Mode and project type provided.
The interactive model may include training, hands-on testing, actual testing, and others (
Bochove & Drijvers (2010).
Patterns are used to organize selective rendering of different properties in the same application.
Scaffolding and tips, for example, can only be used in training and practice test mode, but not in actual test mode.
Appmay allows different project types from static multiple-
Multiple-choice questions are dynamic simulations of real-world environments where students interact with fairly complex data in a multimedia environment.
Different project types allow different opportunities to observe and evaluate students' behavior and reasoning (Gorin, 2006).
In this case, we will look at all the interactions that exist during the use of the application, including interactions with mathematics, scenarios, digital functions, and devices.
The scores and adaptive scores of student achievement are the core components of the assessment development.
Scores can be used to assess the performance of the student and provide true
Time feedback from students, but the meaning of the score depends on how the score is calculated.
For example, when only accuracy is considered, the meaning of the score is different from when accuracy and speed are considered.
Scores can also be managed as part of the student profile and master account (adaptability)
At the same time, the adaptability of the scoring algorithm will also be affected.
We focus on two elements in this category, namely, scoring methods and adaptability.
The scoring method focuses on which variables are included in the scorecard calculation in each application (e. g.
, Correct/incorrect, required hintsneded, time, etc. )
Adaptability is related to whether and how each application provides users
Dependent Content (e. g. , on-Demand prompt andon-
Error prompt, adapt to user ability and
Always add difficulties etc. ).
For two reasons, we decided to focus on the math education app on the Apple AppStore.
First of all, Apple's iOS platform is the most popular tablet platform for the target population (4th-to 5th-
Grade students, see Mainelli, 2013)
Second, the Apple App Store contains more apps and user reviews than other platforms.
We sampled the math education app on the Apple App Store in the summer of 2013 using the following methods.
First, we got a list of the top 100 most popular educational apps, 12 of which are math-oriented.
We then got the name of the 52 additional apps that appear in the math "education Collection" in iTunes or iTunes education spotlight (
From the list of 64 apps, the decision to review the app is based on many factors: the calculator app is excluded, apps that users download with little and/or low ratings for obvious clones of other apps are excluded.
Also, some of the apps are available in the series and each app has a theme (e. g.
Algebra, fractions, counting, etc. ).
Only the most popular apps for each series (i. e.
, The one with the most downloads)was included.
Finally, due to the actual reasons for the budget constraints, we have excluded some paid applications that are costly.
This process resulted in the download of samples of 16 applications for review (
See Appendix for a list of apps viewed).
The selected app was downloaded and installed on the iPad device and all 16 apps were reviewed by a single researcher, allowing 10-
According to the complexity of the design and the variability of the available interactions, 25 minutes of interaction with each application.
Other information about each app is collected from user ratings and written reviews in the App Store.
Video reviews for each app are also collected through keyword searches on Youtube. com.
The second researcher then reviewed 16 applications, as well as all notes on general agreements on comments and interactions.
Notes are made according to the four dimensions outlined in the introduction.
Notes include the positive and negative aspects of the app in each dimension.
Because our primary goal is to learn tablets.
Instead of quantitative analysis of these notes, we are based on mathematical applications rather than evaluating samples of these specific applications.
We will make a qualitative summary of the lessons learned from our interactions with the app.
Initial sample of result range 60 for math application
Four apps, and 16-
App subset reviews a variety of mathematical topics including numbers and operations, algebra, geometry, and statistics and probabilities.
The target age of the app ranges from preschool to adult, and most apps focus on young people, that is, from preschool to primary school --aged children.
Different types of applications are also different.
Some applications are in the form of ofe.
Textbooks, teaching materials and units
Basic Assessment (e. g.
Woot math, HMH Fuse series).
Others provide video clips or program presentations in the form of personal tutors (e. g. Long division; Khan Academy).
However, the vast majority of applications take the form of a game in which players need to solve math problems to get points and achieve the goals of the game (e. g.
, Dragon Box algebra, sports math series, Teachley;
Additional adventures etc. ).
Interestingly, although some apps are tested, we have not found any apps that claim to be evaluating the appprep in nature.
On the other hand, these two electrons
Textbooks and game apps are usually built in
Elements in the assessment.
So the current review will still provide insight into building a tablet
Based on mathematical evaluation.
Evaluate the selected math app math content when reviewing the content of the Sixteen selected apps, the quality of mathematics we teach from (MQI; see Hill et al. , 2008)
: The accuracy of mathematics and the richness of mathematics.
As mentioned earlier, mathematical precision is adjusted according to MQIdimensions of "error" and "inprecision.
"MoyerPackenham, etc. , (2008)
Define mathematical accuracy as "the degree to which a mathematical object is faithful to the basic mathematical properties of the object in a virtual environment "(p. 204).
Mathematical applications appear mostly accurate in mathematical content, but sometimes conscious design decisions are made at the expense of mathematical accuracy to facilitate use or meet the user's expectations.
DragonBox Algebra, for example, is designed to teach students how to solve unknown values.
However, depending on the level of the game, certain effective mathematical solutions are not possible, which may be to eliminate the interference of students performing at a lower level.
For example, sometimes the two sides of the equation can be represented by the same number, but cannot be separated.
Another type of example can be seen in the question: x = x/3.
The user expects to divide the two sides of the equation by x and get the solution x = 1/3.
However, this makes it impossible to find an alternative [valid]
Solution ofx = 0.
These and other examples may create or reinforce mathematical misconceptions.
The richness of mathematics consists of two elements: attention to the meaning of mathematical facts and procedures and participation in mathematical practice (Hill et al. , 2008).
Meaning construction includes the interpretation of mathematical ideas and the establishment of connections between different mathematical expressions.
Most applications do a poor job in this regard.
They usually focus on retrieving mathematical facts using simple response item types (
For example, a digital response consisting of one digit)
Items automatically generated.
They may also focus on how to execute the program (e. g.
, The addition of scores of different denominator, long division, etc. ).
We didn't find any apps in the comments that involve users in activities that they need to explain or think about why aprocedure works or why certain policies don't work.
For the drawing connection between different expressions, it is widely used.
Some applications only provide Arabic numerals and algebraic expressions (e. g.
, Joel Martinez TouchyMath)
, While others present multiple representation without explicitly drawing the connection between them.
DragonBox Algebra, for example, provides a graphical demonstration of the addition inverse relationship.
Object icon and its color-
Once placed on top of each other, the invertedcounter part will cancel each other.
It also provides numeric expressions for addition inversion, such as 5 (-5)= 0.
However, there is no explicit connection between the graphical and numeric representation.
This is part of the philosophy of Dragon Box algebra design, which claims to secretly teach algebra rules in games that do not explicitly reference numbers and variables.
In contrast to the above, some applications explicitly focus on mapping between different mathematical expressions.
For example, in motion math HD: fraction, the user needs to identify the same scores expressed in different forms, including area models, digital line models, scores, percentages, and decimals.
Overall, most applications do a poor job of engaging users in meaningful math practices.
Mathematical practice includes the existence of multiple solutions, the development of mathematical generalization from specific examples, and the smooth and precise use of mathematical language (Hill et al. , 2008).
In our search, we can't find any apps in the Apple App Store that require users to apply multiple policies to solve the problem.
The efficiency of the solution is usually evaluated by the user response speed.
However, some applications adopt different methods to evaluate the efficiency of solving the problem by the number of moves to solve the problem.
For example, DragonBox Algebra sets the maximum number of moves for each item.
The user must isolate the unknown variable on one side of the equation, and then he/she has no action.
In this way, users need to solve each problem in a relatively effective way, not throughand-error approach.
Overall, we were under the impression that most of the applications we reviewed provided accurate mathematical content, however, most of the applications were not sufficient in terms of mathematical richness in any case.
This may be due in part to the limitations of the type of math application ---
In games that emphasize speed, there are often small spatial formulas or reflections.
In other cases, the narrow focus may be an implicit design decision (e. g.
, The author may not know Rich)or explicitly (e. g.
DragonBox Algebra, which insists on-
Game action and algebra).
Feedback and scaffolding although there are few opportunities for reflection in the current math education application, most applications have feedback and scaffolding in different forms.
Timely feedback on user performance can take three forms :(a)
It can give feedback on the problem being solved or the mathematical object being checked; (b)
It can be to correct the feedback, let the user know that a mistake has been made and guide the user to correct it; (c)
It can be conceptual feedback, asking the user questions, causing the user to reconsider his or her view of the object.
Most of the apps explored provide corrective feedback when completing a project.
For example, the algebraic touch of regular Berry SoftwareLLC clearly indicates when an expression is completely simplified or solved.
Invalid actions are immediately recognized as part of the vibration of the equation on sight, hearing and touch to indicate the position and cause of the action being invalid.
As for status feedback, so far, most apps have a scoreboard/bar to provide information about the overall performance of the user.
Users can also choose to go to the astatus board to see how they perform on each project.
Conceptual feedback is less common than corrective and state feedback.
For example, in Dragon Box algebra, invalid actions are clearly identified (
Although as mentioned earlier, some other effective actions are not allowed)
But there is no reason why they are invalid.
On the other hand, some applications provide perceptual feedback when users have problems --
Experience in solving.
For example, BuzzMath Middle School will provide students with a complete question --
Solve the procedure and explain when they answer the question correctly.
Some applications will not only provide feedback on what is the right solution and why it is the right one, but also feedback on why the given solution is wrong.
In the Touchymath of Joel Martinez, when an invalid operation is attempted, the application highlights the term or action to indicate the reason for the invalid operation.
When a user submits a solution that has not yet been simplified, the application will indicate which parts of the solution can still be simplified.
Concept feedback can also take the form of video lectures.
For example, Khan Academy serves as an extension of the video library and website, syncing user activities on the platform and linking users to poorly performing themed video lectures.
Some applications provide scaffolding when students encounter difficulties.
Scaffolding usually occurs in the form of a hint or a guide to a problem, when the student proves evidence that the problem is difficult to solve, such as a long reaction time or an incorrect first attempt.
Tips are also provided when students request (on-demand hints)
Regardless of their current performance.
An example of on-
Brian West can see the demand tips in PizzaFractions, where students can choose "peek atpizzas" to see the area of the score size before indicating their selectionFigure 1a). On-
The demand prompt may not be able to be customized for a specific item.
For example, in Numerosity: Playwith multiplication, a user's request for a prompt will activate the ageneric multiplication table.
An example of-
The tips needed are from Teachley: Additional adventures in Teachley (Figure 1b).
As shown in the figure, Block 2 5 is suspended by two chains.
If noresponse is given after 2 seconds, a chain is cut off.
Once a chain is cut off, the hint of this problem will jump out.
The prompt on this image suggests that the user should apply the "count" policy;
For users who are not familiar with this strategy, tutorials can be obtained outside the game interface.
The apps we review rarely facilitate reflection through sense --
By asking questions to stimulate the user's summary of mathematical laws.
However, some applications do provide a complete history of user issues --
Resolution actions at the end of the task (e. g. , TouchyMath).
The purpose of this is to give students the opportunity to review their questionsSolution process.
However, there are no clear benefits or benefits in the game environment, and users may skip this step and move on to the next task.
The richness of interactions the number of available interactions sets limits for possible user actions when using an application.
The available interactions also affect the way students complete their tasks.
For example, different patterns (i. e.
Study, practice or test)
It will affect students' strategy selection and participation (
Roediger, Agarwal McDaniel, Mike, 2011;
McDaniel, Agarwal, Huelser, Mike, and Roediger, 2011).
In addition, the project type (e. g.
Multiple choice questions, short answer)
It also affects how students handle tasks (
Butler and Rodeo, 2007).
The mode of interaction through interactive mode we refer to the stages of the game or application that trigger different user interaction modes, for example, tutorials, practices, contests, or main task sets, and more.
Each model has different learning goals.
Some apps provide exercises through game tutorials to familiarize students with the types of interactions used in the game.
For example, DragonBox Algebra provides an introduction to set-
On the screen, different elements of the game (e. g.
"Zero" card, "one" card and "dragon" card)
And the goal of the game (i. e.
Isolate the "dragon" box on one side; see Figure 2).
Users need to follow instructions to complete the task before starting to interact freely with the game.
Practice mode is also used in the middle or before
Games, usually when students have difficulty with certain problems.
For example, in the hand
In the equation, students can choose to watch a short videoclip forhelp.
After watching the video, they need to complete two practice questions before going back to the "practice" interface.
The project types in most applications have fairly clear units of interaction, similar to the test items in the evaluation.
They may be arithmetic problems to be solved one after another, or they may be challenges to pass in order to advance to the next level.
There are a variety of projects in these applications, although the response projects built are not common.
Most items are varied.
Select the question.
Some items will vary depending on the content selected;
The selection may include pictures, charts, and even video clips, not numbers, equations, or text.
Changes are also included in the selected layout.
The number of choices may be unlimited, not a limited number of choices for horizontal or vertical alignment ,(e. g.
, Position on the number line).
In addition, the behavior of choice may also change. Some choice-
In addition to tapping, new interactions are needed to make actions.
For example, in the subject math series, students need to tilt the tablet so that the object falls on the selection area located in the lower left corner of the screen. Drag-and-
Drop operations are also used in multiple operations
Select the issue and the user needs to drag the selected option to the desired location.
Some applications use a combination of gridsMultiple-
The choice of ways to ask questions.
In arithmetic, for example: play multiplication!
How many users may encounterdigit numbers.
They need to select the number for the unit first.
After that, the selections will be refreshed and they can choose fortens (see Figure 3)
There are hundreds, if applicable.
The constructed response items typically involve interactions with virtual operations such as score bars and location value blocks.
The program may require students to represent mathematical entities using manipulatives.
For example, in "math: Base Ten", users can drag units, ten, and hundreds into a virtual "tray" to represent multipledigit numbers.
The program may allow students to use manipulatives to represent mathematical operations.
Figure 4, for example, comes from Woot math at Nimbee LLC.
In order to solve this problem, the user needs to drag the corresponding unit score "slice" to the "score circle", in this case, 3 "1/10" slices and 2 "1/5 display 3/10 2/5 processes are required.
In addition, the user needs to drag 2 "1/10" slices at the top of each "1/5" slice to simulate the process of finding an equivalent score.
In reviewed applications, there are very few handwritten inputs.
When used, the user's handwritten input can be kept "as is "(e. g. ,Woot Math)
Or translated into text input for further processing (e. g.
, FluidMath, to-do list K-2 math exercises).
Most applications that allow handwritten input are usually interactive drawing tools.
For example, FluidMath can convert a user's handwritten input to an equation, and then draw a graph of that equation, including multiple "objects" on the same drawing ".
Figure 5 shows the sketch ellipse, line, and parabola converted to a single drawing, while showing their algebraic expressions.
When the user changes the position or shape of the object on the coordinate plane, the algebraic expression changes accordingly, and vice versa.
In conclusion, the rich interactions we find in our math applications are far greater than typical educational assessments, including the latest technology --enhanced items.
Not only does Tabletsafford have more kinds of project introductions and replies (e. g.
, Intuitive gesture command, accelerometer function, potential for handwritten response, including the ability to draw drawings with text response)
, They also create the opportunity to collect response process data that normally does not exist in traditional assessments (e. g.
, Record the order in which the chart is drawn, try/reset the question before submitting the answer).
The scores and adaptive scores of student performance are the core components of the assessment development.
Scores can be used not only to assess students' performance, but also to provide real-time feedback to students in various ways.
A lower score for a particular topic indicates that more difficulties have been experienced and more work needs to be done on that topic.
Scores can also be used to determine which programs are best suited to the student's current level of competence in order to keep the student engaged during the assessment process.
Scoring most applications marks the response as correct or wrong, and uses percentcorrect as an indicator of the overall performance of the user.
Time-sharing can also consider speed.
For example, in the Math Champion challenge, the number of points a project may gain will decrease over time.
When scoring, the difficulty of the project is often not considered, so all items have a certain weight when calculating the final score.
The scores in these applications only make sense in the application itself;
Any application in the sample is not considered as any external validity of its scores, whether as proof of ability or as a basis for making decisions outside the virtual world of the application.
This is very different from the education assessment.
Not all applications can adapt to the user's response.
Usually one of three methods is used to adjust the difficulty of the project touser capability: the first method is to let the user select the difficulty level.
For example, interactive elementary's middle school math HD allows users to select one of the three difficulty levels before starting the game: simple, medium, and difficult.
The second method is to provide more and more difficult projects, and the successful completion of a set requires the next set to be solved (e. g.
Score planet for Playpower labs).
The last approach involves the dynamic presentation of the problem.
The user still needs to complete the current task before unlocking the next task, but the next task will be rendered on which task depends on their performance on the current task (e. g. , Woot Math).
In addition to the notes above, we also observed various usability issues in math applications.
Instead of listing them here, we discuss how to avoid them in the evaluation design in the next section.
We did a survey of the math application in order to understand what we should do (and should not)
Develop innovative mathematical assessments on tablet devices.
While the prospect of mathematical applications seems broad, there are only a few categories of games, interactions and adaptability that are currently working.
These features may affect the way students solve problems and learn.
They also affect how we explain how students interact with the interface and their final answers.
These are critical issues when we design the next generation of tablets.
Based on mathematical evaluation.
In the following content, we will discuss some lessons learned and how they can be applied to tablets
Design based on evaluation.
Many of these proposals may apply to all technology-based assessments (TBA)
But we leave these suggestions to the tablet, because the interactive nature of the tablet interface is fundamentally different from the computer, students have been shown to interact with the tablet in different ways, and the results are different (
See j. Abrams, Davoli, du, Knapp, and Paull, 2008;
Davoli, du, Montana, garwickick, Abrams, 2010;
Davoli and brockmore, 2012).
We review the shortcomings of many mathematical applications in expressing the accuracy and richness of mathematical content.
While this can be forgiven in a game and information learning environment, at the time of evaluation, these flaws may produce systematic errors in scoring and interpretation.
Fortunately, in the field of education assessment, there is a rigorous review process to ensure the accuracy of the content, which is effective for the familiar type of project.
However, one of the challenges we may have to deal with is how to continue this powerful tradition as assessment tasks become more complex, interactive, and gaming --like.
We observe that in some cases, when a math problem is operated in a game,
Just like the settings, the mathematical concept is shot at a manageable object on a tablet, and effective mathematical operations are not always translated into legitimate physical operations on a tablet, for example, if the operation allowed in dragonbox is blocked from obtaining one of the two valid answers.
This may not be a problem with the game, but it may underestimate the effectiveness of the assessment.
So one of the lessons we 've learned is: create an interactive game --
Like evaluating projects, be very careful about how mathematical concepts and operations map to objects and actions in the virtual world;
Thoroughly review the mapping early in the task design phase.
An indisputable attraction of these applications is that they provide a wide range of rich and engaging interactions around mathematical content.
This is the user--
Students, parents and teachers-
Look forward to the iPad experience ".
"There is no doubt that this is also their expectation for tablets --
Based on assessment.
The bar is set up very high and there are plenty of places to cover in the evaluation industry to catch up with the app.
At the same time, however, the level of rigor required for the assessment is lower/No.
For a high quality assessment, a bet game on content for interaction and a certain amount of freedom is not acceptable.
For any serious attempt to create a tablet, we treat high quality Interactive Item types as given-
Based on mathematical evaluation;
They are what the students expect from everything on the tablet, and, as long as it is correct, they involve the students in mathematical thinking.
Nevertheless, we recommend not to create interactive projects for sakeof engagement only.
As we pointed out earlier, without a clear focus and concrete implementation, it is easy to ignore the structure, resulting in an interesting but unexplained distraction.
From an evaluation perspective, the greatest value of interaction is that they provide data on intermediate steps and strategies that students use to solve problems.
It is often difficult to infer why a student answered a project in error just based on the final answer.
Complete problem record
The resolution process helps determine the cause of the error.
This information can help explain the student's performance in the assessment, or identify scaffolding or instructions that people may need.
We provide three illustrations: * The final product and the response process.
Most applications use the user's answer actions to calculate scores.
In addition to being accurate and fast, student changes to answers can also reveal their strategy use and knowledge.
For example, higher
Capable students are unlikely to change their answers, although they are more likely to get more answers from their answer changes (McMorris et al. , 1991).
The order in which the answer actions are recorded may also reveal the context in which the student changes the answer, which may help to identify cognitive processes involved in this change.
For example, after answering the questions that follow, changing the answers to previous projects may suggest that students use the information in the questions that follow to update their ideas.
On the other hand, before submitting, changing the answer at the time of review may suggest that they solve the problem for the second time and find the error.
* Task navigation.
Navigation through a task can also tell us the pre-existing knowledge.
For example, a student may skip a question that he or she was unsure of at first and then go back to it.
There is very little research on test navigation, but current evidence suggests that
Ability students are less likely to move back and forth when completing the assessment (Kim et al. , 2012).
It is worth noting that task navigation not only reflects the task processing of students, but also reflects the structure of assessment.
For example, many formative assessments allow students to go back to previous projects and future projects through arrows on the current screen.
However, some apps allow students to access any previously resolved items from the comment board (e. g.
, Woot math, see figure 6).
The design of the evaluation process will affect (a)
Can students skip a question and come back later ,(b)
Can students carry themselves
Monitoring actions that mark answers as not confident or problematic, and (c)
How do students navigate back to the previous project (See Pan, Keaton-
Hodges and Feng, 2014).
* Solve problems and use tools.
The process of solving problems can be avoided by students using the virtual tools provided in the application.
Virtual tools include calculators, tips, virtual sketch paper, and other tools.
Observable problems
Addressing behavior may include deciding whether to use tools to help solve the problem (e. g.
"I want to paint paper".
"It's easy to do it in my mind ";
For example, see Wilson, 2002 for information about off-
Loading of cognitive work)
Which tool they choose to use (e. g.
, "I did math with a calculator, but the numbers I got don't look right.
I think I have to do it by hand ")
What did they do with these tools (e. g.
Make longke vs on virtual sketch paper.
Create a visual model for this problem).
The interpretation of student interaction with tools should be carried out at the project level, but a comprehensive measure of such interaction can also provide insight into student problem solving.
Kim and others, for example. , (2012)
Check the student's writing order in a tablet-based math assessment.
They reported on the relevance of cross-project writing patterns to student math scores. High-
The students who scored were less erased strokes than their peers.
They also show more of the top
Write down the action in the process they solve the problem.
The researchers explained that multiple erased strokes at the same location may indicate the trialand-
The wrong way for students to solve problems while top-
The way to write it down may reveal the planning and problems of a systemSolution process.
Observe students' problems at the project level
Learning behavior can provide details about the student's specific strategy, as well as the details they model with mathematics.
For example, we conduct usability research through prototype tablet evaluation to achieve an understanding of rational numbers.
In this task, students can activate a virtual cutting board where they can cut and share banana bread on the plate.
Solve the following problems (Figure 7)
Both students started the patch board.
They answered multiple questions.
The problem of choice is the same.
Their written explanation is also very similar, just like the last screenshot they took when they left the cutting board.
But they have adopted different strategies.
The first student said when asking about their plans for how to use the cutting board, "I want to cut a loaf of bread into 5 pieces.
Then I thought I might have more parts but each one was smaller.
So I cut each into four.
Like I did for the library)
See what happens.
There are 3 for everyone, and there is another piece to share between them.
So everyone has a little more than 3 [sic]
The second student was also prompted what he planned to do, when he started the cutting board: "The four are not divided into five.
So I want to cut each into two parts and try it.
It does not work. I'll try three.
Or not. I'll try [
Cut each into four. .
Or not. . .
Wait, this is the same as the library!
But there's one left to eat.
Someone can take the rest and they will get more. " [sic]
These two strategies represent the views of many students.
In this case, we are able to map the number of exploration actions (i. e.
Number of times the student reset the cutting board, recovered from the notes and video records of the interviewee)
Use of strategies for students. In large-
However, it is unrealistic to record all interactions between students and tablets on a large-scale field test.
So it becomes crucial.
Define the events of interest in evaluating the multiple possibilities of the design, and iterate through the development and testing process to adjust these captured events to ensure that they collect the expected evidence.
Summarizing lessons on interactive projects: when designing an interactive project, start by inferring what evidence is required for student performance and design interactions to collect the necessary data.
Developing innovative project types Our review also proposes weaknesses in popular math application design that need to be innovated in tablet design
Based on assessment.
Here we focus on two issues, namely (a)
The thinking and explanation of the students, and (b)scaffolding. * Self-
Explain and reflect.
Literature on Cognitive psychology (
Chi, De Leeuw, Chiu and Lavancher, 1994;
Chi, Siler, Jeong, Yamauchi & Hausmann, 2001)
Education and evaluation (
Osmond, Merry and Leiling, 2002;
Merry, & Callaghan, 2004)
Point out the importance of self
Explain and reflect.
There are very few opportunities available in the reviewed app for users to reflect and self-
Explain, however.
This may be a by-product of the nature of app game design (i. e.
, Let the user's goal is to get points or complete the game as soon as possible).
There is a lot to learn in education and evaluation by listening to students' sexual experiences (
For example, the example of bread above-cutting)
If we do not offer an opportunity for reflection where feasible, it will be the best.
Tips for oral or written explanations can involve students in a process that is different from their initial response.
These tips sometimes act as a re-
When they sometimes adopt a new strategy or model, check to avoid careless mistakes.
In both cases, interpretation involves in-depth processing of information, providing valid evidence for scoring, and providing teachers with a rich explanation of student scores.
And, self
In promoting conceptual understanding and learning migration, the student's interpretation process is also a commendable practice (Rittle-Johnson, 2006;
Berthold, Eysink & Renkl, 2009).
However, the explanation of prompting all projects requires additional time for evaluation, and for young students with limited keyboard operation skills, providing written explanations may be particularly tiring.
These difficulties raise two questions.
First, when should we prompt the explanation;
The other is how we should collect these explanations.
The time to explain the prompt can be project-based or response-based. Item-
In projects that require multiple steps to solve the problem, and when there are multiple policies in solving the problem, prompt-based should be used. Response-
When students provide incorrect answers, it is possible to use hints based to distinguish between careless mistakes and more serious misconceptions.
This tip can also be used when students spend too much time on a question to distinguish between reading comprehension questions, confusion and lack of knowledge and poor planning.
With regard to the method of collecting student explanations, tabletdevices offers a number of possibilities, such as the use of a soft keyboard or an external keyboard, handwriting, drawing, and voice input.
Our pilot studyCayton-
Hodges front, disc, and Vezzu, 2013)
The fourth one
Grade students like to write digital replies on tablets, but prefer to type
Reply answer written on tablet (
While they also appreciate the ability to supplement their written explanations with drawings, it is often difficult to use a keyboard on a pc --Mouse Interface).
In addition to collecting responses through multiple parties
Touch screen, can also use oral input.
For example, Siri is on the iPad as a real
Time language input processor.
You can return the recognized string in real time by evaluating the interface call.
Reasoning and demonstration are the practices required by Common Core national standards (
See Common Core Standards Initiative, standards of practice in mathematics, 2010)
But these self
Interpretation techniques are often diluted in practice.
There is a lot of room for innovation in this area, and the tablet provides a very versatile platform to capture interpretation and thinking in speeches, writing or painting.
Although some natural language processing tools can be used to automatically classify students' explanations, the grading of these evidence is beyond the scope of this article.
We suggest the following: as part of the engaging interactive task design, it also creates opportunities for students to self
Reflect or explain their problemsSolution process.
Consider the use of appropriate means to collect such evidence.
* Auxiliary equipment and scaffolding are provided.
Tutorials, tips, brackets, and other tools are widely used in the math applications we review.
They do this in order to quickly position the user to the task and let the user move along the path of the challenge.
This assistance is particularly important for the game.
Just like the app, it's possible to lose users if they can't get into the game quickly or get to a certain level and can't find a way out.
These tools usually appear on anas. needed basis.
Part of the reason is limited screen real estate
Applications need to make the most of useful interactions and avoid clustering.
Therefore, when and how to provide help to users is a key consideration in application design.
In contrast, in traditional assessments, the pressure on students is to follow instructions and solve problems.
The practice of copying paper-and-
Computer pencil test
Today, the evaluation-based usually prints instructions statically on the screen, regardless of whether the user needs it or not.
If an assessment task is long and complex, students usually need to read the instructions on one or several pages.
Few applications are able to do this;
They must minimize the workload and the workload of the user, otherwise they will be lost.
Reading a lot does have an impact on students.
Through our tablet trials, we found that students tend to be less careful when reading items and sometimes get confused about the requirements of the items (see Cayton-
Hodges front, disc, and Vezzu, 2013).
Because the direction is static, students are not given any feedback or opportunity to confirm or challenge their understanding.
In this case, students usually ask the interviewer for help.
Some students commented that they would be helpless in the summative assessment because no one asked.
The question here is how we can deliver the direction.
"In games and apps, and in daily communication,-needed basis.
The direction in the test is an exception because normalization is necessary.
While it makes sense when project types are simple and familiar, personalized help will become more important as tasks become more complex and creative, especially in formative assessment environments.
Letting students struggle in frustration not only defeats the "iPad experience" but also threatens the effectiveness of the assessment;
If the student is confused about the task requirements, the score is not an appropriate reflection of the mathematical ability.
If we decide to involve students in complex assessment tasks, then it is our responsibility to ensure that all students understand and are able to complete the tasks.
The math app we reviewed provides many positive examples to guide students through animation tutorials, adaptive instructions, and options to overcome challenges. Help-
The search for behavior can also provide valuable evaluation information.
In formative assessment, the teacher may want to know which students have clicked on the help button and under what circumstances. Having context-
In this case, the assistance relied upon creates an opportunity for assessment.
Imagine an interactive instruction (
Or computer agent)
It can read, rewrite, or explain part of the direction, depending on the user's needs.
This is a useful tool for English learners and students with special needs, and will therefore improve the effectiveness of the assessment.
Finally, from the "memo sheet" to the teaching video (
Please look at math. com).
This not only provides an opportunity for students to evaluate (I don't know)
And how much and how fast they can learn.
Although not all of the auxiliary methods are suitable for all assessments, we suggest the following: the evaluation developer needs to adopt the mindset of the application developer: it is the responsibility of the designer to involve the user in the task, move on.
Atest only works within the scope of the student "in game", however, please note that the measured structure has never been sacrificed in the name of participation.
Summarizing the purpose of this report is to summarize the survey of the application of mathematical temptation in the Apple App Store as part of a research project to develop tablets
Evaluation prototype based on elementary mathematics.
The purpose of this survey is to understand the design principles and techniques used in mathematical applications designed for tablets.
Our comments focus on four aspects ,(1)
The quality of mathematical content ,(2)
Feedback and scaffolding ,(3)
Rich interaction, and (4)
Adaptability of application.
These four areas have been cultivated from previous research on mathematical digital tools (e. g.
Digital tools for educational standards in algebra;
Bokhove & Drijvers, 2011)
Design principles of learning objects (e. g.
Evaluation Indicators of learning objects;
Kai and narak, 2008)
And the quality of mathematics teaching (e. g. , Hill et al. , 2008).
This review culminates in developing four suggestions for researchers and evaluation developers on designing tablets
Basic Mathematics evaluation :(1)
In the early stages of the task design phase, a thorough review of the mapping between concepts/operations and objects/operations; (2)
Start by inferring what evidence is required for the student's performance and design the interaction to collect the necessary data; (3)
Create opportunities for students to self
Reflect or explain their problemsProcess of solving; and (4)
Adopt the mindset of app developers, keep users engaged, complete tasks, and move forward to ensure that students evaluate content knowledge "in-game" accurately enough.
Abrams referenceA. , Davoli, C. C. , Du, F. , Knapp III, W. H. , &Paull, D. (2008).
The vision near the hand changed.
Cognition, 107,1035-1047. Bennett, R. E. (2010).
Cognitive-based assessment of learning, for, and aslearning (CBAL)
: A preliminary theory of action for summative and formative assessment. Measurement, 8(2-3), 70-91. Berthold, K. , Eysink, T. H. , & Renkl, A. (2009). Assistingself-
When learning multiple representation, the explanation prompt is more effective than opening the prompt.
Teaching Science, 37 (4),345-363. Blume, H. (2013). L. A.
Apple's ipad school board OKs $30 million.
Los Angeles Times.
Retrieve from C Bokhove, And the driver, P. (2010).
Digital tools for algebra education: standards and assessments.
Computer Journal of International Mathematics Learning, 15 (1), 45-62. Bokhove, C. , And the driver, P. (2011).
The impact of feedback conditions on online algebra tools. In M. Joubert, A. Clark-Wilson,& M. McCabe (Eds. )
Minutes of the 10 th International Conference on Mathematics Teaching Technology (ICTM T10)(pp. 81-86).
From Butler. C. , And Roediger, H. L. (2007).
Test improvement-
Keep the semester in a simulated classroom environment.
European Journal of Cognitive Psychology, 19 (4-5), 514-527. Cayton-Hodges, G. A. , Feng, G. , Pan, X. , & Vezzu, M. (2013).
Primary Math iPad task design: Year-end report to CBALinitiative.
Princeton, New Jersey: Educational Testing Service. Chan, M. S. , & Black, J. B. (2006, April).
Learning Newton mechanics with animation games: the role of presentation format in mental model acquisition.
A paper published at the annual meeting of the American Educational Research Association in San Francisco, California. Chi, M. T. H. , De Leeuw, N. , Chiu, M. -H.
, And C. Lavancher(1994). Eliciting self-
The explanation improved the understanding.
Cognitive Science at the age of 18 (3), 439-477. Chi, M. T. , Siler, S. A. , Jeong, H. , Yamauchi, T. , & Hausmann,R. G. (2001).
Learn from human counseling.
Cognitive Science at age 254),471-533.
Common Core Standards Initiative. (2010).
Common Core Standards Initiative.
From http://www Retrieval. corestandards.
Organization/math/practice Davoli, C. C.
And brockmore, J. R. (2012).
Shieldattion from the hand of visual interference.
74 years old (attention, perception and psychological physics)7), 1386-1390. Davoli, C. C. , Du, F. , Montana, J. , Garverick, J. , & Abrams, R. A. (2010).
When meaning is important, look but don't touch: the effect of posture on reading.
Memory and Cognition, 38 (5), 555-562. Gibbs, G. , & Simpson, C. (2004).
Evaluate the conditions that support the student's learning.
Learning and Teaching in Higher Education, 1, 3-31. Gorin, J. S. (2006).
Test Design considering cognition.
Education Measurement: Problems and practices, 25 (4), 21-35. Haughey, M. And Muirhead, B. (2005).
Evaluate the school's learning objectives. E-
Journal of teaching science and technology, S (1),229-254. Harrison, T. R. (2013).
The IPad app reviews math learning.
Get it back from J Hattie
, And Timperley, H. (2007).
The power of feedback.
Educational Research Review, 77 (1), 81-112. Hieb, J. L. , & Ralston, P. A. S. (2010).
Tablet in Engineering Mathematics courseB.
School of Speed Engineering.
International Journal of Mathematics education in science and technology, 41 (4), 487-500. Hill, H. C. , Blunk, M. L.
Charalambous, C. Y, Lewis, J. M. ,Phelps, G. C. , Sleep, L. , & Ball, D. L. (2008).
Mathematics knowledge of teaching and mathematics quality of teaching: Exploratory Research.
Cognition and teaching, 26 (4), 430511. IDC (2013).
Tablet shipments are expected to reach the top of total PC shipments in 2013 and 2015, according to IDC [PressRelease].
Search from interactive education system design company(2013).
National monitoring of K-mobile technology12 education.
From Isabwe, G. M. N. (2012, June).
Investigate the availability of iPadmobile tablets in formative assessment of math courses.
2012 International Conference on the Information Society (i-Society)(pp. 39-44).
Paper published at 2012 International Conference on the Information Society (i-Society), London, UK. Kay, R. , & Knaack, L. (2008). A multi-
Component model for evaluating learning objects: evaluation metrics for Learning Objects (LOEM).
Australian Journal of Educational Technology, 24 (5), 574-591.
Kim, Y, Lim, C. , Choi, H. , & Hahn, M. (2012, August).
Effect of training math problems
Solve behavior using a tablet.
Papers published at 2012 International Conference on Engineering teaching, evaluation and learning (TALE), Hong Kong. Kowalski, S. E. , Kowalski, F.
V & Gardner, TQ. (2009).
Lessons learned when collecting real information
Use a tablet for time formative assessment in a university classroom.
Minutes of 39 Frontier Conference on Education (pp. 926-930).
San Antonio, Texas
Piscataway, New Jersey: IEEE Press. doi: 10. 1109/FIE. 2009.
5350639 Laughlin Davis, Los Angeles, Strain-Seymour, E. , Gay, H. (2013).
Tablet testing: Part 2 of a series of usability studies on tablet use
12 evaluation items (
Pearson White Paper).
Retrieved from T Mainelli in December 1, 2013(September 2013). U. S.
Part 1: hardware, operating system, branding and replacement cycle. IDC.
Retrieve from M. McDanielA. , Agarwal, P. K. , Huelser, B. J. , McDermott, K. B.
And H. Roediger III. L. (2011). Test-
Intensive learning in the middle school science classroom: the influence of the frequency and position of the test.
Journal of Education Psychology, 103 (2), 399-414. McKnight, L. , & Fitton, D. (2010). Touch-
Kids Screen Technology: give the right instructions and get the right response.
Minutes of the Ninth International Conference on Interactive Design and children (p. 238-241).
New York: ACM Press. doi:10. 1145/1810543.
1810580 McMorris, R. F. , Schwarz, S. P. , Richichi, R. V. , Fisher, M. ,Buczek, N. M. , Chevalier, L. , Meland, K. A. (1991).
Why do teenagers change their answers in exams?
A study by State University of New York at Albany. (
ERIC file copy service, ED 342 803). Moyer-Packenham, P. S. , Salkind, G. , & Bolyard, J. J. (2008).
K-virtual manipulator used
Mathematics Teaching teachers: consider mathematics, cognition and teaching loyalty.
Contemporary Issues of Technology and Teacher Education, 8 (3), 202-218. Nicol, D. J. , & Macfarlane-Dick, D. (2006).
Self-assessment and self-assessment
Standardized learning: The model and seven principles of good feedback practice.
Research on Higher Education, 31 (2), 199-218. Orsmond, P. , Merry, S.
A. Callahan. (2004).
Implementation of formative assessment models incorporating peer and selfassessment.
Innovation in international education and teaching, 41 (3), 273-290. Orsmond, P. , Merry, S. , & Reiling, K. (2002).
When using student-derived marking criteria in peers and self, the use of paradigm and formative feedbackassessment.
Evaluation and evaluation of higher education, 27 (4), 309-323. Pan, X. Cayton-Hodges, G. A. , and Feng, G. (2014, April).
Not all changes are equal: the cause, the background, and the outcome of the response.
A paper published at the annual conference of the American Association for Educational Research. Ren, X. , & Moriya, S. (2000).
Improve pen selection performance
System-based research on pen style
Based on the interaction of selecting tasks.
ACM Transactions on Computers
Human interaction (TOCHI), 7(3),384-416. Rittle-Johnson, B. (2006).
Promoting transfer: the influence of self
Explain and direct guidance.
Development of Children, 77 (1), 1-15.
Rodger III, H. L. , Agarwal, P. K. , McDaniel, M. A. And McDermot, K. B. (2011). Test-
Intensive learning in the classroom: Long-
Questions that have improved in the short term.
Journal of Experimental Psychology: Application, 17 (4), 382-395. Strain-Seymour, E. , Craft, J. , Davis, L. L. , & Elbom, J. (2013).
Tablet testing: Part 1 of a series of usability studies used by K-tablets
12 evaluation items (
Pearson White Paper).
In December 1, 2013, it was retrieved from the Wisdom balance evaluation alliance. (2012, April 25).
Smarterbalanced and PARCC release guidelines for new teaching techniques.
From being smarter-balanced-and-parcc-issue-guidance-for-
New structure-technology-
M. Toby/Toby, Ma, B. , Lai, G. , Lin, L. , & Jaciw, A. (2012).
Comparative validity of the haudon Mifflin Harcourt fuse: Algebra 1--
Random experiments in four California regions.
Palo Alto, California: Experience EducationTsuei, M. , Chou, H. -Y, & Chen, B. -S. (2013).
Usability measurement of mobile mathematics courses
Measurement application based on children. In A. Marcus (Ed. )
User Experience and availability.
Health, learning, games, culture and cross
Cultural user experience (pp. 304-310). doi: 10. 1007/978-3-642-39241-Wilson, M. (2002).
Six viewpoints of physical cognition.
Psychological Bulletin and comment, 9 (4), 625-636.
Appendix review list of applications for liquidity software companies(2013). FluidMath 2013. (Version 1. 2). [
Mobile application software.
Retrieve from the hand equation. (2013). Hands-on Equation (Version 3. 0)[
Mobile application software.
From Helttula, E. (2013). Long Division (version 2. 8). [
Mobile application software.
Get it back from Hutton Mifflin Harcourt. (2013).
Algebra 1 ()version2. 1). [
Mobile application software.
From INKids. (2013).
Math Champion challengeversion 1. 1). [
Mobile application software.
Retrieve from interactive Primary School. (2012).
High school math HD (version2. 6). [
Mobile application software.
Retrieval from Khan College (2013). Khan Academy (Version 1. 3. 2). [
Mobile application software.
Get it back from J Martinez(2011). touchyMath (Version 1. 2. 7).
Mobile application software.
From semerise. (2013). Map Ruler (version 1. 3)[
Mobile application software.
Retrieve from sports mathematics. (2013).
Sports math HD: score! [
Mobile application software.
Retrieve from Nimbee LLC. (2013). Woot Math. [
Mobile application software.
Communication from Nuance (2013).
Dragon Dictation (version 3. 0. 28). [
Mobile application software.
Retrieve from Play Power Labs. (2013).
Fraction planet (beta). [
Mobile application software.
From Scola (2013).
BuzzMath Middle School (version 1. 3. 1). [
Mobile application software.
Retrived from Teachley. (2013).
Teachley: extra Adventureversion 1. 2). [
Mobile application software.
From Thoughtbox. (2013).
Numbers: play multiplication (Version1. 0). [
Mobile application software.
Retrieve B from the West. (2013).
Pizza score: start with a simple score (Version 1. 5). [
Mobile application software.
From what we want to know(2013).
Dragon Box algebra (Version 1. 1. 6)[
Mobile application software.
Gabrielle's inversion. Cayton-Hodges (1)*, Gary Feng (1)
Pan Xingyu (2)(1)
Princeton Educational Testing Service Center, New Jersey, USA (Zip code: 08541)2)
University of Michigan, Ann Arbor cognitive science research group/gcayton-hodges@ets. org//gfeng@ets.
Org/xypan @ umich.