Our exercises are seeing tons of use, but as you might have noticed they are really bite-sized chunks that closely match our video content. It’s not a one-to-one relationship, but it’s usually one exercise to just a few videos. They’re intended to provide students and coaches with timely feedback about how well a student understands an individual concept. There are no “grades” handed out, even though progress is measured, tracked, and encouraged. In other words, they are formative assessments. Rewards given for these exercises are not, generally speaking, based on the difficulty of the exercise (e.g. you can get a “Nice Streak” badge on any exercise).
The balance to formative assessments is summative assessments. For the curious, summative assessments are designed to summarize the progress/development of a student at particular points in time. Our version of summative assessments is called “Challenges”, and we’ve designed them to represent milestones in mastering the different subject areas of the knowledge map.
Challenges are summaries of the most challenging exercises in each of the major subject areas of the knowledge map. To see them, you zoom all the way out on the map.
After playing with this for a while, we landed on a very literal representation of the relationship between the Challenge and the exercises that it covers. While we’re happy with the feel of the interaction (zooming out to get to the summaries), we’re not convinced that this is the best visualization. This is something we will be watching/considering closely in the near term.
How they work
Because a challenge is supposed to be, well, challenging, and we need to assess students on a bunch of different types of questions, challenges require you to get multiple streaks of 10. The header tells you exactly how many streaks you need to get, what exercise the current problem was taken from, and shows you your progress on the new multi-streak bar. Each streak will have problems from the various exercises covered by the challenge. One important thing to note here is that we won’t give students two questions from the same exercise in a row. The added context switching increases the overall level of difficulty of the challenges.
Understanding exactly how difficult these challenges are, we’ve built in one other feature to make the process a little more forgiving, while keeping up the same level of rigor, breaking your streak will only drop you back to your last complete streak (not to 0). That means if you get 23 questions in a row correct, and get the 24th wrong, your streak will reset to 20 instead of 0. Hints and videos work the same as they did in regular exercises with videos maintaining and hints breaking your streak.
Each challenge that we create will have a unique reward that we’re calling “Challenge Patches”. We’ve borrowed the color scheme from the “proficiency” stars on the knowledge map to create an implicit connection between obtaining these rewards achieving mastery in a particular subject area. In addition, we wanted them to feel more “real” than the other badges, so they’ve been given a very tangible appearance inspired by the mission patches that an astronaut might have sewn onto their uniform. Who knows, maybe some day we can get someone to make them into real patches or stickers?
We’re learning along with our students
Although it’s very early, it only took a few minutes of the new exercises being live for students to discover and start working on them, and even easiest challenge, the arithmetic challenge, is time consuming. We’ll be watching over the next days/weeks to see how people react to the overall implementation, the visualization of the map, the user interface within the challenges, the overall difficulty of the challenges, and the rewards.
So head on over to the site and give it a try. As always, comments, questions, and suggestions are welcome!