top of page

Implementation&evaluation

Prototype | Evaluation | Project Management | Reflection

LEARNING DESIGN SOLUTION

solution 1 : Tutorials

To link concrete phenomena to abstract analysis:

01 Videos to give examples

We use videos to give examples of dog's body language and context. Videos can enable learners to recognize naturally occurring phenomena and analyze a situation.

02 interactive widgets to reinforce comprehension

We use interactive widgets such as scenario and process to further explain concepts and have learners apply what they've learned to real-life situations. Interactive widgets can:

  • be carefully scrutinized and constantly checked

  • enable the linear sequencing of information

  • provide concrete examples of abstractions

  • ask for participation from learners and bring social cues

  • create emotional context

03 knowledge check

We use knowledge checks such as sorting, flashcards, and sequencing to test learners' knowledge. Knowledge check can create friction to have learners better remember and also have them aware of their own knowledge gap. 

solution 2: Interactive Game

01 Knowledge check

Untitled design.png

We will let the player finish the knowledge check first to test their knowledge. If they can't pass the knowledge check, we would recommend players check the tutorial first.

02 Play the scenario-based game

Finish

Do the test

Make observations based on instruction

Scenario

Untitled design (1).png
  1. Game-based Learning: Apply what they learn to the real-life scenario

  2. Guiding players to make the right observation and identification steps

  3. Help players identify their misunderstandings 

​​

Beagle Puppy

Executive Summary

We work with four individual “proxies” learners to gather data to revise the instruction. The purpose is to evaluate whether the learning experience is both engaging and effective. Based on the results, we remove some obvious errors and omissions in terms of usability and instructions, adjust the content to make it less overwhelming.

Purpose
(Evaluation issues): 

Dog wearing Costume
  • Does every step make sense for them?

  • Can they understand what each button and page mean?  

  • Did task performances align with our learning objectives and goals? (What do they learn from each task?)

  • Do we have enough content or too complicated for learners? 

  • Did learners keep up with their pace or lose their patience to learn?

Corgi

Methodology

  • Methods

  • Participants

  • Instruments

Methods: 
  • Think-aloud

  • Interview

  • Observation

Participants:

We asked 4 classmates with varying degrees of experience of having a dog and prior knowledge of dog’s body language as the proxies for the learners.

Instruments:
  • Google docs to take notes

  • Mural board to organize notes

Result

The tutorial is effective but could be more fun and engaging:

  1. Participants who finished tutorials first can successfully complete the tasks in the game. 

  2. Participants suggest the content could be chunked, or there could be less text and more videos. They felt overwhelmed and lost patience when they read through all the content. 

  3. There could be more feedback.

 

The game is effective and engaging, but with minor omissions in terms of usability.

  1. The game flow is smooth and instruction is clear.

  2. The icons of the backpack, magnifier, and hints on the “Congratulations” page are confusing.

  3. The user experience of observation of the dog's body parts could be more smooth.

  4. The feedback on the wrong answer in the game could be more clear. Instead of only providing a link of the tutorial content they can refer back to, there could be more actionable feedback or more granular information that explains the scenario.  

 

The connection between the tutorials and game:

  1. The format of both tutorials and game is similar as they are both scenario-based questions and tasks. Participants suggest the game could be more interactive and given more autonomy to explore the “Dog City” to differentiate from the tutorials.

  2. The tutorial and game could be on the same platform.

  3. The visual is consistent. 

Conclusion And Recommendation

Our goal

For the tutorials and games, we need to find the balance between effectiveness and engagement, a.k.a. user-centered design and learning design.

 

Tutorials:

  1. The intro part could be replaced with a video that hooks learners first. The scenario with a lot of text might have learners lose interest.

  2. While learners prefer videos as they are more engaging, the text is particularly good at handling the linear structure of information so we use it to explain the procedure of reading a dog’s body language. The text is effective in this way but it is also overwhelming for learners without much prior knowledge. We can add videos first and then use text or interactive widgets to help learners review the procedure.

  3. Content needs to be chunked.

  4. Provide more autonomy for learners with different levels of prior knowledge.

 

Games:

  1. We will correct obvious errors and omissions regarding usability. 

  2. Provide more actionable feedback on wrong answers. 

 

The connection between tutorials and games:

  1. In the future, we will keep both tutorials and games on the same platform.

  2. We will try to make the game more interactive with wide choices.

                                    Rationale

 

The Kemp Design Model (Morrison et al, 2004) ’s circular structure allows us a significant degree of flexibility, as there is no required order to consider the components, meanwhile, it consists of most of the elements across our milestones. As this model is pretty flexible, we can address some phases simultaneously, do some revisions or even skip some phases. We believe that this model can make our design process open to revisions and adjustments.

Applications

Instructional Problems:

We did primary and secondary research to emphasize our learners to find out the main problems dog lovers face. We further narrow the topic down to teaching learners how to dog’s body language. Then we interview SMEs to know about learners’ misconceptions and the knowledge gap and how to effectively intervene.

Task Analysis:

We referred to online learning contents and made a flow chart regarding how to identify the dog's current state such as behavioral and health problems through reading dog's body parts and postures.

Learner Characteristics

We distributed surveys to identify learners’ preference, prior knowledge, knowledge gap, etc. We made two personas and a storyboard to further emphasize with them. Our learners are mainly dog lovers, with different experiences of having a dog, but all of them want to have a good relationship with dogs.

Instructional Objectives:

Then we go back to the predetermined instructional goals to further define learning objectives. We mainly focus on helping learners to prevent dog’s behavioral and health problems through reading dog’s body language. Learners will be able to have a good attitude towards their dogs while establishing good communication with them.

Content Sequencing:

To show how certain learning activities affect behaviors and help learners achieve the goal, we create logic model to guide our design. Then we outline the course content and sequence learning modules by interviewing SMEs and referring to learning materials such as videos, courses and books.

Instructional Delivery:

Based on instructional strategies, we created learning modules in Articulate Rise 360, and prototype the interactive game in Figma.

Instructional Strategies:

We come up with three design ideas such as tutorials, interactive games and discussion forums based on design implications and learning theory, and the implemented technology affordance.

Evaluation Instruments:

We conduct the formative evaluation by creating an evaluation plan and writing an evaluation report based on participants’ feedback. During evaluation, we mainly focus on the usability and learning outcomes as we want to make your learning design both engaging and effective.

The Kemp Design Model

截屏2022-05-14 13.37.34.png

  Reference

Morrison, G. R., Ross, S. J., Morrison, J. R., & Kalman, H. K. (2004). Designing effective instruction. John Wiley & Sons.

Project management

Screen Shot 2022-05-12 at 3.08.23 PM.png

Our group members regularly meet on zoom once or twice a week, and during each meeting, we share what we have done and assign the next tasks to each team member.

Reflection

YUhan

  • Instructional design is a quite new field to me this challenging project really helped me to break down the complex learning design process into smaller chunks. During the development of the prototype, I realized that making deliberate choices about what to teach and how learners will perceive each step is necessary to support the overall learning experience. 

  • I have learned that evaluation is important for continuous improvement and assists us to understand the gap between theory and real practice. For example, there might be a gap between learners’ expectations and our design model since some learners mentioned the interactive widgets with images and texts may be confused. Therefore, I think it is important for instructional designers to balance the interactive experience with learners’ cognitive process. 

  • Within the limited time, our final learning design model may still have more space to improve. However, I am very delighted to see the consistency of our design visuals and our effort in each learning design phase. Working together allows us to develop and maximize our strengths in this project. 

Kemeng

  • I think during the prototyping phase, I found that sometimes it’s not easy to visualize all the ideas, also, design an engaging game but at the same time, players can learn from the game is also a challenge for us.

  • I think the evaluation is very important for us to figure out the usability of our prototype, the learning experience as well as the flow and engagement of our game. Evaluations help us determine which functions in out prototype are effective and which can be improved.

  • I think our group made a complete prototype in a limited amount of time, and although there is still much room for improvement and refinement, we did a good job of using the prototype to showcase our ideas. In the future development, I think we can focus on how to make this game more engaging and interesting, also, continue developing our teaching tutorial based on our SME’s advice.

Yi Lu

  • I think there might always be a gap between design ideas and what we finally developed. For example, creating videos would require a lot of money and work. So at this time, we changed the format of the tutorial a little bit, with more interactive widgets in Rise but less videos as we imagined before. 

  • We need to find the balance between user-centered design and learning design. Videos are engaging and also more preferred by our target audience, but they are not necessarily effective when it comes to learning the process of reading dog’s body language. The interactive widgets with more text and images will make learners lose patience but they can reinforce their understanding. The gist of learning design is to balance learners’ germane load and extraneous load. 

  • I came to realize that we don’t necessarily need to polish the project and make it perfect, as it is just a student project with limited time, resources and experience. What is important is that everyone is learning.

Tianyu yang

  • As the first time I came into contact with instructional design, it was a very big challenge. I went through a lot of steps in the design process, which made me understand the tediousness of the design process more deeply, and also made me realize that continuing integration between content and learners to design a learning experience that supports the whole.

  • In the whole process of learning design, I think that continuous evaluation tests have given me a deep understanding of the differences between theory and reality. We need to continuously interview and understand learners, and summarize them through big data to solve the problems that arise. questions, balancing learning interaction and learner cognition.

  • In a very limited time, I think there is a lot of room for improvement in our final learning design. But I am very happy that all the team members have the same idea and the design effect is very consistent with our original idea. All of us put a lot of effort into the whole process, which also allowed us to maximize our strengths in this project.

bottom of page