👩‍💻Lesson 5 - Mouse Theremin: UX Design

How can we make changes to our project to improve the experience for the user?

Overview

This lesson will have students examine the simple mouse theremin that they made in the previous lesson by analyzing data collected about the user’s experience while interacting with this instrument. They will then consider how to make changes/modifications to improve the user experience.

Lesson Objectives

Students will be able to:

  • Articulate specific feature modifications needed for their project based on user feedback

  • Identify how a part of their code connects to a specific functionality or output of the overall project

  • Design coding solutions needed to make feature modifications

  • Add visual elements to their project which correspond to the specific audio output.

Suggested Duration

1-3 periods (45 - 135 minutes) - Depending on how far into the design process you choose to go

NYS Standards

9-12.IC.7 Investigate the use of computer science in multiple fields.

9-12.CT.2 Collect and evaluate data from multiple sources for use in a computational artifact.

9-12.CT.5 Modify a function or procedure in a program to perform its computation in a different way over the same inputs, while preserving the result of the overall program.

9-12.CT.9 Systematically test and refine programs using a range of test cases, based on anticipating common errors and user behavior.

9-12.DL.1 Type proficiently on a keyboard.

9-12.DL.2 Communicate and work collaboratively with others using digital tools to support individual learning and contribute to the learning of others.

Vocabulary

User Experience (UX) Design: The design of the elements that determine the interaction a user has with a product or service

Planning Notes

Main Objective

The objective for this lesson is to create an instrument which is functional, intuitive and built with care and accessibility in mind. We want to consider that we are trying to replicate the experience of playing a real instrument (Theremin) within the constraints of the coding environment we are using. We want to try and view our project through the lens of this objective and focus on how we can modify it to make the user experience align more closely to our goal.

UX Design

The process we will engage in with this lesson is called UX Design which stands for User Experience. UX is a process of creating things (products, services etc) that provides a meaningful experience for the user. Making something that is easy to use and understand can make for a more pleasant and efficient time for the user. Inversely, a user may not be able to extract the most value possible out of something that is not user friendly.

The UX design process involves a mixture of creative arts and scientific reasoning. You are designing and building something that requires you to use your creativity but a lot of the choices you are making are informed by research, data and best practices.

You can choose where you prefer to introduce this concept into the lesson. You could introduce this in the beginning of the lesson ("The process we are going to go through is called UX Design.....") and discuss what that means or you can simply frame this lesson as an exercise where you will be taking user feedback and using it to make your instrument better and easier for people to use. Once students have gone through the process, you can use the wrap up time to explain that this is a process called UX design which many people do as a career.

User Feedback

A big part of the process (as the name suggests) is getting input from the user. Ideally, we want to get input from people who were not involved in the designing and building process of our project. There can be a big difference between how a designer interacts with their project and how a user interacts with their project. You can gain so much insight about your project by seeing how someone else uses it.

Therefore, to try and make this a meaningful and authentic experience, it would be beneficial to have people who were not involved in the process of creating the initial version of this project interact with it and provide feedback on their experience. See the Intro to UX Design section of the previous lesson for more on how you can do this.

Lesson Structure

This lesson has been broken up into steps that take you through the UX process. It is not necessary that you complete all these steps in order for students to have a meaningful experience engaging with this design process. Each step can allow students to demonstrate their knowledge and understanding. How far you take students through these steps should depend on your student’s abilities, your own comfort level in leading students through this process, time constraints etc.

These steps are meant to mirror the different roles involved in product and software design.

  1. Project Manager: Have students identify the issues based on an analysis of the feedback which was collected from your users and develop a to-do list of changes/modifications that they want to implement for this project. This by itself can be an acceptable deliverable for this assignment.

  2. Designer: Have students plan out how they would implement the specific modifications for the items on the to-do list. You could choose to have students focus on just one item or all items. You could have students choose which item they want to plan for or you can assign items based on student ability level. This does not have to include the exact coding solution needed. It could be a written explanation or pseudocode.

  3. Engineer: Have students execute their plan for implementing the change in the project by modifying the code.

You can choose to incorporate these into your lesson as roles for students doing group work, different phases in the lesson itself or however you might see it to be beneficial.

Modeling Solutions

You can determine the degree to which you want to model this process for the students.

For example, you could just have students pick an item and try to plan a solution, design how it will be implemented and then execute that in code all on their own.

OR

You could go through this process more collaboratively as a class. Model the steps in the process including how to come up with a possible solution, take suggestions on how this change could be implemented and then have students execute that change in their code and reflect on the results as a class.

From here, you can choose the next steps for students to take more independence. You could help students come up with a possible solution, but have them come up with how to implement that solution in their code. For the next item, have them complete all the steps themselves.

OR

You could give them another issue and turn them loose to find a solution and implement it on their own. This approach can also lend itself well to differentiation to accommodate the different ability levels that may be present in your class.

Multiple Solutions

Something else to consider is that there is always more than one solution to a problem that you identify. Think about it: if there was only one solution to each problem, there would be no need for UX design. You can choose how you want your students to work through this aspect of the process. You could take the time to discuss the pros and cons of different designs as a class before implementing them or you could have different groups of students try to implement different solutions and then compare them side by side afterwards. This approach could also be another opportunity to gather more user feedback. You could have an additional group of students working on a separate issue so they can test these features with a more unbiased opinion.

Any of these methods can provide a meaningful experience in UX design. It will be up to you to choose how you want to do this in your class as well as what/how you want to assess the work the students do.

Accessibility

This lesson does look at incorporating corresponding visuals for accessibility purposes. However, sequentially in this lesson plan, it comes later. It is possible that during the initial brainstorming session, the students may bring up that we did not begin this project with Care in mind in regards to including accessibility from the inception of the project as was discussed in a previous lesson. This would be an excellent observation and a sign that discussing accessibility has already gotten students to consider it more within their own projects. You can choose to have students try to consider the accessibility part first and modify their creative choices as more issues are tackled. However, the other design aspects we want to look at also take Care into consideration and could have an affect on the types of visuals we want to incorporate. You can inform students of this and make the visual design choices after these other issues have been addressed. This demonstrates that we have considered accessibility from the beginning, but have made a conscious choice on when to start implementing it into our design. If you are allowing students to take more ownership over the path and sequence they choose to implement these changes, it may be interesting to have students discuss and reflect on when they started incorporating the accessibility part of the design, how that process went and what were the pros/cons of trying to implement the accessibility into the design at that point.

Materials

N/A

Resources

Video: What is UX Design: An Introduction

Assessments

The types of assessments you may choose for this lesson will depend on what you are having students do. For example, if you are just focusing on creating a To Do list which identifies issues based on user feedback, your assessments will look very different from students writing code to try and implement a feature modification.

Students should be asked to document the steps in their planning process regardless of how far they go trying to implement these features in their projects. This documentation can allow them to reflect on the process later on. It can also be used as an artifact on which you can choose to based part of their grade.

New Code in this lesson

Since this lesson has multiple sections and is meant to be adapted and structured to meet the needs and logistics of your individual classroom, all new code that is introduced is covered in the specific section where that code is applicable.

Do Now/Warm Up

Use of the Do Now time should be determined on how you are approaching this lesson in regards to obtaining feedback about the initial design of the instrument.

If you have solicited outside feedback, you can use the Do Now time to examine the responses to the questions about user experience from the participants, look for common trends in the responses and determine how this can inform your decisions for what changes to make in the instrument design. One way to do this could be in the form of the prompt: "What do you notice? What do you wonder?"

If you are using feedback from the students in class, you could take time during the Do Now to have them engage with the instrument as a first time user and record their responses to the guiding questions or you could have students do this in the preceding class and use this time to analyze their responses.

While looking at the responses, once you have identified an issue or area of improvement, provide students the opportunity to first consider what could be done to improve this issue in a simple explanation. From there, you can start to consider how this could be implemented with code. You could choose to create a list of issues, solutions and coding implementation first and tackle each one or you could start by identifying one and modeling the design process for the class.

Issue - Sound constantly plays - No way to stop or start

As you have probably already noticed after working with the oscillator object in class, the constant sound emitted from the sketch once it has started can get irritating very quickly. It is important that we provide the user with the ability to turn the sound on when they want it and off when they don't want it.

One solution that is likely to come up when considering solutions is to add some sort button that can be pressed to turn the sound on or off.

You can take this opportunity to think about the logistics of this solution for how it would be implemented in your code. For example, is there only one button that works for both on and off? Is there an on button and a separate off button? Where would you place the button? Consider the functionality for the user. Since we need to move the mouse to get to the button, the pitch and volume will change every time we go to change the button. This means the user can never start or stop the theremin on a specific pitch or volume.

A more straight forward approach would be to use the mouse button to control when the sound goes on and off. This can be done with the mousePressed( ) and mouseReleased( ) functions.

function mousePressed(){
    osc.start();
}

function mouseReleased(){
    osc.stop();
}

Discuss some of the benefits of this solution.

  • The default setting is OFF, so we only hear sound when the user chooses to make sound

  • There is no need to move the location of the mouse to turn the sound on or off

  • The user has the ability to play two separate pitches without needing to hear the frequencies in between.

Issue - Amplitude gets louder when going mouse goes down

Something that should stick out in the original design for this project is correlation between the movement of the mouse on the Y axis and the change in volume. The way that our project is designed now, as we move the mouse further down the canvas, the louder the volume gets. Considering how the p5.js canvas is laid out, this makes perfect sense: the further down the canvas we go, the higher the Y axis value is. However, when we consider this layout within the context of the theremin, it doesn't make sense. As students have seen in the video about how the theremin works, as the hand that controls the volume moves higher, the volume increases. Beyond this, we can also consider other real world product designs that control volume. Have students think about their phones or the volume bar on their TV. You don't press the down button on your phone when you want to make it louder. The volume bar on the TV doesn't get lower as you increase the volume. Sometimes when making a design choice, the best choice is to mimic a pattern that people are already familiar with.

The solution to this problem lies in the map( ) function. If students are struggling to find this solution, you can ask them if there is anything in the documentation for the map( ) function that says the arguments we provide for a range of numbers have to start with a low number and end with a high number. This should lead them to realize that they can just switch one of the ranges to start with the higher value and end with the lower value.

let vol = map(mouseY, 0, height, 1, 0);

OR

let vol = map(mouseY, height, 0, 0, 1);

Issue - Frequency continues to change once you have moved beyond the width of the canvas

In an earlier lesson, we have already discussed how certain high pitched frequencies can be painful for the listener. This led us to use Care in our choice for setting the maximum value for the theremin pitch when we used the map( ) function. However, even though we have set the range of the pitch to hit the maximum value when we get to the right edge of the canvas, it is clear that the pitch continues to go up even when the mouse has gone past the edge of the canvas.

There are multiple coding solutions that can be used to solve this problem. You can choose to just implement one or discuss the pros and cons of each one. Two solutions involve modifying the code we already have to work the way it is intended while a third solution allows students to re-imagine what the final result of their project can look like.

Solution #1 - withinBounds: optional parameter to map( ) function

When looking at the documentation of the map( ) function, we know that there are five parameters we need to provide, but there is also a sixth, optional parameter called withinBounds. The explanation for this parameter in the documentation says "constrain the value to the newly mapped range" This parameter is, therefore, seemingly custom made to solve the issue we are having. It will make sure that the newly mapped value will never go outside of the range of values we have provided. All that is needed is to provide a boolean value of true as a sixth argument in the map function!

let pitch = map(mouseX, 0, width, 200, 800, true);

Solution #2 - constrain( ) function

There is another way that this issue could be approached which includes using the constrain( ) function. Exploring how to execute this solution may seem unnecessary since the map( ) function already includes a parameter that easily solves the problem, however it could be a good exercise for more advanced students to try and figure out by looking at the documentation for this function.

You could provide students with prompts to help guide them as they look at the documentation for the constrain( ) function. For example:

  • How many parameters does the constrain function take?

  • What are the parameters we want to pass to this argument?

  • Explain what each parameter is for

The next step would be for students to consider where in their code this function should go –

Things to consider:

  • Does this function return a value?

  • Should we assign the return value to a variable?

  • What should we do with constrained value?

  • Where in our code should we put this function?

  • Does it require other functions to be executed first?

 let pitch = map(mouseX, 0, width, 200, 800);
  pitch = constrain(pitch, 200, 800); // constrain pitch variable value between 200 and 800
  osc.freq(pitch);

Once they have figured out how it works, you could also challenge them to try and find another situation where you could use the constrain function outside of mapping a range of values.

Solution #3 - Resize the canvas size to fit the entire computer screen

Even if students are successfully able to implement a fix that constrains the range of the frequency to within the canvas, have them think about how this design will be perceived by the user. Do they even know what the canvas is? Will it be frustrating to move the mouse outside the canvas and not hear any change in volume or pitch when they are expecting that to happen? Do we need to include some type of label or instructions to make them aware of this? How would we implement that? How would that impact the overall design? Emphasize how even though we seemingly found a solution, it raises new questions and issues with our design.

If we think back to the theremin website that students played around with in the previous lesson, we'll remember that there were no issues of going outside of the canvas because the we had the entirety of the computer screen to move around on. This gave the user a lot more room to move around and provided more fine-tuned control over the pitch. It also makes the experience more straight-forward as there is no confusion about where the area to move the mouse begins and ends.

There is a straightforward solution to solve this problem. Just like there is a built in system variable that holds the value of the width and height of the canvas, there are variables which also hold the value of the size of the window width and window height. When we use these variables for the arguments of the createCanvas function, it will resize the canvas to fit the full size of the screen. As a result, the width and height variables that we use in the map( ) function will also correspond to the size of the full screen and allow the user to move the mouse across the entire screen to control the theremin.

function setup(){
createCanvas(windowWidth, windowHeight);
}

Be aware that in the edit mode of the web editor, it will only take up the size of the space next to the text editor. However, when we share the project using the full screen link, the canvas will take up the space of the entire computer screen.

To share a project in full screen, go to file -> share then select the link for full screen. You can paste this link into another window or click the go to button to automatically open the project in another tab.

Accessibility

Since we are working with sound, the focus should be for making this project more accessible for the d/Deaf and hard of hearing users.

There are countless ways that students can choose to add visuals to this project to go with the sounds produced by the oscillator, so you should not provide specific step by step instructions for how to go about doing this. The main thing to consider is how students’ choices of visuals will correspond to the two characteristics of sound that will be changed by the theremin: volume and pitch. For example, the change in volume could be reflected in the visual by a change in size or height of a shape. A change in pitch could be reflected by a change in color of the shape or background.

Example - Mouse Theremin w/ Visual

Since we want to connect the changes in visuals to the changes in sound, we need to keep in mind that all of the changes in our project are affected by the movement of the mouse on the X and Y axis which is stored in the mouseX and mouseY variables. We used the map( ) function to convert the values of the mouseX and mouseY variables into ranges of values that are appropriate for the changes in amplitude and frequency that we want to hear from our theremin. Therefore, we should also assume that we can use the map( ) function to convert the mouseX and mouseY variables into ranges of values that are relevant for whatever changes we want to happen in the visuals we create.

If students are considering mapping changes in color, have them consider using the HSB color mode instead of RGB. This will allow them to have a single value which will affect the Hue of the color (a value between 0 and 360) instead of 3 separate values for RGB. This will require students to use the colorMode(HSB) function in setup. Students could also consider changing the saturation or brightness to reflect a change in either of the pitch or volume.

function setup(){
createCanvas(400, 400);
colorMode(HSB); // set color to Hue, Saturation, Brightness
}

Example - Using HSB colorMode to correspond to changes in pitch

However, we can also consider that in some cases, the range of values for mouseX or mouseY could also be the values we want for a visual. For example, if we want to have a rectangle move from the bottom to the top of the canvas to shows the changes in volume, we could just use the value of the mouseY variable (although we may need to do some work to invert the values just as we do for mapping the volume so it gets louder as the mouse moves upwards on the canvas.)

Example - Rectangle changing size to correspond to changes in volume

Whatever students decide to do, they should consider whether it would still be engaging for a user if they were unable to experience the sound portion of the project.

Wrap Up

Reflection

Have students take time to reflect on how this process went. Possible questions you can ask them:

  • How did reading user feedback change how you viewed your project?

  • What step of the design process did you find the most enjoyable? Why?

  • What step of the design process did you find the most difficult? Why?

  • How might being aware of this type of design process affect the way you appraoch making future projects?

Real World Application

If you did not choose to introduce this lesson as being related to UX Design, this wrap up time would be beneficial to make students aware that the process they just went through is one that is used for designing countless products and something that many people do as a career.

UX Design is a very broad field in which there are several types of careers which fall under that label. You can take this time to have students explore the different job titles that people can have in the field of UX Design. They could examine the skills required for each one and compare/contrast those with another title.

See these resources for more about different job titles in the field of UX Design:

Extensions

There could be certain ideas/features/modifications that students come up with that go beyond the scope of this lesson. It is up to you to determine how open ended you want to make this lesson based on things like class time, student ability level, your comfort level of being able to address and teach unforeseen concepts etc. If you feel you have students who are capable, you can certainly give them space to try and add their own feature ideas to this project.

PreviousLesson 4 - Mouse Theremin: CodeNextLesson 6 - For Loops && Conditional Statements

Last updated