Design for the Senses

To create an immersive experience, the successful games design for more than one sense.
The more senses you want to design for, and the more immersive experience you want to create, the more planning you have to do. Because using the senses of taste and smell in games is not mainstream yet, we’ll focus on the other three senses: touch, sound, and sight.

Touch

Most console games use a controller that has a set number of buttons and analog joysticks. Because interaction through these elements isn’t as fast or precise as through a mouse or direct touch, you should understand the limitations of your controller device.

The controller’s layout can make or break the players’ experiences within a game. Design your controls to feel as natural as possible. One of the first things to consider is how each button will be used in the gameplay.

Console games have been around for decades, so a number of interactions have become standard and are even expected by players today.

Microsoft’s Xbox One

Menu navigation controls of the Xbox One controller

• A: enter, select, confirm
• B: back, cancel
• X: details
• Y: details
• Menu: menu toggle
• View: map or chat
• D-Pad: navigation
• Left Stick: navigation
• Right Stick: text or list scroll
• Left Bumper: sub-menu, left navigation
• Right Bumper: sub-menu, right navigation
• Left Trigger: page-up navigation
• Right Trigger: page-down navigation
• Nexus: Xbox Home

Sony’s Playstation Controller

The layouts of the PlayStation and Xbox controllers are very similar. One major exception is the touchpad, which was introduced in PlayStation 4 and offers additional opportunities for input, such as touch and gesture control.
Menu navigation controls of the PS4 controller

• CROSS: enter, select, confirm
• CIRCLE: back, cancel
• SQUARE: details
• TRIANGLE: details
• Options: pause menu, map
• Share: share
• D-Pad: navigation
• Left Analog: navigation
• Right Analog: text or list scroll
• L1 Bumper: sub-menu, left navigation
• R1 Bumper: sub-menu, right navigation
• L2 Trigger: page-up navigation
• R2 Trigger: page-down navigation
• PS Home: PS Home
• Touch Pad: chat keyboard, directional cursor, map

Nintendo’s Switch Controller

The Switch controller is similar to the Xbox and PlayStation controllers but can also be split into two controllers that have a similar but limited layout.
Menu navigation controls of the Switch controller

• B: enter, select, confirm
• A: back, cancel
• X: details
• Y: details
• - Button: text/list scroll
• + Button: navigation
• Directional Buttons: navigation
• Left Stick: navigation
• Right Stick: text or list scroll
• L Button: sub-menu, left navigation
• R Button: sub-menu, right navigation
• ZL Trigger: page-up navigation
• ZR Trigger: page-down navigation
• Home: Nintendo Home
• Capture Button: map/chat

Microsoft’s Xbox Adaptive Controller

Designed primarily to meet the needs of gamers with limited mobility, the Xbox Adaptive Controller is a unified hub for devices and is designed to make gaming more accessible. It works with a range of external devices (such as switches, buttons, mounts, and joysticks) to create a customizable controller experience unique to a player.

Layouts of controllers

Different layouts of controllers have been designed to suit different genres of games, and in some genres, several layouts of controllers are used. Use the following common, genre-specific layouts as a guide in your game design:

Mouse and keyboard

Interaction through the mouse is similar to interaction through touch, in that the user can perform any action with pinpoint accuracy, on any part of the screen. The keyboard is a powerful tool in its own way, because it offers a wide variety of button and key inputs (and an even greater variety of combinations) for communication and navigation.

Converting the Mouse & Keyboard to the Controller

Many games are first developed for a mouse and keyboard, which offer many buttons and are relatively easy to implement. The mouse also makes it easy to master the skill of moving and aiming the camera with pinpoint accuracy. In combination, the mouse and keyboard allow for a great development platform on which new features of a game can be tested out. However, converting complex interactions that are possible with a mouse and keyboard to interactions with a controller can be difficult, because the latter has a fixed number of button inputs. In addition, an analog joystick is not as fast or accurate as the mouse. The following are some of the common mouse and keyboard input designs adapted for a controller:

Tip: Use the face buttons to test your core gameplay action. Use the shoulder and trigger buttons to modify actions. Use the D-Pad to select modes and abilities.

Sound

Up to 50% of UX comes from sound, yet many games do not utilize sound as effectively as they could. Watching a favorite movie without sound would quickly convince you that the impact is not quite the same.

Sound design for a game is often an afterthought and implemented at the very end of production, with minimal resources and budget dedicated for proper sound production. Instead, sound design should be taken as seriously as any other element in production.

Use sound when it is most impactful to gameplay, such as setting a mood or drawing attention to important information on the screen.

Here are a few examples:

• A player uses the frontend menu to enter a Microsoft Store page. Rather than having the user transition through all the elements without sound, use a bell or a subtle ring at the end of each transition, and highlight new items for sale. Not only would the sound draw attention to a new item on the screen, but it would also teach the user that the sound of a bell means a new feature is available in the store.

• A player uses the HUD to look down the scope of a gun. To increase the feeling of tension, trigger the sound of a heartbeat while the player is trying to focus on a target. In a sports game such as basketball, trigger the same heartbeat sound while the player is about to take that critical last shot, with seconds remaining in the game.

The sound effects illustrated in these examples do not stop gameplay; in fact, they make the experience much more immersive.

Sight

Sight is the sense that video games exploit most extensively. Visuals set the mood and help tell a story. Take advantage of the composition theory to come up with imagery that can help you get ahead of your competition.

Dead Space took traditional flat UI and unified gameplay with narrative through immersive, diegetic presentation of the UI. The 3rd person perspective allows the display of health and ammo information on the character’s back and weapon which can be seen by both the character and player. This opens up the entire gameplay view for a completely immersive experience allowing players to take in the survival horror atmosphere. The high contast color palette between UI and game world also provides clarity and readability. Dead Space’s pioneering use of diegeic UI has become a source of inspiration for the industry.

An example from DeadSpace

This image also works in black and white. The features remain clear and UI is legible.

Tip: Do not overload the player with information and visuals.

At the other end of the spectrum are images like the one shown below. The image is vibrant and saturated with color, and it has interesting elements. However, there is too much to look at, and no obvious resting point. The impact is not as great as it could have been.