SXSW 2012: Teaching Touch: Tapworthy Touchscreen Design

Josh Clark
Principal, Global Moxie


Presentation Description

Discover the rules of thumb for finger-friendly design. Touch gestures are sweeping away buttons, menus and windows from mobile devices—and even from the next version of Windows. Find out why those familiar desktop widgets are weak replacements for manipulating content directly, and learn to craft touchscreen interfaces that effortlessly teach users new gesture vocabularies.

The challenge: gestures are invisible, without the visual cues offered by buttons and menus. As your touchscreen app sheds buttons, how do people figure out how to use the damn thing? Learn to lead your audience by the hand (and fingers) with practical techniques that make invisible gestures obvious. Designer Josh Clark (author of O’Reilly books “Tapworthy” and “Best iPhone Apps”) mines a variety of surprising sources for interface inspiration and design patterns. Along the way, discover the subtle power of animation, why you should be playing lots more video games, and why a toddler is your best beta tester.

Presentation Notes

The ability to interact directly with an object lowers the need for complexity. Designing for touchscreen is not only a complex for developers and designers, but also for consumers.

Fitts Law

The presenter, Josh, says he hates the iPad’s back button “with the heat of a million suns.” Fitts Law is a rule to test how long it takes a user to move an object to a target. The rule of thumb is the closer something is, the easier it is, but the further away the target it then the harder it is to hit. Although the buttons on an iPad are the same size as the buttons on an iPhone, it’s actually physically harder to hit.

The motto as a designer should be “Let people be lazy.” Why can’t people just hit a massive easy to hit button? When Apple released iOS5 they made it so you can easily swipe out the little drawer in your email instead of using the back button.

Gestures are the keyboard shortcuts of touch

Big screens invite big gestures. You don’t have to keep hitting that little button all of the time.

Buttons are an “inspired” hack

Even in the real world we have physical buttons and switches, such as light switch. A light switch when you enter a room turns on a light from a far distance away, and this type of intuition needs to be learned by the user. These types of “controls” add a layer of abstraction. We can think about interface design in the same way we do with real-life buttons and switches. With touch we now have an opportunity to close this gap. Designers need to start looking at new interface models of touch, facial recognition, etc, and ask themselves if we still need the “classic” way of doing things? Can these “classic” ways be replaced by these new interface models?

Whither the Web?

There are two things we need to make gestural interaction on the web plausible:

Real support for gestures fails because Javascript can do touchstart, touchend, etc, but it’s lacking built in pinching, rotating, etc.

Gesture conventions are not even defined well on apps, so how can we move it to the web yet? On the web the only gestures you really have to work with are “tap” and “swipe.” It’s also hard to come up with sophisticated gesture conventions on the web because from browser-to-browser what the interaction does may be overridden by it.

Both JQuery Mobile and Sencha Touch are adding functionality for additional features such as doubletap, drag, pinch, and rotate.

See also: Touchy.js

Good Examples:, Touch Up (changes brush size by zooming in/out because your finger doesn’t change size).

Finding What You Can’t See

So great, we’re making gesture stuff. How do users know these advanced (or even basic) gestures exist? Well, people should be able to figure out simple gestures they’re using to. For instance with Google Maps people figure out the double-click to zoom in because you can do that on the desktop app. But no one is ever going to figure out a two finger tap will zoom you out.

A lot of apps will make you look at a screen involving all of the various gestures you can use. It’s like a massive complex user’s manual, and you haven’t even seen the app yet. It makes the app sound like it is much harder than it actually is. Upfront instruction manuals make your apps seem harder.

Nature Doesn’t Have Instructions

The best interfaces don’t need instructions. However, even nature took time to learn when we’re first born. Even Apple makes mistakes. Their Address book looks like a book you should be able to swipe. However, when you swipe you actually delete content and you don’t switch to the next page. The fact Apple hasn’t even gotten it 100% right yet just goes to show that it’s very difficult to get it right, and no one quite has yet.

Love the one you’re with

If it looks like a physical object, people are going to try to make it work like one. The interface should be the instructions to use it. Although digital newspapers are nice, and they’re just basically a PDF, don’t neglect to add what the digital advantage can give us: table of contents.

The iPad is the awesome love child of many parents

Watch how toddlers use an iPad. It’s amazing how quickly toddlers get it. The wont get your multi-level menu system, but neither will your adult users. People who have very limited computing experiences (elders and children) seem to figure out these types of devices pretty quickly.

Play more video games

Many times when you start up a video game, you don’t even know what your goals are. Games teach you while you go and they bring you along from novice to expert to master. So how do we do this?

  1. Coaching: simple demonstrations such as prompts. Pointing things out as you go. You learn while you’re doing it. You don’t learn how to play a piano through a manual, you learn it through practice. Gmail does this with little popups explaining new features and information for more. But don’t be like Microsoft’s clippy and pop up at inconvenient times. A suitcase without a handle is useless. A gesture without a visual aid is the same way. You have to provide visual queues.
  2. Leveling up: Once a user engages one of your features, you offer to teach them more. Users are often most engages when they first try something. OSX Lion does this in that you have to scroll the window down to get to the Continue button, essentially learning the new way it scrolls.
  3. Power ups: Give a shortcut or advantage. Perhaps show the user how to get to the spot they’re trying to get to faster after doing it for the 10th time.

This is a time to be generous, so share the knowledge of how this new platform of touch should interact. Throw out ideas, reasons why things work, and why things don’t. It’s exciting!

SXSW 2012: Designing for Context

Andrew Crow
VP, Experience Design, GE

Ben Fullerton
Director, User Experience, Method

Leah Buley
Design Strategist, Intuit

Nate Bolt
Pres, Bolt|Peters

Ryan Freitas
Co-founder, AOL/

Presentation Description

As designers take on new problems of convergence and ubiquity, we find ourselves facing new challenges. The products we create are accessed through multiple devices, different channels and a wide audience. How do we accommodate the context of use?

Whether you design mobile apps, services or web experiences, you know that people have different needs and desires. Those issues are complicated further by a landscape of technology.

This discussion will highlight these new challenges and offer solutions based on years of design experience. Topics include:

  • What should you be aware of when designing a product or service for use in various locations and environments?
  • How does motion and distraction affect interaction and content design decisions?
  • Do you provide for casual use vs. urgent need?
  • How does the form factor or input method of your device steer your design efforts?
  • What happens in an ecosystem of products?
  • How does social and cultural context play into the strategy of your design?

Twitter Hash Tag: #DforC

Presentation Notes

In 1995 the Decision Theory and Adaptive Systems Group was created at Microsoft (Dr. Eric Horvitz) who thought they found out exactly when a user would become frustrated, hence the paperclip, which had very poor acceptance and was hated.

Context is the situation people are in when using products. These can be based on context, time, or the circumstances when the user is using the product.

What should we be aware of when designing for Context?


  • Designing interactions for different lengths or instances of time.
  • People are doing other things while using your products. Not only are people doing other things while using your product, they’re also turning your product off (or not using it) and then doing other things in the “real world” and then later come back to it (in-progress tasks).
  • Right “now” is an important time in which it is the best time to survey someone in how they are using your product, and what they’re doing while using it.
  • The context of time can change. For instance, what are the things that have to happen when you pull out your phone in a grocery line real quick compared to the things you have to do when you have hours to do something? What things are possible? What things are not possible?


  • Prioritizing one platform (such as developing a beautiful app on the iPad) and then making the other devices and website carry through the beautiful aspects of it.


  • How do we accommodate and embrace various locations?
  • If you’re designing an app that is used when you’re outside, make sure it isn’t dark as it’s hard to see with sunlight.

Form & Technology

  • What about screen size, input methods, technical constraints?
  • Intuit has their app SnapTax which allows you to take a photo of your W2 and it automatically populates the fields of your W2. However, and ironically, doing this via a photo takes actually longer than it does to just key-in your values. This aspect actually drives people to the app.

Brand and Relationships

  • How you feel about a brand is going to affect how you feel about their products and services.

What have we learned?

If you’re going to extend your existing product, make sure you break out the pieces of the product you really want and what are most important (1:1 mapping). You have to do the research, design well, and understand your audience.

Adopt a service design mentality to understand where people are intersecting on common needs.

Exposing the matrix. What do we know when we add and look at all the aspects? Well, just matrix those ideas out to find what works best.

Use research that doesn’t take a lot of time or money.

SXSW 2011: Hacking RSS: Filtering & Processing Obscene Amounts of Information

Dawn Foster
MeeGo Community Manager

Information Overload

There is an obseen amount of data in the world we live in today. Right now we have a mass of 600+ Exabytes of data today (1 Exabyte = 1,073,741,824 Gigabytes).

Most of this information is…

  • Complete Crap
  • Out of Date / Obsolete
  • Not Relevant

So, what techniques can you use to find the information you want?

RSS is a start. Sources you care about delivered right to you, but do you care about everything in each feed? What about feeds you do not subscribe to? Can you keep up with what you have?

Prioritizing your reader

  • Put things you care about at the top
  • Categorize
  • Don’t try to read everything

Outsource / Crowdsource New Sources

The Real Magic is Filtering RSS

  • PostRank: Finds the best posts in a feed ranked on engagement (links, sharing, comments). You can then get an output as an RSS feed, and the feed includes the postrank number as a field.
  • Yahoo! Pipes: Allows you to filter based on any field in the RSS file, not just title and description. The downside is it takes a long time to learn and muddle through.
  • Feed Rinse:Easy to use, not as flexible. Import RSS feeds, and filters, then get new RSS feeds out.
  • BackTweets lets you search Twitter based on a URL regardless of it’s short link.
  • …and many more!

Things to use this with

  • Personal Productivity
  • Understanding the Possibilities
  • Creating prototypes for something you want to build

When not to use it

  • Don’t use in critical or production environments
  • Typically all of this can be done in most programming languages with caching and error checking


Dawn Foster’s Blog Post

SXSW 2011: Better Living Through Cloud Computing

Jon Wiley (@jonwiley)
Designer, Search

Cloud computing is still just computing as it’s simply manipulation of storage and data. Cloud computing isn’t really a new thing as it’s been around for quite a while. It’s really an ongoing evolution from the day the internet was born.

SalesForce: No client software was harmed (or used). Basically it’s all on the server, and no clients have the software on their machine.

Jon’s presentation was brought to us by the Cloud via the use of his Google Chrome Laptop, and the internet.

Examples of the cloud are: Google Voice, Dropbox, Evernote, Netflix Streaming, Kindle, Google Maps, Google Translate, Picnik, Aviary’s Phoenix, Pixlr, Mint, PayPal, ING Direct, JayCut, OnLive and OpenTable.

However, there are some limitations to Cloud Computing. The biggest limitation is bandwidth: typically the up-link speed is more important than the download speed. In the USA bandwidth speeds are quite slow. In South Korea they’re proposing having 1 GB/second by 2012. Google proposed their fiber-optics line for 50K-500K people.

The maximum human sensory badwidth is 100,000,000 bits per second. That means South Korea is almost at the speed of human brain power! Woah!

Cloud-augmented: taking previous objects that were not connected to super computers but enable them to do so. For example, the Eye-Fi SD card.

There are some risks/bad things with Cloud Computing:

  • Security: Passwords, Hacking, Phishing.
  • The Patriot Act requires only a supina to access your Cloud. However if it was stored on your physical computer they’d need a Search Warrant.
  • There’s no FDIC for your data.
  • What if the Cloud site closes? Your data is then no longer accessible.
  • What happens when you die with your passwords and data?

Other Links

Guillaume Nery base jumping at Dean’s Blue Hole, filmed on breath hold by Julie Gautier

SXSW 2011: Conserve Code: Storyboard Experiences with Customers First

Joseph O’Sullivan
Lead, Design Innovation

Rachel Evans
Principle Research Scientist, Chief Innovation Catalyst

Design Thinking

  • Deep Customer Empathy: knowing your customers more than they know themselves
  • Go Broad to go Narrow: if you’re going to get a great idea, you’ll need a portfolio of existing great ideas.
  • Rapid Experimentation with Customers


Storyboards are quick visual steps in how a user in a system executes. The first documentation in history of a storyboard was in 1930 in Hells Angels.

Intuit uses Storyboards in terms of:

  • Web Applications
  • Mobile Applications
  • Customer Care: even before the first phone call, or during it
  • Human Resources: their employees’ experiences throughout their job from the first day
  • Community Support

An Example

Snap Tax is an iPhone application to file your taxes with the ability to take a photo of your W2 and have it instantly populated, ask them additional tax questions, and then pay and file your taxes. When creating this application originally they made a Storyboard which gave them their estimated tax right off the bat with just their W2 and marital status. This Storyboard was also just 6 slides long, and very generalized and simple. If you show a customer a finished product they’re less likely to give you negative feedback because they feel bad. But if you give them a storyboard right off the bat then they’re more likely to. In the end they discovered customers didn’t care if they were getting a refund, or for how much, but rather or not the phone system would actually speed up their process. So instead of showing them their number right away they showed them how easy it was to enter your W2.

  • Pitty Begets Honesty: Customers tend to react to rough sketches more honestly than with finished click throughs.
  • Narcissus Antidote: We’ve never seen anyone fall in love with their storyboard and not change it.

Creating Storyboards

When you’re creating a Storyboard you’re just wondering what’s good or bad with your idea, and what works well with the customer. A Storyboard needs to be aligned with the customer’s problem, solve that it is a solution for their issues, and finally wow them. In every cell of a Storyboard there is something to learn:

  • Do you understand the problem? Is it an important problem?
  • Does your solution solve the problem completely?
  • Lastly, the benefit. What is good about your idea from the customer’s perspective? Will it delight them?

A 6-cell storyboard should have these cells: goal, problem, solution, solution, solution, benefit

In setting up a storyboard you should consider these things:

  • What’s the project?
  • Who’s the customer? Get specific: age, gender, experiences with similar products, etc.
  • What’s the problem? “I’m trying to __(goal)__ but __(problem)__.”
  • What’s the solution? Three most important moments that need to occur for the solution to execute
  • What’s the benefit? This isn’t the feature list. It’s what’s beneficial to the customer, and doesn’t include any of the words how you would describe the features to your boss.
  • Now that you have the customer problem, the solution, and the customer benefit, what do you want to learn about it? Your goal is to gather as much new feedback as possible. It’s what’s not clear to you about what you have completed so far.
  • Now it’s time to draw.