Don Rickert Design

What Does Ethnography Look Like?

Much of the value we add comes from years' of experience in observing REAL PEOPLE doing REAL THINGS in the REAL WORLD...this is often called Ethnography. The skill, experience and formal training for doing ethnography correctly is ALMOST NEVER found in-house; although, you may feel that it is. It would be an exceedingly rare organization that has qualified in-house ethnography expertise.  See our postings on Ethnography by clicking the Ethnography category on the right of the screen.

So what does Ethnography look like?: The short answer is that it looks like real life. The photos that follow with accompanying commentary may help to communicate the different variations of Ethnography.

The ‘Fuzzy Front End’ of New Product Development

The Front End of New Product Development is where:     

• Users and stakeholders are identified
• Personas (i.e. users) and Scenarios (i.e. what they do) are created
• Gaps between current products and user needs/desires are identified
• Potential market size and competitors are assessed
• The right questions for surveys and the right tasks for usability tests are identified
• The all-important prototyping, usability testing, depth interviews, appeal assessment and ideation occur
• The Product Concept is defined and refined
• The Market Definition is refined
• Pricing Analysis is done
• Technical feasibility is assessed

The result of the Front End of NPD is a well-refined Product Concept that is certified as having Appeal and Value to users—in other words: is useful, usable and desirable, as well as technically feasible.

In many ways, the most difficult part of NPD is the beginning, invariably called the ‘Front End’ in contemporary NPD literature, with variations such as the ‘Front End of Innovation’ or the ‘Fuzzy Front End.’ One reason for the difficulty is that Front End is inherently qualitative in nature. Except for those disposed to and trained in qualitative inquiry, doing work without hard-and-fast rules is extremely difficult.

The insights gained during a well-run NPD Front End process are absolutely essential input (either directly or indirectly) to so many other downstream NPD activities, that ignoring the Front End has the potential to doom a project to failure.

Patterns for UX Design

Patterns are formal descriptions of best practices within a given design domain. They capture common solutions to design tensions (usually called "forces" in the pattern literature). The idea of patterns actually started in the field of architecture (as in physical buildings). The first book (in the 1970s)
on patterns and pattern languages (collections of related patterns) was Christopher Alexander's A Pattern Language.

The “Alexandrian” format for documenting patterns is still very popular today. Patterns were introduced to the field of commercial software engineering in the mid-1990s by the publication of Design Patterns by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides.

In recent years, patterns have taken hold in the field of Human-Computer Interaction (HCI). A very good introduction to UI patterns is (Common Ground: A Pattern Language for Human-Computer Interface Design) by Jennifer Tidwell at MIT. There is a growing body of literature on the topic, as well as a number of peer- reviewed UI pattern websites, such as—

• UI Patterns and Techniques
• Interaction Design Patterns

Some Important Points About Design Patterns

UI design patterns do not separate system behavior from “look”. The traditional approach to “Information Architecture” views visual identity as decoration for functional behaviors. A pattern, on the other hand, describes an entire solution to a task (e.g. choose an account from a list), including the way it should look and feel. A pattern can optionally describe the best way to code the solution. In other words, UI architecture standards and style guide can ultimately be combined in the same document in an easy to understand format.

Patterns...

1. Are observed through either formal user research or experience.
2. Are typically written in a structured format (see below for S1 UI pattern format).
3. Are reusable artifacts (i.e. prevent “reinventing the wheel”).
4. Exist at different levels of abstraction (e.g., structural patterns vs. behavioral patterns).
5. Undergo continuous improvement.
6. Communicate design and best practices.
7. Can be used together to solve a larger problem.

The Important Design Patterns for Online Systems

Our experience and years of research have taught us that the following UI Design Patterns are extremely critical to positive User Experience:

1. Accessibility
2. Visual Identity
3. Screen Layout
4. Forms & Form Elements
5. Terminology
6. Text Usage
7. Navigation
8. Lists
9. Paging & Scrolling
10. Searching
11. Sorting
12. Hiding and Showing Information
13. Icon usage
14. Error Handling
15. Messages and Other User Feedback
16. Common Functions & Actions
17. Buttons & Hyperlinks
18. Selecting Objects and Executing Actions
19. Online Help
20. Multi-part Actions
21. Required Field Indicators

We have developed the above-listed patterns for a client. As you might suspect, these patterns are closely-guarded intellectual property; therefore, we cannot share them (see are information on 'Our Corporate Values' and on 'Ethics').

Method & Technique Expertise

---

Appeal & Attractiveness Assessment: Assessing the initial and long-term 'curb appeal' of a product

• Visual Preference Surveys
• Appeal Assessment Surveys

---

Design, including—

• Industrial Design
• User Interface Design
• Information Design/Information Architecture

---

Design Patterns: The ‘blueprints’ from which engineers and developers build the real product

---

Field Studies and Related Observational Research

• Ethnography
• Depth-Interviews
• Journaling
• Behavioral Sampling

---

Human Factors Analysis

Human Factors is the discipline that deals with optimizing the relationship between people and things. At this stage, Human Factors Analysis would involve looking at known ergonomic issues with the type of product that is still in the conceptual stage. For instance, if the product concept is beginning to look like a chair, there are known standards for chair dimensions.

It will sometimes be possible at this stage to perform a cursory Task Analysis. Task Analysis is a low-level description of steps required to use a ‘thing’. Task Analysis can be thought of as Scenario creation, but at a far more detailed level. For example, the beginning of a Task Analysis for watching TV might start with:    

1. Subject picks up the remote control device.
2. Subject looks for a button labeled ‘Power’
3. Subject presses the ‘Power’ button while pointing the remote control device in the general direction of the television set.

   

And so on...

---

Ideation & Brainstorming

Ideation refers to any deliberate intervention to elicit as many ideas from a group of people in a relatively short period of time. Brainstorming is a particular type of ideation; a facilitated session where a group of product stakeholders, having looked at the available data from other methods, identify as many problems and solutions, pros & cons and so forth as possible. A hallmark of brainstorming is that criticism of any idea put forth is explicitly forbidden.

• Brainstorming
• Mind-mapping
• Charettes (a creative process akin to visual brainstorming used by the Design Professions, particularly Architecture, Urban Design and Industrial Design)

---

Market Research (focus groups, structured interviews, conjoint, etc)

---

Naming & Categorization: Discovering the right names for UI elements (e.g. buttons) or physical controls and how they should be grouped so that people can easily find them

• Word-Concept Association Studies
• Online Card-Sorting
• Cluster Analysis

---

Persona & Scenario Development: Modeling the types of people who use your product (or you want to use your product) and how they actually use it

---

Prototyping & Modeling

Prototyping is a critical aspect of all product development, online or physical products. The types of prototypes vary according to whether the task is repairing a bad user experience, enhancing a good user experience or developing a completely new product. Some prototyping methods are only used for online systems and some are only used for physical products.

Prototypes play a fundamental role in iterative Usability Testing.

---

Surveys/Questionnaires

• Design
• Pilot Testing
• Statistical Validity and Reliability Assessment
• Implementation (online and paper-based)
• Quantitative Data Analysis
• Analysis of text data (e.g.‘additional comments’ questions)

---

Usability Evaluation & Testing

• Remote (on location)
• Remote (via web)
• Laboratory-based

Appeal

Common to both the Value and the Quality perspectives is the notion of Desirability or Attractiveness. If not exactly one in the same, these are both highly related to the construct of Appeal, the overall attribution of goodness that a person associates with a product.

Appeal, as defined by Hassenzahl (2004), is the combined effect of usefulness and usability, plus attractive qualities such as novelty, surprise, fun, visual attractiveness, perceived high ratings on adoption factors.

Adjectives that describe an appealing product are:

• Pleasant
• Pleasant
• Good
• Inviting
• Attractive
• Motivating
• Desirable

   

This is very important, as appeal is emerging as one of the top known reasons for why people decide to buy or adopt new products. Appeal is usually more important than usability and is sometimes more important than usefulness. Indeed, one of the primary problems with physical products, web sites and online applications is that their ‘look' is either outdated or just plain ugly and unappealing.      

Most people think of appeal as something that is totally subjective and cannot be measured. This was true until fairly recently. There are methods, adopted from the worlds of academic research and urban design, have been shown to reliably measure product appeal.   

Enragers: It only takes one! 

Just as important as appeal as a product attribute is its opposite—disgust or even rage. Bad experiences with a product chase people away permanently. It only takes one thing about your product that fails to meet customer expectations to the degree that they are angered. Once that happens, you've lost them forever.

Usability Evaluation

Usability Evaluation can simulate relatively unobtrusive observation of semi-realistic behaviors. Generally, Usability tests do a good job of uncovering egregious problems with conventional quality--things such as bad links, buttons that stick, etc.

The 5 Levels of Usability Assessment

Following the practice of Dr. Robert Bailey of Computer Psychology, Inc., we use up to 5 levels of Usability Assessment, depending on the project. These levels of assessment range from a simple Usability Review to a full-blown scientifically controlled comparison of two products or two versions of a single product.

Level 1 Inspection : review by two experts against common usability rules of thumb (i.e. 'heuristics')

Usability Inspection is used to make a quick assessment of the quantity and magnitude of commonly-observed usability problems. Because actual users or consumers are not part of Usability Review, it must never be the sole criterion on which the goodness of a product is based.

It is critical to keep in mind at all times, that real users always turn up surprises that are missed by the Reviewers.

Level 2 Expert Review: review by at least 4 experts against a comprehensive list of usability guidelines

Expert Review is used to make a quick assessment of the quantity and magnitude of commonly-observed usability problems. Because actual users or consumers are not part of Expert Review, it must never be the sole criterion on which the goodness of a product is based.

It is critical to keep in mind at all times, that real users always turn up surprises that are missed by the Experts.

Level 3 Expert Review: review by 12 -16 (combination of usability experts and domain experts). The larger number of reviewers allows statistically valid assertions (but still preliminary) to be made about the product.

Level 4 Usability Test: formal test involving 12–16 actual or potential users

Level 5 Usability Test: A scientific comparison of two versions or two different products

Usability Evaluation Planning

The idea is to ensure that usability test protocols are based on insights discovered through real world observation and depth interviews. This way, the usability tests will test the right tasks

Lacking a clear notion of who the users are and what they do, how could one even know what questions to ask or what to test in the usability lab? In science jargon, asking irrelevant or just plain wrong research questions is called 'Type III Error'.

The common practice of conducting usability testing prior to rigorous Ethnographic study, such as Field Observation and Depth Interviews, is a principal reason for so much usability testing being a waste of time and money.

By analogy, conducting a usability test before Personas and Scenarios are developed is like booking a recording studio before any of the songs are written--very expensive and very little valuable output.

This is why it is essential that Usability Evaluation Planning be done as an integral part of Researching the Opportunity and take full advantage of the insights from Ethnography, Initial Market Research, Human Factors and Persona/Scenario Refinement.

Observation

All credible notions of the New Product Development (NPD) process stress the importance of forming, early on, an understanding the people who will use products, how they will use them and the context in which usage is most likely to occur.

This absolutely requires real-world observation of real people on their own turf, doing whatever it is that they do. This real-world observation as part of New Product Development is often called "Ethnography", a term borrowed from several other fields, especially Anthropology.

Is Ethnography the "new core competency", as claimed by a fairly recent article in Business Week (www.businessweek.com)? We think YES.

Ethnography in New Product Development

While being inspired by the observation of people on their own turf that is the hallmark of traditional ethnography, New Product Ethnography is specifically focused on New Product Development. The emphasis is on providing actionable insights into user needs and desires.

What is Ethnography?

The term ‘Ethnography’ seems to have achieved the dubious status of buzz-word. There are many definitions, depending on who is doing the defining. A good definition of traditional ethnography is—

‘...the art and science of of describing a group or culture. It is a form of cultural anthropology using fieldwork to observe the group and derive patterns of behavior, belief, and activity’ (http://www.creatingbreakthroughproducts.com/).

No matter what definition you get, there are several signature attributes:

• Observation of people doing whatever it is they do on their own turf–Going out into the “real world” and observing is critical.
• Qualitative in approach–For instance, statistical analyses are not normally  performed on the discoveries of studies deemed to be ‘ethnographic.’
• Inductive in nature–does not start with hypotheses
• Interaction with ‘Research Subjects’ is encouraged–Many, especially those with legitimate claim to the title of ‘Professional Ethnographer’, stress that ‘Participant-Observation’ is a necessary attribute of ‘true’ ethnography.
• Depth Interviewing (Long-Open-ended Interviews) is often used
• Hypothesis generation and prediction–There is not universal agreement about whether hypothesis generation, based on observations, should be part of ethnography or whether it should be purely descriptive.

Ethnography and Business Meet

Business decision makers are always seeking better ways to answer questions like:

• Who are the target customers?
• What are their days like?
• What are the real problems they wish they could solve with a product that your company might make?
• What are the opportunities for new products that these unmet needs present?
• Will people pay enough for these new products for the company to make money?

The New 'Ethnographic' Methods Arrive on the Business Scene

In the 1990s, various methods began to pop-up that resembled traditional Ethnography in that they were qualitative in style and emphasized:

• Observation of people on their own turf (e.g. their homes, offices, in stores, and so forth)
• Induction rather hypothesis-testing
• Interaction with people being observed
• Depth Interviewing

Some of the names by which the new methods are known:

• ‘Shadowing’
• ‘Day-in-the-Life Studies’
• ‘Ethnography Lite’
• ‘Consumer Ethnography’
• ‘Field Observation’
• ‘New Product Ethnography’ (Cagan and Vogel)
• ‘Observational Research’ (Abrams)

'New Product Ethnography'

• A blend of traditional ethnographic methods with new technology and interdisciplinary research teams for observing, recording, and analyzing social situations.
• More than just applied anthropology
• Predictive as well as descriptive
• Emphasis on “actionable insights” into consumer behavior and lifestyle activities.
• Has emerged into a powerful new area for predicting–
• Consumer preferences for product features, form, material and color
• Patterns of use and purchase

Ethnography Helps Us with Our Other Research Methods

• Quantitative Market Research

• • Helps us construct hypotheses
  • Helps to identify the right questions on surveys
  • Who to survey

• Focus Groups
• • Choosing the right issues to discuss
  • Who to include?

• Usability Testing
• • Selecting the right types of participants
  • Having them perform the most relevant tasks
     

Depth Interviews

Depth Interviews are actually one of the many methods used in the course of Ethnography. They are lengthy interviews that are unscripted. To the User, they are simply a long conversation about them, their needs and desires.

The purpose of Depth Interviews in the current context is to ensure that the prototypes being evaluated in Usability Testing really are perceived as valuable and appealing (i.e. useful, usable and desirable) to customers.

Design Research

Design Research has been used to designate a number of activities, including the study of the designers and the design process itself. We use the phrase 'design research' in a more practical way.

When we say 'design research' we mean the application of behavioral and social science research approaches to inspire the design of the best products possible.

We use the best of Marketing/Consumer Research, Human Factors, Ethnography, Usability Testing and Engineering methods to uncover insights needed to develop great products that people value enough to buy at a price that makes a fair profit.

Prototyping

Prototyping is a critical aspect of all product development, online or physical products. The types of prototypes vary according to whether the task is repairing a bad user experience, enhancing a good user experience or developing a completely new product. Some prototyping methods are only used for online systems and some are only used for physical products.

Prototypes play a fundamental role in iterative Usability Testing. Some of the different types of prototypes are as follows:

Paper Prototype: A 'pretend' prototype of a new product using pen (or pencil) and paper.

    

Rapid Prototype: A prototype of a new product or features with partial and/or simulated functionality. Microsoft PowerPoint is often used for rapid prototyping, as many users are familiar with how to use it.

Physical Prototype: When the product is a device, physical models of new products or features is more effective than Paper Prototyping or Rapid Prototyping. Often, modeling clay and/or foam or wooden blocks are used for initial physical prototypes.

Physical prototypes should ultimately evolve into artifacts that look convincingly real. This often requires the skills of a professional model-builder

Blank Model Prototyping: Similar to drawing the experience but with physical models instead of drawings. The physical models are often made with foam or wooden blocks, on which research participants draw controls.

'Wire Frame' Prototype: A simplified working model of a product, with apparent functionality

High-Fidelity Prototype: A prototype that looks and acts enough like an actual product that it can be submitted to unfacilitated usability testing

Working Models: A prototype that looks and acts enough like an actual web site that it can be submitted to unfacilitated usability testing.

Value and Quality

Two related perspectives from which to judge the goodness of a product are value and quality—actually it would be more accurate to refer to perceived value and quality. These perceptions happen very quickly, apparently faster than previously thought; even faster than purported by Malcolm Gladwell in his recent book, Blink.

For instance, in a recent study, Gitte Lindgaard of Carleton University in Ottawa found that first impressions of a visual stimulus are made in the first 50 milliseconds of viewing (see Nature: Web users judge sites in the blink of an eye)

Value

What has come to be known as the Fitch perspective (Fitch is a very successful Industrial Design firm), a valuable product is one that a customer will buy or a user will adopt. The Fitch model states that valuable products possess three critical attributes:

• Usefulness: Do people perceive it as having a useful purpose?
• Usability: Is it relatively easy to learn and use?
• Desirability: Is the product appealing enough on an emotional level for people to go to the trouble to buy/adopt?

Quality

Quality is another way of looking at goodness. The product marketing thought leader, Noriaki Kano1, says that products have three components of quality:

• Basic Quality: "Must-have" requirements that the customer expects in the product (e.g. the car starts every time)
• Linear Quality: Product attributes that customers can identify they want more or less of, such as good gas mileage, easy to use, etc.
• Attractive Quality: Surprises, delighters, exciters. Unexpected attributes that distinguish the product from its competitors, such as novelty, beauty, pleasant sound, “Wow!” factor, “coolness”

The Dimensions of Appeal

Marc Hassenzahl and his colleagues are the undisputed leaders in the measurement of Appeal. Appeal is divided into ‘Pragmatic Quality’, which is roughly equivalent to what we think of as Usability, and ‘Hedonic Quality.’ Hedonic refers to factors that are appealing for their own sake, not for any practical reason. Hedonic Qualities include attractiveness, fun, emotion, and so forth. Hassenzahl and others caution that, for any given product, a balance between Pragmatic Quality and Hedonic Quality must be achieved.

Pragmatic Quality Scales (roughly equivalent to Usability)

    

• Technical – Human
• Complicated – Simple
• Impractical – Practical
• Cumbersome – Direct
• Unpredictable – Predictable
• Confusing – Clear
• Unruly – Manageable

Hedonic Quality – Identification Scales

    

• Isolating – Integrating
• Amateurish – Professional
• Gaudy – Classy
• Cheap – Valuable
• Non-Inclusive – Inclusive
• Takes me distant from people – Brings me closer to people

Hedonic Quality – Stimulation Scale

    

• Typical – Original
• Standard – Creative
• Cautious – Courageous
• Conservative – Innovative
• Lame – Exciting
• Easy – Challenging
• Commonplace – New Commonplace – New

Overall Appeal Scales

• Ugly – Beautiful
• Bad – Good

Hassenzahl, M. (2000). Hedonic and Ergonomic Quality Aspects Determine a Software’s Appeal. Presented at ACM CHI conference, April 2000.

Hassenzahl, M. (2004). The Interplay of Beauty, Goodness and Usability in Interactive Products. Human-Computer Interaction, Vol. 19, pp. 319- 349.

   

Appeal Assessment

Whereas Depth Interviews are a qualitative method for ensuring that a prototype or product is perceived as valuable and appealing (i.e. useful, usable and desirable) to customers, Appeal Assessment is a quantitative method for doing the same thing. It is important, whenever possible, to use both qualitative and quantitative methods to converge on the truth.

Appeal, as defined by Hassenzahl (2004), is the combined effect of usefulness and usability, plus attractive qualities such as novelty, surprise, fun, visual attractiveness, perceived high ratings on adoption factors.

Adjectives that describe an appealing product are:

• Pleasant
• Pleasant
• Good
• Inviting
• Attractive
• Motivating
• Desirable

Appeal is very important, as it is emerging as one of the top known reasons for why people decide to buy or adopt new products. Appeal is usually more important than usability and is sometimes more important than usefulness. Indeed, one of the primary problems with physical products, web sites and online applications is that their ‘look' is either outdated or just plain ugly and unappealing.

Most people think of appeal as something that is totally subjective and cannot be measured. This was true until fairly recently. There are methods, adopted from the worlds of academic research and urban design, have been shown to reliably measure product appeal.

Balance & Interdisciplinary Synergy

Many disciplines are in involved in successfully bringing a product to market. Developing products is not simply a matter of requirements definition, nor is it just an engineering problem, a quality assurance problem, a design problem, a marketing problem or a consumer research problem alone. Product Development is a multi-faceted endeavor that requires the single-minded cooperation of all of the essential core competencies, forming a powerful interdisciplinary balance.  

In accord with the latest thinking in New Product Development, we view the essential core competencies as:

• Marketing
• Design
• Engineering
• Research

The proper balance, of course, varies according to the type of product and its target market. The balance is quite different for developing an enterprise software system than it is for creating a new carbon fiber violin or designing a new medical instrument.

 

Interdisciplinary Cooperation is Essential in NPD

Developing products is not simply an engineering problem, a design problem, a marketing problem or a research problem alone. It is a multi-faceted problem that requires the single-minded cooperation of all of the involved core disciplines.

It is essential that four core competencies and the many disciplines of which they are comprised work in an uncommonly close collaboration during the Front End of NPD. The core competencies are:

Marketing

• Product Marketing
• Marketing Communications
• Product Management

Design

• Industrial Design
• User Interface Design
• User Experience Design
• Graphic Design
• Packaging Design

Engineering

• Information Architecture
• Software Engineering
• Industrial Engineering
• Electrical Engineering
• Mechanical Engineering

Research

• Marketing Research
• Operations Research & Logistics
• Consumer Research (sometime called ‘Consumer Insights’)
• Human Factors & Ergonomics
• Usability Testing
• Ethnography (a.k.a. Observational Field Research)

Other Specialty Disciplines

It is often necessary to bring in, as needed, experts from specialty disciplines or sub- disciplines, such as—

• Traditional Anthropology/Ethnography
• Acoustics
• Architecture
• Medical Devices
• Affective Human Factors
• RF Engineering
• etc.

The cultural change to make collaboration rather than competition among the core competency areas the norm is challenging but critical!

   

Don Rickert Research & Design Front End Design Research Services

Solve the Right Problem with the Right Product!

Product development is often about solving a problem or fulfilling an unmet consumer need through innovation.  But, how do you know you are  solving the right problem?  Ethnography and other front-end product development research are tools that have been carefully developed to identify which problems and unmet needs offer the greatest opportunities.

Front-End Product Development Research reduces the cost and duration of product development programs, while minimizing the risk of launching the wrong product.

Our services include:

• Ethnographic Study and Analysis
• Depth Interviewing
• “Trendspotting” and Analysis
• Group Idea Generation
• Opportunity / Needs Analysis

At the core of all of these services is rigorous observation and engagement of sufficient numbers of real people (customers, consumers and end-users) to support actionable product development insights.

Benefits of Front End Product Development Research include:

1. Accurate understanding of the people who might buy your new product or products
2. Identification of the gaps between current products (yours as well as your competitors’) and consumers’ unmet needs and desires
3. Practical insights into how to fulfill theses unmet needs and desires with a better product or products
4. Project teams empowered with critical knowledge to ensure a more efficient and effective product development program

What is Ethnography?

The term ‘Ethnography' or its many synonyms often comes up in the context of New Product Development. For example, Cagan and Vogel state that “the most powerful area of new product research is an emerging field of new product ethnography.” (Cagan and Vogel, 2003). Ethnography also plays an important role in our model; therefore, it will be useful to describe the method and its central role.

What is Ethnography?

The term ‘Ethnography' seems to have achieved the dubious status of buzz-word. There are many definitions, depending on who is doing the defining. A good definition of traditional ethnography is—

…the art and science of of describing a group or culture. It is a form of cultural anthropology using fieldwork to observe the group and derive patterns of behavior, belief, and activity (Cagan and Vogel, 2002, p. 183).

No matter what definition you get, there are several signature attributes—

• Observation of people doing whatever it is they do on their own turf —Going out into the “real world” and observing is critical.
• Qualitative in approach —For instance, statistical analyses are not normally performed on the discoveries of studies deemed to be ‘ethnographic.'
• Inductive in nature —does not start with hypotheses
• Interaction with ‘Research Subjects' is encouraged —Many, especially those with legitimate claim to the title of ‘Professional Ethnographer', stress that ‘Participant-Observation' is a necessary attribute of ‘true' ethnography.
• Depth Interviewing (Long-Open-ended Interviews) is often used
• Hypothesis generation and prediction —There is not universal agreement about whether hypothesis generation, based on observations, should be part of ethnography or whether it should be purely descriptive.