London Web Design & SEO Services

Questions to Ask at Kick-Off Meetings

1.  Define purpose/vision for the site

• What is the purpose of the site?
• What are the goals of the site?

2.  Develop goals for the site

• How would you define a successful Web site for your organization? 
• What does success look like?  How will you know when you have been successful?
• How would you describe the site?
• From an organization’s viewpoint?
• From a user’s viewpoint?

3.  Define audiences & goals

• Who are the users of the site?  (Primary and secondary users)
• How would you describe the users?  (User characteristics, i.e., age, experience, education, etc.)
• Why will they come to the site?  (User needs, interests, and goals)
• When and where will users access the site?  (User environment and context)
• How will users access the site?  (User computer settings, i.e., connection speed, resolution, etc.)

4.  Conduct task analysis and prioritize tasks

• What will users do on the site?  (User tasks, content, features and functionality)
• Which tasks are critical to users’ success on the Web site?  (Criticality)
• Which tasks are most important to users?  (Importance)
• Which features of the site will users use the most?  (Frequency)
• Which features are prone to usability issues?  (Vulnerability)
• Which tasks are critical to the organization’s success on the Web site?
• How often will users frequent your Web site? 
• What will compel users to return to your Web site?

5.  Determine measurable usability objectives

• Which tasks should users be able to accomplish easily with few errors?  (Efficiency)
• Which tasks should users be able to finish quickly and efficiently?  (Effectiveness)
• What level of satisfaction should users have after using the site?  (Enjoyability) 

6.  Discuss expectations, requirements & preferences

• What is your vision of what the site should do? 
• Describe your initial view of the project.  What do you think the project should entail?
• What prompted the redesign?
• Who will be the key point of contact?
• Are there any restraints, mandates, or guidelines for the site? 
• Are there any sites you would like to model or a particular style that you prefer?
• What characteristics/attributes/attitude should the site convey to users? 

7.  Determine accessibility requirements and needs

• Is the site currently accessible?
• What type of accessibility testing has been done? 
• What types of accessibility tools are being used?
• Who is the key point of contact on accessibility issues?

8.  Identify available resources and training needs

• What level of resources is available for site updating and maintenance?
• Do you have content writers skilled in writing for the Web?  
• Are there graphic designers on staff?
• Who will be responsible for programming and maintaining the site?
• Who is in charge of site marketing and promotion?

9.  Discuss initial technology needs

• What are your hosting needs?
• Do you currently have a domain name or do you need a new one?
• Are you currently using a content management system?  If so, which one?
• Are you currently logging Web metrics?  If so, what metrics are you currently capturing? 
• Do you currently have a search engine?  If so, what type of search are you using? 

10.  Timeline and Project Plan

There are a variety of issues that tend to recur in the life of a user interface designer. I recall the first time I was asked to write performance requirements for a user interface. How should I go about deciding on an acceptable level of errors or an acceptable speed of accomplishing a standard task? How do I know whether I have improved an interface design enough? Of the many problems that need fixing, which ones should take priority? How do I even know whether improving the user interface of a product is going to have an impact on sales? At one company, we sold usability so successfully one of the business units declared they wanted to label each product with a “usability seal of approval.” How would one go about determining when to award such a seal?

Over the years I have worked with colleagues at Ameritech (where the work began), U.S. WEST Advanced Technologies, and most recently Sapient to create a tool that has helped in dealing with some of these questions. The tool that we developed is called the USE Questionnaire. USE stands for Usefulness, Satisfaction, and Ease of use. These are the three dimensions that emerged most strongly in the early development of the USE Questionnaire. For many applications, Usability appears to consist of Usefulness and Ease of Use, and Usefulness and Ease of Use are correlated. Each factor in turn drives user satisfaction and frequency of use. Users appear to have a good sense of what is usable and what is not, and can apply their internal metrics across domains.

General Background

Subjective reactions to the usability of a product or application tend to be neglected in favor of performance measures, and yet it is often the case that these metrics measure the aspects of the user experience that are most closely tied to user behavior and purchase decisions. While some tools exist for assessing software usability, they typically are proprietary (and may only be available for a fee). More importantly, they do not do a good job of assessing usability across domains. When re-engineering began at Ameritech, it became important to be able to set benchmarks for product usability and to be able to measure progress against those benchmarks. It also was critical to ensure resources were being used as efficiently as possible, and so tools to help select the most cost-effective methodology and the ability to prioritize design problems to be fixed by developers became important.

Finally, it became clear that we could eliminate all the design problems and still end up with a product that would fail in the marketplace. It was with this environment as a background that a series of studies began at Ameritech. The first one was headed by Amy Schwartz, and was a collaboration of human factors, market research in our largest marketing organization, and a researcher from the University of Michigan business school.

Building on that research, I decided to develop a short questionnaire that could be used to measure the most important dimensions of usability for users, and to measure those dimensions across domains. Ideally it should work for software, hardware, services, and user support materials. It should allow meaningful comparisons of products in different domains, even though testing of the products happened at different times and perhaps under different circumstances. In the best of all worlds, the items would have a certain amount of face validity for both users and practitioners, and it would be possible to imagine the aspects of the design that might influence ratings of the items. It would not be intended to be a diagnostic tool, but rather would treat the dimensions of usability as dependent variables. Subsequent research would assess how various aspects of a given category of design would impact usability ratings.

The early studies at Ameritech suggested that a viable questionnaire could be created. Interestingly, the results of those early studies were consistent with studies conducted in the MIS and technology diffusion areas, which also had identified the importance of and the relationship between Usefulness, Satisfaction, and Ease of Use. Furthermore, the rich research tradition in these other areas provides theory that may be extended to explain the relationships. This is an area that provides a link between academic research and practice, and it is informed by several disciplines. Some work has already been published suggesting that at least one publicly available tool drawn from earlier research can be applied effectively to software interfaces.

How It Developed

The first step in identifying potential items for the questionnaire was to collect a large pool of items to test. The items were collected from previous internal studies, from the literature, and from brainstorming. The list was then massaged to eliminate or reword items that could not be applied across the hardware, software, documentation, and service domains. One goal was to make the items as simply worded as possible, and as general as possible. As rounds of testing progressed, standard psychometric techniques were used to weed out additional items that appeared to be too idiosyncratic or to improve items through ongoing tweaking of the wording. In general, the items contributing to each scale were of approximately equal weight, the Chronbach’s Alphas were very high, and for the most part the items appeared to tap slightly different aspects of the dimensions being measured.

The questionnaires were constructed as seven-point Likert rating scales. Users were asked to rate agreement with the statements, raging from strongly disagree to strongly agree. Various forms of the questionnaires were used to evaluate user attitudes towards a variety of consumer products.

Factor analyses following each study suggested that users were evaluating the products primarily using three dimensions, Usefulness, Satisfaction, and Ease of Use. Evidence of other dimensions was found, but these three served to most effectively discriminate between interfaces. Partial correlation calculated using scales derived for these dimensions suggested that Ease of Use and Usefulness influence one another, such that improvements in Ease of Use improve ratings of Usefulness and vice versa. While both drive Satisfaction, Usefulness is relatively less important when the systems are internal systems that users are required to use. Users are more variable in their Usefulness ratings when they have had only limited exposure to a product. As expected from the literature, Satisfaction was strongly related to the usage (actual or predicted). For internal systems, the items contributing to Ease of Use for other products actually could be separated into two factors, Ease of Learning and Ease of Use (which were obviously highly correlated). The items that appeared across tests for the three factors plus Ease of Learning are listed below. The items in italics loaded relatively less strongly on the factors.

Usefulness

• It helps me be more effective.
• It helps me be more productive.
• It is useful.
• It gives me more control over the activities in my life.
• It makes the things I want to accomplish easier to get done.
• It saves me time when I use it.
• It meets my needs.
• It does everything I would expect it to do.

Ease of Use

• It is easy to use.
• It is simple to use.
• It is user friendly.
• It requires the fewest steps possible to accomplish what I want to do with it.
• It is flexible.
• Using it is effortless.
• I can use it without written instructions.
• I don’t notice any inconsistencies as I use it.
• Both occasional and regular users would like it.
• I can recover from mistakes quickly and easily.
• I can use it successfully every time.

Ease of Learning

• I learned to use it quickly.
• I easily remember how to use it.
• It is easy to learn to use it.
• I quickly became skillful with it.

Satisfaction

• I am satisfied with it.
• I would recommend it to a friend.
• It is fun to use.
• It works the way I want it to work.
• It is wonderful.
• I feel I need to have it.
• It is pleasant to use.

Work to refine the items and the scales continues. There is some evidence that for web sites and certain consumer products there is an additional dimension of fun or aesthetics associated with making a product compelling. For the dependent variables of primary interest, however, these items appear to be reasmonably robust. A short form of the questionnaire is easily constructed by using the three or four most heavily weighted items for each factor.

Conclusion

While the questionnaire has been used successfully by many companies around the world, and as part of several dissertation projects, the development of the questionnaire is still not over. For the reasons cited, this is an excellent starting place. The norms I have developed over the years have been useful in determining when I have achieved sufficient usability to enable success in the market. To truly develop a standardized instrument, however, the items should be taken through a complete psychometric instrument development process.

A study I have been hoping to run is one that simultaneously uses the USE Questionnaire and other questionnaires like SUMI or QUIS to evaluate applications. Once a publicly available (i.e., free) standardized questionnaire is available that applies across domains, a variety of interesting lines of research are possible. The USE Questionnaire should continue to be useful as it stands, but I hope the best is yet to come.

Author: Arnold M. Lund

Designers on Design

A designer knows that he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away - Antoine de St-Expurey

The public is more familiar with bad design than good design. It is, in effect, conditioned to prefer bad design, because that is what it lives with. The new becomes threatening, the old reassuring. ­ Paul Rand (Design, Form, and Chaos)

We are searching for some kind of harmony between two intangibles: a form which we have not yet designed and a context which we cannot properly describe. - Christopher Alexander’s Notes On The Synthesis Of  Form concerning the design process…

However beautiful the strategy, you should occasionally look at the results. - Winston Churchill

When I’m working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong. - R. Buckminster Fuller

Questions about whether design is necessary or affordable are quite beside the point: design is inevitable. The alternative to good design is bad design, not no design at all. Everyone makes design decisions all the time without realizing it­like Moliere’s M. Jourdain who discovered he had been speaking prose all his life­ and good design is simply the result of making these decisions consciously, at the right stage, and in consultation with others as the need arise - Douglas Martin (Book Design)

The details are not the details. They make the design. - Charles Eames

It’s better to design the user experience than rectify it. It’s the difference between a cathedral and the Winchester House. - Jon MeadsIf there’s a ‘trick’ to it, the UI is broken. - Douglas Anderson

Things that look different should act different
Things that look the same should act the same. - Larry Marine

Design without the ego.

In theory, theory and practice are the same,
but in practice, they’re not.”If I can see it, it’s a failure” - Bill Buxton

“We will go into your houses and redesign them the same way your web sites are designed. The basement will be the first thing you see, the kitchen will be unreachable except through the bedroom and both bathrooms, the bedroom will be on six different floors, and the dog will be in every room at once.” - Ann Feeny, Information Architect’s Manifesto

Discovery and Invention

…you’ll never have all the information you need to make a decision. If you did, it would be a foregone conclusion, not a decision - David Mahoney

I have not failed. I’ve just found 10,000 ways that won’t work. - Thomas Edison

Hell, there are no rules here - we’re trying to accomplish something. - Thomas Edison

I believe that a scientist looking at nonscientific problems is just as dumb as the next guy. - Richard Feynman

Just because it isn’t done doesn’t mean it can’t be done. Just because it can be done doesn’t mean it should be - Barry Glasford

Nothing is impossible for the man who doesn’t have to do it it himself - A. H. Weiler

People’s behavior makes sense if you think about it in terms of their goals, needs, and motives. - Thomas Mann

Always listen to the experts. They will tell you what can’t be done, and why. Then do it. - Robert A. Heinlein

Rigorous reasoning from inapplicable assumptions yields the world’s most durable nonsense. 

For every piece of durable nonsense, there is an irrelevant frame of reference in which it makes perfect sense. 

“We’ve all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the Internet, we know this is not true. ” - Robert Wilensky

Simplicity, Consistency and Other Hobgoblins

Complexity is the Problem; Ease of Use is the Solution; Productivity is the Impact - an executive of a large computer firm, (quoted in Kelley, John Falk, “Natural Language and computers: six empirical steps for writing an easy-to-use computer application”, University Microfilms International #8321592, 1983)

For every problem, there is one solution which is simple, neat and wrong. - H. L. Mencken

Don’t make me think - Steve Krug

Easy is Hard - Peter Lewis, NY Times

Every time we get it idiot-proofed, Ma Nature produces cleverer idiots. - Robin Kinkead

I would not give a fig for the simplicity this side of complexity, but I would give my life for the simplicity on the other side of complexity. - Oliver Wendell Holmes

Make it as simple as possible. But no simpler. - Albert Einstein

The best journey is the one with the fewest steps. Shorten the distance between the user and their goal. -

The main thing is that everything become simple, easy enough for a child to understand; that each act be ordered, that good and evil be decided arbitrarily, thus clearly. - Albert Camus

There are two ways of constructing a software design: one way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. - C.A.R. Hoare

It’s easy to make things difficult, but it’s difficult to make things easy.

A foolish consistency is the hobgoblin of small minds - Ralph Waldo Emerson

Consistency requires you to be as ignorant today as you were a year ago. - Bernard Berenson

Learn the rules so you know how to break them properly - Dali Lama

Users

Know thy user, and YOU are not thy user.

If the user can’t use it, it doesn’t work. - Susan Dray

If something is hard to use, I just don’t use it as much. - Melanie Sokol, quoted in Steve Krug’s ” Don’t Make Me Think”

Those who cannot tell what they desire or expect, still sigh and struggle with indefinite thoughts and vast wishes. - Ralph Waldo Emerson

We see what we look for, not what we look at - Ulrich Neisser (paraphrase)

…pay attention to what users do, not what they say. - Jakob Nielsen

…This is so simple a five-year-old child could understand it. “Quick, run out and bring me a five-year-old child.” - Groucho Marx

Users don’t know what they want, and users can’t always say what they know.

It is impossible to design anything that is foolproof because fools are so ingenious 

To err is human, to admit having erred is not human.

If I made an error, at least let me finish my thought before I have to fix it.

If the user can’t find it, it doesn’t exist - HFI button

Even experts are novices at some point.

The user is NOT a lower life form - Ken Becker

Whadya mean, they’re not all computer scientists?

Communication and Design

A picture is worth a thousand words and that’s the problem!

A picture is worth a thousand words, but it will take longer to download

A picture is worth a thousand words, unless of course, you’re talking about a picture of a thousand words.

For any given thousand words, it’s hard to come up with a picture - Yuri Englehart

It takes less time to do a thing right than to explain why you did it wrong. - H.W. Longfellow

There is no urge so great as for one man to edit another man’s work - Mark Twain

When your work speaks for itself, don’t interrupt - Henry J. Kaiser

What you do speaks so clearly that I don’t have to hear what you say. - Chuck Knox (Seattle Seahawk Coachs)

Just because nobody complains doesn’t mean all parachutes are perfect. - Benny Hill

There’s a limit to the usability problems you can document your way out of. Things beyond that are training problems.

If you can’t describe it simply, you can’t use it easily.

If it’s very difficult to write about then it probably doesn’t have quality usability.

No you can’t just explain it in the manual.

Process

If you don’t care about quality, you can meet any other requirement - Gerald M. Weinberg, “The Zeroth Law of Software Engineering”

You can use an eraser on the drafting table or a sledge hammer on the construction site. - Frank Lloyd Wright

No shortcuts today; I’m in a hurry. - Swiss saying

Planning is essential, but plans are useless - Dwight D. Eisenhower

Interface Design: “Design first and code later.”
Carpenter: “Measure twice and cut once.” -

Designing to Requirements and Walking on Water are Easy if Both are Frozen. -

If you can’t afford the time to do it right, how are you going to find the time to fix it up?

Rigorous reasoning from inapplicable assumptions yields the world’s most durable nonsense. -

The chief cause of problems is solutions. - Eric Severeid

The only thing more expensive than hiring a professional, is hiring an amateur. - Red Adair

The sooner you start to code, the longer the program will take. - Roy Carlson

Rules are made for people we don’t like.
New rules are made for people we REALLY don’t like.
Brand new rules are for you. Poor usability is often the result of a thousand cuts. - Greg Hoskins

Usability testing is the killing field of cherished notions. - David Orr

Technology and Computers

If the automobile had followed the same development as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year killing everyone inside. - Robert Cringely, InfoWorld

It is a far better to adapt the technology to the user than to force the user to adapt to the technology - Larry Marine

The fault is not in thyself, but in thy system.

To err is human, to really foul things up you need a computer - Paul Ehrlich

Coding is long. Design is short. Paper is cheap.

No system is so foolproof that it can’t be brought to its knees by a well-intentioned novice.

The joy of an early release lasts but a short time. The bitterness of an unusable system lasts for years.

Cute is not a good adjective for systems.

Words to Live By

Since our problems have been our own creation, they also can be overcome. - George Harrison

We can’t solve problems by using the same kind of thinking we used when we created them. - Albert Einstein

All the good ideas never lie under one hat - Dale Turner

We know very little, and most of what we know is wrong. - George Casaday

Don’t ever take a fence down until you know why it was put up. - G.K. Chesterton

For every piece of durable nonsense, there is an irrelevant frame of reference in which it makes perfect sense.

I don’t know the key to success, but the key to failure is trying to please everybody. - Bill Cosby

If you always do what you always did, you always get what you always got.

Let us think the unthinkable, let us do the undoable. Let us prepare to grapple with the ineffable itself, and see if we may not eff it after all.

Never attribute to malice what incompetence will explain.

Sometimes you just need a bigger hammer. - G Casaday Sr

Supposing is good, but finding out is better. - Samuel Clemens

There is no free lunch. But sometimes if you eat a good breakfast, you won’t need to spend as much money on lunch. - Cameron Hayne, CRIM, on cost justifying usability testing.

You can learn at least one principal of user interface design by loading a dishwasher. If you crowd a lot in there, nothing gets very clean.

You can’t prevent people from putting beans in their noses. But you shouldn’t stuff beans in their noses. - Stan Schwartz

It’s a jungle. Be careful out there.

Common sense is an uncommon commodity.

Everything in its place, and a place for everything.

Pre Test Questions

1. Do you ever research issues related to Web accessibility for people with disabilities? (If no: skip to 11)
2. What resources do you use to learn about Web accessibility? (Probe: Web sites, books, classes)
3. (May not need to ask) What Web sites do you use to learn about Web accessibility?
4. How often do you use them?
5. What do you like about each of the Web sites?
6. What do you dislike about each of the Web sites?
7. What types of Web accessibility information do you look for on the Internet?
8. What are the last three Web accessibility topics or questions you researched on the Internet?
9. What drives your interest in Web accessibility?
10. How long have you been involved in Web accessibility?
11. Do any of the authoring tools you use have features to help make your Web sites accessible?
12. How long have you been involved in Web development or management?
13. If you were to envision your ideal Web accessibility Web site, what sorts of information would it contain? What would it look and act like? How would it be organized?

Participant Tasks

Task 1

This is the homepage of a Web site dedicated to Web-related accessibility issues. Please give me your initial reactions to this page. Feel free to explore this page as you normally would. You can scroll around with your mouse, but please don’t click on anything just yet.

Faciliator will ask:

• Have you ever seen this Web site before?
• Please give me your initial impressions about the layout of this page and what you think of the colors, graphics, photos, etc.
• Without clicking on anything yet, please describe the options you see on the home page and what you think they do. Feel free to move around the page, but again I’ll ask you not to click on anything right now.
• Without clicking on anything yet, if you were exploring, what would you click on first?
• What do you think is the purpose of this site?
• Who do you think this site is intended for?
• Whose Web site is this?

Task 2

I’m going to give you five minutes to freely explore this Web site. You may go anywhere you would like to go on the Web site, but please remember to speak aloud as you do so. I will tell you when the five minutes are up.

Task 3

Your friend Kevin mentions hearing about something called “the Web Accessibility Initiative” but he isn’t sure what it is. Using this Web site, determine whether or not it contains information that would address Kevin’s question.

When you feel you have completed this task, please say so.

Task 4

Your team at work is developing a Web site and you have some concerns about how accessible the Web site might be to persons with disabilities. Using this Web site, determine whether or not it contains general hints about what Web developers need to know about Web accessibility.

When you feel you have completed this task, please say so.

Task 5

A few of your colleagues are interested in finding out how to be a part of WAI’s effort to develop guidelines for Web accessibility. Using this Web site, determine whether or not opportunities exist for becoming involved in WAI guideline development.

When you feel you have completed this task, please say so.

Task 6

You have just been handed a report, generated by a Web accessibility evaluation tool, which informs you that your company Web site contains graphs that do not meet “Checkpoint 1.1.”

Using this Web site, a) determine what Checkpoint 1.1 is, and b) determine an appropriate strategy for representing these complex pictures.

When you feel you have completed this task, please say so.

Task 7

Your company is revising the online forms on its Web site. Find specific information on how to make the online forms accessible.

When you feel you have completed this task, please say so.

Task 8

A company with many global divisions and Web sites in many languages has asked for your opinion on how Web accessibility laws differ around the world. Using this Web site, determine whether or not it contains relevant information.

When you feel you have completed this task, please say so.

Task 9

You have been invited to be a presenter at a local conference on Web accessibility. Find information on this Web site that you would want to use to help you prepare your talk.

When you feel you have completed this task, please say so.

Task 10

The company division you work in is responsible for making sure that your corporate Web site is accessible. How can your team use the WAI Web site to determine the accessibility of your company Web site?

When you feel you have completed this task, please say so.

Post Test Interview

1. What are your overall impressions of the Web site?
2. If you had to give the site a grade, from A to F, where A was exemplary and F was failing, what grade would you give it, and why?
3. Name three words or characteristics that describe this Web site.
4. What are the three things you like best about the Web site?
5. What are the three things you like least about the Web site?
6. If you could make one significant change to this Web site, what change would you make?
7. Would you return to this Web site on your own in the future? Why/why not?
8. What wou