User Research Smoke and Mirrors Reflections

The Die Die Must Read Article.
1 2 3 4 5

Even though I think we are only required to read part 5, but reading all parts put the author argument in clearer light. :) So I will suggest everyone read all.

I especially like the argument about in Part 2 about the eyetracking device with the example shown. Sometimes these so-called scientific testing devices just do not work, there is always such a time when exceptional cases come up. If such scientific devices are all so wonderful, we actually do not need the humans at all. Producing and designing a website can be all done with machines without the human eye. Machines can use the test results to organise and hi-light information the 'correct' way. So why not just let machines take over? In scientific research, interpreting results is often a subjective matter. We often see that different TV stations have different ratings from each other, and most of the time each TV station will show itself as having the best ratings. Scientific research can be not scientific at all. Crunching numbers to suit each of our own agenda, to prove ourselves right are just ways to make us feel better about our products. By arguing that red spots are the 'hot' spots people look at longest and hence the best element of the site, it is just to suit one's agenda. Like the article states, the red spots can both mean that something is interesting or something is so difficult that people must ponder over it for a very long time. Besides, the scientific research disregard any environmental elements that might be at work, I may be thinking about something else and thus staring at one spot in a daze for no reason. Probably it could mean that the website did not catch my attention at all, yet the eyetracking result will show that I am interested in this random spot. Of course, I do not mean that these tools are useless, they could still provide some valuable insights when used with other non scientific researches. The point is we should not rely on such tools blatantly and management must also think with their brain and not just feel happy because such tools reflect that their sites are full of red spots. We can deceive ourselves but not users. When users visit and decide to come back to the site, they look at the whole site subjectively in their own ways and not at how popular the eyetracking tool shows the site to be. In a nutshell, like what Fahey says "an eyetracker can tell you what people are looking at, but not necessarily what they are seeing (or why they are looking at it)".

I was laughing abit when I read about the persona rooms in Part 4. The first thing I thought of is how filthy rich these people are. I imagine the designers trying to think and live the life of a persona they have created and eventually become the persona. The point is no amount of persona rooms is going to make a designer into a real "Bob, the single dude” or “Mary, the homemaker”. It may help one live and create the imagination more vividly, but just one room doesn't bring you into the world of a real Bob and Mary. I think it will be more fruitful to seat down, talk to a real Bob and Mary, know them personally, ask them about their dreams, ask them about their lives, ask them what they hate, ask them about all places they lived before, try to think in their shoes and keep those money for something more meaningful.

This bring to the discussion of the less rigorous methodologies like card sorting, focus groups, usability testing, and user personas. I have done card sorting and did usability testings, heard about focus groups, but the first time I have come across user personas as a form of user research is in NM4210. I remember writing for the first time my first user persona Skye. We were suppose to come up with a user persona using one type of product and then choosing a phone to suit the persona. The hard part is that I tend to want to write Skye with a particular phone in mind, but I try to think Skye in the terms of sunglasses that she would buy. Because Skye is whatever I made her up to be, I didnt really get the gist of persona yet until the final project. That was when we interviewed real target users and talk to them. Then we came up for the personas, which are partly true and with a little of our imagination. Then we tried hard to think in terms of these users, of what their expectations of the learning portal would be. I guess the personas feel more real with a real persons behind them. I will think back about the people we have interviewed and imagine their expressions or exclamations when faced with certain product elements. But of course what I imagine is totally based on my assumption about how I feel they will feel, with the background information (sociological pleasures, physiological pleasures, psychological pleasures, ideological pleasures, need, wants, appreciations) I have on them. It may not be 100% accurate, but it is something for me to base on.

I like how Fahey used the analogy of the work a novelest does when researching her characters, historical events, and locations in preparation for their to the impact and value of these non scientific kind of researches. I mean everyone can get the facts of history from textbooks, scout the location, interview people who are similar to the characters they are building, but ultimately the novel is purely his creation by weaving together his imagination with research data. Although he has created a more convincing world to readers. This does not mean that everyone might agree with what he wrote, they will still use their own experiences and encounters to judge the book. They may love it, but they may hate it too.

Even after we do user testings, personas and what nots and build a product with users in mind, there is still no 100% guarantee that the final product will be a success, because no one user is the same and everything is subjective. Like what Seldman said "If web design were not an art, then we would always get every part right. But it is an art, and, like all arts, it deals with the subjective. The subjective is something you can never get 100% right." Furthermore, no matter what, the product is still build with one's impression of target users. There will never be a time when we can totally be in other people shoes, we can be very close to, but not totally. Our encounters and expertise would have interfered with all these. Hence I feel most products, even those which claim to be totally build around users, will still have a little shadow of what we like ourselves. And it is easy to just fall into the trap that I think I know what is best for users.

I feel that qualitative research is more useful than quantitative research. However, sometimes we are so used to the numbers, to the nicely charted graphs that we tend to need quantitative research to justify findings. Moreover, qualitative results can differ so extremely, so I think a lot of people would rather stay safely on the figures and charts. Some more these impressive charts can be shown to the management. I myself is more used to qualitative research like surveys because they can give the number you want. But now, I feel that I must try to do more qualitative research, especially with users experience design so hot now.

I find this line by Zeldman very meaningful. "The user is never wrong because experience is experience, not fact." But I think as designers, we are prone to think that the user is wrong, especially when something so obvious is put right smack in front of them. It is hard to think that users are never wrong, because more often than not, we will think of thinks our way and not as others. Like in real life, being in a waitress position, we may think that the customer is over demanding. But when we switch position to the customer position, we may think that we have every right to demand good service and that the waitress should have better attitude. I think that sometimes we need to think ourselves as users too and not as designers. Sometimes, when we design something, we are so bent on seeing our own way that we neglect some elements. It takes a user to point out the mistake, and a designer to listen for changes to be made accordingly. Many a times, designers don't take bad comments well, and its easier to say that the users don't know anything. Well they might not know all the technical jargons, but at the end of the day they are the one who used a product most. This is a point we must remember and take suggestions, good or bad and however hard openly. I have encountered this myself a few times as web designer and video editor, and it hard real hard to hear someone criticize my hard work. It easier to bitch about to friends who agree with you that those people don't know anything about web pages or video editing. But at the end of the day, I try to take it in my stride and see it from their point of view. And sometimes, not all, those suggestions are not that bad after all.

"The user is never wrong because experience is experience, not fact". I will remember this ^^.