A diary study involves asking a number of people to record their experiences related to a particular subject over a period of time. It is a useful tool to help learn about user behaviour as it provides a record of thoughts and actions in context. Diary studies aren't always the right choice but if you were studying the use of mobile devices for instance, it is particularly challenging due to their location-independent nature so diary studies are well suited. Field studies are another option but often prove to be too impractical, particularly for a service or product that is used all day. Lab based tests can give good insight but for some areas of interest, just won't give the real world accuracy.
A diary study is flexible and needs no special materials or equipment. However, this can also be a potential pitfall. Without careful planning, itï¿½s easy to lose track of what exactly you want to get out of it and the data can then be incorrectly focused.
There are many ways in which you can ask participants to capture diary entries.
Paper diaries are most traditional type of diary recording method and are considered the most natural and personal. They are good because they can be used by participants with all levels of technical ability, they can be used at the same time as digital devices, e.g. while browsing the Internet, and they're very portable and can be used to capture events as soon as they happen.
However, they do have some downfalls. Firstly, the researcher must wait until the diary is returned from the participant before starting analysis. Also, rich experiences can't be captured in the same way that audio or video can, handwritten notes aren't always easy to read and transcribing, if required, can be time-consuming.
Quick tip: Paper diaries should include the participants name, instructions for capturing entries and details for when and where to return the diary so they are continually reminded of it during the process.
Participants can be asked to email all diary entries to the researcher (usually at the end of each day) in a similar manner to a blog. This is useful as the researcher can view the diaries each day, rather than having to wait until the end of the process. The user can choose how they record their thoughts during the day, i.e. notes on mobile, pen and paper or just recalling the day straight into the email. Usually, the later is what the user opts for which means diary entries are usually made sometime after the event. This is an issue as participants may not remember the exact details of what happened, what they thought and how they felt at the time.
Using a twitter feed allows participants to send a text message or tweet online and make entries to a private twitter account. This is a much newer method and fits in with modern living, making it natural for many users. Also, the researcher can view the diary entries as they occur and can adapt the focus of the study as it goes as an insight into the results can be seen straight away. Short entries can be focussed and to the point, and a user may be more inclined to write them more often. However, the 140 character limit may reduce the amount of information captured and its success will depend on the audience, is this is a comfortable way of communicating for all users?
Quick Tip: twitter feeds will also allow you to reply to participants if they require advice, but beware of interacting too much with them as it may alter their thoughts and actions.
Being flexible and allowing different participants to use different diary capture methods is better for the participants. However, bear in mind that having to analyse different types of diaries can complicate the analysis process.
Deciding on the type of diary study is an important step in the process but there is a lot to consider so here are another 8 questions to think about before setting off.
There are a few items that should be prepared in advance to make sure the smooth and successful running of a diary study. Firstly, an analysis guide, a set of criteria against which each diary will be analysed. Include sections for everything you wish to observe. Similarly, create an interview guide outlining questions about the activities you're studying that you can ask to all participants. You can add points from an individual's diary which you want to expand on or clarify when you are reading the diaries. Finally, a management schedule planning agreed dates/times for check-points, noting progress and managing who has been contacted and when.
Always recruit more participants than you need. A 15-20% drop-out rate is quite normal and of those who complete the study because they last longer than other types of research and need more involvement. Also, some just might not carry out the activities you're interested in and people's plans can change so it's best to include some extra participants. A rule of thumb is to recruit about a third more participants than you need to get good data from. Provide a participant study reference letter which outlines what's required of participants overall and what they need to do each day.
The period of time can either be set (e.g. 4 weeks) or you can monitor the collected data until you have what you need. In practice, a set period of time often works better as it may be difficult to find participants to sign up to an open-ended study.
Make sure participants know what's expected of them in terms of the kind of information to record and when to record it. It is important though to avoid leading the participants by being too specific. Make sure they only record their natural behaviour and don't deliberately do things which they may feel would be interesting to you. It's a difficult skill to master but it's important you get this balance right otherwise the results may end up being skewed.
Participants should be required to be available at a set number of check-points during the study. This is important to check their progress and helps minimise any drop-outs by incentivising the completion of each check-point (and if possible having increasing incentives for each check-point). The number of check-points depends on what is being studied, but a minimum of 3 is recommended, for instance one at the start, one halfway through and one at the end which includes an interview.
If using paper diaries, near the end of the study, remind participants to return their diary as soon as possible. Incentivising the receipt of completed diary content will help ensure content is returned as soon as possible. Also try to keep the time delay between completing the diary and coming to interview to a minimum to make sure the experiences are still fresh in their mind.
Record key observations from the analysis of diary content using the analysis guide you prepared in advance. Start by summarising the key steps taken by the participant throughout the study. Next, try and identify any patterns between participants. Patterns to look for include: common behaviour (what they did), differences in behaviour, context of use, situated action, tasks, user needs, information requirements, successes/failures and barriers/problems encountered.
Essentially, talk the participant through the completed analysis guide. Start by going through the steps of each diary, asking why each step was done will help clarify user behaviour. Try to clarify any patterns of behaviour and expand on areas you found interesting. Where possible, try to cover the same topics in each interview so that you can compare experiences across participants. People generally love to talk about their experiences so ask as many questions as you need to. Make sure the atmosphere is conversational rather than formal. Trying to formalise a discussion around something which is everyday behaviour will negatively impact the responses you receive. Interviewing participants' face-to-face works better if possible, as it is more relaxed and less structured than via the telephone.
Having completed the diary study you will now have a wealth of information about your target group which you can report on or feed into a design process. Make sure you keep everything created during the study, from the diaries themselves to your analysis of the diary content and interview notes. This will act as a handy reference to help answer subsequent questions about your target group.
A diary study is a quick and inexpensive way of obtaining real-world data about user behaviour. Careful management of your target group together with studying these guidelines will enable your diary study to run smoothly and provide useful results.