Theme 6 - Post reflection

This week we had our last lecture and final seminar in the course, where we discussed both qualitative methods and case studies. At the seminar, we also talked about our reflections of the course overall and what improvements that could be applied for next years students. However, most of the time at the seminar we tried to define what a case study was and how it can be applied in the real life, which would turn out being quite difficult.

Even though i thought I understood the concept of a case study before the seminar, what i learned during it changed my interpretation a bit. As i understand the meaning of a case study research now afterwards, is that you want to combine different methods in order to investigate what happened in a specific field or situation. The field that you want to do the research on can either be created yourself, or found in the real life. Leif talked about the custom service at the airport as an example of finding a field for a case study research, where the researchers defines a scene and what’s happening there. I also learned about some advantages as well as disadvantages with case studies. One positive thing is that you’ll never be short of data, since you are combining information from several different methods into one. On the other hand, this could lead to a point where you have too much information, and considering the amount of work you’re doing, is it necessary to put that much effort into it? It was also exciting to clarify why the number of 4-10 case studies is the optimal. The group had some different aspects, but it turned out to be as basic as we’re considering 4 as “many”, and therefore it’s good to do “many” case studies in order to get the best result as possible.

Finally, I really think this was a good theme of the last week, where we actually could get usage of our earlier experiences and knowledge from the course.

Theme 5 - after reflections

This week, Eva-Lotta Sallnäs Pysander from the institution of HCI visited us and held a lecture about collaborative haptics. We were also suppose to have a lecture with Haibo Li but that one got cancelled and moved to next week instead. There was no seminar this week either.

I was excited to listen to Eva-Lotta and what she had to say about design research and the concept itself but unfortunately she focused most on her own studies about haptics. She began with some brief explanations about how a study can be constructed and that it is preferable to use both quantitative and qualitative methods in order to get the best output from it. Worth mentioning is that choice of method can depend on the extent of present research in the specific area.  

However, even though she didn’t talk very much about the methods which we wrote about in the pre-reflections she had some interesting points and reflections from her earlier studies. The way a collaborative research regarding haptics can be performed was exciting to hear about. I really liked the assistive technology research that investigated how group work in school could be supported in order to make sure that the learning process is fair and equal for visually implied kids and the ones with sighted pupils. it was instructive to get it explained from an experienced researcher, how the selection of target group is made, which kind of people you should invite for the interviews and how haptics can be used in research. Another thing i learned from the lecture regarding collaborative researches is that it’s possible to get either qualitative or quantitative into the bargain. Since the participants of the research may come up with ideas or answers that never were discovered before, it could lead collaborative research into new perspectives.

Theme 6: Qualitative and case study research (pre)

Part 1
I chose a qualitative researched paper named Publicly Private and Privately Public: Social Networking on YouTube written by Patricia G. Lange. The paper is from the Journal of Computer-Mediated Communication which has an impact factor of 2.019.

The paper analyzes the Social Network Site (SNS) YouTube and how the users who’re uploading and sharing their videos, select different approaches depending on what part of their social circle they want to reach. The author investigate especially two different cases when it comes to sharing videos and get publicity. The first one is called “Publicy private” which means that the publisher of a video shows its identity but make sure that the video is hard find on YouTube. The other one is called “privately public” that is basically the opposite. The uploader wants to reach as many viewers as possible but at the same time, the identity should be hidden for the viewers.

Which qualitative method or methods are used in the paper? Which are the benefits and limitations of using these methods?
The paper used qualitative methods combined with a quantitative one. It contained interviews (N=54) of people in primarily the United States of an age distribution from 9 to 43 and each interview lasted for 1-3 hours. There were three different types of interviews; phone-interviews, interviews conducted over messaging chat and finally face-to-face interviews. In order to find the focus group that was most suited for the interviews, Lange used a quantitative method in the shape of a pre-interview survey. The survey had the aim to understand the media use and demographic background of the user. The author also collected field notes from observations made on YouTube over one year. Published comments and uploaded videos were analyzed. The benefits of a large amount of interviews is that the researcher may get inputs and new experiences from the interviewees that never would appear from an online survey etc. Thoughts and viewpoints from interviews also has the potential to lead into new fields that the researcher later can investigate deeper. On the other hand, since there were three different interview methods, one might answer differently or be misunderstood based on how the interview is designed. For example, it’s easier to interpret body-language in a face-to-face interview than a text-chat interview and how the answers are perceived may differ.

What did you learn about qualitative methods from reading the paper?
One thing i noticed and learnt from this paper and its methods is how time-consuming it actually can be to fulfill a study like this. It requires a lot of discipline and time to perform one year of observations, analyze a pre-interview survey and finally complete all the interviews. I also learned that it’s important to be adaptive when doing interviews for example. In this research, the interviewer had prepared different question depending on the the participants’ backgrounds, interests and media choices etc.

Which are the main methodological problems of the study? How could the use of the qualitative method or methods have been improved?
One methodological problem of the study is to find the most suitable group to the interviews. Since Lange partly wants to compare two different type of user groups on YouTube which are quite hard to define, it may occur that the participants perceive themselves as one type of  user while the researchers understand them as something different. This could lead questionable results. Another problem is that the author wants to investigate too many questions in the research. To narrow down the scope of the survey a bit could lead to a better and clearer result.

Part 2
For this part i chose the paper Social media competitive analysis and text mining: A case study in the pizza industry from the journal named International Journal of Information Management.

Briefly explain to a first year university student what a case study is.
A case study is one way of obtaining and analyzing conclusions by using one or a several real-life phenomena/persons/events. Furthermore, a case study can include different cases and many analysis based on their structure. It also used to be done in an empirical way, which means that the researchers collect data from the reality and experiences.

Use the "Process of Building Theory from Case Study Research" (Eisenhardt, summarized in Table 1) to analyze the strengths and weaknesses of your selected paper.
Firstly, the paper is getting started by defining three different research questions:
*What patterns can be found from their Facebook sites respectively?
*What patterns can be found from their Twitter sites respectively?
*What are the main differences in terms of their Facebook and Twitter patterns?
I like that fact that it’s done early in the research which makes it easier for the reader to understand what parts that are going to be investigated. Hence, the authors selected cases by picking three of the largest pizza chains in America; Domino’s Pizza, Pizza Hut and Papa John’s Pizza. They’re clear with why they want to do a case on these three specific Pizza chains which is good. I also think they were clear with the crafting instruments and protocol step since they collected quantitative data regarding the Pizza business from their social media flow, e.g. by looking at the number of fans/followers and comments. They also analyzed the patterns and use of social media from the three Pizza chains by applying text mining on the text messages that were posted on their sites.

I think the study more all less included all the steps from Eisenhardts’ table. However, this study only take the three largest chains into consideration. It would have been interesting to compare this with a couple of smaller and probably unknown Pizza companies to find out differences in their social media use. Another thing that could have been done differently or improved is that they maybe could have spoken to the companies themselves and get their viewpoint on how they work with social media instead of just emanating from the analyzed data collected.

Theme 4: Quantitative research (after)

The theme of the previous week was quantitative research and i prepared myself by reading a research paper which used a quantitative method and answered the associated questions. The seminar this week, or workshop as Olle Bälter would call it, differed a bit from the earlier ones. This time we were divided into groups and the entire workshop consisted a competition among us where we could collect points by answering questions about quantitative and qualitative methods. Everyone in the class were really engaged and it was very stimulating to combine learning with a small competition. The workshop though was initiated by Olle who taught us some basic about how a web-survey should look like and easy mistakes that are often made. I learnt that one common mistake that sometimes appears in questionnaires is “two questions in one”, which often makes the answerer really confused. Which of the two questions should I answer? Another thing that was interesting is how people are thinking when it comes down to a question with uneven alternatives where their answer could be considered as “neutral”. Is it better to always have even numbers of alternatives or not?

Furthermore, we discussed a lot of pros and cons with web survey versus paper surveys. I haven’t really thought about it before, but you may get a clear difference in the result of a survey depending on which method you use. The advantages with a web based survey is that you can make it more interactive, it is easy to distribute and it’s flexible (you can adjust a question just by a few clicks) for example. On the other hand, some disadvantages is that you need a computer and internet connection in order to answer a web survey. This leaves out a certain group (often older people without this technique) to be able to answer. Hence, a paper questionnaire is much more pleasant for the eye but it is not as environmental friendly as a web survey. These were just some of the pros and cons that we discussed during the workshop.

It was also interesting which factors that were important in order to make a successfull web survey. First of all you should send an invitation where you introduce your survey, briefly explain the purpose behind it and give the respondent a password for the questionnaire. Later on you send a link with the url to the survey where he/she uses the password to get access to it. It is also important to remind the respondent after a while if he has forgotten to complete the survey. This could be done through phone, e-mail or letter.

Pre-reflection of Theme 5: Design research

Part one

The paper Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration by Haibo Li and his colleagues was about in which way we should synchronize a live football game and the moving ball to a cell phone. Depending on where the ball is, which team who has it for example, different kind of vibrations will be sent to the phone.    

How can media technologies be evaluated?

Since there are endless amounts of different products and services (things we use by looking at them, interacting with it for example) in several areas regarding media technologies, that leads to a couple methods of evaluating each of them. In this specific article where we want to connect a live football game to a cell phone, they’re using a usability evaluation, that of course can be applied on other studies as well. This method evaluate the usability of a product according to three different aspects:

• Effectiveness
• Satisfaction
• Efficiency

The effectiveness part is investigating if a specific task could be accomplished with the specified system. One way of measuring effectiveness is by calculating the error-rate. Satisfaction is measured by looking how comfortable and accepted a system is according to the user. In this study this was measured by questionnaires answered by the participants. Finally, efficiency is how much effort one is needed to put in to complete a task. In this case, this was done by measuring the users reaction time.

What role will prototypes play in research?

When it comes down to research, i think prototypes have a big role in how you should go on and develop your work. Since we today have modern techniques and can use 3D-printers etc. for prototypes, its even more current and the way we can find out how our product works, which parts that doesn’t and how we should improve it in an early stage could be very time saving. To detect problems early, like the way they found that the product was uncomfortable in this specific study, helps a lot in the developing process and you’ll have plenty of time adjusting it until the release of the final product.

Why could it be necessary to develop a proof of concept prototype?

As i understood “a proof of concept prototype” is that you do it before the actual creating of the prototype in order to investigate what possibilities and disadvantages there are. The researchers has the chance to adjust and give input before the prototype is made, and therefore make sure that the prototype itself will be as good as possible. It can be used when the idea of a product is hard to imagine and we want to test it on some users.

What are characteristics and limitations of prototypes?

A prototype could be seen as a foretaste of the final product which leaves us with some limitations but also good characteristics of it. What limits a prototype is that it could be very expensive if the user find some problem that needs to be adjusted and thereafter create a new prototype and so on. This is also very time inefficiently. What’s good though is that a prototype is formed like the idea of a final product. The tester can therefore find errors and problems that can be removed. This help the producers a lot in the developing stage.

How can design research be communicated/presented?

To present a design research, by using a prototype for example, is a good way for the audience to get a preview of the final product. I think that visualizing and demonstrating a prototype is far more efficient than printing out a manual regarding the product for the consumer. However, some sort of combining text with the prototype is of course necessary in order to get a good overview.

Part two

How does a collaborative setting differ from a single user setting as regards methodology used and the results obtained?

What differs a collaborative setting to a single user setting is that when you consider a collaborative setting it is a group of two or more testers and the result you get will be a compilation of all the users. The advantage of this though, is that in a collaborative setting you can get a several different views of the exactly same study since a group of people often perceive things differently. On the other hand, analyzing a single user is more comfortable for the researcher because only one person has to be examined and evaluated.

How can qualitative and quantitative methods in the same study complement each other?

Since qualitative and quantitative methods both have different advantages and limitations, combining these two into one study could lead to a better and more credible result than just using one of them. We should also be aware of that this is not always the case. However, combining these two methods could be done by starting of with a survey for a large number of people in order to get an useful overview and thereafter move on to a qualitative method regarding something you want to investigate deeper as a result of the survey, using interviews for example.

How can using both subjective and objective methods give a better understanding of a phenomenon?

This could almost be quasi to the comparison of quantitative versus qualitative methods and how they as one could lead to a better understanding of something you investigate. In this case, using subjective method is directly based on perceptions and thoughts from a person while objective methods is not based on a person’s thoughts, but more on independent experiments and collected data for example. By combining subjective and objective methods and complement a study with one or the other where it’s needed to fill the gap, a clearer overview and better understanding of a phenomenon could be achieved.    

   

Theme 3 reflections

I really found the theme of this week a lot more graspable and it felt pleasant to finally leave two weeks of more philosophical elements behind (even though some earlier terms and concepts were mentioned during the seminar). I prepared myself for the “research and theory” week by reading the papers which we were assigned and answered the questions according to them. Unfortunately i missed the lecture with Leif on monday since I had a mandatory lesson in another course but the seminar itself was really instructive and i learnt a lot during it.

During the seminar we were divided into groups and talked about our papers and what different kind of theories we have discovered but also about how we define theory. I actually thought I had grasped the definition of theory correctly but when Leif started to describe for us what theory really is, almost the entire room looked surprised. How we should see the difference between theory, knowledge, hypotheses and statements is something that came up during the entire seminar and was hard to completely understand. As i understand the word theory and its meaning now afterwards is that it is propositions we create ourselves that tries to explain and describe different phenomenon and it can be either strong or weak depending on different circumstances. Furthermore, to build a strong theory it has to be supported by a set of proven hypotheses and if the hypotheses are reformulated as a result of a strong theory, it can end up with new theories. Therefore, even if it’s hard to convince people about a new theory and why it should be accepted, it’s still possible and basically could be changed interminably.

When i read my text “The benefits of Facebook “Friends”: Social Capital And College Students” which was about the relationship between Facebook and social capital i never realized that social capital was a theory itself until Leif explained it for us. There are so many different theories we’re witnessing in our daily lives without noticing. It’s fascinating with this course how our perceptions of different concepts that we’re examine could change radically in a second and I will carry with me the knowledge of theory into the coming up weeks.

Theme 4: Quantitative research - before

Part 1

The quantitative article i chose to read was Measuring Mobile Phone Use: Self-Report Versus Log Data published by Jeffrey Boase and Rich Ling. The paper could be found in the Journal of Computer-Mediated Communication which has an impact factor of 2.019.

Two quantitative methods were used in this paper, self-report measures and logged data.

The aim of this paper was to investigate and compare self-report measures regarding voice calls and SMS (Short Message System) usage from cell phones, with logged data in order to find out their correlation. Hence, to be sure that the quantitative methods they were going to use were the most relevant for this survey the authors examined previous researches in the area and found out that investigating the frequency and duration of mobile phone use in the self-report measures were predominantly. Measuring a user's frequency is about finding out how often one is using their cell phone, how many calls or SMS that are received each day. The duration part is about estimating how much time a participant uses his/her cell phone a day.

The method used in this case was a survey including 1382 Norwegian citizens in late 2008 focusing on what i earlier mentioned, finding out their frequency and duration of cell phone use. The self-report done by the participants were later compared with logged data from telecommunication companies collected from the month which the survey took place. The method was divided into two separate parts. The first part was about asking the participants how many times they used their cell phone for outgoing calls and SMS “yesterday”. In the this case the respondent was free to report any number that they considered matching their usage. In the other part of the self-report measure the authors wanted to find out “how often” the participant used their cell phone in order to receive/send voice calls and SMS. Unlike the first part of the survey, this time the respondents had to select from a predefined set-list of responses-categories, for example “2-4 times a day” or “1-2 times a week” etc.

The benefit with this method is that the investigator easy can compare the self-report measures with the logged data and thereafter analyze and point out under- and overreporting with different demographic traits for example. There are some limits as well. The phone usage in Norway may differ from other countries and cultures, and the matter to under/overreport in this kind of survey could be different around the world. Another concern is that this survey does not measure phone activity for a specific day, but for a more general frequency like in this case “yesterday”. People with day-to-day activities may have a mobile phone usage that varies a lot. I also think the credibility of the logged data could be questioned since we’re not sure that family or friends of the participants didn’t borrow their cell phones.

There are some methodological problems in this paper as well. We should be aware of and question the self-measured data in three aspects which are cognitive burden, social desirability and conceptual validity. There may be some occasions when the participant does not recall their exact frequency or duration of time spent using their cell phone. Therefore, the accuracy of these respondents’ answers needs to be considered if a survey is made in order to collect exact numbers of something since it’s easy to over- and underreport an answer. Mainly overreports may also occur since the usage of mobile phones often is jointly with sociability, and the interest of high sociability among friends could give uncertain answers.

Finally, in this particular study I think it’s hard to improve the self-report measures since people find their phone usages as a part of their integrity and therefore are very aware of which information they share (further information than just numbers/short comments could be too intimate for certain people).

Part 2:

1. Which are the benefits and limitations of using quantitative methods?

One benefit of using quantitative methods is that you often make a survey included a lot of participants which leads to plenty of data that you later on easily can analyze, finding patterns depending on demographic aspects etc. Quantitative methods are often cheap to implement and it is time efficient compared to qualitative methods. There are some limitations as well regarding quantitative methods. First of all, acquiring a good result depends on plenty of participants. It’s hard to obtain a result that’s reliable and where you can get an accurate analysis of the data when only a few participated. It’s also worth mentioning that misunderstood questions can lead to an arbitrary survey.

2. Which are the benefits and limitations of using qualitative methods?

Qualitative methods differ a lot from quantitative ones. The benefits with qualitative methods is first of all that you can get a deeper understanding on an individual level and can analyze specific people’s thoughts. This also leads to a limitation. Since you collect data in a way that’s very comprehensive and different from participant to participant it takes a lot of time to analyze all of them. Therefore you have to do the survey on the less amount of participants compared to quantitative methods unless you have unlimited time.