torsdag 28 november 2013

Theme 4: Quantitative Research


This week I have read the article Facebook and texting made me do it: Media-induced task-switching while studying by Larry D. Rosen, L. Mark Carrier and Nancy A. Cheever (2013).

In this study 263 Middle School, High School and University Students were observed during a study session of 15 minutes in their own home. They did this by educating 263 student observers, in how to take minute-by-minute notes on what the subject of study was doing (on-task studying, texting, browsing the internet, eating etc.). Each observer could chose up to three friends to observe while studying for a period of 15 minutes in his/her own home. The authors hoped to come up with strategies on how to cope with technical stress and how to make studying more efficient and on-task.  

All these minute-by-minute notes were collected, and by using regression analysis they could see several interesting patterns regarding study attitude and how likely a student would be off-task while studying during these 15 minutes. For instance: students, who had a preference to finish one task before moving on to the next task, were significantly more likely to be off-task while studying, and that students who checked their Facebook at least once during the 15 minute study period did in fact have lower grade-point average than the average study group person.

The limitation to this study is that they had to depend on the observers, who had no other experience than the received training session on how to take notes. The authors had no means to check whether notes were taken in a correct or uniform way.  It is also likely that the subjects who were observed tried harder than usual to stay on task during these 15 minutes of study as compared with if they would have been alone in their room. The benefit with being observed by a friend, and this was important for the study, was that the studies could take place in an environment as normal as possible for the students, and without feeling uneasy about being observed by a stranger.

In this paper, as well as the paper I read last week, I found it interesting how they excluded tests persons from the study and on what grounds they did so. I guess you have to exclude people, because not everybody will conform to the study or respond to questionnaires, it just had not occurred to me that you need to declare also those people in the research paper.  I also realized that I am not up to date with all the statistics calculations they perform.

As discussed above I think the main methodological problem in this study is that the test persons who are being observed while studying might not behave like they normally would, even though they tried their best to make them feel comfortable at home. If they had filmed them during a week with a video camera, it might be more realistic as the test people might forget about being observed. This however would make the study more expensive and it would be more difficult to collect all the data (like number of open pages on the computer during the study session).  

Quantitative methods:

In a quantitative research the goal is to determine the relationship between one thing (an independent variable) and another (a dependent or outcome variable). In my study above the independent variable was "distractions during studying" and the outcome variable the "grade point average".

The benefit of using a quantitative method is that it tries to control the study environment in order to isolate and recognize the variables in study. It aims to collect a large amount of data to study, and when it comes to study people, it tries to use a large and random group in order to have results which are representative over the entire population. The limitations lie in setting up the test case not tainted with outside-the-study variables, as well as finding a test group that is in representative of everybody.   

Qualitative methods:

Qualitative research method try to investigate the how and why of question, and generally investigates the human side to problems and peoples experiences. It generally generates rich, detailed data that leave the participants' perspectives intact. With this kind of study the difficulty is making sure your results are broad enough to make general recommendations. It is often more difficult to collect a large amount of data using this method, and you need to rely on a narrower field of view or results. The smaller data-references might make it more difficult detect false or fabricated and unrealistic answers.   

References:


8 kommentarer:

  1. Hi Jenny, your chosen paper sounds very interesting since it is a very valid topic these days with all the smartphones which makes it so easy to multitask and procrastinate. I find it also interesting how they have done the method of research with all the observation. I am wondering though if this can be called a quantitative method? I tried to google examples of quantitative and qualitative methods before i looked up my paper for this theme because i was not sure what to look for. The perception i got then was that observation was a qualitative method of research. Maybe i got it wrong or that observations can be done differently depending on if you want quantitative or qualitative data?

    SvaraRadera
  2. Hi Filip, you have a valid point and I am not absolutely sure myself if I have correctly classified this study as quantitative.

    I have been thinking about this a lot and here are my thoughts: although the data collection method in this case was observation (which, just like you say, is normal in a qualitative study) the data they collected in this study I believe is of quantitative type. Every minute the observers noted down what websites were visited (if any), how many computer windows were open, did the person take breaks, were they on-task (studying) or off-task etc, and these observations-numbers were the base of the study. They did not ask the students how they felt about their study or why they visited different websites etc. also the purpose of the study was to determine if there were any patterns between number and type of off-task-breaks, and the how that affected the grades of the students.

    On the other hand, the study did involve human behavior and they wanted also discussed ways on how to improve staying in focus for longer periods of time, which is more of a qualitative study.

    Well, I am not absolutely sure about this. This might be a good question to bring up on the seminar next week.

    SvaraRadera
    Svar
    1. There is an interesting conversation going on here and I plan to take part in it!

      I have to admit that in the beginning I was not sure on whether observation could be considered as a quantitative method. But as you say through observation they collected statistical data and numbers. So, I would argue that in this study there are quantitative measurements. Or maybe, that a combination of quantitative and qualitative methodologies were used.

      Radera
    2. That is my opinion too, that this was a quantitative study with some qualitative elements. Thanks for supporting my case! :-)

      Radera
  3. Hi Jenny!
    The used method in your chosen paper was an interesting idea that I don't think I have thought of before, letting the students do the observation themselves by taking note of a friend. That way it could eliminate the risk of the observer having an influence on how the study object behaves. With a friend that they know and are comfortable with, they would likely be able to behave (somewhat?) more "normally", as you wrote.
    On the other hand you mentioned that there's no guarantee that the notes are correct. The students aren't experienced observers. I would feel doubtful leaving it up to them. They could have excluded an activity for example.
    Observations is a good method to use when analyzing behaviours, but I think that as long as the respondents are aware of them being observed, the results will be doubtful. I think observations might be best suited for observations from a distance and "secretly", for example how the general public behave on the public transportation, out on the streets, in stores, etc.Though thinking about the other different methods (interviews, questionnaires, etc), there's never a guarantee of the accuracy of any delivered data.

    SvaraRadera
    Svar
    1. Hej Zahra!

      You made a valid point when discussing the limitations of observation. Observation always involves the risk of influenced results, even if the observer is a friend. If the subject of observation is aware that it is being observed, it might behave differently. And since our conclusions are based on collected data, it could be said that the results are doubtful.

      You propose that secret observation might provide us with accurate results. But since some of the participants were not adults, I guess that secret observation might lead to legal issues. Regardless of the age, I would have to say that even though I accept the importance of secret observation, I would not feel comfortable using this method. If I was a subject of research, I would like to be aware that I was being observed.

      Radera
    2. Hi Zahra and Katerina,

      Even though the data is surely affected by the test person being observed behind his/her back, I believe that it is fair to say one thing: if you cannot resist yourself from checking your Facebook account at least once during a 15 minute study session when you are being observed by somebody it is likely happening at least the same amount of times when not observed (if not a lot more often).

      Radera
    3. Hi Katerina,
      I would like to comment on your comment to my comment. :)

      You pointed out that a secret observation might lead to legal issues, but I'm wondering if that's really the case? You seem to have been thinking of applying it into the case of Jenny's research paper, to secretly observe the students' study behaviors. In that case I understand of course and I would want to know too (but it would be very difficult to secretly do an observation there because you will have to sneak around behind the students while they are studying at home/school and look at their phones if they are on Facebook or not.. and without getting caught by them. Might require some ninja skills.).

      But I was thinking of more general open behaviours of the public, like out on the streets for example and not as private. You could stand somewhere far away with a pen and paper and take note of how many times people choose to cross a certain street on a certain day and a certain time, or something. The people are random, they are anonymous and the only notes you take are numbers or how/what they do with something or whatever the subject is. Another example of an observation is standing by a street and counting how many red cars pass by during one hour. I doubt such studies are illegal. That's why I thought such cases would be best suited for an observation where only you are aware of the study. That's my explanation. :)

      Radera