torsdag 30 oktober 2014

Combining different methods in order to answer complex research questions

This course touches a variation of different topics all from philosophy, theories, methods and so on. This reflection is about how a researcher should think before “combining different methods in order to answer complex research questions” and why this approach can be beneficial.
In the first theme we discussed a statement from the German philosopher Immanuel Kant:
“Thus far it has been assumed that all our cognition must conform to objects. On that presupposition, however, all our attempts to establish something about them a priori, by means of concepts through which our cognition would be expanded, have come to nothing. Let us, therefore, try to find out by experiment whether we shall not make better progress in the problems of metaphysics if we assume that objects must conform to our cognition.” KANT.
One way to analyse this statement is that Kant challenge and urge us to change the way we reflect and to think in a “reverse order” from the norm. Thus giving us new perspectives and allowing us to learn more.


In multimethod research, that is including more than one method in the research, the use of different methods gives new perspectives - exactly what Kant wants us to do. This is a philosophical thought that supports combining different methods.


In theme 5’s guest lecture, Haibo Li claimed that a well thought problem definition for research is crucial. If the problem statement is well-defined it is easier to spot the solution, and I would also argue that this helps a researcher to choose correct methods for the study. It is also essential for a researcher to find “easy” and straight-forward solutions instead of complicating the results to make it more complex-looking. This also applies for choosing methods. It is not important to find and combine the most complicated methods but instead to use methods that are appropriate for the study but also feasible for the researcher.  
When doing research it is important to identify what kind of theory or theories to use. Theory is system of ideas intended to explain something. In theme 3 we analysed a text from Shirley Gregor which distinguishes five types of theory: (1) Theory for analysing “what is” , (2) Theory for explaining “how and why”, (3) Theory for predicting  “what will be not why, (4) Theory for explaining and predicting, EP - theory “what is, how, why, when and what will be” and (5)  Theory for design and actionhow to do something”. Theory 1-4 build on each other with theory 1 being the most basic type of theory that only aims to describe a subject.
After identifying what type of theory this research aims to achieve, the researcher can choose methods that are the most appropriate to fit the goals of the research.

When choosing methods it is crucial to be knowledgeable about them. When reading about qualitative and quantitative methods I learned that they are almost like opposites. In Table 1 some benefits of both methods are mentioned, these were discussed at the seminar, for theme 4, with Olle Bälter.
Table 1: Some of the pros with Qualitative and Quantitative from the seminar with Olle Bälter
Qualitative
Quantitative
  • information about individuals thought and experiences
  • in-depth results
  • smaller nr of participants
  • very specific answers
  • whole picture answer/results
  • useful for describing complex phenomenon
  • easy to compare and compile result
  • easy to visualise (graphics)
  • involves large nr of people
  • generic results


Qualitative methods, such as interviews, gives specific and in-depth result. However, quantitative studies mostly collects data and commonly results in statistics which gives an overview of the subject and is also easier to visualise in e.g. table or graphs. Through their differences, several perspectives can be presented by using both methods. Since these two methods complement each other very well using both has become fairly common and is called “mixed methodology”.


Eva-Lotta Sallnäs argued, during the guest lecture, that the results of a research are dependent on the research method. I agree with this since for example qualitative methods concludes in complex answers while quantitative is commonly compiled in numbers. Thus, the researcher needs to be aware of the results from various methods to be able to use and combine methods in such a way that it enhances and improves the results.


Further, Sallnäs gave some examples of different types of research and which methods that can be appropriate. For example, the researches should use other methods, than those used in previous similar studies, to gain new insight and perspectives. For very new areas, especially those with few possible participants, qualitative methods are suitable as future research can use these results as a base. When creating prototypes and ultimately products design research should be one of the methods used in the research.


During theme 6 case studies were discussed and which aims at solving “real” problems in the world. The more cases included in a case study, the more legit the theories that the study generates, according to Eisenhardt. Since case studies uses several methods they return a lot of data and it is a challenge to compile all of it.

Conclusion

What I have learned from this course is the importance and value of understanding that various methods are used for different reasons and generate different types of results.


Diverse methods can be used to complement one and other. Therefore, it is important to be aware of the pros and cons, for the methods, to be able to combine them in a way so they will complement one and other leading to maximising the final result. When combining methods the researcher needs to consider if he or she has the resources to execute the study in an appropriate and satisfactory way, since combining methods is complex and commonly generates a large amount of data. However, this gives a fuller and more comprehensive answer to the problem statement, which directs me to the conclusion that combining methods is an appropriate way to tackle complex problems - if it is feasible for the researcher.


The information that I have collected gives me a heads-up for the challenges I will face when writing my master thesis. The knowledge that I now have about theories and methods will be of utmost help in the future.






tisdag 28 oktober 2014

Comments

I have collected all my comments in a google docs since it is more structured that way (I have a clickable table of contents). However, I will also post all my comments here just in case. I do recommend google doc since all the comments make around 17 pages. Here is the link: https://docs.google.com/document/d/1-xt6-7AAblpdbis1MEFxtiJY0nt7LaJZT55FL5A8sXA/edit?usp=sharing

I have collected 5 post for every theme which makes the total to 30 comments. However, since I did not know from the beginning that we were supposed to collect the comments some have disappeared since I couldn't trace and find them. 

Since I've enjoyed writing comments and tried to make "qualitative" comments, my comments have become quite long. 

Here are my comments:

Theme 1

Comment 1.1:

http://dm2572emelie.blogspot.se/2014/09/reflections-on-lecture-seminar-on.html?showComment=1413403946342#c6901960841901977410
Me: I like the way you make a differentiation between "listening" and "hearing". I understand the reason why you do this differentiation and think this applies to more than just unknowing babies. How many times in our lives do we only "hear" instead of truly "listening"? I thought this was something worth thinking about…

Comment 1.2:

http://dm2572emelie.blogspot.se/2014/09/reflections-on-lecture-seminar-on.html?showComment=1413403946342#c6901960841901977410
I believe that I understand what you mean. And as you say - teachers, politicians, police etc should have this personality trait to be able to see and listen to people and at the same time being patient with them. However, I believe that too many in these occupations don't have this. Which is really sad, especially when it comes to teachers who are supposed to encourage learning and be some kind of security point for children... I've heard too much about kindergartens where the teachers have no patience with the children and just yells at them…

I think this comment kind of got into a new subject, but yeah. Just wanted to share my thoughts with you!

Comment 1.3:

As you wrote in your reflection you say that a newborn child collects impressions already in the womb and when she finally sees her mother she connects the visual image with her experiences. Is this the way you perceive knowledge? That our knowledge is built on experience? Or can it be attained in other ways?

Comment 1.4:

Hi Johan! I also had a very difficult time to understand the text. I guess it is because they are so different from earlier material that we have read as engineering students on KTH. However, even though they were really hard, I can in hindsight understand that they were useful for us to read. It opens our perspective and forces us to realise that there is other ways to see the world - not only scientific or technological.

Good reflection,
/Stephanie

Comment 1.5:

Hello Gabriella! Good reflection!
During my seminar we talked a lot about Kant's saying "Perception without conception is blind". I found this really interesting since I have never thought about it. I believe that it means that most people and even babies are able to see with their eyes. However, if we don't gain experience or reflect upon what we are seeing we can never learn or gain knowledge about the item. For example babies learn language through other people who point at objects and say the name. The babies thus see the object and start to connect it with words and it learns to say for example "bed". Next time the baby sees a bed, it also knows that the beds are soft and that you can lay in them. This is because they have learned something about the item more than just the word by perceiving it - that is by touching, seeing it and gaining experience over it. I hope this made any sense.
/Stephanie

Theme 2

Comment 2.1:

I thought about nominalism as trying to get away of grouping objects. That there is no "real" horse that all horses should look like. In a nominalistic way of thinking no objects should be grouped together but individually "analysed". I think this still can be seen in today's society in e.g the feminist debate. Then the picture of the "real" woman would be a housewife - cleans, cooks and takes care of the children and ultimately is inferior to her husband. Today, at least in the Swedish society we have moved away from this way of thinking but are trying to move away from it further. I guess I believe that the best way of viewing men and females are actually viewing them as individuals rather than what sex they have... which is a bit nominalistic.

Wow, that was a long comment...

Comment 2.2:

Hi!
I also felt that this weeks lectures and seminars were more understandable than the first week. In general, I prefer seminars over lectures since one participates more actively in those and therefore through communication and discussion learns more. The first weeks seminar was just like another lecture, and I believe that is one of the reasons that I never really got the hang of theme 1. That's also the reason why I liked that Henrik connected the concepts of theme 2 to theme 1, which made it a bit more understandable.

The discussions on the seminar were also kind of political-oriented which made me see politics with "new eyes". I wonder how much that has really changed from the media today and the media from 50 years ago...

Comment 2.3:

That's interesting! I also believe that historical circumstances like politics, affects the way that the politicians use mass media and that in turns affect the society and the people living in that society. Therefor also changing the culture in the society

Comment 2:4

I agree with what you are saying about stepping out of old ideas and leave them. However, I believe that is the difficult part of it all. Changing people's and most importantly society's way of thinking is very difficult. In some very modern countries (like for example the U.S) the housewife-thinking style still exist. The most effective way to change this would probably be by offering parental leave to both parents (it's not
offered for neither). After some time, both fathers and mothers being home would become normal. However, this would mean that change is happening by force of the state. I wonder, how much is the state really allowed to go in an dictate how to live our lives in our homes? I mean, in this case, it would be a good change - but that is only in our perspective. Having a state that dictates this thing could be misused very badly. But if not this way then which? The other ways are so slow, just as you say. In Sweden we've had the state kind of helping us in the way of gender equality when it comes to taking care of children by having parental leave divided to mothers and fathers. Many disliked the fact that the fathers could not "give" all their days to the mothers. But today it's kind of normal, right? So what is the correct way of making change?...
I believe that having a nominalistic way of seeing some things are vital due to structure. We need structure in our lives and our languages and that happens by defining things in a nominalistic way - like the fact that all chair are called chairs. Through this definitions, communication is possible through language and words since we both have the same interpretation of the words.
Sorry for my very long answer.

Comment 2.5:

Hi!
I also liked the parallel between to the political world. We talked a lot about communism, nazism and capitalism and also how they are characterised in media. Media had a really big impact when providing propaganda to make people believe in the different ideals, e.g the Nazis and their famous propaganda about the Jewish. This made me wonder to what extent this is used today, and how much can media affect people today?
Maybe it is difficult with ideals but I still believe that media and marketing has a big impact on people's decision especially for purchasing goods. The commercials make us believe that we need their products, for example all the hype about the latest iPhone 6. How do you think the situation is today?
And you also discuss a bit about the "world of the senses" and the "conceptualised world". Which one do you think is most accepted today? And which one should be the most accepted? Or should it be a mix of both?

Nice reflection by the way! :)


Theme 3

Comment 3.1:

Hi!
Good reflection and nice pictures!
I just wanted to add that on my seminar we talked about different levels of theories and that the higher levels contained and included the lower levels of theory. So for example if the research paper that you investigated had theory 2 (theory for explaining) it automatically includes a bit of theory 1 (theory for analysing). I had not understood this before the seminar, but during the seminar I realised that this way of seeing the different types of theories made a lot of sense.
It also arguments for you saying that the higher "nr" of theory type the more complex the paper is - because it includes more types of theories. However, as you also mentioned, it depends of the researchers and if they've made a good job etc, but I guess you could say that this is valid for papers that are written by the same author and then are compared.

However, I do not agree with you that the higher "nr" of theory the more valuable the research paper is. I believe this depends on the problem at hand. Some problems or areas to research are very new - and therefor maybe only the first type of theory is applicable. This doesn't make this research less valuable. Or at least that's what I think.

Sorry for my super long comment.
/Stephanie

Comment 3.2

Hi! Nice reflection!
I agree with you, I also thought that I understood the concept of theories etc and what they really meant but as you say they are far more complex than I initially thought.

I really liked your explanation of the differences of hypothesis and statement. That the statement is "harder" and more matter-of-fact. When a person says a statement it is often taken as the truth. A statement is e.g - "that purse is black". But a hypothesis would maybe be more like "All people think black purses are pretty" - which could be proven wrong.

Good reflection! :) /Stephanie

Comment 3.3

Hello!
Just like you, I also had a very difficult time in interpreting what kind of theory that was used in the research paper that I chose to read. On the seminar it was mentioned that the types theories was build upon each other and therefore kind of mixed. Theory of analysing is the most basic one. However, theory for explaining (the second theory) is a bit more complex and it also includes a bit ot theory of analysing. Later the third theory includes a bit of theory two and one, and so on. This kind of made it a bit easier for me to understand the different types of theories.

The question that you mention, "how many trees represent a forest" was really abstract for me. Sure, we can have a theory about how many trees that are needed, but what is actually correct? Since it is not stated how many trees that represent a forest in the definition of a forest one can not truly find the true answer... or?

Good reflection!
/Stephanie

Comment 3.4:

Hi!
I agree with you that this weeks readings were a lot easier to understand and also felt more connected and useful to the master thesis.

I really liked your expression that theory is the bridge between raw data and hypothesises! We hadn't talked about this on our seminar so this helped me to get some further understanding about what theory really is. I still think that it is difficult to truly grasp the full meaning of the word "theory". It's easier to say what it isn't, (like for example raw data), or what do you say?

As you say it is truly a complex word with many different definitions and interpretations. However, I'm glad that the seminar helped you gain more understanding of the word.
/Stephanie

Comment 3.5:

Hi!
Good reflection.
For me it was easier to grasping the different types of theories when it was explained that they included each other. That is that, theory 2 (theory for explaining) includes theory 1 (theory for analysing). This makes a lot of sense since explaining "how and why" (theory 2) can't really be explained without stating "what is" (theory 1).

I liked how you connected theory 3 (prediction) with earlier concepts that we learned. You say that the outcome of the theory 3 should be an "a priori" prediction, which should be pretty legit. This helped me understand a bit more what "a priori" actually means, since I thought it was a difficult concept.
Thanks!

/Stephanie






Theme 4

Comment 4.1:

Hi!
I did interviews for my bachelor thesis so I've never really learned how to correctly do questionnaires or surveys. I always kind of thought that they were pretty easy to do. However, it seems like I was pretty wrong. When doing questionnaires it is very easy to write ambiguous question that can be misunderstood. In an interview, usually the interviewee would ask if he/she didn't understand the question and then the interviewer can explain. This is not really the case for questionnaires, and even impossible for the online ones..
Thanks for your comment!

Comment 4.2:

Hello Sofia!
Thank you for your comment.
I think that when you do a "think-aloud" it is important to try to set an environment that is as close to what the "real" environment will be for the "real" respondents. Of course, it's not going to be identical since the “think-aloud” respondent will actually be sitting with you.

The respondent should be in your target group and should have similar knowledge as the people in your target group. If you tell the respondent more information (than what you would tell the "real" respondents) it is possible that he/she has less misunderstandings when doing the survey. This way it's possible to miss some possible problems that the "real" respondents may encounter. I think that the “think-aloud” and “real” respondents should have the same information about the questionnaire/survey for the best results.

I hope this made any sense!

Comment 4.3:

Hello Emelie!
I really agree with the part where you say "I’ve understood the importance of thinking and analysing when choosing method(s)". I think choosing methods for a research paper is very essential because this will affect the outcome and the result of the research. Therefore it is very important to choose methods carefully. Also one must consider which of the methods are best suited for the research question. The data collected from different methods are very different (qualitative vs quantitative) and the researcher should think about what kind of data that is most fitted for the paper and what kind of data that he/she wants and has the tools or knowledge to analyse.

Good reflection!  

Comment 4.4:

Hello! I agree with you that the information in this course would have been useful for the bachelor thesis! However, in the bachelor thesis we received a lot of help and guidance. We also had a course book about "how to write a good essay". At least for me and my partner this was extremely helpful. But, we used qualitative methods and therefore didn't read anything about quantitative methods. I believe that this course and the bachelor thesis (the success and the mistakes that we can learn from) together will give us a good asis for the master thesis!
Nice comments! :)


Comment 4.5:

Hello Malin.
Great reflection!

I like the key points that you are taking with you from this seminar. Especially the second one which essentially is that statements/questions should be as clear as possible and not contain ambiguity. I think a good way to avoid ambiguity is by not having several statements in one question. So instead of writing as your example "Some people wander aimlessly through life, but I’m not one of them” which is very difficult to answer since the participant doesn't know if he/she is answering to the statement that "some people wander aimlessly through life" or "am I one of them?" or both? The researcher should write something like "I wander aimlessly through life" or "I live life without aim and goals" and then the participant can answer if he agrees or doesn't. This seems more straightforward and I believe that the answers still gives the same information for the researcher.

I also agree with you that the seminar was a bit confusing but fun :)



Theme 5

Comment 5.1:

Hello!
I also liked Haibo's lecture. And as you say the clip with Johnny English was really brilliant when showing that solving a problem can be so much easier than what we think.
As you said, probably most people would benefit from trying to improve their problem definition instead on just focusing on the actual solution. I also believe that students/researchers should try to find the "easiest" solution and not just be most "cool"-looking. Especially, us technology students that want to solve everything with the newest and coolest technology but sometimes it's better to not over-complicate things and just as you say - take a step back and look at what the problem really is...

Good reflection! :) /Stephanie

Comment 5.2:

Hi!
As you say this topic is a bit easier to understand than the philosophy topics. It's probably because we are more used to these kind of theories.

However, I still believe that we could have learned more by having seminars as usual. I also believe, like you, that they are the best sources of gaining new "knowledge" (I'm a bit careful using that word now...). Discussing with others forces you to see and reflect from different points of views and aspects. I believe that challenging your own thoughts and ideas can only be beneficial.

Good comments! /Stephanie

Comment 5.3:

Hello!

I'm sad to hear that you didn't learn so much from this seminar. I really enjoyed this seminar and learned a lot from it. Most of the things that Haibo talked about may seem "basic", but it is often the basic things that people forget or do "wrong" because it is supposed to be so easy. However, I believe it is very important do the basic thoroughly since this is the basic for the rest of the project.

One key point that I thought was really important that Haibo mentioned was that the researchers should switch the focus from the solution to actually defining the problem. Often if a problem is well-defined the solution will be much easier to see or visualise.

I also liked the fact that Haibo separated the process from idea to prototype in different steps since it makes the process a bit more easy.

Good reflection!

Comment 5.4:

Hello! Good reflections!
I didn't really get the part that qualitative methods are better if one is researching a new subject or area. I guess this kind of makes sense since there probably is few participants to include in the research if it's a very new subject. In these cases where the subject is new - then the target group are probably very nished and few, therefore a quantitative method wouldn't be sufficient to really investigate the subject.

I also liked the part of "flamming". I've never heard it in this context before. For me "flamming" meant when people rage when playing computer games for example if it's a team game (but with strangers) and someone gets super mad because one of the team members are bad. However, I guess it is used in different contexts.
I also recognise myself in your description of flamming as people misunderstanding each others' text because they can't see each others faces. I usually like to use exclamation marks because I think that dots seems so serious. But once, my friend thought I was angry at her because of my exclamation marks. Which is kind of funny because it was the opposite of what I want to with my exclamation marks. I guess everyone uses emojis or other punctuations marks differently.

Keep up the good work!

Comment 5.5:

Hello!
As you mention there is the possibility to videotape participants when answering surveys or being in interviews so that the researcher later can investigate and analyse the participants questions, conversations and even expression and gestures. However, I believe that this is sort of observation technique is very difficult to master since the researcher needs a lot of experience and knowledge about how people act and move. For example, I would never be able to do a good research through observation techniques.

You say that an interview can lead to deeper results than the videotaping technique. I both agree and disagree with you. Interviewing is easier than analysing videotapes. However, many gestures and facial expressions happen unconsciously and since the participants are unaware of them they can't always explain them.

Good reflections!  





Theme 6

Comment 6.1:

Hey Sofia!
I agree with you, it was very difficult to find research papers that only used one or the other method. I believe this is because the two methods each other very well since it gives two very different perspectives. Quantitative is visualised by numbers and statistics which gives an overview mostly answering what (at least how I see it)- while qualitative gives more in-depth answers based on personal experience which can answer why a lot better.

All of the 25 participants were superparticipants so they had a lot of data to interpret. I was also quite shocked, but I believe it was a bit more smooth and faster to interpret this data than actually having interviews or something similar.

Thanks for your comment!

Comment 6.2:

It was very interesting!
No he didn't go in-depth in describing a specific case study about ADHD it was more of an example of what a case study is. Leif also said that even though ADHD today is an accepted term (and getting an ADHD diagnosis), in the future it might disappear (that is no longer seen as a "condition") or turn into more complex terms (like dividing up ADHD since there are different types of it) .

I think this is very interesting since it means that "things" (diseases, politics, science etc) that is accepted today may not be considered legit in the future. That means that the "truth" today may change in the future. I never really thought of science developing in that kind of way, but I suppose it is correct.

Hope it made some kind of sense! Thanks for your comment!

Comment 6.3:

Hi Malin!
I really like that you have summarised the 3 key points from the seminar.
As you, I also had a hard time understanding what a case study is and how it is different from other forms of studies.

As you say case studies can sometimes help to give a new concept a definition and I would say also some trustworthiness. One example that Leif mentioned was about ADHD. Before ADHD did not exist and no one had that diagnosis. However, when ADHD was discovered case studies helped to establish the definition.

I was also shocked about the fact that case studies should include more than 3 cases to be seen as valid. But it makes sense because the more cases included - the more valid and trustworthy the results are.

Good job!

Comment 6.4:

Hi!
I was also very confused about what a case study is and how it is different from other studies. What I got from the seminar is that a case is:
* something that exists in real life with boundaries or limits
* it includes different methods (e.g both qualitative and quantitative)
* it is inductive - as you also mentioned.

Case studies generates theories and therefore it is important to have several (more than 3) to make the theories valid and credible.

Good reflection :)

Comment 6.5:

Hi Johan! Great reflections!
In my seminar group we also spent by far most time discussing what a case study really is. I like how you define a case study in the end and as you say case studies need to use various methods. It also needs to include several cases for it to be valid, preferably more than four. This way the case study becomes more valid and legit according to Eisenhardt.

One important thing that I would like to add is that case studies are always about real phenomenas in the world. It can for example be a company experiencing some problem that they want a solution to or it can even be solving a crime. Since the definition of case study's are pretty broad I believe it makes it more difficult to understand.


/Stephanie :)