researcher
Active member
- Messages
- 25
- Reaction score
- 0
>>In general my view is that when possible a deductive stance is better because it implies that you know what you are looking for and the outcome can be predictive.<<
Research Methods
- Thread starter Hugo
- Start date
- Replies 164
- Views 32K
British Wholesales - Certified Wholesale Linen & Towels | Halal Food Gastronomy | vBulletin 4 php 8.0 patch
researcher
Active member
- Messages
- 25
- Reaction score
- 0
I know what I'm looking for and why and how however the outcomes are not predictive - I do not know what my findings might be although I have a fair idea in relation to two of the data sets. In terms of one data set I'm not sure what the outcomes might be... I might be suprised or it might simply validate what I already knew...
inductive or deductive?? can research have elements of both?
inductive or deductive?? can research have elements of both?
Things like positivism are just ways, loosely of thinking about what would constitute date. Is data just things we can see, touch etc or can it be things that are just in our minds.
With induction/deduction its about how you want to think about something and as we have discussed that itself will drive what you want as data.
They key idea is I suppose whether you want to predict or describe something and mostly in social research it seems to me you are describing and hoping that the description hold in general. For example, suppose you were looking at marriage break up and you interviewed 10 couples then it would be hard to say with certainty that you know unequivocally why marriages break up but you could described in detail why these 10 marriages broke up and that might be useful information but it would never become a certainty.
Not really because then what you are saying is that you both have a theory and you don't have a theory and that must be an absurdity.
If you say start out inductively then think of it as a task to describe something, that will be your outcome and based on that description you may be able to make inferences. In the marriage break up case you end up with a general description of marriage break up and out of that you could say the most common causes of marriage break up is X therefore you recommend doing Y.
That is the best you can do because next week, next month, next year it is quite possible that things will be different so an inductive outlook is the best you can do. So in a way you can have ideas about causes and try to follow them up but the situation will always be on the move
Last edited:
researcher
Active member
- Messages
- 25
- Reaction score
- 0
Thanks Hugo,
Understood... mine is most definitely inductive!
Another question can you combine two epistemological stances i.e. critical realism and positivism for example. or positivism and social constructionism?
Understood... mine is most definitely inductive!
Another question can you combine two epistemological stances i.e. critical realism and positivism for example. or positivism and social constructionism?
Yes it is allowed to have mixed methods as you suggest but I would normally in a student project suggest you stick to one simply because of time. So one might argue the case for one or the other as being most suitable and then choose one that one will help you choose the data you want.
In all research if you make you methods and choices plain then you can do no more and a reader can judge it fully.
Re: Research Methods: Problem, Outcome and Actor
Here is just a short set of very simple examples to illustrate the notions of problem, actor, outcome and target (they are not meant to be examples of real projects)
The actor is anybody who can use the outcome but it must be credible: if you said your actor was the Prime Minster or President then no one will take you seriously of if you say my actor are ALL managers of SMEs in Hong Kong people will just say you are stupid.
Similarly, if say your outcome was a specification for some software application then the actor cannot be the sales manager or accounts manager or training manager as in this case it is obvious the actor must be a computer programmer.
Target and outcome are linked because we USE the outcome to GET the target effects. Think of it like this - if I (I am the actor) were riding a horse and the horse was going too slow (problem) I would use the whip (outcome) to make the horse go faster (target)
What you are saying here is that YOU (the actor) will use something called "reduced waistline" to get a target of a reduced waistline and better looks. But that does not make any sense does it?
What you are saying here is that YOU (the actor) will use healthy food recipes get a target of a reduction in calories. But it’s not just a reduction in calories we want is it so you are mixing up a means with an outcome.
Now it makes sense and we can see that many different outcomes could bring about the change we want –
In real life situation of course you will have to convince a lot of people that what you propose will in fact work. Here is a full example of a real project and as you can see it sums up the whole problem very concisely
The list and rationale (outcome) of suspected redundant processes (problem) will be used by area managers (actors) to decide what vestigial manual processes can safely be removed without affecting company business throughput and in this way generate the target of increased effectiveness centred on the full use of the new CRM system.
Here is just a short set of very simple examples to illustrate the notions of problem, actor, outcome and target (they are not meant to be examples of real projects)
The actor is anybody who can use the outcome but it must be credible: if you said your actor was the Prime Minster or President then no one will take you seriously of if you say my actor are ALL managers of SMEs in Hong Kong people will just say you are stupid.
Similarly, if say your outcome was a specification for some software application then the actor cannot be the sales manager or accounts manager or training manager as in this case it is obvious the actor must be a computer programmer.
Target and outcome are linked because we USE the outcome to GET the target effects. Think of it like this - if I (I am the actor) were riding a horse and the horse was going too slow (problem) I would use the whip (outcome) to make the horse go faster (target)
Problem: getting bigger around the waist
Target: reduced waist and improved looks
Outcome: reduced waistline
Actor: myself
Target: reduced waist and improved looks
Outcome: reduced waistline
Actor: myself
What you are saying here is that YOU (the actor) will use something called "reduced waistline" to get a target of a reduced waistline and better looks. But that does not make any sense does it?
Problem: my excessive weight
Target: reduction in calories
Outcome: healthy food recipes
Actor: myself or could be my wife depends who cooks and will use the recipe
Target: reduction in calories
Outcome: healthy food recipes
Actor: myself or could be my wife depends who cooks and will use the recipe
What you are saying here is that YOU (the actor) will use healthy food recipes get a target of a reduction in calories. But it’s not just a reduction in calories we want is it so you are mixing up a means with an outcome.
Problem: getting bigger around the waist
Target: reduced waist and improved looks
Outcome: diet sheet or an exercise plan or a set of wholesome recipes etc
Actor: myself or my husband etc
Target: reduced waist and improved looks
Outcome: diet sheet or an exercise plan or a set of wholesome recipes etc
Actor: myself or my husband etc
Now it makes sense and we can see that many different outcomes could bring about the change we want –
the actor uses the diet sheet to control food intake and this leads to the target of a reduced waistline and better looks or
the actor uses the exercise plan to go to the gym and work out to burn off excess fat, leading to the target of a reduced waistline and better looks or
the actor uses the wholesome recipes to make calorie controlled meals and eating them leads to the target of a reduced waistline and better looks.
the actor uses the exercise plan to go to the gym and work out to burn off excess fat, leading to the target of a reduced waistline and better looks or
the actor uses the wholesome recipes to make calorie controlled meals and eating them leads to the target of a reduced waistline and better looks.
In real life situation of course you will have to convince a lot of people that what you propose will in fact work. Here is a full example of a real project and as you can see it sums up the whole problem very concisely
The list and rationale (outcome) of suspected redundant processes (problem) will be used by area managers (actors) to decide what vestigial manual processes can safely be removed without affecting company business throughput and in this way generate the target of increased effectiveness centred on the full use of the new CRM system.
Re: Research Methods Selection
In research the outcome will mainly set out to do one of the following: understand something, explore something, describe something, explain something, improve something, build something or prove something. To do any of this you need a method, a best way of working and this is referred to as a Research Method.
A Research Method is a model or framework in which you set your research design – this is useful because each model will have features that suit what it is you are doing. Here are some very common Research Methods:
Choosing a research method will depend on many factors and you can see from this list it is not a simple matter:
The simplest guide is to think about your basic intention – ask am I setting out to: understand, explore, describe, explain, improve, build or prove. Here are some methods that are well suited to particular research intentions:
In research the outcome will mainly set out to do one of the following: understand something, explore something, describe something, explain something, improve something, build something or prove something. To do any of this you need a method, a best way of working and this is referred to as a Research Method.
A Research Method is a model or framework in which you set your research design – this is useful because each model will have features that suit what it is you are doing. Here are some very common Research Methods:
Case Studies,
Vignettes,
Action Research,
Experiments,
Quasi-Experiments,
Surveys,
Biographies/History,
Grounded Theory,
Ethnography,
Requirements Gathering
Vignettes,
Action Research,
Experiments,
Quasi-Experiments,
Surveys,
Biographies/History,
Grounded Theory,
Ethnography,
Requirements Gathering
Choosing a research method will depend on many factors and you can see from this list it is not a simple matter:
Context,
Time available,
Skills available,
Practicalities,
Access to data,
Reason for the study,
What kind of outcome you want,
Cost,
Quantitative/qualitative,
Scope and scale,
Control,
Sensitivity of the data and so on.
Time available,
Skills available,
Practicalities,
Access to data,
Reason for the study,
What kind of outcome you want,
Cost,
Quantitative/qualitative,
Scope and scale,
Control,
Sensitivity of the data and so on.
The simplest guide is to think about your basic intention – ask am I setting out to: understand, explore, describe, explain, improve, build or prove. Here are some methods that are well suited to particular research intentions:
Case Studies - understand a situation
Vignettes - explain a situation or phenomenon
Action Research - improve a situation or process
Experiments - prove a nominal theory of some kind
Surveys - describe a situation
Grounded Theory - explore a situation or setting
Example - Suppose my Research was about looking at the trustworthiness (problem theme) of computer users in a situation where personal data is being handled such as in Banking. Here we are trying to explore trustworthiness and the scale is large and the data is very sensitive in terms of accuracy, potential loss or improper disclosure. I decide therefore that I need and exploratory study to try to identify key points and ideas in trustworthiness. This make me think of Vignettes. Vignettes are like tiny case study, an outline, a sketch, a cartoon that just illustrate ONE important point at a time so a collection of these would indicate several important aspects of Trustworthiness and those aspects could then form the basis for a more extensive study or to initiate debate about the problem theme.Vignettes - explain a situation or phenomenon
Action Research - improve a situation or process
Experiments - prove a nominal theory of some kind
Surveys - describe a situation
Grounded Theory - explore a situation or setting
Last edited:
For some reason this post got shunted during the break and although I only posted it a few days ago it is showing as being several weeks ago so I have copied it here in case any missed it.
There are two broad types of research study; which are usually named as follows.
Notice that the type of study has nothing directly to do with how you actually collect data; indeed one can use any data collection method that seems appropriate in the sense that it will help you get accurate and reliable data - in the above examples I used a questionnaire in one case and a seminar in the other. Be imaginative as well as practical, don’t lazily assume in every case that a questionnaire is the only possibility. Here is a list of collection vehicles; but don't assume you know what they mean because the words are familiar - do some reading and find out how they are used and when they are used:
Primary Data Collection Vehicles
There are two broad types of research study; which are usually named as follows.
Interventionist – that is when you set up a trial or experiment of some kind that is likely to change the setting and then observe its consequences. As a simple example one could set up a new business process and then see how well it is performing by using for example a questionnaire to gather relevant data.
Observational – here one simply selects a situation and observe it as is stands in some way. For example, if one wanted to find out the general attitude to training in a company one might conduct a series of interviews with both general workers, to see if they felt the training had any value and relevance to their work and managers, to consider if they felt things such as productivity or morale had improved because of the training.
Observational – here one simply selects a situation and observe it as is stands in some way. For example, if one wanted to find out the general attitude to training in a company one might conduct a series of interviews with both general workers, to see if they felt the training had any value and relevance to their work and managers, to consider if they felt things such as productivity or morale had improved because of the training.
Notice that the type of study has nothing directly to do with how you actually collect data; indeed one can use any data collection method that seems appropriate in the sense that it will help you get accurate and reliable data - in the above examples I used a questionnaire in one case and a seminar in the other. Be imaginative as well as practical, don’t lazily assume in every case that a questionnaire is the only possibility. Here is a list of collection vehicles; but don't assume you know what they mean because the words are familiar - do some reading and find out how they are used and when they are used:
Primary Data Collection Vehicles
Activity logs/skill sheets/Diaries
Document searching
Focus groups
Interviewing
Observation
Portfolios
Questionnaire
Role Playing/Simulation
Seminars
Life Histories
Tests
Document searching
Focus groups
Interviewing
Observation
Portfolios
Questionnaire
Role Playing/Simulation
Seminars
Life Histories
Tests
Last edited:
Re: Research Methods: Proof?
I posted this in another thread but it might also be useful here. Its a short note on possible way to deal with the idea of proof.
Hypothesis
With regard to proof, many like to write out a hypothesis. There are two stages: write the null and alternative hypotheses and then write down the two (usually) variables involved (dependant and independent variables). This is usually the basis of an experiment of some kind so implies repeatability and so in context of this thread might not be all that useful
Setting Standards/Definitions
It is easily acknowledged that in normal life we can almost never get what one might call absolute proof. In courts of law for example they talk about the evidence being “beyond reasonable doubt” or based on “the balance of probabilities” – in other words you get enough information to convince (but not absolutely prove) you of the truth. This may be done in many ways but usually one lists the things one wants to see. For example, if I wanted to prove that Manchester United is the best football team in the world (they are not because everyone knows that is Arsenal!) then I might lay out my standard or definition for proof: no of goals scored, championships won, number of world class players and so on. There are three problems with this approach: firstly whoever you are talking to has to agree to your standards or definition, secondly if I can prove it today will it still be true tomorrow and thirdly once one knows the standards, it is all too easy to find the necessary evidence (one might often say manufacture the evidence).
A final vignette may help you here. I came across a true story of a man who had been married for many years but regularly had nightmares as to whether his wife really loved him. He was so bothered by this that he went to see a Church Minister and was told one way (standard/definition) to find proof (evidence) of love was to consider all the little things you wife does for you: wash you shirts, clean you shoes, cook your meals, look after you when you are ill and so on. You can see the “proofs” would be observable but is it really proof, will that man accept the standard or not, will it still be true in a year’s time and if the wife knows the standard will she just manufacture the evidence? The fact is that one can never know the answer here for certain and there will be many things in life that simply cannot be absolutely proved in any observable or rational manner and all that can be done is to feel convinced.
Falsification
This means that a proof is like a chain with links – break one link and the chain fails. So in proof if you can find one contrary example then the proof fails. In life of course it is all too easy to just ignore contrary opinion or examples and go on only looking for supporting ones. No doubt you have come across many people like this (you or I may be one of them!!), who no matter what you say to them they refuse to be shifted from their own view even when the evidence is overwhelming. Sometime you see this very strong idea distorted. Where you see this is with authors who try to show that X is untrue and then say that Y is therefore true. I am not talking here about a hypothesis because often there is no link at all between X and Y. The argument is something like “I can show that Ford is a bad car therefore Volkswagen is a good one” or even more starkly and absurdly, “you are wrong therefore I am right.” One might say here that if you cannot find at least in principle a way to falsify something then it can never be regarded as a proof - that is it means your proof cannot be tested.
Proof by the Unexpected
Most often when we are working on a topic we have an expectation about the answer so that when we actually see it we feel sort of reassured about it. Now, sometimes we can be surprised and startled by an answer because it is just “too good to be true” and that is most often that somewhere along the line you have made a mistake
Proof by Example or Illustration/Vignette
Many authors try argument as a way of proof. One often seen this in religion and politics but it is also present in many technical papers we see. The idea is that I present my view on something and then proceed to “prove” it by instancing examples that endorse its truth. Often those who use this idea challenge you to find a contrary example (falsification). I rather like this approach but one just needs to be ultra careful that we are not taken in by our own arguments and get to a stage where we just want to keep convincing ourselves that we are right and fail to see weaknesses in our own thinking. When you use this form your logic must be impeccable and you must get evidence that can be checked and always keep in mind that your arguments are almost bound to be constrained.
Does it Work
It is always very strong when you can show that something works. That is you have a sort of theory and although you perhaps cannot prove it in any absolute sense you can show that it works by citing examples. For example, some project management techniques are like this as we can see them working but if one is sensible one just recommends them as likely to work as no one would be willing to offer a guarantee that they always work no matter what the circumstances or project.
Reading
Monk, R and Raphael, F (ed), (2000), The Great Philosophers published, Phoenix ISBN 0-75381-136-7
Popper, K (2005), the Open Society and Its Enemies Volume 1, Routledge, ISBN 0-415-23731-9
Popper K, (2006), the Logic of Scientific Discovery, Routledge, ISBN 0-415-27844-9
Lipton, P, (2004), Inference to the best Explanation, Routledge, ISBN 0-415-2424-09
Forstater, M., “The Living Wisdom of Socrates”, Hodder Headliner Audio books.
Blackburn, Simon, (2001), Think, Oxford University Press, ISBN 0-19-285425-9
Blackburn, Simon, (2006), Truth, Penguin, ISBN 0-141-01423-3
Talib, N,N (2007), The Back Swan, Penguin ISBN 978-0-1410-3459-1
Talib, N,N (2007), Fooled by Randomness, Penguin ISBN 978-0-141-034148-4
Kaye, S. M., (2009), Critical Thinking, Oneworld Publications ISBN 978-1-85168-654-4
I posted this in another thread but it might also be useful here. Its a short note on possible way to deal with the idea of proof.
Hypothesis
With regard to proof, many like to write out a hypothesis. There are two stages: write the null and alternative hypotheses and then write down the two (usually) variables involved (dependant and independent variables). This is usually the basis of an experiment of some kind so implies repeatability and so in context of this thread might not be all that useful
Setting Standards/Definitions
It is easily acknowledged that in normal life we can almost never get what one might call absolute proof. In courts of law for example they talk about the evidence being “beyond reasonable doubt” or based on “the balance of probabilities” – in other words you get enough information to convince (but not absolutely prove) you of the truth. This may be done in many ways but usually one lists the things one wants to see. For example, if I wanted to prove that Manchester United is the best football team in the world (they are not because everyone knows that is Arsenal!) then I might lay out my standard or definition for proof: no of goals scored, championships won, number of world class players and so on. There are three problems with this approach: firstly whoever you are talking to has to agree to your standards or definition, secondly if I can prove it today will it still be true tomorrow and thirdly once one knows the standards, it is all too easy to find the necessary evidence (one might often say manufacture the evidence).
A final vignette may help you here. I came across a true story of a man who had been married for many years but regularly had nightmares as to whether his wife really loved him. He was so bothered by this that he went to see a Church Minister and was told one way (standard/definition) to find proof (evidence) of love was to consider all the little things you wife does for you: wash you shirts, clean you shoes, cook your meals, look after you when you are ill and so on. You can see the “proofs” would be observable but is it really proof, will that man accept the standard or not, will it still be true in a year’s time and if the wife knows the standard will she just manufacture the evidence? The fact is that one can never know the answer here for certain and there will be many things in life that simply cannot be absolutely proved in any observable or rational manner and all that can be done is to feel convinced.
Falsification
This means that a proof is like a chain with links – break one link and the chain fails. So in proof if you can find one contrary example then the proof fails. In life of course it is all too easy to just ignore contrary opinion or examples and go on only looking for supporting ones. No doubt you have come across many people like this (you or I may be one of them!!), who no matter what you say to them they refuse to be shifted from their own view even when the evidence is overwhelming. Sometime you see this very strong idea distorted. Where you see this is with authors who try to show that X is untrue and then say that Y is therefore true. I am not talking here about a hypothesis because often there is no link at all between X and Y. The argument is something like “I can show that Ford is a bad car therefore Volkswagen is a good one” or even more starkly and absurdly, “you are wrong therefore I am right.” One might say here that if you cannot find at least in principle a way to falsify something then it can never be regarded as a proof - that is it means your proof cannot be tested.
Proof by the Unexpected
Most often when we are working on a topic we have an expectation about the answer so that when we actually see it we feel sort of reassured about it. Now, sometimes we can be surprised and startled by an answer because it is just “too good to be true” and that is most often that somewhere along the line you have made a mistake
Proof by Example or Illustration/Vignette
Many authors try argument as a way of proof. One often seen this in religion and politics but it is also present in many technical papers we see. The idea is that I present my view on something and then proceed to “prove” it by instancing examples that endorse its truth. Often those who use this idea challenge you to find a contrary example (falsification). I rather like this approach but one just needs to be ultra careful that we are not taken in by our own arguments and get to a stage where we just want to keep convincing ourselves that we are right and fail to see weaknesses in our own thinking. When you use this form your logic must be impeccable and you must get evidence that can be checked and always keep in mind that your arguments are almost bound to be constrained.
Does it Work
It is always very strong when you can show that something works. That is you have a sort of theory and although you perhaps cannot prove it in any absolute sense you can show that it works by citing examples. For example, some project management techniques are like this as we can see them working but if one is sensible one just recommends them as likely to work as no one would be willing to offer a guarantee that they always work no matter what the circumstances or project.
Reading
Monk, R and Raphael, F (ed), (2000), The Great Philosophers published, Phoenix ISBN 0-75381-136-7
Popper, K (2005), the Open Society and Its Enemies Volume 1, Routledge, ISBN 0-415-23731-9
Popper K, (2006), the Logic of Scientific Discovery, Routledge, ISBN 0-415-27844-9
Lipton, P, (2004), Inference to the best Explanation, Routledge, ISBN 0-415-2424-09
Forstater, M., “The Living Wisdom of Socrates”, Hodder Headliner Audio books.
Blackburn, Simon, (2001), Think, Oxford University Press, ISBN 0-19-285425-9
Blackburn, Simon, (2006), Truth, Penguin, ISBN 0-141-01423-3
Talib, N,N (2007), The Back Swan, Penguin ISBN 978-0-1410-3459-1
Talib, N,N (2007), Fooled by Randomness, Penguin ISBN 978-0-141-034148-4
Kaye, S. M., (2009), Critical Thinking, Oneworld Publications ISBN 978-1-85168-654-4
Re: Research Methods: Cautions
Having looked through a lot of student work it is common to see no serious attempt at proof so I thought I might post some things to think about if you are serious about the idea of proof. The ideas I give here are common in all kinds of research and can unquestionable be weaknesses of huge significance. Here I use the terms typical to the scientific community but they are of course not necessarily universal.
1. Cherry Picking - this occurs when you are selective or very selective about the data so you only choose examples that support your particular case or stance - in simple terms you did not do any research of your own, you took a short cut instead.
2. Torturing the Data - "torture the data and it will confess to anything', as they say at Guantanamo Bay. That is you don't care at all what the data actually said you will make it say just what you want it to.
3. Methodology - often in students work you cannot find a full description of the methodology, the research method, the research plan used to extract information from the data. Be honest, would you trust a research study outcome if the study owners refused to tell you how they got their results? There are ways of assessing methodologies - for example, in medical research there are the so called Jadad scores
4. Authority - are you taken in by people who generate a claim because they say they are experts, well qualified so it must be right? Now of course we want to check on credentials but if we simply rely on those you will be making a big mistake. Sadly, the literature in almost every discipline it littered by well-qualified charlatans. By all means check our qualifications but don't fall into the trap of thinking that is enough for a result to be correct.
5. Journals and Review Sites - Often in student work you cannot find a single reputable journals mentioned and that alone would lead me to consider a fail grade. Since I used a medical example above, what I would like to see is a review site such as the Cohrane Collaboration.
6. Interpretation - in research it is often said that getting the data is easy, precessing its is hard and interpreting is where we give up and lie down in a dark room and hope the problem will go away. Finding meaning is always going to be hard work because the results may not be all that clear, they may be far too clear which should always make you think you have made a mistake - some thing are just too good to be true, if you look at any set of data long enough you will find patterns, it is all to easy to be biased or lazy and look for what we want to see - so finding meaning means you need to be really knowledgeable in your area and you have to be absolutely honest. Be very wary of statistics and always get an expert to help you decide what stats you want and how to make sense of them - sadly this is often not done.
Richard Feynman, undoubtedly one of the finest brains in the world started a lecture with a very salutatory story. If you cannot understand the point he is making here with respect to this thread and more generally to research then you really do need to do a lot of reading and thinking.
To give a more mundane example, this faulty logic would lead to you say after research: Ford cars have good brakes, therefore Honda cars must also have good brakes - this might be true but it does not logically follow. follow.
Having looked through a lot of student work it is common to see no serious attempt at proof so I thought I might post some things to think about if you are serious about the idea of proof. The ideas I give here are common in all kinds of research and can unquestionable be weaknesses of huge significance. Here I use the terms typical to the scientific community but they are of course not necessarily universal.
1. Cherry Picking - this occurs when you are selective or very selective about the data so you only choose examples that support your particular case or stance - in simple terms you did not do any research of your own, you took a short cut instead.
2. Torturing the Data - "torture the data and it will confess to anything', as they say at Guantanamo Bay. That is you don't care at all what the data actually said you will make it say just what you want it to.
3. Methodology - often in students work you cannot find a full description of the methodology, the research method, the research plan used to extract information from the data. Be honest, would you trust a research study outcome if the study owners refused to tell you how they got their results? There are ways of assessing methodologies - for example, in medical research there are the so called Jadad scores
4. Authority - are you taken in by people who generate a claim because they say they are experts, well qualified so it must be right? Now of course we want to check on credentials but if we simply rely on those you will be making a big mistake. Sadly, the literature in almost every discipline it littered by well-qualified charlatans. By all means check our qualifications but don't fall into the trap of thinking that is enough for a result to be correct.
5. Journals and Review Sites - Often in student work you cannot find a single reputable journals mentioned and that alone would lead me to consider a fail grade. Since I used a medical example above, what I would like to see is a review site such as the Cohrane Collaboration.
6. Interpretation - in research it is often said that getting the data is easy, precessing its is hard and interpreting is where we give up and lie down in a dark room and hope the problem will go away. Finding meaning is always going to be hard work because the results may not be all that clear, they may be far too clear which should always make you think you have made a mistake - some thing are just too good to be true, if you look at any set of data long enough you will find patterns, it is all to easy to be biased or lazy and look for what we want to see - so finding meaning means you need to be really knowledgeable in your area and you have to be absolutely honest. Be very wary of statistics and always get an expert to help you decide what stats you want and how to make sense of them - sadly this is often not done.
Richard Feynman, undoubtedly one of the finest brains in the world started a lecture with a very salutatory story. If you cannot understand the point he is making here with respect to this thread and more generally to research then you really do need to do a lot of reading and thinking.
You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won't believe what happened. I saw a car with the licence plate ARW 357. Can you imagine? Of all the millions of licence plates in the state, what was the chance that I would see that particular one tonight? Amazing....
7. Over or Inappropriate Generalizations - this is just another way of making sure you understand the notion of not arguing from the particular to the Universal. That is you get one result and conclude it now applies everywhere and sadly it usually occurs when you are desperate to prove your point at any cost. A good example was created in this thread by skye and czgibson in another thread.Skye - Certainly the magnitude of work from that time speaks volume, if historians can claim we have wiped out banu quryza using Islamic primary sources, then by the same token, they can find the man or men who have dictated the Quran to the prophet in such an unparalleled style!
Czgibson - So because there is historical evidence for one event, there must be historical evidence for all events in the Prophet's (pbuh) life? Is that what you're saying?
Czgibson - So because there is historical evidence for one event, there must be historical evidence for all events in the Prophet's (pbuh) life? Is that what you're saying?
To give a more mundane example, this faulty logic would lead to you say after research: Ford cars have good brakes, therefore Honda cars must also have good brakes - this might be true but it does not logically follow. follow.
Last edited:
researcher
Active member
- Messages
- 25
- Reaction score
- 0
Hi Hugo,
I haven't kept up with your posts but will catch up at some point.
A question I'm in the midst of completing an assignment which is due is very soon. There's something I don't quite understand...
On the question of what makes a theory a genuinely scientific one, Karl Popper’s criterion of demarcation, as it is called, has now gained very general acceptance: namely, that every genuine scientific theory must be testable, and therefore falsifiable, at least in principle – in other words, if a theory is incompatible with possible observations it is scientific; conversely, a theory which is compatible with all possible observations is unscientific (Popper, K. The Logic of Scientific Discovery).
Im a bit confused... what does the red and pink bit mean.. I thought it would be the other way round i.e. if a theory whch is compatible with all possible observations is scientific as opposed to the other way round... why is it unscientific if it is compatible with all possible observations??
would appreciate any insight you can provide...
many thanks...
I haven't kept up with your posts but will catch up at some point.
A question I'm in the midst of completing an assignment which is due is very soon. There's something I don't quite understand...
On the question of what makes a theory a genuinely scientific one, Karl Popper’s criterion of demarcation, as it is called, has now gained very general acceptance: namely, that every genuine scientific theory must be testable, and therefore falsifiable, at least in principle – in other words, if a theory is incompatible with possible observations it is scientific; conversely, a theory which is compatible with all possible observations is unscientific (Popper, K. The Logic of Scientific Discovery).
Im a bit confused... what does the red and pink bit mean.. I thought it would be the other way round i.e. if a theory whch is compatible with all possible observations is scientific as opposed to the other way round... why is it unscientific if it is compatible with all possible observations??
would appreciate any insight you can provide...
many thanks...
Here it is just the way you understand what he means when he says 'incompatible'; it is just another way of stating that it is falsifiable, that is, it is in principle possible to show that the theory does not match (incompatible with) the observations. Another way of putting it, as some have done is to re frame it into a question 'What does the theory imply which, if false, would show the whole theory to be false?'
Think about the famous white swan question - if we had a theory that all swans are white then we can easily see immediately that IF we can find ONE non-white swan then the theory fails - that is we KNOW when the theory fails if a certain thing occurs and if we know that we know how the theory might be falsified. Be careful, this does not mean that the theory will fail but it does mean we can recognise it if it does. It also does not mean that no such falsification method exits but until we find one, some sort of test then its not scientific.
Being topical here, suppose we say that God exists then that only become a scientific theory IF we can say unambiguous when it fails to be true. Thus, someone might say "earthquakes" prove there is no God but although this MAY be true there is no way any one would feel certain of this test.
Again, suppose someone says "he is lying because he is demon possessed" now this might be a true explanation of his lying but I cannot think of any way to show that it is UNTRUE (incompatible) so we therefore say it is in essence a logical fallacy.
Last edited:
Re: Research Methods: Research Plans/Design
Here is an interesting outline example and my thanks to Gossamer Skye for the basic details and checking the medical aspects. It illustrates many research principles of importance and you would do well to discover what they are. This is a medical example but for our purposes it is easy to understand although reading this will not qualify you as a clinical practitioner so please do not go off and try this out.
Let's say someone is suspected of suffering from depression in medicine it is not something you can quantify in on any fixed scale; there is no machine you can sit in front of or any physical test one can do and end up with some measurements of your psyche on a computer printout. So the question is how can we get reliable diagnoses and begin treatment? One possible way is to set criteria or you can think of them as standards instead such as those shown below devised by the Mayo Clinic based in Rochester, Minn. Thus, if a person exhibits the majority of these said criteria then you can reliably give them the diagnosis of depression.
Basics – at the start of any research project it is best to think through certain aspects so that you have a clear view of what you are doing: Problem, Target, Plan and at this stage particularly:
Research Style – at this stage you need to consider if your style is quantitative or qualitative. It is easy to confuse these two and simply think of them as describing data types but to do so means you are missing the whole point. In general, if you outcome is quantitative then it is in some way intended to be predictive whereas if it qualitative then it is intended to be mostly descriptive.
Study Type – broadly speaking there are two types; the first is interventionist when you make a change in a situation and then study its consequences and the second is observational when you simply record what is currently going on.
Thinking Process – begin by asking how were these standards/criteria set since this obviously has to be done with stringency as people’s lives are involved? However, they did not just appear out of the ‘air’; some sort of rational process was used to get them so what could those thinking processes have been? In most cases those processes will have been either deductive, meaning they were generated from some theoretical standpoint or inductive meaning that they are a kind of best guess at this stage, you do not have enough information to form a theory so in a way you are hoping these results will show that these criteria are good at describing or even predicting depression. Part of this will also be consideration of some or all of the following points:
Research Method – to test the list for acceptability we need to do some kind of test on it and so we need a framework to confine and guide our work. Possible Research Methods are: experiment, survey, case study, vignette, grounded theory, action research, etc. For our purposes here let us assume that the survey method is chosen but as a question for you what are the factors you might take into account in such a choice? Once the research Method is decided and justified we must go through all the following stages.
Design of Questionnaire – this will be dealt with later but will also include methods of checking reliability and validity. Do not be tempted into thinking that the design of a questionnaire is simple, in practice it is one of the most difficult instruments to design and use. In this case the questionnaire is designed on one dimension that of depression and this is expressed by listing several suspected aspects along some nominal and bi-polar scale.
Pilot – if the study has any significance it should also be preceded by a pilot to make sure that our design does not have any flaws.
Processing – once the data has been collected it can be processed and this is most often done in two stages. The first just summarises the data into some convenient form based on the raw original data: tables, catalogues, charts, statistics etc whatever seem appropriate or get it into a form that will allow you to generate you outcome. One might start using Cronbach's alpha to check the consistency of the questions (are they all measuring the same thing) and then generate simple measures like standard deviation and more complex ones such as seeing if there is any correlation between questions.
The last stage is to bring all you finding together and generate your outcome, To do this you can look at the general format used to generate survey reports, if your outcome was model you might start by considering the universal process model, you might want to generate a best practice so to do that you might us a best practice model to guide you and so on.
Here is an interesting outline example and my thanks to Gossamer Skye for the basic details and checking the medical aspects. It illustrates many research principles of importance and you would do well to discover what they are. This is a medical example but for our purposes it is easy to understand although reading this will not qualify you as a clinical practitioner so please do not go off and try this out.
Let's say someone is suspected of suffering from depression in medicine it is not something you can quantify in on any fixed scale; there is no machine you can sit in front of or any physical test one can do and end up with some measurements of your psyche on a computer printout. So the question is how can we get reliable diagnoses and begin treatment? One possible way is to set criteria or you can think of them as standards instead such as those shown below devised by the Mayo Clinic based in Rochester, Minn. Thus, if a person exhibits the majority of these said criteria then you can reliably give them the diagnosis of depression.
Loss of interest in normal daily activities, Feeling sad or down, Feeling hopeless, Crying spells for no apparent reason, Problems sleeping, Trouble focusing or concentrating, Difficulty making decisions, Unintentional weight gain or loss, Irritability, Restlessness, Being easily annoyed, Feeling fatigued or weak, Feeling worthless, Loss of interest in sex, Thoughts of suicide or suicidal behaviour, Unexplained physical problems, such as back pain or headaches
Be aware here that none of these things are absolutely measurable and the best we could do is say use a 5-point nominal scale from totally agree to never get that feeling to assess each criteria. If you do not know what is meant by nominal scale there is a post in this thread that deals with all types of scales. So from a research point of view you may learn a good deal from this example if you can answer the following questions: Basics – at the start of any research project it is best to think through certain aspects so that you have a clear view of what you are doing: Problem, Target, Plan and at this stage particularly:
Outcome – having thought through the above one now has to decide what you will produce at the end of the project. Some possibilities are: a survey report, a set of recommendations, a plan, some qualification on the use of the criteria etc
Actor – it is very important to think about whom, meaning a person or persons will receive and use your outcome and so as what will they do with it.
Actor – it is very important to think about whom, meaning a person or persons will receive and use your outcome and so as what will they do with it.
Research Style – at this stage you need to consider if your style is quantitative or qualitative. It is easy to confuse these two and simply think of them as describing data types but to do so means you are missing the whole point. In general, if you outcome is quantitative then it is in some way intended to be predictive whereas if it qualitative then it is intended to be mostly descriptive.
Study Type – broadly speaking there are two types; the first is interventionist when you make a change in a situation and then study its consequences and the second is observational when you simply record what is currently going on.
Thinking Process – begin by asking how were these standards/criteria set since this obviously has to be done with stringency as people’s lives are involved? However, they did not just appear out of the ‘air’; some sort of rational process was used to get them so what could those thinking processes have been? In most cases those processes will have been either deductive, meaning they were generated from some theoretical standpoint or inductive meaning that they are a kind of best guess at this stage, you do not have enough information to form a theory so in a way you are hoping these results will show that these criteria are good at describing or even predicting depression. Part of this will also be consideration of some or all of the following points:
Qualifications - Must these criteria of necessity be set by an ‘expert’ or could anyone do it or at least suggest possible criteria?
Errors - Is it possible that the person or persons setting the list could make a mistake or mistakes even if they are unquestionably expert?
Aspects of Depression - the focus is on the pathology of depression and this list of criteria might be thought of as aspects of the condition. It is therefore important to ask is the list complete, are there any other relevant aspects that are missing or perhaps the list is too long, or some aspects included but have no real bearing and so they can be discarded?
Criticality - consider if this list might lead to false diagnoses because these symptoms may also indicate other conditions.
Validity – meaning here that the criteria do indicate depression and not something else?
Reliable - meaning here that the criteria are consistent over time and so useful in diagnoses and therefore the research is worth it?
Ethics - Suppose we use this list to collect data then are there any ethical issues that we might need to deal with and if there are how can they be accommodated or are they insurmountable? This will cover everything from data collection, to processing to storage to dissemination.Errors - Is it possible that the person or persons setting the list could make a mistake or mistakes even if they are unquestionably expert?
Aspects of Depression - the focus is on the pathology of depression and this list of criteria might be thought of as aspects of the condition. It is therefore important to ask is the list complete, are there any other relevant aspects that are missing or perhaps the list is too long, or some aspects included but have no real bearing and so they can be discarded?
Criticality - consider if this list might lead to false diagnoses because these symptoms may also indicate other conditions.
Validity – meaning here that the criteria do indicate depression and not something else?
Reliable - meaning here that the criteria are consistent over time and so useful in diagnoses and therefore the research is worth it?
Research Method – to test the list for acceptability we need to do some kind of test on it and so we need a framework to confine and guide our work. Possible Research Methods are: experiment, survey, case study, vignette, grounded theory, action research, etc. For our purposes here let us assume that the survey method is chosen but as a question for you what are the factors you might take into account in such a choice? Once the research Method is decided and justified we must go through all the following stages.
Population – can we identify where our possible respondents come from. This is not a simple task and careful thought and a practical outlook are needed?
Sample Frame – can we obtain a list of some kind that can be used to select respondents who we might collect data from? This is a KEY point in the whole process and unless this is done well the sample may well be totally invalid.
Classification - classification means that we collect data about the respondent themselves (as opposed to data about the medical condition on the depression criteria listed above) - this might be age, ethnicity, job profile and so on. We need an exact protocol/set of criteria to be sure we collect data from a relevant set of people? One final aspect of classification is to decide if we should just use people who have already been diagnosed with depression, those suspected of the condition or just anyone?
Selection Method – once we have the sample frame it will be necessary to select names from it and there are many ways to do this: random, purposive, cluster etc but ideally, we would want to use a random number generator to select people from this list. Do not be tempted to ‘invent’ ways to randomise because they invariably turn out to be systematic and may therefore invalidate all your results – many research efforts are ruined by inappropriate randomisation.
Blinding – one might consider blinding here were neither the clinician nor the patient knows what intervention, if any, is involved. However, blinding to be useful is totally dependent on the randomisation process. One also may have to consider the placebo effect.
Choosing a Sample Size – the population might be quite large and it can be shown that statistically a sample of sufficient size will give us the required level of confidence in our results – so how can we calculate a sample size, how many respondents do we need?
Data Collection - Suppose we decide to collect data with our survey then what might be the most practical way to do that: interviewing patients, examining clinical records, a patient questionnaire or it might even be possible to do it by observation?
Mode of Collection - can data be collected by anyone such as you, a nurse or must it be of necessity a clinician? Just work through the list of possibilities above and suggest who might do it but also consider if it may be automated?
Research Plan – since we are trying to establish if these criteria accurately and reliably indicate depression then we have to have some way of knowing whether clinically the respondent is thought to be depressed or not. So we might proceed in several ways: Sample Frame – can we obtain a list of some kind that can be used to select respondents who we might collect data from? This is a KEY point in the whole process and unless this is done well the sample may well be totally invalid.
Classification - classification means that we collect data about the respondent themselves (as opposed to data about the medical condition on the depression criteria listed above) - this might be age, ethnicity, job profile and so on. We need an exact protocol/set of criteria to be sure we collect data from a relevant set of people? One final aspect of classification is to decide if we should just use people who have already been diagnosed with depression, those suspected of the condition or just anyone?
Selection Method – once we have the sample frame it will be necessary to select names from it and there are many ways to do this: random, purposive, cluster etc but ideally, we would want to use a random number generator to select people from this list. Do not be tempted to ‘invent’ ways to randomise because they invariably turn out to be systematic and may therefore invalidate all your results – many research efforts are ruined by inappropriate randomisation.
Blinding – one might consider blinding here were neither the clinician nor the patient knows what intervention, if any, is involved. However, blinding to be useful is totally dependent on the randomisation process. One also may have to consider the placebo effect.
Choosing a Sample Size – the population might be quite large and it can be shown that statistically a sample of sufficient size will give us the required level of confidence in our results – so how can we calculate a sample size, how many respondents do we need?
Data Collection - Suppose we decide to collect data with our survey then what might be the most practical way to do that: interviewing patients, examining clinical records, a patient questionnaire or it might even be possible to do it by observation?
Mode of Collection - can data be collected by anyone such as you, a nurse or must it be of necessity a clinician? Just work through the list of possibilities above and suggest who might do it but also consider if it may be automated?
a. Take a sample of the general population and collect results and come to diagnoses. At the same time take a sample from clinical records of people who have been diagnosed as depressed and collected the same data. If on processing these two sets of data we find no significant differences then one would question the usefulness of such a test.
b. Use clinical records and extract answers to each query that way we can later process the data as a whole and see if there is any correlation between clinician’s findings as recorded in the patient record and the result we might obtain from the collected data. If a correlation exists we might then argue that the criteria are a useful guide. Alternatively, you might use the same patient records but then go to each patient and fill in the questionnaire and after that the work is the same.
Data Type - What kind of data is it that we are trying to collect here and what kinds of processing can we apply to it? Will it be opinions, factual, etc. One consideration is to ask is the data nominal or ordinal because will influence the way we process the data. In this case it is defiantly nominal as there can be no scale were say ‘irritability’ can be measure with precision – what we mean here is that we are not able to say things such as ‘I have zero irritability’ or ‘I am twice as irritable as I was yesterday’ and of course there is no way to be sure that different people are registering the same levels.b. Use clinical records and extract answers to each query that way we can later process the data as a whole and see if there is any correlation between clinician’s findings as recorded in the patient record and the result we might obtain from the collected data. If a correlation exists we might then argue that the criteria are a useful guide. Alternatively, you might use the same patient records but then go to each patient and fill in the questionnaire and after that the work is the same.
Design of Questionnaire – this will be dealt with later but will also include methods of checking reliability and validity. Do not be tempted into thinking that the design of a questionnaire is simple, in practice it is one of the most difficult instruments to design and use. In this case the questionnaire is designed on one dimension that of depression and this is expressed by listing several suspected aspects along some nominal and bi-polar scale.
Pilot – if the study has any significance it should also be preceded by a pilot to make sure that our design does not have any flaws.
Processing – once the data has been collected it can be processed and this is most often done in two stages. The first just summarises the data into some convenient form based on the raw original data: tables, catalogues, charts, statistics etc whatever seem appropriate or get it into a form that will allow you to generate you outcome. One might start using Cronbach's alpha to check the consistency of the questions (are they all measuring the same thing) and then generate simple measures like standard deviation and more complex ones such as seeing if there is any correlation between questions.
The last stage is to bring all you finding together and generate your outcome, To do this you can look at the general format used to generate survey reports, if your outcome was model you might start by considering the universal process model, you might want to generate a best practice so to do that you might us a best practice model to guide you and so on.
Last edited:
1. Things can be hard to measure because it is often the case that we cannot find a suitable scale to use. In cases like this one defines a nominal measure.scale that looks as if it will be suitable. So you might ask is there a link between flexitime and performance. So you could just compare employee on felexitime and those not but to do it you have to find a measure of productivity.Hamza81 said:
This could be many things such as calls taken per hour, customer feedback, number of re-calls that kind of thing and of course you can take all of them. You can also ask what management regard as a productivity measure and you might find that is not the same for every manager and finally, see what the literature says. Out of all that you might be able to say what might be best practice and that can be you outcome.
2. With regard to emotional exhaustion then you have to be able to say what that might be but it is a good route to take because one might argue that it is the underlying reason of good work or poor performance. There is a thing called Emotional Intelligence and there is plenty of literature on it so one might look at emotional exhaustion and emotional intelligence and then as an outcome can suggest a training plan for developing emotional intelligence skills/competences that will counter emotional exhaustion. You could do that by finding out what the factors are for sound emotional intelligence and then say interview operators based on those factors or of course you could use a questionnaire. One might also interview managers to see if they have any idea how to promote emotional well-being on the same lines. Here is a short summary:
Emotional Intelligence
This is an area of interest at present because it offers a way for individuals to become more aware of themselves and others and hence make them in some sense more competent at their job. It is not entirely useful to try to define EI but Eaton and Johnson suggested it might be summarised as “the ability to inform our decisions with an understanding of our own and others’ emotions so that we can take productive action”.
EI literature talks about various emotional competences but we must not take this too far as then we end up as some sort of robot stoic, those who have no emotions at all, cannot feel another’s pain, or sympathize with another’s predicament, feel love or hate, joy or misery; who wants to be around people like that? The essential point is that emotionally we are all different and that is a strength; the fact is we all have emotional defects/strengths of one sort or another and we cannot get rid of/do not want to get rid of them but we can be AWARE of them and in some sense manage them and recognize them in others. For what it’s worth the literature usually defines 5 competences in this area but they are all essentially premised on the idea of a deep sense of self-awareness. These competences are essentially the variables in your study.
Self-Regulation - the management and control of one’s impulses and resources so as to regulate one’s self against impulsive actions, delaying instant gratification in order to remain focused.
Self-Awareness - or consciousness/sensitivity to our own emotional states and intuitions leading to recognition of their limitations and paradoxically therefore maximizing strengths.
Motivation – loosely these are emotional tendencies that facilitate the achievement of goals or you might think of it as a way of focusing internal energies and impulses on a mission to achieve excellence though any presented opportunity coupled with a considered inclination exploit them.
Empathy – strictly, this is to feel another’s pain by attuning our emotions to those of others so as to derive the knowledge and understanding of how and why other people feel, act and react the way they do in given situations, particularly when significant stress is involved.
Social Skills – enables the individual to “read” the intentions and actions of others and so adjust to or influence the operational ethos of groups so fitting into the mood, atmosphere and trust of the other team members.
Self-Awareness - or consciousness/sensitivity to our own emotional states and intuitions leading to recognition of their limitations and paradoxically therefore maximizing strengths.
Motivation – loosely these are emotional tendencies that facilitate the achievement of goals or you might think of it as a way of focusing internal energies and impulses on a mission to achieve excellence though any presented opportunity coupled with a considered inclination exploit them.
Empathy – strictly, this is to feel another’s pain by attuning our emotions to those of others so as to derive the knowledge and understanding of how and why other people feel, act and react the way they do in given situations, particularly when significant stress is involved.
Social Skills – enables the individual to “read” the intentions and actions of others and so adjust to or influence the operational ethos of groups so fitting into the mood, atmosphere and trust of the other team members.
Last edited:
Re: Research Methods: Proof
I came across a very nice way of thinking about proof and falsification when reading Bill Bryson's book called 'Mother Tongue'. The root of the word proof comes from the word we use for 'test' and the idea of 'rules'. Now, in English we have a very common expression which says:
This is very like the idea of falsifiability, that is can we find a test (an exception) that would conclusively show the rule or theory to be false; because if there are no exceptions then we can with some degree of confidence accept the proposition, theory to be true.
I came across a very nice way of thinking about proof and falsification when reading Bill Bryson's book called 'Mother Tongue'. The root of the word proof comes from the word we use for 'test' and the idea of 'rules'. Now, in English we have a very common expression which says:
'The exception proves the rule'
But how can an exception prove a rule, surely, if there is an exception to the rule the rule does not work does it? The answer lies in the earlier meaning of 'prove' which was to test. Now things are clear, if we find an exception or what sounds like an exception then we can use it to test the rule and if it works for the exception then we can have a stronger sense of faith in the rule.This is very like the idea of falsifiability, that is can we find a test (an exception) that would conclusively show the rule or theory to be false; because if there are no exceptions then we can with some degree of confidence accept the proposition, theory to be true.
Last edited:
Sample Frame
This can take many forms but is an absolutely indispensable idea in research and your research may stand or fall based on its choice or often no choice. In its purest form a sample frame is a list from which you can choose in some way, usually randomly. Remember, you can sample things as well as people but here I will just give examples for people.
In samples we often speak of the level of precision. Here precision means how well does the sample represent the population and it is obvious that precision is largely determined by the sample fame and the calculated sample size. However, it is obvious that sample frame is of the first importance since if one is selecting sample points from an unsuitable list then it really does not matter what sample size you use it will never be representative.
1. The phone book - if you were sampling people in a particular area then this might provide you with a useful list of names. It is not perfect of course as not everybody in the area will be listed but it is the best perhaps you can do.
2. A prepared list of names - if you were doing a study in a company then you could ask for a list of names that meet certain criteria to be prepared or of course you could ask for a list of everybody and then go though it excluding the ones who do not fit.
3. A collection of business cards that people have given you say at a product convention. This can often be useful because such cards give you lost of information.
4. Undefined list - this might occur if say you were sampling people booking into a hotel so you stand in the lobby and every time you see someone book in or out you ask if they will be willing to take part in your study so in a way you are defined the frame as the ones who agree. Here it would be unlikely the hotel management would give you access to their files but they might allow you to do it as I have described.
Well that probably gives you the idea but then you must think about how you choose people from the sample frame (list) and there are many way to do this and if you look through the post you will find such a list.
Ideally, it would likely be best to do it randomly using a random number generator on your computer. So you number everyone on your list from say 1 to 500 (or however long the list is) and then if you want a sample of say 50 you ask the computer to generate 50 random numbers between 1 and 500.
WARNING - I strongly caution you not to try to think up your own randomisation process as more often than not they will bias the data. Be guided here and get it fixed in your memory that any process of selection that sounds remotely systemic is likely to add bias - tossing a coin sounds random but it has often been found that lead to bias occurs so be warned.
This can take many forms but is an absolutely indispensable idea in research and your research may stand or fall based on its choice or often no choice. In its purest form a sample frame is a list from which you can choose in some way, usually randomly. Remember, you can sample things as well as people but here I will just give examples for people.
In samples we often speak of the level of precision. Here precision means how well does the sample represent the population and it is obvious that precision is largely determined by the sample fame and the calculated sample size. However, it is obvious that sample frame is of the first importance since if one is selecting sample points from an unsuitable list then it really does not matter what sample size you use it will never be representative.
1. The phone book - if you were sampling people in a particular area then this might provide you with a useful list of names. It is not perfect of course as not everybody in the area will be listed but it is the best perhaps you can do.
2. A prepared list of names - if you were doing a study in a company then you could ask for a list of names that meet certain criteria to be prepared or of course you could ask for a list of everybody and then go though it excluding the ones who do not fit.
3. A collection of business cards that people have given you say at a product convention. This can often be useful because such cards give you lost of information.
4. Undefined list - this might occur if say you were sampling people booking into a hotel so you stand in the lobby and every time you see someone book in or out you ask if they will be willing to take part in your study so in a way you are defined the frame as the ones who agree. Here it would be unlikely the hotel management would give you access to their files but they might allow you to do it as I have described.
Well that probably gives you the idea but then you must think about how you choose people from the sample frame (list) and there are many way to do this and if you look through the post you will find such a list.
Ideally, it would likely be best to do it randomly using a random number generator on your computer. So you number everyone on your list from say 1 to 500 (or however long the list is) and then if you want a sample of say 50 you ask the computer to generate 50 random numbers between 1 and 500.
WARNING - I strongly caution you not to try to think up your own randomisation process as more often than not they will bias the data. Be guided here and get it fixed in your memory that any process of selection that sounds remotely systemic is likely to add bias - tossing a coin sounds random but it has often been found that lead to bias occurs so be warned.
Last edited:
Here is a project summary note similar to the one I posted a while ago but it is a little more concise and you may find it helpful. In Research Methods the following key words are defined but these terms are not universal but the underlying ideas are so DO NOT assume that you know what these mean else you are likely to get into considerable difficulties.
Problem – this must define a single core problem for which you are going to find a solution route.
Target – these are the effects that will be evident in the real world if the problem can be solved. It is permissible to list more than one effect but it is best to look for the principle one.
Outcome – the object you will generate as the final product of your MSc project. Possible outcomes are characterised by nouns so might be: models, frameworks, policies, strategies, position papers, reviews, procedure description, best practice descriptions, dictionaries, lexicons, concordances, protocols, dossier, diagrams, charts, plans, etc.
Actor - It is normal when you define your outcome to say who the actor or actors are (meaning persons) who will use your defined outcome to bring about change by using your outcome leading to the target effects.
Thinking – it is important to be aware of how you think about the problem because that will help you decide what data is needed. In simple terms; you may have a theory that you want to test so the work is deductive, consequently you only define data that serves to test the theory. Alternatively, you may have no fixed views and you will be inductive and draw inferences from the data when you get it so in a sense you would more or less guess what data might be useful.
Activity and Data Spotlight – focusing on exactly the primary data that you need and nothing else. There are two parts, Activity: account for, analyses, collate, appraise and so on and the spotlight: the place where the data can be found.
Research Question – this is intended to be a lucid question that connects the various features and expresses the direction of your research and summarises your whole project. Research Questions have 7 features: Interrogative (I), Outcome (O), Actor (A), Problem (P), Target (T), Spotlight (S) and Method (M) and you might find it useful to remember them by using the word “IMPASTO” meaning way or approach. The correct order of these features in a sentence depends almost entirely on the interrogative if you are to produce a valid sentence in English. Possible interrogatives are: Whose, who, whom, what, which, where, whence, whither, when, how, why, wherefore, does/is, s/are, and can.
Research Style – either quantitative; meaning mostly numbers make up the data, the intention is to process that data in order to make predictions and such studies are often deductive in nature. Alternatively, the data may be largely text and so qualitative in nature and such studies are inductive, designed to look for understanding of a situation or phenomena.
Research Type - meaning is you study observational or interventionist.
Research Method – method selection depends on many factors: context, time available, skills available, practicalities, access, reason for the study, what kind of outcome you want, cost, nature of the study quantitative or qualitative, scale, control, sensitivity of the data, etc. Basic purpose of any study is assumed to provide as an outcome one of the following forms: express an understanding, an exploration, a description, an explanation, an improvement suggestion, build something or prove something. Common Methods and typical uses are:
Population – the set of people or things from which you will derive your data. You must try to be as specific as you can and also attempt to estimate the number of people or things involved.
Sample Frame – this refers to the mechanism you will use to select sample points. Ideally, it would be a list of some kind and then by a process (ideally randomised process) select sample points. The sample frame determines the level of precision (meaning how well does you sample represent the population) you can get so its importance cannot be underestimated.
Sample Selection – with an acceptable sample frame you need a rational way of selecting a sample of the size you calculate. Typical ways of sampling are: random, Systematic, Stratified, Cluster, Stage, Convenience, Voluntary, Quota, Purposive, Dimensional, Snowball, Event and Time sampling.
Sample Size – it is always necessary to calculate statistically a sample size and there are many formulae for doing this and almost always they are based on what you expected prevalence of the kind of thing you are looking for.
Collection Protocol – the means by which the classified (data that tells you who or what you are to collect from) primary data is collected: interview, questionnaire, observation, role playing, seminar, focus groups, document or record searching
Pre-Processing – This stage must always begin with a consideration of the sample size and was it as calculated and if not then consider how that might have affected the results. In addition the classification data must be examined to see if the sample is suitable or has it in the collection phase become biased in some way. Finally, you must describes how the primary data in its raw collected form is structured into a suitable entity ready for generating the outcome.
Outcome Processing – describes how the structured primary data collection is used to generate the intended outcome noting if there were any difficulties over the sample size or its classification and if so decide if some qualifications or limitations must be applied to the outcome.
Target – these are the effects that will be evident in the real world if the problem can be solved. It is permissible to list more than one effect but it is best to look for the principle one.
Outcome – the object you will generate as the final product of your MSc project. Possible outcomes are characterised by nouns so might be: models, frameworks, policies, strategies, position papers, reviews, procedure description, best practice descriptions, dictionaries, lexicons, concordances, protocols, dossier, diagrams, charts, plans, etc.
Actor - It is normal when you define your outcome to say who the actor or actors are (meaning persons) who will use your defined outcome to bring about change by using your outcome leading to the target effects.
Thinking – it is important to be aware of how you think about the problem because that will help you decide what data is needed. In simple terms; you may have a theory that you want to test so the work is deductive, consequently you only define data that serves to test the theory. Alternatively, you may have no fixed views and you will be inductive and draw inferences from the data when you get it so in a sense you would more or less guess what data might be useful.
Activity and Data Spotlight – focusing on exactly the primary data that you need and nothing else. There are two parts, Activity: account for, analyses, collate, appraise and so on and the spotlight: the place where the data can be found.
Research Question – this is intended to be a lucid question that connects the various features and expresses the direction of your research and summarises your whole project. Research Questions have 7 features: Interrogative (I), Outcome (O), Actor (A), Problem (P), Target (T), Spotlight (S) and Method (M) and you might find it useful to remember them by using the word “IMPASTO” meaning way or approach. The correct order of these features in a sentence depends almost entirely on the interrogative if you are to produce a valid sentence in English. Possible interrogatives are: Whose, who, whom, what, which, where, whence, whither, when, how, why, wherefore, does/is, s/are, and can.
Research Style – either quantitative; meaning mostly numbers make up the data, the intention is to process that data in order to make predictions and such studies are often deductive in nature. Alternatively, the data may be largely text and so qualitative in nature and such studies are inductive, designed to look for understanding of a situation or phenomena.
Research Type - meaning is you study observational or interventionist.
Research Method – method selection depends on many factors: context, time available, skills available, practicalities, access, reason for the study, what kind of outcome you want, cost, nature of the study quantitative or qualitative, scale, control, sensitivity of the data, etc. Basic purpose of any study is assumed to provide as an outcome one of the following forms: express an understanding, an exploration, a description, an explanation, an improvement suggestion, build something or prove something. Common Methods and typical uses are:
Case Studies – useful when trying to understand a situation or practice
Vignettes - useful for exploring a situation in order to illustrate its major features
Action Research – useful when it is desirable to improve a situation by working within it
Experiments – useful when one is trying to prove or more usually indicate the truth of some proposition
Quasi-Experiments – as for experiments but the experiment can only be simulated
Surveys – useful when trying to describe a situation or effect
Biographies/History – useful when one wants to explore a situation in order to replicates it or improve it
Grounded Theory – useful when the area under study is barely understood but needs to be explored
Ethnography – useful when one wants to describe a situation of some kind involving behaviour
Requirements Gathering – useful when one wishes to build a real world object
Vignettes - useful for exploring a situation in order to illustrate its major features
Action Research – useful when it is desirable to improve a situation by working within it
Experiments – useful when one is trying to prove or more usually indicate the truth of some proposition
Quasi-Experiments – as for experiments but the experiment can only be simulated
Surveys – useful when trying to describe a situation or effect
Biographies/History – useful when one wants to explore a situation in order to replicates it or improve it
Grounded Theory – useful when the area under study is barely understood but needs to be explored
Ethnography – useful when one wants to describe a situation of some kind involving behaviour
Requirements Gathering – useful when one wishes to build a real world object
Population – the set of people or things from which you will derive your data. You must try to be as specific as you can and also attempt to estimate the number of people or things involved.
Sample Frame – this refers to the mechanism you will use to select sample points. Ideally, it would be a list of some kind and then by a process (ideally randomised process) select sample points. The sample frame determines the level of precision (meaning how well does you sample represent the population) you can get so its importance cannot be underestimated.
Sample Selection – with an acceptable sample frame you need a rational way of selecting a sample of the size you calculate. Typical ways of sampling are: random, Systematic, Stratified, Cluster, Stage, Convenience, Voluntary, Quota, Purposive, Dimensional, Snowball, Event and Time sampling.
Sample Size – it is always necessary to calculate statistically a sample size and there are many formulae for doing this and almost always they are based on what you expected prevalence of the kind of thing you are looking for.
Collection Protocol – the means by which the classified (data that tells you who or what you are to collect from) primary data is collected: interview, questionnaire, observation, role playing, seminar, focus groups, document or record searching
Pre-Processing – This stage must always begin with a consideration of the sample size and was it as calculated and if not then consider how that might have affected the results. In addition the classification data must be examined to see if the sample is suitable or has it in the collection phase become biased in some way. Finally, you must describes how the primary data in its raw collected form is structured into a suitable entity ready for generating the outcome.
Outcome Processing – describes how the structured primary data collection is used to generate the intended outcome noting if there were any difficulties over the sample size or its classification and if so decide if some qualifications or limitations must be applied to the outcome.
Last edited:
Sample Size
Unfortunately there is no single answer to this and in general, the larger the sample size, the more closely your sample data will match that from the population. However, it can be show statistically that a particular sample size will give you the necessary level of precision. In practice, you need to work out how many responses will give you sufficient precision (meaning how well does the sample represent the population) at an affordable cost. Calculation of an appropriate sample size depends on a number of factors unique to each survey and it is a matter for you to make the decision. The most important are:
One possible formula to find an appropriate sample size for a population-based survey is determined largely by three factors: n = 1500p(1 - p)/r, where:
Example - Suppose we expect that 85% (0.85) of the returned forms meet the criteria and we estimate a poor return of just 50% of questionnaires then we have:
Note. It is often assumed in student work that getting at least 35 will be good enough for the data to be regarded as just about Normal but this is a very rough estimate but certainly anything much below that is a cause of concern.
Unfortunately there is no single answer to this and in general, the larger the sample size, the more closely your sample data will match that from the population. However, it can be show statistically that a particular sample size will give you the necessary level of precision. In practice, you need to work out how many responses will give you sufficient precision (meaning how well does the sample represent the population) at an affordable cost. Calculation of an appropriate sample size depends on a number of factors unique to each survey and it is a matter for you to make the decision. The most important are:
How accurate do you wish to be?
How confident do you need to be in the results
what budget/time you have available?
How confident do you need to be in the results
what budget/time you have available?
One possible formula to find an appropriate sample size for a population-based survey is determined largely by three factors: n = 1500p(1 - p)/r, where:
Estimated Sample Size 
– this is the estimate of the number of sample points needed
Prevalence (p) - prevalence of the variable of interest; how many of the returned questionnaires meet the sample criteria or if you like what is the percentage of people, in the population that fit the criteria well. It is always hard to know what this value might be so one might decide to use say 85% and that is what is used in the above estimating formula.
Expected Rate of Return (r) – not every questionnaire you send out will be returned so one builds in an estimate so that at least you have some assurance of a minimum sample size.
– this is the estimate of the number of sample points neededPrevalence (p) - prevalence of the variable of interest; how many of the returned questionnaires meet the sample criteria or if you like what is the percentage of people, in the population that fit the criteria well. It is always hard to know what this value might be so one might decide to use say 85% and that is what is used in the above estimating formula.
Expected Rate of Return (r) – not every questionnaire you send out will be returned so one builds in an estimate so that at least you have some assurance of a minimum sample size.
Example - Suppose we expect that 85% (0.85) of the returned forms meet the criteria and we estimate a poor return of just 50% of questionnaires then we have:
N = 1500*0.85(1-0.85)/0.5 = 1275*0.15 = 190 is estimated required sample size rounded
N* = 0.5 * 190 = 95 expected return and as this is greater than 35 it is a reasonable
N** = 190/0.5 = 380 questionnaire to be sent out if we hope to get the full sample size.
N* = 0.5 * 190 = 95 expected return and as this is greater than 35 it is a reasonable
N** = 190/0.5 = 380 questionnaire to be sent out if we hope to get the full sample size.
Note. It is often assumed in student work that getting at least 35 will be good enough for the data to be regarded as just about Normal but this is a very rough estimate but certainly anything much below that is a cause of concern.
Last edited:
I have had some queries about sample frame and it may help you appreciate why it is such an important idea if I speak of another idea called sample PRECISION.
Precision means or refers to how accurately your sample mirrors or represents the population. For example, if you made a several cauldrons of soup of different sorts for say 100 people and you wanted to test it to see if it had enough salt then its obvious one takes a sample, but how can we be sure that sample will give us the precise level of salt in the whole pot - in other words how much do we need to take out and taste to give us confidence that we got it right.
If we take too little then we might feel unsure because it might not be precise enough. If we take too much we waste time and soup and don't really increase our confidence that we got it right.
So in general how can we be sure at a certain level of confidence - say 98%? Well it all depends on your sample frame, if you get that right and pick a sample of sufficient size from it then that is the best you can do.
Just as an illustration, I am not saying you test soup this way!! In the soup example I might take out a litre from the cauldron into a saucepan (my sample frame) and then from that take two spoonfuls (my actual sample).
Here it is obvious if you get the wrong sample frame (from the wrong cauldron) the results will be wrecked.
Precision means or refers to how accurately your sample mirrors or represents the population. For example, if you made a several cauldrons of soup of different sorts for say 100 people and you wanted to test it to see if it had enough salt then its obvious one takes a sample, but how can we be sure that sample will give us the precise level of salt in the whole pot - in other words how much do we need to take out and taste to give us confidence that we got it right.
If we take too little then we might feel unsure because it might not be precise enough. If we take too much we waste time and soup and don't really increase our confidence that we got it right.
So in general how can we be sure at a certain level of confidence - say 98%? Well it all depends on your sample frame, if you get that right and pick a sample of sufficient size from it then that is the best you can do.
Just as an illustration, I am not saying you test soup this way!! In the soup example I might take out a litre from the cauldron into a saucepan (my sample frame) and then from that take two spoonfuls (my actual sample).
Here it is obvious if you get the wrong sample frame (from the wrong cauldron) the results will be wrecked.