Sample Size Question

[dnvrbw]

Hello all,
I'm not a statistics expert so I am wondering if someone can give some guidance on need sample size. Here's the scenario:

I have some distinct markets (e.g. LA, NYC, etc...) that we want to be able to compare to an overall national sample. Also would like to be able to compare the markets vs. markets (i.e. LA vs. NYC). If I use one of the sample size calculators I get that I need roughly 400 responses whether an individual city or national. So that would seem to require 400 national plus 400 per market. Problem is that there are over 10 markets making the cost of such a project prohibitive.

Does anyone have a sample size/distribution suggestion for doing such a project?

Thanks.

[Ian Straus]

......
Does anyone have a sample size/distribution suggestion for doing such a project? ....

OK, so information has a price, and you don't want to pay for 4,000 interviews.

That 400 figure assumes you want a 95% confidence level that your results are plus or minus 5% of the '"real" value in a given market, also assuming that responses will be split about evenly on your question.
If you can't afford that, then what confidence level will you settle for?
Note that plus or minus 10% takes a sample of about 100. If you can also settle for 90% confidence in that your answer is down to less than 70.
Do you have reason to believe people's answers are or are not split about evenly on your topic?
How sensitive to the numbers you will be producing are your business decisions? How much is at stake?

Answer these questions on both an individual-market and a comparison between two markets basis, because the sample size calculations are not the same. Are there really two different decisions, one national and the other between-markets, and what if you don't research the between-markets one?

Even though you don't want to pay for 4,000 interviews, is that the same as not being able to afford them?

You haven't told us anything about your proposed methodology or any screening criteria / description of who is to be sampled, so you won't get any suggestions about method. Which can make a big difference in cost.

[Statman]

See the Research Aids link at www.surveysystem.com for some calculators (or even at my Web site, Site Map page the links toward the bottom). Also look into G*Power, an excellent free application for sample size under different assumptions and models

[sdobney]

Unfortunately as Ian says there are no easy ways around this - you want to dive down into the detail of individual markets so you will need sufficient sample in each of those markets to be able to resolve differences between the local and national picture. If you're already operating in 10 markets, you'll already be budgeting for 10 lots of activity and be carrying the overhead of the geographic distribution, so local sampling is just a cost of doing business like local ads or local promotions and local employee costs.

If you're not yet operating in the markets, for instance you're looking at where to start up, one method, which has upsides and downsides, is to start with a smaller sample size and then scale up with a later sample boost in the markets where you start to feel you need to examine greater detail. Essentially the smaller sample size will only be able to resolve big differences in markets - but this might be sufficient to show that all markets 'look' sufficiently similar and that other factors (eg costs of set-up) can take priority. This way round you may get two or three 'more likely' candidates. In which case you can extend the sample for these areas to help choose between these more likely alternatives.

Now the downside is that strictly speaking if there are no significant differences, but some variations within the smaller sample sizes, the 'no significant differences' is essentially implying that the variation could be a pure sampling effect and the 'candidates' are just picked because of the random noise. And in fact, the boost sample might demonstrate this by resolving out the previous differences and taking the candidate back to the average. However, in the light of the budget, it may be a gamble that you're willing to take.

If you are really looking for candidates start-up markets, you might also consider rolling out the sample - so starting with three or four markets to see if one stands out and if not add two more etc. This might not show the best of the 10 markets, but it might find something sufficiently good, particularly when other factors are included.


Saul