Post Weighting and Sample Size

[LucyMoore]

Hi,

Looking for some advice regarding the sample size requirements for weighting data. I have a sample of 1100 approx and would like to weight the data to reflect the true population for gender, age, and state. I have adequate sample numbers in each gender, age and state. However I am using SPSS and it only lets me apply one weight at a time. I would like to apply all three weighting variables at once.

My questions are:
1. Is it possible to multiply my three weighting variables together for each case to determine a combined weight?
2. Do I need to have a decent sample (n>30) in each possible gender, age, and state combination?

Any comments or discussions regarding these issues would be appreciated.

Thanks,
Lucy

[Statman]

Yes, the combined weight is via multiplication. However, be prepared to have some variations if the various segments aren't truly interactive. That is, if the relationship between gender, age & state isn't "logical" then the combined weight's response base might not be reflective. As an example, I had a study across several countries and within each there were market segments. We needed a combined weight but the distribution of segments by country wasn't parallel. Some countries had all segments and some didn't so some logic was faulty and the combined weight didn't work "100%."

Sample by weighting segment - Yes, there needs to be a sufficient base for each segment so that the data are "stable." Note that this is different from "reliable." The rule of thumb seen over the years is 100 cases. Not really sure where this came from but Gallup comes to mind.

One final comment. If your sample is not based on simple random or some form of simple stratified proportional then you could be in the complex sampling arena. Here there is selection based on strata & stratum and not purely random. If this is the case then special tools have to be used for analysis. Check out www.westat.com & www.spss.com as well as http://www.rti.org/sudaan/.

S

[vlad simion]

you should try rim weighting instead of combined weight via multiplication

[Statman]

vlad, how would he apply rim weighting

[vlad simion]

hi,

if you have quantum it's quite easy, as a matter of fact, it does the rim weighting for you, otherwise it's a little bit more complicated, you have to implement the rim weighting algorithm in spss (this can be done with the matrix language in spss), if you have spss 14 or 15 and have the programability plugin-in, there is a python program developed by Jon Peck, which calculate the weights by raking procedures.
here is the algorithm that quantum use to do rim weighting :
"The following references to target[] [], work[][] and sum[][] refer to the various arrays as they are named in the C code which calculates rim weights.

Target[D][E] is the given target value for element E of dimension D.

Work[D][E] contains an adjustment value, updated on each iteration, by which to adjust the weighted number of records in element E of dimension D. Thus, the actual weight value, as iterated so far, for one particular cell, is Product-over-dimensions-D-for the element E of dimension D which the cell corresponds to.

Sum[D][E] contains, on each iteration, the weighted number of records in element E of dimension D.

How the code performs the iterative calculation:
Firstly, set work[D][E]=1.0 for all E within all D. This says that our first estimate of how much each record falling into element E of dimension D should contribute to the total weighted sample is 1.0, ie, we start with each record
being unweighted.

for each iteration, I,
Set rms=0.0 - this will accumulate the root-mean-square, which is the sum
over E within D of the squared difference between how many
records we asked for in E within D and how many we currently
have, allowing for iterations so far.

for each dimension, D,
Initialize sum[D][E]=0.0 for each E, this D for each record, R,

Calculate the product, P, over all dimensions, X, of work[X][Y],
where Y is the element of dimension X that record R belongs in.

Add P to sum[D][Y], where Y is as above, for each element, E, of dimension D

Adjust work[D][E] as follows:
work[D][E] = work[D][E] * target[D][E] / sum[D][E]

Add (target[D][E] - sum[D][E]) squared to rms"
"Quantum Rim Weighting Algorithm
(Workings of rim weighting, originally written prior to March 1990)" from spss/quantum support.
i have also sent you the paper “On a Least Squares Adjustment of a Sampled Frequency Table when the Expected Marginal Totals are Known” by W. Deming and F. Stephan, which is the generalized algorithm for iterative cell frequency estimating
hope this helps,

cheers,

vlad

[Statman]

Tks

[phil_hearn]

You need to be careful with rim weighting as it is more volatile than standard target weighting. It should not be used as a 'quick fix' without bearing in my mind some of the consequences.

For example:

1) If any of the rims have any relationship with each other, it could cause volatile weights to be generated. For example, if you used rims of social grade and owners/non-owners of luxury cars, this could be dangerous.

2) If the one or more rims are very unevenly distributed, you may get extreme weights applied. For example, users/non-users of a product where 96% of people are users.

3) If you have too many rims (someone recently asked me to apply 15 rims to 200 respondents!), you are asking for trouble.

As with any weighting procedure, you should always check the weights that are being used. However, it is absolutely essential that the spread of factors applied are checked when applying rim weighting. It is tempting to trust the data just because the sample sizes look right after applying rim weighting. My instinct is to be distrusting and cautious.

Do not imagine that rim weighting is a 'magic black box'. It is scaling your data to each target one by one and then repeating this process in an iterative manner until it achieves the targets you want. To understand this properly, it is worth playing with some data in Excel. The software you are using should be intelligent enough to tell if the rims are not achievable, as this can happen too.

[vlad simion]

hi,

-i'm not sure i understand what you meant by volatile weights, could you be more specific please and explain, are you referring to the fact that social grade and owners/non-owners might be highly correlated?
-for the situation when the rims are unevenly distributed, you get extreme weights only if the it's a big gap between the observed values and the target you must end up with, when this happens you get a very poor weight efficiency,
-i have applied rim weighting for 20 concepts acrossed 4 demographic variables, trying to reach for each concept a specific target, and i have succeded with a weight efficiency of more than 90%, i must mention that within the demographics, the concepts were overlapping, meaning that a respondent might have seen 2 concepts or more, or being user of 2 or more concepts, so i dissociated them in terms of user/non-user per concept, trying not to weight one respondent several times. what are the perrils in this case?
-about the iterative process of rim weighting, it is done untill a convergence point is achieved, more specific in case of quantum, this point is when the root mean square is below 0.05, this point can be set within some ranges, depending the situation.

if i'm wrong or i said stupid things, please correct me and explain where i might get errors

many thanks,

vlad

[Vanina]

Hello,
I'm peruvian (sorry for my english) , I'm trying to apply the Rim weighting method to some panel data for an audience study, but i'm really concerned about the Rim weighting when it's been applyed a complex sample.
I 've built the weights whith the rim weighting (cells of a table), but how can distribute the total weight of the cell in the persons of my sample if each of them were selected independently with different probabilities?
In my opinion, all the person in a cell can't have the same weight (mean).
I hope, you understand my doubt.
Thank you so much for your answers.