Earlier on this blog and through admonsters, we’ve introduced the idea of audience efficiency, shared key findings about 3^rd party data and spoken about the variations between industries and brands. In the fourth part of this audience efficiency series, I want to look at some of the limitations of this methodology primarily because overlap matters.

Why Overlap Matters was published on admonsters on 14^th August but you can also read it below:

Audience Efficiency: the limitations

Now we’ve explained (http://blog.tribalfusion.com/?s=audience+efficiency) the possibilities of ‘audience efficiency’ to drive the highest lift for any target reach, we ought to explain its limitations. They are borne out of the methodology that has to be used to identify the anonymous interests and intentions shared by users – and therefore audience segments – most likely to work for any one advertiser.

Remember that, to identify the behaviors that are characteristic of a user more likely to be in the target audience for a brand, the advertiser must first provide their own data on that target audience, whether it be people that visit their site, register or buy online – or even via offline shopping data. Those audiences are then found on the targeting provider’s own data network and their behaviors across the platform identified. From this we can see what behaviors an advertiser’s customers demonstrate and build out audiences of users that show those same behaviors.

However, this methodology has some limitations. Mainly because, for it to work, there has to be meaningful overlap between the audiences an advertiser provides with the users on the advertising provider’s platform.
There are three major challenges:

1.The whole methodology does not deliver insight into the combination of different behaviors that would form the best audience for an advertiser to target. It only provides insight into the relative lift of individual behaviors. Our Audience Efficiency model is how we get the former from the latter.

2.Because of the need for overlap it can be applied only to advertisers with large enough site audiences or that are able to identify proxies to their customers for analysis. Small advertisers or advertiser in smaller international markets may not be large enough to run this analysis without dropping the statistical significance test which will, by definition, let through more noise.

3.If the data network has insufficient breadth or depth it will neither have enough overlap or relevant audiences to show up. Data networks in smaller markets will need to work harder to maximize scale.
Let’s look at how all this works in greater depth.

Comparing audiences

We can visualize a simple data network as a nested set of audience segments. The overall data network is the outer circle, or parent, with “nested” audience interests and intentions. The size of the circles represents the relative reach of the topics with increasingly granular topics nested within their parents.

Unless the data network has near complete population coverage, any advertiser audience will have only partial overlap with the data network. The larger the advertiser audience and the data network then the more likely it is that there will be an overlap between the audiences.

The lift of an audience is just a measure of the overlap between a single data network audience and the advertiser audience relative to the average. However, this methodology includes a 90% significance test, i.e. is the sample size big enough to say with any confidence that the lift is really any different to the average? In our graphic, the advertiser audience has greater overlap with Audience A than the data network overall so it is above average lift.

Audience B is smaller than A but its relative overlap is even greater. It is a more relevant audience to the advertiser’s target so has high lift (provided that it passed the significance test). Audience C, however, while of the same size as B, has no overlap. Clearly it is not relevant to the target but there is also not enough information to even measure that lack of relevance. This is why it is hard to identify poorly related segments: by definition you won’t see enough overlap to know how bad it is!

If the advertiser target audience itself is small we can see that the overlap with the data network and any of its audiences becomes decreasingly significant and therefore increasingly hard to say anything meaningful. At best there might be some overlap with very high level branches like “sport” or “technology” but nothing like “fishing” or “xbox 360”, which is a more insightful and valuable signal.