One of favorite sayings is ‘data in of itself is useless.’ That may sound odd coming from a data strategy and engineering firm, but has turned out to be a central tenant of our work. What we mean is that data, as a series of numbers, is incomplete on its own. Data has value largely in relation to things, what we call context.
An example everyone can relate to is a stock price. If I tell you a particular stock has a share price of $33.50, you’d probably have some follow up questions: how does that compare to the 52 week high/low? What are competitor’s share prices? How does the share price compare to earnings?
All of these are questions that add context to the original number - its not until you find these additional answers that you can understand what the original data point means.
In the context of population health, CHNAs, and other assessments, the key piece of context is a benchmark. A benchmark is simply a comparison value that tells you how a particular community relates to its peers; is it better or worse? Is it getting better or worse over time?
There are a couple of different types of benchmarks that are commonly used.
National Averages
The most commonly used benchmark is the national average (usually the mean). This can be calculated based on a few different data sets, depending on the source and granularity available. The national average is useful in that is easy to understand and broadly relevant. For most population health data sets, its calculated based on county-level values, but can be based on tract-level values if available.
The downside of a national average is, well, general and not particularly relevant in some cases. If you are in an urban environment, the national average for things like housing cost, transportation access and median income will be dramatically lower than what you see.
State/Regional Averages
Less common, but usually more relevant, are state and regional averages, usually also derived from county values. Often these benchmarks can provide a more relevant comparison, especially for things like economic indicators which are determined, in part, by state-level tax policies.
However, depending on the size of the state, these averages may be only slightly more relevant for smaller or more urban communities.
Index Benchmarks
A new technique for calculating benchmarks to compute ‘index’ values, which are simply a combination of areas or indicators selected using some common critieria. For example, we’ve created an urban cities index for use with our larger city and county customers; it provides comparison values for the 100 most populous cities in the US. This is a great benchmark to use with large urban centers where the most relevant comparison is other large urban areas. The same approach works very well with rural communities.
Index benchmarks aren’t limited to geography - because there are causal relationships between things like poverty and various health outcomes, creating index values for other places with similar poverty rates can provide an accurate picture of health outcomes by controlling for confounding variables.
How to use a benchmark
There are two approaches to using a benchmark value when analyzing population-level health data for assessments.k
Percentage variance from mean
The first is to calculate what the percentage difference is between a value and the benchmark. This is the simplest approach, and gives you a ‘quick and dirty’ sense of how you compare. Values over 0 mean a score that is better than the benchmark, values below 0 mean a score that is lower than the benchmark.
Unfortunately, percentage variance can be misleading, especially when comparing across indicators. A 20% difference in poverty level doesn’t necessarily mean the same thing that a 20% difference in stroke incidence, because you don’t know the significance of the percentage - is it good or bad compared to all the other values?
Standard deviation from mean
The second approach is to the use Standard Deviation from the mean to compute a Z Score value. In short, this uses a complex statistical technique to tell you both what the deviation from the average is, but also how significant that deviation is by comparing it to the distribution of all the other values. Therefore, you know that a Z Score of -1 in one indicator is comparable to a Z Score of -1 in another indicator, and that those values are worse than about 66% of peers.
Though a little more complicated, a Z Score approach to analysis is a much more robust and useful method.
Benchmarks are key to adding context to your data, turning numbers into actionable information.
