Wednesday, May 26, 2021

You can't manage what you don't measure: Improving OIA statistics

Back in 2017 the then-SSC published the first set of performance statistics on the Official Information Act. The statistics were crude: just the number of requests handled, the number and percent on-time, the number published on the agency's website, and the number of complaints and final views issued by the Ombudsman. But it immediately allowed some problems to be identified, and there was a commitment that:

Over time the information on performance that is gathered and published will increase to provide a more comprehensive picture of compliance with the letter and spirit of the Act.
That hasn't happened, and five years on, we're still using the same crude stats, with a crude focus on timeliness we were using in the beginning. Its not like they didn't think about it: in late 2017 they published agency guidance on Selection and Reporting of Official Information Act Statistics, encouraging agencies to collect and report additional statistics to give a better view of their performance. But there was no real followup: Te Kawa Mataaho Public Service Commission does not even bother to monitor what statistics agencies collect. In September / October 2019 they cautiously proposed a limited expansion of statistics to include whether information was released (granted in full / granted in part / refused / publicly available), and possibly some information on extensions and transfers, but nothing has come of it. And in the meantime agencies have learned to juke the stats, with an increasing number of extensions for "consultations" to meet timeliness targets, or in the case of the police, packing the stats with a vast number of requests which are not legally OIAs to hide their utter failure to obey the law. And they are able to get away with this because of the crudeness of our statistics. And you only have to look at the UK, which publishes full annual statistics on timeliness, outcomes, and withholding grounds used, to see that we could be doing a lot, lot better.

So what could they do to produce immediate improvements? Back in February I used the OIA to survey core government agencies on what OIA statistics they collected, using their 2017 guidance as a template. The raw data (including links to responses) is here. The good news is that most government agencies already collect or report the information TKM-PSC proposed in 2019, and most also measure the time taken to process requests or to extend them. Which means that with a bit of leadership for TKM-PSC we should be able to get full UK-level stats on timeliness, extensions, and outcomes. Which would show us a bit more where agencies are falling down and how they're juking things.

So will they do it? Its literally their job, and I've been told by TKM-PSC staff that their Minister, Chris Hipkins, is fully committed to open government. If that's the case, then fixing this seems to be an obvious and easy way to show it. You can't manage what you don't measure. So start measuring this, and let public expectations start managing it.