What happens when we stop asking people what they think?

Accessibility, the ease with which an individual can get to destinations, is now well established as a social justice issue in transport planning. In 2003, Making the Connections, a report by the Social Exclusion Unit highlighted poor accessibility as one of the multi-faceted aspects leading to social exclusion. This resulted in many transport practitioners engaging with issues of transport-related social exclusion and issues of inequality through the process of Accessibility Planning. However, efforts were largely focused around the quantification and measurement of access to destinations in order to produce threshold based accessibility targets. A database of accessibility statistics covering the whole of England at zone level was also produced. Such attempts at quantification have been critiqued, especially in the context of understanding social exclusion, for their black box approach, which fails to understand the lived experience of (in) accessibility and its potential impacts.

However, another source of data, the National Travel Survey (NTS) has been largely underutilised in this regard. Between 2005 and 2012 there was a bank of questions in the NTS, which asked how long it would take respondents to reach the nearest of each of a set of destinations using public transport or walking; this has recently been replaced by a computer generated ‘objective’ measure [i],[ii].

Increasingly, moves towards big data and the ability to link to administrative records means that we may be able to replace information previously asked in social surveys with official records. For example, rather than asking people about their health, we might be able to access their health records. But there may be some cases, where understanding an individuals’ perception of their health is more important than a record of visits to the GP. It is therefore crucial to be clear about what we can and cannot understand by asking certain questions in a survey, or by replacing them with information from large scale datasets. The same applies to the replacement of this set of questions in the NTS with objective measures.

It seems timely to consider what might be gained from including the question regarding self-reported journey times to destinations and what the implications of excluding it might be in terms of our understanding of accessibility. What does this change in measurement technique mean for the way in which we understand accessibility to local services based on the NTS data and does the move away from self-reported data have any implications?

According to the NTS guidance, the previous question in the survey was an objective measure:

“These questions are measures of fact, not of opinion. Interviewers are encouraged to make use of information other than that provided by the household if this provides a more accurate indication of the true position. ‘Walk time’ assumes a walk speed of three miles per hour, and takes no account of, for example, any infirmity or disability of the respondent” (NTS 2006, p.60)[iii].

On this basis, the move in the latest NTS to include ‘objective’ measures of accessibility calculated using TRACC, utilising geo-coded destinations and public transport timetables based is a positive development towards including a measure which is more comparable across the sample and is not affected by individual’s experience of the time taken or definition of what might constitute a grocery store, for example. However, while the objective potential accessibility provided by the transport and land use system is important for understanding the opportunities an individual does or does not have, when trying to address issues of transport and social inclusion it might be that how an individual perceives or really experiences accessibility is more important in determining their travel behaviour, and subsequently outcomes which may or may not contribute to processes of social exclusion.

In the National Travel Survey, there is however no indication of whether interviewers did override respondents answers or not and given that responses are self reported (either by the respondent or the interviewer) they are by nature subjective and likely to be influenced by individuals’ abilities and constraints, rather than being an indication of any ‘objective’ or ‘factual’ level of service provision. I have previously compared the NTS survey question to another nationwide objective dataset of public transport journey time access to destinations, the DfT Accessibility Statistics using data from the same year and at comparing at the postcode sector level and found socio-spatial differences between self reported and objective measures. The very fact that differences exist, suggests that the NTS was not as ‘accurate’ as an objective measure as was intended, again supporting the change of methodology towards one which does not rely of self-report or interviewer reports. However, the fact that differences exist also highlights that there is variation in self reported levels of accessibility compared with objective measures, which has implications in terms of policy interventions and targets to improve accessibility.

The new linked data includes a caveat that it is not directly comparable to the previous questions due to the change in methodology. This is clear from Figure 1, which shows clear shifts in the proportion of the population with access to each type of destination between 2012 and 2014. While not dramatic they do show a clear departure from previous trends and extrapolated to the population might have considerable impact. For example, the increase from 22% to 25% of the population having access to a hospital within 15 minutes, equates to 1.8 million people (based on 2012 and 2015 mid-year population estimates for England). This might have substantial implications for planning and policy decisions around accessibility to hospitals, and lead to a possibly incorrect assumption that accessibility has improved against targets.  It is clear that the change in methodology has affects destinations differently, some showing and increase and some a decrease. It is also likely that there change has differential impact geographically and across socio-demographic groups. Unfortunately, by not including the two approaches in the same survey year, it is not possible to evaluate the impact of this change in methodology, although we can speculate as to what some of the implications might be, it is not possible to control for any actual changes over time.

Figure 1 suggests that there may be a difference in the definition of destinations between self-reported responses and the dataset used to calculate objective measures. The increased proportion of the population who can reach a hospital, GP or post office according to the new measure might be interpreted as meaning that there are more destinations in a national dataset than would be considered in a subjective choice set. On the other hand, we might assume that, particularly for hospitals, these are infrequent journeys which might be reported times are longer due to unfamiliarity. Familiarity has previously been reported as a reason explaining the ‘accuracy’ of journey time estimations.

NTS_figureAC
Figure 1 – Percentage of population who can access a given destination within 15 minutes by public transport or walking based on the National Travel Survey 2005-2014 (excluding 2013). For some destinations the data for 2008 has been imputed by taking the mean of 2007 and 2009 as the data is not available for all destinations for this year.

Individual difference is one of the main reasons for disparities between the two types of measure. A place based measure cannot encapsulate the different travel experiences of all those living in a particular place. In my thesis I found that older people and car drivers reported longer journey times than measured objectively.  There are two potential mechanisms at play here: in some cases differences occur because of inaccuracies in objective measures, for example because of the spatial scale used or the destinations included; differences may also occur because self reported measures are not accurate, relative to the objective measure. Differences may occur because an individual is not aware of a particular journey or destination, issues of familiarity or perceiving unfamiliar journeys to be longer than they are, or because a journey actually does take longer for an individual than an objectively measured average, because of mobility constraints such as children or older age.

Therefore, for some population groups, the differences between the TRACC and survey question data may be much larger than others, which leads us to think about what implications this might have for issues of social inclusion in transport planning. While contour mapping and cumulative accessibility measures might demonstrate that certain population groups are more likely to be at risk of transport related social exclusion, the fact that some of these groups may perceive accessibility to be worse than an objective measure would suggest could mean that they are at even greater risk of social exclusion. On the other hand, people may adapt to adverse circumstances and therefore not report problems, highlighting the need to compare self reported measures to objective measures.

If we can calculate public transport accessibility to destinations at the population level (using either the CAI or TRACC) then the value of attaching this data to a representative sample and then using this to extrapolate to the population seems limited. Would it not be better to have the national dataset available separately and keep the NTS as a survey which can be linked to a whole range of other data sets? One clear benefit, of course, is the ability to link data regarding accessibility to destinations to the rich information available in the NTS, which presents a range of research opportunities outwith the scope of the discussion here. The NTS had value in understanding how individual’s experiences might differ from an objective ‘norm’ (or at least a self reported measure, which did not intend to represent an objective reality, would).

The NTS is a continuous survey, designed to measures change over time. The change of methodology undermines this role. It should be remembered that the DfT reverted to traditional travel diary methods following a trial of using GPS to replace the traditional travel diary element of the NTS due to large discrepancies which meant that the use of the NTS as an ongoing measure of travel behaviour would be limited. This is not to say that the GPS data, or in this case, objective measures of accessibility are not useful, but it must be remembered that they measure something different to questions asked in a social survey. The nature of the knowledge is different, and evidence showing differences should make it clear that we need both kinds of knowledge to address issues of inclusion in transport.

The focus here has been on journey time, which is an important component of accessibility to destinations. However, for those at risk of transport related social exclusion, it may not be the most important factor and more work is needed to understand the importance of other aspects of accessibility across the social spectrum. A greater understanding of how perceptions differ from objective measures, for whom and why is crucial to addressing exclusion as it might illuminate why some groups feel unable to access destinations, despite measures which otherwise suggest reasonable levels of accessibility.

Transport planning has been criticised for its positivist approach, considering averages and not individuals’ experience. In this regard the change in methodology would appear to be a backwards step, moving away from understanding how individuals experience access to destinations. Of course, both approaches to measurement are needed if we are to make any progress in improving accessibility against a baseline. But while much academic research and professional practice has focussed on objective measurement of accessibility, perceptions have recently been highlighted as an important area for future research, at the very same time as the NTS is moving away from asking people what they think in favour of trying to capture the ‘reality’.

[i] NTS, 2014, Technical Report: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/457690/nts2014-technical.pdf

[ii] https://www.linkedin.com/groups/4246233/4246233-6045492862064353284

[iii] NTS, 2006. National Travel Survey ( NTS ) data 2002-2006 : User guidance.

Thanks to Julie Clark for comments and proofreading

Advertisements