Challenges and opportunities for big data in the Digital Railway

Source: RTM Dec/Jan 17

Dr John Easton and Dr Shruti Kohli from the University of Birmingham write for RTM about the role big data can play in transforming the country’s railways.

Over the last few years there has been a great deal of discussion within the industry around the shape the railways will take in the decades to come. Much of the debate has focused on the arrival of new, digital technologies that will replace the systems the industry has relied on until now, and what the impact of those changes might be in terms of passenger experience, operation of the network, safety and security, and business processes. 

At first glance, it is easy to assume that the digitalisation of the railway is all about train control. However, the challenge that new digitally-enabled infrastructure and systems present to the railway industry is about far more than in-cab signals. It’s about creating a virtually-connected ‘system of railway systems’, where the timely provision and analysis of data on the state of the network enables smoother running, increased flexibility, and, critically, greater use of the physical capacity available. 

Nowhere is the early application of these principles more obvious than in the research arena, where the EU-funded Shift2Rail lighthouse projects In2Rail and IT2Rail are blazing new trails in the development of next generation Traffic Management Systems and Passenger Information Systems respectively. These new systems share a common theme, drawing together information from around the industry, and delivering it to the right people, in the right place, at the right time to support effective decision-making. Work of this calibre does not take place in isolation, and both projects are building on a huge body of previous work from within the industry and in the research sector, including projects such as AUTOMAIN, ON-TIME, and Capacity4Rail. 

Big data analytics and cyber security 

Developing a deeper understanding of the asset base is obviously a key win for digital technologies, and while condition monitoring may grab the lion’s share of the headlines, the potential for improvements in defect reporting and sharing of best maintenance practice are both easy wins in this area. By combining large volumes of data from similar fleets across the network, for example, big data analytics approaches will enable the industry to identify patterns that represent best practice in operation and maintenance for those vehicles and, perhaps more importantly, show how day-to-day activities can be improved on poorly-performing routes. 

Data collection is also evolving, with IoT technologies offering exciting new possibilities for very low-cost and short-term sensor installations. The potential application of big data techniques within the industry will not be limited to the physical assets it owns and operates. Understanding the movements of passengers around station buildings, for instance, will become increasingly important as headways decrease and station dwell times become more critical. The automatic analysis of footage from station CCTV systems will be an important additional tool in this area, and will be able to supplement existing approaches, such as passenger surveys, by providing continuous footfall figures across the network. 

Cyber security game changer 

The digitalisation of the railway is a game changer for cyber security. If the industry is to achieve the potentially huge business benefits that could arise from greater interconnectivity between systems, then we need to change the way we think about security, and in particular we must address the soon-to-be-flawed assumption that railway ICT is protected because it runs on closed networks. 

New systems will need to be secure by design, the workforce will need to receive training in security best practice, and as an industry we will need robust procedures in place to ensure that when – not if – problems arise the network can continue to run safely and with a minimum of disruption. 

A key component of the successful adoption of the new systems will lie in the adaptation of business processes to the new way of working, and particularly supporting the exchange and integration of data owned by the various stakeholders across the industry. Fortunately, there is already some great work going on in this area in projects such as the RSSB’s T1010 project on enabling the use of cross-industry remote condition monitoring. Business processes will also need to evolve to accommodate rail’s role in the wider, multimodal transport system, where the challenge will be to adapt technologies such as electronic ticketing that have been shown to work well in closed systems, such as those run by Transport for London, to the national system. 

The scale of the changes that digital technologies will bring to the industry mean that this is an exciting time to be working in rail, and the topics discussed here are just the tip of the iceberg. As a final thought, it’s important to remember that if digitalisation is going to succeed, then we need to make sure that the change is led by the passengers and staff, and while the possibilities are almost endless, we need to focus on meeting the needs of the industry, and not the needs of the technology.



James Palma   01/02/2017 at 20:46

Very interesting and much needed. There appears to be a general consensus that old imformation for assets which are now well over 150 years old is readily available. As a reseacher in transport related infrastructure academically and proffessionally i can testify that with decades of budget cuts and a lack of understanding as to retention of archive material and documenting changes to assets, that there is a hidden cost in unravelling exisitng data. This hidden cost caused by misunderstanding of asset ownership, rights and responsibilities for decades and lack of requirements for projects, contractors and business units to clearly document what was done how when and why and to what. This leads to lack of understanding and additional costs to unravel the tangled web. Often too late to make amends. A good way to understand what is involved is a recent article in the instituition of civil engineers journal of asset management Dember issue which discusses the complexity of infrastructure ownership with regard to glagow subway. It is entitled a conceptual framework for land use and metro infrastructure and demonstrates some of the complexities at hand.

Cyberh   02/02/2017 at 15:57

Very informative approach when tackling Big Data security. When considering to implement a big data strategy, HPCC Systems is an open source data-intensive supercomputing platform that can help companies derive actionable insights from their data. While security is definitely a major concern for anyone using, manipulating and transporting big data, the HPCC Systems platform has added a wide array of extensive, configurable and customizable security features, the choice of which depends on the risk tolerance of the HPCC Systems data stakeholders. An administrator can quickly and easily configure simple user authentication via LDAP or HTPASSWD. Additionally, our LDAP security manager can be configured in minutes, and it implements a very robust set of features where you can guard access to the many HPCC Systems features, as well as protecting access to the data and services. Finally, we expose a robust pluggable security manager framework, where software developers can implement their own custom security managers tailored to their unique requirements. Learn more at

Add your comment


rail technology magazine tv

more videos >

latest rail news

View all News

rail industry focus

The most complex signalling phase in Britain

23/03/2017The most complex signalling phase in Britain

RTM caught up with Richard Scott, programme m... more >
Bond Street: The importance of clear and consistent stakeholder communication

21/03/2017Bond Street: The importance of clear and consistent stakeholder communication

Chris Pateman, Crossrail’s Bond Street ... more >

editor's comment

08/03/2017A celebration of rail achievement

Welcome to this special UK Rail Industry Awards (UKRIA) review issue of RTM. The fourth edition of the prestigious awards took place on 9 February at London’s Battersea Evolution and, once again, was a huge success.  With over 1,100 rail decision-makers in attendance, the black-tie event was a great place for networking and celebrating the success of the rail industry in the last 12 months. To see who won at this year’s event, we have a special section that starts on... read more >

last word

Reasons to be cheerful

Reasons to be cheerful

Ahead of the major imminent reforms to the apprenticeship system, Simon Rennie, general manager of the National Training Academy for Rail, outlines the industry’s reasons to be positive about the future of skills. Those of a particular age will recall with a warm glow the prestige of the British Rail Apprenticeship scheme and how th... more > more last word articles >

'the sleepers' daily blog

Digital service to reunite passengers with lost valuables trialled in Wales

11/04/2017Digital service to reunite passengers with lost valuables trialled in Wales

Customers pulling their hair out at constantly leaving their valuables on the train may be in luck, as an innovative new system may help them track down their possessions. The system, FindMyLost, is a digital service which allows passengers and railway staff who find an item to put it online with a description, along with a photo and deta... more >
read more blog posts from 'the sleeper' >


Intertrain: ready for the future

23/02/2017Intertrain: ready for the future

RTM recently attended Intertrain’s ‘Driving for Success’ event in Doncaster, where leaders from major players such as Carillion... more >
Tackling regulation at its routes

24/01/2017Tackling regulation at its routes

John Larkinson, the ORR’s director of railway markets and economics, speaks to RTM about the move to regulating Network Rail at a route lev... more >
Investing in the future of Scottish Rail

15/11/2016Investing in the future of Scottish Rail

Phil Verster, managing director of ScotRail Alliance, speaks to RTM’s Luana Salles about the recently-published ‘Investing in the Fut... more >
Maintaining the safety momentum

12/09/2016Maintaining the safety momentum

Britain’s railways are currently the safest they have ever been, but there is still room for improvement, Ian Prosser, HM chief inspector o... more >