Challenges and opportunities for big data in the Digital Railway

Source: RTM Dec/Jan 17

Dr John Easton and Dr Shruti Kohli from the University of Birmingham write for RTM about the role big data can play in transforming the country’s railways.

Over the last few years there has been a great deal of discussion within the industry around the shape the railways will take in the decades to come. Much of the debate has focused on the arrival of new, digital technologies that will replace the systems the industry has relied on until now, and what the impact of those changes might be in terms of passenger experience, operation of the network, safety and security, and business processes. 

At first glance, it is easy to assume that the digitalisation of the railway is all about train control. However, the challenge that new digitally-enabled infrastructure and systems present to the railway industry is about far more than in-cab signals. It’s about creating a virtually-connected ‘system of railway systems’, where the timely provision and analysis of data on the state of the network enables smoother running, increased flexibility, and, critically, greater use of the physical capacity available. 

Nowhere is the early application of these principles more obvious than in the research arena, where the EU-funded Shift2Rail lighthouse projects In2Rail and IT2Rail are blazing new trails in the development of next generation Traffic Management Systems and Passenger Information Systems respectively. These new systems share a common theme, drawing together information from around the industry, and delivering it to the right people, in the right place, at the right time to support effective decision-making. Work of this calibre does not take place in isolation, and both projects are building on a huge body of previous work from within the industry and in the research sector, including projects such as AUTOMAIN, ON-TIME, and Capacity4Rail. 

Big data analytics and cyber security 

Developing a deeper understanding of the asset base is obviously a key win for digital technologies, and while condition monitoring may grab the lion’s share of the headlines, the potential for improvements in defect reporting and sharing of best maintenance practice are both easy wins in this area. By combining large volumes of data from similar fleets across the network, for example, big data analytics approaches will enable the industry to identify patterns that represent best practice in operation and maintenance for those vehicles and, perhaps more importantly, show how day-to-day activities can be improved on poorly-performing routes. 

Data collection is also evolving, with IoT technologies offering exciting new possibilities for very low-cost and short-term sensor installations. The potential application of big data techniques within the industry will not be limited to the physical assets it owns and operates. Understanding the movements of passengers around station buildings, for instance, will become increasingly important as headways decrease and station dwell times become more critical. The automatic analysis of footage from station CCTV systems will be an important additional tool in this area, and will be able to supplement existing approaches, such as passenger surveys, by providing continuous footfall figures across the network. 

Cyber security game changer 

The digitalisation of the railway is a game changer for cyber security. If the industry is to achieve the potentially huge business benefits that could arise from greater interconnectivity between systems, then we need to change the way we think about security, and in particular we must address the soon-to-be-flawed assumption that railway ICT is protected because it runs on closed networks. 

New systems will need to be secure by design, the workforce will need to receive training in security best practice, and as an industry we will need robust procedures in place to ensure that when – not if – problems arise the network can continue to run safely and with a minimum of disruption. 

A key component of the successful adoption of the new systems will lie in the adaptation of business processes to the new way of working, and particularly supporting the exchange and integration of data owned by the various stakeholders across the industry. Fortunately, there is already some great work going on in this area in projects such as the RSSB’s T1010 project on enabling the use of cross-industry remote condition monitoring. Business processes will also need to evolve to accommodate rail’s role in the wider, multimodal transport system, where the challenge will be to adapt technologies such as electronic ticketing that have been shown to work well in closed systems, such as those run by Transport for London, to the national system. 

The scale of the changes that digital technologies will bring to the industry mean that this is an exciting time to be working in rail, and the topics discussed here are just the tip of the iceberg. As a final thought, it’s important to remember that if digitalisation is going to succeed, then we need to make sure that the change is led by the passengers and staff, and while the possibilities are almost endless, we need to focus on meeting the needs of the industry, and not the needs of the technology.



James Palma   01/02/2017 at 20:46

Very interesting and much needed. There appears to be a general consensus that old imformation for assets which are now well over 150 years old is readily available. As a reseacher in transport related infrastructure academically and proffessionally i can testify that with decades of budget cuts and a lack of understanding as to retention of archive material and documenting changes to assets, that there is a hidden cost in unravelling exisitng data. This hidden cost caused by misunderstanding of asset ownership, rights and responsibilities for decades and lack of requirements for projects, contractors and business units to clearly document what was done how when and why and to what. This leads to lack of understanding and additional costs to unravel the tangled web. Often too late to make amends. A good way to understand what is involved is a recent article in the instituition of civil engineers journal of asset management Dember issue which discusses the complexity of infrastructure ownership with regard to glagow subway. It is entitled a conceptual framework for land use and metro infrastructure and demonstrates some of the complexities at hand.

Cyberh   02/02/2017 at 15:57

Very informative approach when tackling Big Data security. When considering to implement a big data strategy, HPCC Systems is an open source data-intensive supercomputing platform that can help companies derive actionable insights from their data. While security is definitely a major concern for anyone using, manipulating and transporting big data, the HPCC Systems platform has added a wide array of extensive, configurable and customizable security features, the choice of which depends on the risk tolerance of the HPCC Systems data stakeholders. An administrator can quickly and easily configure simple user authentication via LDAP or HTPASSWD. Additionally, our LDAP security manager can be configured in minutes, and it implements a very robust set of features where you can guard access to the many HPCC Systems features, as well as protecting access to the data and services. Finally, we expose a robust pluggable security manager framework, where software developers can implement their own custom security managers tailored to their unique requirements. Learn more at

Add your comment


rail technology magazine tv

more videos >

latest rail news

View all News

rail industry focus

Risk management for railway engineering

22/05/2017Risk management for railway engineering

RTM talks to Ben Shirley, health and safety r... more >
Get ready for cyber safety

15/05/2017Get ready for cyber safety

It’s time the rail industry turns its a... more >

editor's comment

08/05/2017All set for Railtex

As the rail industry prepares for the biggest UK rail show of the year, Railtex 2017, we have produced a specially expanded edition of RTM for you.  Our exclusive show guide, which starts on page 67, provides you with everything you need to know about the things to see, hear and do at the three-day show. A look through our preview pages will give you a good idea of which stands to visit, as well as meetings to set up.  The RTM team – as ever – will be at... read more >

last word

Collaborative working is the key to the future of rail infrastructure

Collaborative working is the key to the future of rail infrastructure

David Hawkins, operations director at the Institute for Collaborative Working (ICW), on why ISO 44001 is a new evolution in collaborative working. The launch of BS 11000 in 2010 heralded a step change in thinking for the UK rail industry led by Network Rail. Since those early days, the benefits of a structured approach to collaboration ha... more > more last word articles >

'the sleepers' daily blog

Opinion poll: HS3 must take priority over Crossrail 2

19/05/2017Opinion poll: HS3 must take priority over Crossrail 2

Fresh from their victories in the metro mayor elections earlier this month, new leaders of Greater Manchester and Liverpool City Region Andy Burnham and Steve Rotheram were quick to make transport a top priority for their cities last week. In a post on Twitter, both called on the government to prioritise plans for an HS3 link connecting t... more >
read more blog posts from 'the sleeper' >


Intertrain: ready for the future

23/02/2017Intertrain: ready for the future

RTM recently attended Intertrain’s ‘Driving for Success’ event in Doncaster, where leaders from major players such as Carillion... more >
Tackling regulation at its routes

24/01/2017Tackling regulation at its routes

John Larkinson, the ORR’s director of railway markets and economics, speaks to RTM about the move to regulating Network Rail at a route lev... more >
Investing in the future of Scottish Rail

15/11/2016Investing in the future of Scottish Rail

Phil Verster, managing director of ScotRail Alliance, speaks to RTM’s Luana Salles about the recently-published ‘Investing in the Fut... more >
Maintaining the safety momentum

12/09/2016Maintaining the safety momentum

Britain’s railways are currently the safest they have ever been, but there is still room for improvement, Ian Prosser, HM chief inspector o... more >