Challenges and opportunities for big data in the Digital Railway

Source: RTM Dec/Jan 17

Dr John Easton and Dr Shruti Kohli from the University of Birmingham write for RTM about the role big data can play in transforming the country’s railways.

Over the last few years there has been a great deal of discussion within the industry around the shape the railways will take in the decades to come. Much of the debate has focused on the arrival of new, digital technologies that will replace the systems the industry has relied on until now, and what the impact of those changes might be in terms of passenger experience, operation of the network, safety and security, and business processes. 

At first glance, it is easy to assume that the digitalisation of the railway is all about train control. However, the challenge that new digitally-enabled infrastructure and systems present to the railway industry is about far more than in-cab signals. It’s about creating a virtually-connected ‘system of railway systems’, where the timely provision and analysis of data on the state of the network enables smoother running, increased flexibility, and, critically, greater use of the physical capacity available. 

Nowhere is the early application of these principles more obvious than in the research arena, where the EU-funded Shift2Rail lighthouse projects In2Rail and IT2Rail are blazing new trails in the development of next generation Traffic Management Systems and Passenger Information Systems respectively. These new systems share a common theme, drawing together information from around the industry, and delivering it to the right people, in the right place, at the right time to support effective decision-making. Work of this calibre does not take place in isolation, and both projects are building on a huge body of previous work from within the industry and in the research sector, including projects such as AUTOMAIN, ON-TIME, and Capacity4Rail. 

Big data analytics and cyber security 

Developing a deeper understanding of the asset base is obviously a key win for digital technologies, and while condition monitoring may grab the lion’s share of the headlines, the potential for improvements in defect reporting and sharing of best maintenance practice are both easy wins in this area. By combining large volumes of data from similar fleets across the network, for example, big data analytics approaches will enable the industry to identify patterns that represent best practice in operation and maintenance for those vehicles and, perhaps more importantly, show how day-to-day activities can be improved on poorly-performing routes. 

Data collection is also evolving, with IoT technologies offering exciting new possibilities for very low-cost and short-term sensor installations. The potential application of big data techniques within the industry will not be limited to the physical assets it owns and operates. Understanding the movements of passengers around station buildings, for instance, will become increasingly important as headways decrease and station dwell times become more critical. The automatic analysis of footage from station CCTV systems will be an important additional tool in this area, and will be able to supplement existing approaches, such as passenger surveys, by providing continuous footfall figures across the network. 

Cyber security game changer 

The digitalisation of the railway is a game changer for cyber security. If the industry is to achieve the potentially huge business benefits that could arise from greater interconnectivity between systems, then we need to change the way we think about security, and in particular we must address the soon-to-be-flawed assumption that railway ICT is protected because it runs on closed networks. 

New systems will need to be secure by design, the workforce will need to receive training in security best practice, and as an industry we will need robust procedures in place to ensure that when – not if – problems arise the network can continue to run safely and with a minimum of disruption. 

A key component of the successful adoption of the new systems will lie in the adaptation of business processes to the new way of working, and particularly supporting the exchange and integration of data owned by the various stakeholders across the industry. Fortunately, there is already some great work going on in this area in projects such as the RSSB’s T1010 project on enabling the use of cross-industry remote condition monitoring. Business processes will also need to evolve to accommodate rail’s role in the wider, multimodal transport system, where the challenge will be to adapt technologies such as electronic ticketing that have been shown to work well in closed systems, such as those run by Transport for London, to the national system. 

The scale of the changes that digital technologies will bring to the industry mean that this is an exciting time to be working in rail, and the topics discussed here are just the tip of the iceberg. As a final thought, it’s important to remember that if digitalisation is going to succeed, then we need to make sure that the change is led by the passengers and staff, and while the possibilities are almost endless, we need to focus on meeting the needs of the industry, and not the needs of the technology.


[email protected]


James Palma   01/02/2017 at 20:46

Very interesting and much needed. There appears to be a general consensus that old imformation for assets which are now well over 150 years old is readily available. As a reseacher in transport related infrastructure academically and proffessionally i can testify that with decades of budget cuts and a lack of understanding as to retention of archive material and documenting changes to assets, that there is a hidden cost in unravelling exisitng data. This hidden cost caused by misunderstanding of asset ownership, rights and responsibilities for decades and lack of requirements for projects, contractors and business units to clearly document what was done how when and why and to what. This leads to lack of understanding and additional costs to unravel the tangled web. Often too late to make amends. A good way to understand what is involved is a recent article in the instituition of civil engineers journal of asset management Dember issue which discusses the complexity of infrastructure ownership with regard to glagow subway. It is entitled a conceptual framework for land use and metro infrastructure and demonstrates some of the complexities at hand.

Cyberh   02/02/2017 at 15:57

Very informative approach when tackling Big Data security. When considering to implement a big data strategy, HPCC Systems is an open source data-intensive supercomputing platform that can help companies derive actionable insights from their data. While security is definitely a major concern for anyone using, manipulating and transporting big data, the HPCC Systems platform has added a wide array of extensive, configurable and customizable security features, the choice of which depends on the risk tolerance of the HPCC Systems data stakeholders. An administrator can quickly and easily configure simple user authentication via LDAP or HTPASSWD. Additionally, our LDAP security manager can be configured in minutes, and it implements a very robust set of features where you can guard access to the many HPCC Systems features, as well as protecting access to the data and services. Finally, we expose a robust pluggable security manager framework, where software developers can implement their own custom security managers tailored to their unique requirements. Learn more at

Add your comment

rail technology magazine tv

more videos >

latest rail news

View all News

rail industry focus

Versatile coating system enhances Indestructible Paint rail industry role

12/08/2020Versatile coating system enhances Indestructible Paint rail industry role

A highly versatile and robust epoxy coating s... more >
Network Rail partners with Cycling UK for new initiative

03/08/2020Network Rail partners with Cycling UK for new initiative

Network Rail and Cycling UK have launched a p... more >

editor's comment

23/01/2018Out with the old...

Despite a few disappointing policy announcements, especially for the electrification aficionados amongst us, 2017 was, like Darren Caplan writes on page 20, a year generally marked by positive news for the rail industry. We polished off the iconic Ordsall Chord (p32), hit some solid milestones on Thameslink (p40), progressed on ambitious rolling stock orders (p16), and finally started moving forward on HS2 (p14) ‒ paving the way for a New Year with brand-new infrastructrure to... read more >

last word

Encouraging youngsters to be safe on the railway

Encouraging youngsters to be safe on the railway

This summer, Arriva Group's CrossCountry and the Scout Association joined to launch a new partnership to promote rail safety among young people. Chris Leech MBE, business community manager at the TOC, gives RTM an update on the innovative scheme. Recognising that young people are more likely to take a risk trespassing on railway tracks, C... more > more last word articles >

'the sleepers' daily blog

On the right track, Sulzer is awarded RISAS accreditation for Nottingham Service Centre

29/06/2020On the right track, Sulzer is awarded RISAS accreditation for Nottingham Service Centre

Following an independent audit, Sulzer’s Nottingham Service Centre has been accepted as part of the rail industry supplier approval scheme (RISAS). The accreditation reinforces the high-quality standards that are maintained by Sulzer’s network of independent repair facilities across the UK and further afield in its global network. ... more >
read more blog posts from 'the sleeper' >


Andrew Haines, CE of Network Rail, tells BBC News his organisation could issue future rail franchises

24/06/2019Andrew Haines, CE of Network Rail, tells BBC News his organisation could issue future rail franchises

Andrew Haines, the Chief Executive of Network Rail, has told the Today programme on Radio 4's BBC’s flagship news programme that he wo... more >
Advancing the rail industry with management degree apprenticeships

08/05/2019Advancing the rail industry with management degree apprenticeships

In answering the pressing questions of how current and future generations of managers can provide solutions to high-profile infrastructure projec... more >
Women in rail - is the industry on the right track?

12/03/2019Women in rail - is the industry on the right track?

RTM sits down with Samantha Smith, sole female member of the TransPennine Route Upgrade Alliance Leadership Team, to find out more about encourag... more >
TfN Strategic Transport Plan: not just for transport's sake

22/01/2019TfN Strategic Transport Plan: not just for transport's sake

Peter Molyneux, Transport for the North’s (TfN’s) strategic roads director, has been leading on the development of the seven economic... more >