Challenges and opportunities for big data in the Digital Railway

Source: RTM Dec/Jan 17

Dr John Easton and Dr Shruti Kohli from the University of Birmingham write for RTM about the role big data can play in transforming the country’s railways.

Over the last few years there has been a great deal of discussion within the industry around the shape the railways will take in the decades to come. Much of the debate has focused on the arrival of new, digital technologies that will replace the systems the industry has relied on until now, and what the impact of those changes might be in terms of passenger experience, operation of the network, safety and security, and business processes. 

At first glance, it is easy to assume that the digitalisation of the railway is all about train control. However, the challenge that new digitally-enabled infrastructure and systems present to the railway industry is about far more than in-cab signals. It’s about creating a virtually-connected ‘system of railway systems’, where the timely provision and analysis of data on the state of the network enables smoother running, increased flexibility, and, critically, greater use of the physical capacity available. 

Nowhere is the early application of these principles more obvious than in the research arena, where the EU-funded Shift2Rail lighthouse projects In2Rail and IT2Rail are blazing new trails in the development of next generation Traffic Management Systems and Passenger Information Systems respectively. These new systems share a common theme, drawing together information from around the industry, and delivering it to the right people, in the right place, at the right time to support effective decision-making. Work of this calibre does not take place in isolation, and both projects are building on a huge body of previous work from within the industry and in the research sector, including projects such as AUTOMAIN, ON-TIME, and Capacity4Rail. 

Big data analytics and cyber security 

Developing a deeper understanding of the asset base is obviously a key win for digital technologies, and while condition monitoring may grab the lion’s share of the headlines, the potential for improvements in defect reporting and sharing of best maintenance practice are both easy wins in this area. By combining large volumes of data from similar fleets across the network, for example, big data analytics approaches will enable the industry to identify patterns that represent best practice in operation and maintenance for those vehicles and, perhaps more importantly, show how day-to-day activities can be improved on poorly-performing routes. 

Data collection is also evolving, with IoT technologies offering exciting new possibilities for very low-cost and short-term sensor installations. The potential application of big data techniques within the industry will not be limited to the physical assets it owns and operates. Understanding the movements of passengers around station buildings, for instance, will become increasingly important as headways decrease and station dwell times become more critical. The automatic analysis of footage from station CCTV systems will be an important additional tool in this area, and will be able to supplement existing approaches, such as passenger surveys, by providing continuous footfall figures across the network. 

Cyber security game changer 

The digitalisation of the railway is a game changer for cyber security. If the industry is to achieve the potentially huge business benefits that could arise from greater interconnectivity between systems, then we need to change the way we think about security, and in particular we must address the soon-to-be-flawed assumption that railway ICT is protected because it runs on closed networks. 

New systems will need to be secure by design, the workforce will need to receive training in security best practice, and as an industry we will need robust procedures in place to ensure that when – not if – problems arise the network can continue to run safely and with a minimum of disruption. 

A key component of the successful adoption of the new systems will lie in the adaptation of business processes to the new way of working, and particularly supporting the exchange and integration of data owned by the various stakeholders across the industry. Fortunately, there is already some great work going on in this area in projects such as the RSSB’s T1010 project on enabling the use of cross-industry remote condition monitoring. Business processes will also need to evolve to accommodate rail’s role in the wider, multimodal transport system, where the challenge will be to adapt technologies such as electronic ticketing that have been shown to work well in closed systems, such as those run by Transport for London, to the national system. 

The scale of the changes that digital technologies will bring to the industry mean that this is an exciting time to be working in rail, and the topics discussed here are just the tip of the iceberg. As a final thought, it’s important to remember that if digitalisation is going to succeed, then we need to make sure that the change is led by the passengers and staff, and while the possibilities are almost endless, we need to focus on meeting the needs of the industry, and not the needs of the technology.



James Palma   01/02/2017 at 20:46

Very interesting and much needed. There appears to be a general consensus that old imformation for assets which are now well over 150 years old is readily available. As a reseacher in transport related infrastructure academically and proffessionally i can testify that with decades of budget cuts and a lack of understanding as to retention of archive material and documenting changes to assets, that there is a hidden cost in unravelling exisitng data. This hidden cost caused by misunderstanding of asset ownership, rights and responsibilities for decades and lack of requirements for projects, contractors and business units to clearly document what was done how when and why and to what. This leads to lack of understanding and additional costs to unravel the tangled web. Often too late to make amends. A good way to understand what is involved is a recent article in the instituition of civil engineers journal of asset management Dember issue which discusses the complexity of infrastructure ownership with regard to glagow subway. It is entitled a conceptual framework for land use and metro infrastructure and demonstrates some of the complexities at hand.

Cyberh   02/02/2017 at 15:57

Very informative approach when tackling Big Data security. When considering to implement a big data strategy, HPCC Systems is an open source data-intensive supercomputing platform that can help companies derive actionable insights from their data. While security is definitely a major concern for anyone using, manipulating and transporting big data, the HPCC Systems platform has added a wide array of extensive, configurable and customizable security features, the choice of which depends on the risk tolerance of the HPCC Systems data stakeholders. An administrator can quickly and easily configure simple user authentication via LDAP or HTPASSWD. Additionally, our LDAP security manager can be configured in minutes, and it implements a very robust set of features where you can guard access to the many HPCC Systems features, as well as protecting access to the data and services. Finally, we expose a robust pluggable security manager framework, where software developers can implement their own custom security managers tailored to their unique requirements. Learn more at

Add your comment


rail technology magazine tv

more videos >

latest rail news

View all News

rail industry focus

Flexible skills for a changing industry

18/09/2017Flexible skills for a changing industry

Beth Curtis, head of partnerships and communi... more >
Keeping an eye on cable theft

14/09/2017Keeping an eye on cable theft

RTM’s David Stevenson reflects on recen... more >

editor's comment

04/09/2017A series of incredible events

There are a couple of announcements I’d like to make in this editor’s comment, which I believe could be very beneficial to many of our readers.  Firstly, the deadline is fast approaching to enter the UK Rail Industry Awards (UKRIA), which will take place on 22 March 2018 at Battersea Evolution in London.  Now in its fifth year, UKRIA continues to bring together the leaders and decision-makers running the rail industry for the sector’s biggest and most... read more >

last word

How do tram and rail passengers compare?

How do tram and rail passengers compare?

David Sidebottom, director at Transport Focus, analyses the drivers in performance of passenger satisfaction in tram compared to rail. Results published in our recent Tram Passenger Survey (TPS) have seen remarkable levels of satisfaction by the six operators we covered in Blackpool, Birmingham, Edinburgh, Manchester, Nottingham and Sheff... more > more last word articles >

'the sleepers' daily blog

UKRIA 2018: Meet the judging panel

19/09/2017UKRIA 2018: Meet the judging panel

With just under a week left to get your applications in for the rail industry’s biggest awards event, UKRIA, it seems like the perfect time to introduce the huge names that are making up our judging panel this year. The most exciting addition to the senior pan-industry panel this year is one of the most recognisable names in the ind... more >
read more blog posts from 'the sleeper' >


A game changer for Wales and Borders

17/07/2017A game changer for Wales and Borders

Andy Thomas, managing director for Network Rail’s Wales route, describes how the infrastructure owner will work more collaboratively than e... more >
‘In the rail business, we’re leading the way’

13/07/2017‘In the rail business, we’re leading the way’

Andy Milner, the chief executive of Amey, tells RTM about the transformation programme that has been taking place in the company since his takeov... more >
Expanding an unprecedented level of style and comfort

13/07/2017Expanding an unprecedented level of style and comfort

At the end of May, Eurostar introduced its new e320 train on the London-Brussels route. Rob Haycocks talks to RTM about what passengers can expec... more >
A development opportunity across the national network

10/07/2017A development opportunity across the national network

Simone Bailey, asset management director at Abellio Greater Anglia, talks to RTM about overcoming the different challenges facing stations of var... more >