Challenges and opportunities for big data in the Digital Railway

Source: RTM Dec/Jan 17

Dr John Easton and Dr Shruti Kohli from the University of Birmingham write for RTM about the role big data can play in transforming the country’s railways.

Over the last few years there has been a great deal of discussion within the industry around the shape the railways will take in the decades to come. Much of the debate has focused on the arrival of new, digital technologies that will replace the systems the industry has relied on until now, and what the impact of those changes might be in terms of passenger experience, operation of the network, safety and security, and business processes. 

At first glance, it is easy to assume that the digitalisation of the railway is all about train control. However, the challenge that new digitally-enabled infrastructure and systems present to the railway industry is about far more than in-cab signals. It’s about creating a virtually-connected ‘system of railway systems’, where the timely provision and analysis of data on the state of the network enables smoother running, increased flexibility, and, critically, greater use of the physical capacity available. 

Nowhere is the early application of these principles more obvious than in the research arena, where the EU-funded Shift2Rail lighthouse projects In2Rail and IT2Rail are blazing new trails in the development of next generation Traffic Management Systems and Passenger Information Systems respectively. These new systems share a common theme, drawing together information from around the industry, and delivering it to the right people, in the right place, at the right time to support effective decision-making. Work of this calibre does not take place in isolation, and both projects are building on a huge body of previous work from within the industry and in the research sector, including projects such as AUTOMAIN, ON-TIME, and Capacity4Rail. 

Big data analytics and cyber security 

Developing a deeper understanding of the asset base is obviously a key win for digital technologies, and while condition monitoring may grab the lion’s share of the headlines, the potential for improvements in defect reporting and sharing of best maintenance practice are both easy wins in this area. By combining large volumes of data from similar fleets across the network, for example, big data analytics approaches will enable the industry to identify patterns that represent best practice in operation and maintenance for those vehicles and, perhaps more importantly, show how day-to-day activities can be improved on poorly-performing routes. 

Data collection is also evolving, with IoT technologies offering exciting new possibilities for very low-cost and short-term sensor installations. The potential application of big data techniques within the industry will not be limited to the physical assets it owns and operates. Understanding the movements of passengers around station buildings, for instance, will become increasingly important as headways decrease and station dwell times become more critical. The automatic analysis of footage from station CCTV systems will be an important additional tool in this area, and will be able to supplement existing approaches, such as passenger surveys, by providing continuous footfall figures across the network. 

Cyber security game changer 

The digitalisation of the railway is a game changer for cyber security. If the industry is to achieve the potentially huge business benefits that could arise from greater interconnectivity between systems, then we need to change the way we think about security, and in particular we must address the soon-to-be-flawed assumption that railway ICT is protected because it runs on closed networks. 

New systems will need to be secure by design, the workforce will need to receive training in security best practice, and as an industry we will need robust procedures in place to ensure that when – not if – problems arise the network can continue to run safely and with a minimum of disruption. 

A key component of the successful adoption of the new systems will lie in the adaptation of business processes to the new way of working, and particularly supporting the exchange and integration of data owned by the various stakeholders across the industry. Fortunately, there is already some great work going on in this area in projects such as the RSSB’s T1010 project on enabling the use of cross-industry remote condition monitoring. Business processes will also need to evolve to accommodate rail’s role in the wider, multimodal transport system, where the challenge will be to adapt technologies such as electronic ticketing that have been shown to work well in closed systems, such as those run by Transport for London, to the national system. 

The scale of the changes that digital technologies will bring to the industry mean that this is an exciting time to be working in rail, and the topics discussed here are just the tip of the iceberg. As a final thought, it’s important to remember that if digitalisation is going to succeed, then we need to make sure that the change is led by the passengers and staff, and while the possibilities are almost endless, we need to focus on meeting the needs of the industry, and not the needs of the technology.



James Palma   01/02/2017 at 20:46

Very interesting and much needed. There appears to be a general consensus that old imformation for assets which are now well over 150 years old is readily available. As a reseacher in transport related infrastructure academically and proffessionally i can testify that with decades of budget cuts and a lack of understanding as to retention of archive material and documenting changes to assets, that there is a hidden cost in unravelling exisitng data. This hidden cost caused by misunderstanding of asset ownership, rights and responsibilities for decades and lack of requirements for projects, contractors and business units to clearly document what was done how when and why and to what. This leads to lack of understanding and additional costs to unravel the tangled web. Often too late to make amends. A good way to understand what is involved is a recent article in the instituition of civil engineers journal of asset management Dember issue which discusses the complexity of infrastructure ownership with regard to glagow subway. It is entitled a conceptual framework for land use and metro infrastructure and demonstrates some of the complexities at hand.

Cyberh   02/02/2017 at 15:57

Very informative approach when tackling Big Data security. When considering to implement a big data strategy, HPCC Systems is an open source data-intensive supercomputing platform that can help companies derive actionable insights from their data. While security is definitely a major concern for anyone using, manipulating and transporting big data, the HPCC Systems platform has added a wide array of extensive, configurable and customizable security features, the choice of which depends on the risk tolerance of the HPCC Systems data stakeholders. An administrator can quickly and easily configure simple user authentication via LDAP or HTPASSWD. Additionally, our LDAP security manager can be configured in minutes, and it implements a very robust set of features where you can guard access to the many HPCC Systems features, as well as protecting access to the data and services. Finally, we expose a robust pluggable security manager framework, where software developers can implement their own custom security managers tailored to their unique requirements. Learn more at

Add your comment


rail technology magazine tv

more videos >

latest rail news

View all News

rail industry focus

Keep cool and carry on

20/07/2017Keep cool and carry on

RTM’s Luana Salles looks at the reasons... more >
HS2: the catalyst for diversity

17/07/2017HS2: the catalyst for diversity

RTM’s Josh Mines reports on how HS2 Ltd... more >

editor's comment

03/07/2017Rapid progress needed

As RTM went to press, the National Infrastructure Commission outlined a list of the ‘top 12’ immediate priorities on which ministers must make rapid progress in the next year. Unsurprisingly, major rail schemes, including HS2, Crossrail 2 and HS3, featured highly in the projects that needed speedy development.  Lord Adonis stated that all of these have been agreed in principle, “but require decisive action to get them moving in the new Parliament. They ought to... read more >

last word

How do tram and rail passengers compare?

How do tram and rail passengers compare?

David Sidebottom, director at Transport Focus, analyses the drivers in performance of passenger satisfaction in tram compared to rail. Results published in our recent Tram Passenger Survey (TPS) have seen remarkable levels of satisfaction by the six operators we covered in Blackpool, Birmingham, Edinburgh, Manchester, Nottingham and Sheff... more > more last word articles >

'the sleepers' daily blog

Londoners given peek at abandoned Mail Rail Underground line

28/07/2017Londoners given peek at abandoned Mail Rail Underground line

Londoners will soon be able to enjoy a unique rail experience – by riding a train going along the disused Mail Rail line. The 6.5-mile track runs deep under the capital, criss-crossing Tube lines and formerly linking six sorting offices with mainline stations at Liverpool Street and Paddington. At most, the small, electric and drive... more >
read more blog posts from 'the sleeper' >


A game changer for Wales and Borders

17/07/2017A game changer for Wales and Borders

Andy Thomas, managing director for Network Rail’s Wales route, describes how the infrastructure owner will work more collaboratively than e... more >
‘In the rail business, we’re leading the way’

13/07/2017‘In the rail business, we’re leading the way’

Andy Milner, the chief executive of Amey, tells RTM about the transformation programme that has been taking place in the company since his takeov... more >
Expanding an unprecedented level of style and comfort

13/07/2017Expanding an unprecedented level of style and comfort

At the end of May, Eurostar introduced its new e320 train on the London-Brussels route. Rob Haycocks talks to RTM about what passengers can expec... more >
A development opportunity across the national network

10/07/2017A development opportunity across the national network

Simone Bailey, asset management director at Abellio Greater Anglia, talks to RTM about overcoming the different challenges facing stations of var... more >