Scarce resources, an ever-growing population, natural disasters and many other factors mean we need to be smart about the way we manage the environments we live in.

CIO Australia spoke to several organisations in Australia and abroad to get insights into how they are tapping into advanced analytic tools, data modelling, the Internet of Things and machine-to-machine communications to help build smarter, data-driven cities.

Creating smart transport systems

Port Botany's rail freight operation was experiencing capacity and congestion issues so the organisation considered investing in new infrastructure.

A team at National ICT Australia (NICTA) collected real-time data from trains and shipping containers and interviewed various stakeholders to build a computer model of trains coming in and out of Port Botany. They were able to use clever scheduling to show the rail line didn't need to be upgraded for another 10 or 20 years, saving hundreds of millions of dollars.

"This is an area of say 300 hectares, and it can move up to 1 million containers per year. We have a forecast of 3 million containers until 2030. So the first reaction from everybody would be 'we have to build new infrastructure'. That means $100 million just for new infrastructure because you would also have to build new railways, new terminals, etc," says Thomas Vitsounis, project leader, total port logistics at NICTA.

"We said 'let's see how we can make it 2 million with the infrastructure as it is'. So we need to change the layout, we need to change the operational rules, we need to change the flow of goods in and out, etc."

The team also changed the length of the train carriages, and looked at each interaction within the supply chain from the loading of the containers through to train schedules to better optimise the whole network.

"Looking at the whole system in something as complex as a port can yield important insights. The state-of-the-art model we developed [showing] how all the parts interact showed that the rail infrastructure does not appear to be the main bottleneck. That is very forward thinking of the port [workers] to ask us to analyse the system in this way," says Dean Economou, technology strategist at NICTA.

"The conventional techniques for analysis just don't reveal the subtlety in the interactions. Now it's actually possible to get visibility of the entire supply chain, whereas before that was quite difficult.

"You know where stuff is and you can see two stages up the supply chain if there's a delay. Because you know what's happening further up the chain, you can actually adapt your own situation."

The NICTA team is also working on optimising Canberra's bus network because there's a shortage of bus drivers and services to support growing demand, particularly on weekends.

Instead of the government throwing money at the problem, the NICTA team looked at how the transport system could be transformed to create efficiencies. Using a new model, the system showed that people could get to their destination faster without having to pay more for their ticket.

"You get a taxi to pick you up and take you to the bus hub. Then you get on a high frequency bus, go to the next bus hub and get on another taxi. You're thinking 'that's very expensive'. But it turns out that because of the savings you make by having the buses running nearly full, keeping the buses fully utilised, you actually have enough money to pay for the taxis at either end," says Economou.

"And instead of just waiting at the bus stop where you have a service say every two hours, you can just ring up a taxi with 15 minutes warning and say 'I'd like to catch the bus at this bus stop'. Yeah, you do have to change, but you don't have to wait as much."

NICTA has sent a proposal to the ACT government to trial the new system.

"It's the ability to build these detailed models with lots of understanding of the interactions and the costs; and now using more advanced mathematics and computation, you can actually run scenarios and understand them," says Economou.

NICTA is also doing machine learning and predictive analytics in addition to conventional simulation planning for a new light rail in George St, Sydney. The team looked at historical data to build a model that can predict how the traffic will behave under certain conditions, and build future action plans based on that data.

"We are looking at how to complement and extend it with machine learning [using] the counters from under the road. So there are these little things that detect cars, and those things are constantly counting ... and inferring how to adjust the green, red and amber cycles to try and get through as many cars as possible," says Economou.

"We built a model of how the Sydney CBD is working and then from that, made predictions on what would happen when George St was closed."

Preventing incidents with smart safety and emergency services

There were 1,188 road deaths let alone crashes in the year ended March 2014, according to Bureau of Infrastructure,Transport and Regional Economics.

John Wall, manager of road safety technology at Transport for NSW, reflects on an incident from a few years ago that stands out in his mind: A young women was driving in severe wet conditions, hit a wet patch on the road and crashed into a pole.

At the time, Wall asked: How could smart, connected technologies help prevent crashes like this, as well as make the job of emergency services easier?

Wall says gathering weather data, as well as information about the driver and car through an opt-in form, could be used to encourage drivers to avoid certain roads during severe wet conditions or to take public transport when they are planning their journeys.

"In the future, hopefully we will prevent those kids from getting on the road in the first place. The driver who hit this pole was a P-plater, she was in an old vehicle, and the weather was absolutely terrible. She hit a very wet patch of road in really heavy rain, and lost control," says Wall.

"If the system was able to know a little bit more about the driver, perhaps the journey systems could of taken into account all of this information on the weather plus the fact she was young, driving a car that doesn't have stability control and recommend she go by train or bus."

Connected vehicles also have a role in preventing crashes on the road. The idea is to have sensors on the wheels that detect a sharp bump in the road or a when the wheels start to get slippery so that information can be relayed to other connected vehicles behind and warn the drivers to slow down, Wall says.

"The car could even prepare itself to hit that wet patch by getting its breaking systems ready, and electronic stability control. So it's not just being able to warn the driver, it's also warning the vehicle management systems that control the breaks and the accelerator and steering -- all of those sorts of things -- that there's a potential hazard ahead."

The NSW Centre for Road Safety is trialling a Cooperative Intelligent Transport Initiative (CITI) where trucks are fitted with anti-collision devices. Information such as the truck's position and speed is sent from the truck to roadside devices. The devices then send alerts, such as warnings about potential crashes, back to truck drivers.

"The Dedicated Short Range Communication radios that we will have installed in some trucks in the Illawarra from July are doing this 10 times a second -- sending information out to other vehicles if they detect hazards that are on the road," says Wall.

"The vehicle talks to the driver only when the situation becomes more urgent or critical ... because we don't want drivers getting 10 messages a second. The system itself that sits in the vehicle takes care of that information and then decides through business rules what it should actually advise the driver on."

Transport for NSW is also working on Automatic Crash Notification technology that lets a connected vehicle automatically call emergency services and inform staff of its location once it has crashed.

"In the US, there are a number of people who have been working at George Washington University and they have taken it one step further. They have also looked at sensor systems within the vehicle to determine how many passengers are in it. It's the kind of sensors that cause a light to flash when you are not wearing your seat belt. So [emergency services] can plan for the number of causalities that may be at the crash scene," Wall says.

"I have been to a number of night time crashes ... where vehicles have gone over cliffs and those sorts of things, and it's been very difficult to find where these people are. We have heard of people who have crashed not being found until the following morning, and they maybe would have survived if we got early notification of exactly where the crash happened."

Getting to the crash scene as quick as possible can also be assisted by connected vehicles. Wall says in the near future traffic systems will be able to detect an emergency vehicle coming towards an intersection and readjust the traffic flow to give the vehicle a green light. The emergency vehicle could also talk to other vehicles nearby to alert drivers to give way.

"When the emergency vehicle arrives on scene, then we have to look at things like where's the best place to cut through parts of the car to rescue a person trapped in a vehicle.

"If they had information at their fingertips, maybe through Google Glass and augmented reality, they would be able to look at the vehicle and know where potential hazards are, where they can't cut because there's a high voltage cable that runs in a particular compartment or there's an airbag that hasn't gone off that's at risk of going off," Wall says.

The NICTA team did an analysis on how fast they could get all people residing in Sydney safely evacuated during a flood. The team looked at which Sydney regions they would evacuate first, which modes of transport should be used first and where.

The analysis found that the current algorithms would not get 40 per cent of the population out in time, whereas the new model showed it can get everyone out safely. "And that it is still possible to do so even of the population increases by 40 per cent," says Economou.

Next page: Building smart utilities

Building smart utilities

Smart grids play a key part in running a smarter, efficient city. The Smart Grid, Smart City project trial, led by the Australian government and Ausgrid, recently finished, with the cost/benefit analysis currently being reviewed. Paul Budde, executive director of Smart Grid Australia, says 30 to 40 per cent of energy savings can be made across Australia using smart grid technology.

Budde says the data generated through smart grids can be combined with data coming from the weather forecast, TV networks, city cultural event planners and so on as they all affect the electricity networks.

"If you combine all of that data and analyse it then you know if there is a big event and everybody is going to be watching the television tonight, so you know what to do with your network. If you know there's a storm coming, you know that has an effect on use of the electricity, as people stay home and use heaters to stay warm," he says.

"Also, the mobile companies can say to the electricity companies 'hey, there's a big event in town so expect 100,000 people using their mobile phone at once and make sure you network in the city is able to handle that capacity."

Smart appliances will be able to connect with the smart grid to schedule power-consuming tasks during off peak periods of the day and save on energy. The customer would instruct the energy retailer when to automatically turn on their washing machine, for example.

The NICTA team is using machine learning and predictive analytics to anticipate when a water pipe in Sydney is about to break. Water utility companies can only afford to inspect about 1 per cent of their pipes, and there is 20,000 kilometres of underground pipes in Sydney alone. Busted water pipes overall cost the Australian economy more than $1 billion a year.

"If they break, the people who own the pipes and run them are responsible for the damage. So if you flood a basement of a building and block a road and that stops some deliveries happening, the companies affected will come and get you and will want compensation. If a really big one in the middle of a city breaks, it could cost $5 million in compensation in economic damage," says Economou.

Some 326,571 pipes in Sydney were analysed, with 75,000 failure records. The team looked at the correlations between the age of the pipe, location, material and water pressure. In Wollongong, for example, location/geology and material had a strong correlation, showing that older pipes don't necessarily break earlier.

The team analysed the first nine years of 10 years worth of historical data to predict what would happen to the pipes in the 10th year. They then looked at the data from the 10th year to see if it matched their predicted result and it almost did completely.

Carly Perry, business manager in infrastructure, transport and logistics at NICTA, says the organisation compared its analysis with that of water utility companies and found it could predict twice the amount of pipe breakages using the same maintenance budget.

Monitoring Rio de Janeiro

Rio de Janeiro in Brazil has been developing a smarter city since 2010 when it opened an around-the-clock operation centre after experiencing damaging floods. The centre was first developed to do more sophisticated data analysis to help emergency services better prepare and respond to emergencies before being used for additional purposes.

The centre brings together 30 agencies into one central command centre where data from sensors, video feeds and social media is collected and analysed. Video data streaming is done in real time from 570 cameras from utility providers and the Secretariat of Public Security.

It includes an 80 square metre video wall -- the biggest in Latin America -- made up of 80, 46-inch screens. A smart map of the city has more than 120 layers of data. Around 30,000 metres of fibre optic cable connects the infrastructure in the centre.

"It's not unusual for Rio to have extreme weather; it's famous for its sharp and steep hills, mudslides and related heavy rains. Rio "narrowcasts" weather in its operations centre.

"We've used our weather modelling systems and weather forecasting to 'narrowcast' down to very small areas to predict how much rain will appear, when and under what circumstances," says Michael Dixon, general manager of smarter cities at IBM.

Using this narrowcasting, emergency response times have improved by 30 per cent, and it can predict 48 hours in advance where it is going to rain and how much rainfall will appear.

The city has also been working with the FIFA World Cup planning team on logistics to help prepare for the big event and ensure the city's infrastructure can support the surge in population.

"The Mayor of Rio, Eduardo Paes, had a vision some years ago when he thought that there was potential for Rio to host either the World Cup and/or the Olympics.

"He started to realise a higher level of management of the city would be essential to both convince authorities that Rio was the right city to host these events. They [Rio] started off with a very clear view that they wanted to have an understanding of what was happening in the city at any point in time," Dixon says.

The city is monitoring Twitter feeds during events to get a pulse of what's happening in the city and respond to people's concerns quickly. For example, there might be a huge amount of waste in a particular area that many people are tweeting about.

The team in the operations centre can identify common words in those tweets, the amount of tweets and locate where those tweets are coming from. Surveillance cameras can be used to confirm if there is a major problem, and then a waste disposal facility could act on it right away.

Next page: Improving New York City's transport system

Improving New York City's transport system

NYC is also a leading smarter city, and is using data to better assess how it should introduce new infrastructure in its transport system.

"We have something called the Traffic Information System, managed by the Department of Transportation. What they'll do is request traffic studies and gather all the data available about an intersection [to assess] whether they will develop an intervention for it, whether they want to put in new markings, or to remove or add a lane, add a bike lane, or add a pedestrian marking - something like that," says Nicholas O'Brien, acting chief analytics officer for the City of New York.

"So they will put additional traffic count monitors there, and take data from the taxi cabs that are sending out GPS signals to determine how quickly they are moving through there and where the potential problems are."

The city is also doing a proof of concept to apply machine learning to its traffic cameras for car and pedestrian counts to better plan interventions on its streets.

The city is also using microtargeting in its 'Notify NYC' system for emergency notification. For example, it is able to geolocate specific alerts to mobile phones to notify people in the relevant area that a child has gone missing, or there's a major weather event coming.

"It's still early stages in determining what we can do with the information coming from phones. And right now we have a lot of privacy concerns around it," says O'Brien.

IoT, M2M challenges

Standardisation and interoperability are still challenges many infrastructure companies in Australia and around the world are trying to wade through when it comes to building a smart city.

Smart Grid Australia's Budde says the smart meter rollout in Victoria is an example of this. "What you now see in Victoria with the smart meter rollout is the various energy operators are using different smart meters that are not interoperable," he says.

"If you are a retailer across several of those organisations, you cannot find what is the best solution for one of your customers because you have to go through three different systems. So how smart is that? I think interoperability is one of the key issues in machine-to-machine and Internet of Things."

NICTA's Vitsounis agrees that disparate systems make data sharing a challenge, using the example of Port Botany and how trust can be an issue when it comes to the various people sharing data.

"You always have various interactions, various players that have to coordinate. But at the same time they are also competitors and each one has to define its own competitive advantage. It takes time, you need to build the relationships to be a trustworthy party and get this data."

NICTA's future logistics living lab leader, Neil Temperley, adds that there's a human aspect to building a connected and integrated supply chain around dealing with concerns about confidentiality and commercial information.

"They all need to be educated and have a vested interest in sharing information with their partners, their team -- they are a team, but they may not realise it -- to work together to keep things flowing as efficiently as possible. You can't just take your own information and play in your own space."

Access to open data is another challenge. Budde says there is still reluctance by the Australian government to provide free access to data. This could be due to issues around data quality, for example, or it could be that there is a closed culture among some agencies.

The website has collected 3,514 data sets since it first launched in 2011. In comparison, the US website holds 105,257 data sets since its 2009 launch. Communications Minister Malcolm Turnbull has also pointed out that the government is playing catch up when it comes to making its data publicly available.

"We need a change of mind in government. You see already with the real-time transport applications that the benefits are enormous if you open up the data that governments have," Budde said.

Ensuring there will be strong broadband and wireless connectivity in future is also key to building a smart city. Budde says we have wireless connectivity "pretty well covered" in Australia, but "always more can be done".

"When you talk about the Internet of Things, you really talk about billions of sensors that suddenly have to be linked to the network and that then requires a whole new level of networking - street lamps, parking meters, and so on all need to be linked to the wireless network," he says. "Going forward, more and more investments are needed."

O'Brien says it is a challenge to support billions of sensors in a city, all feeding off the wireless Internet network. "We are living in the future but not that far in the future where it's no real cost to handle 20 billion data points a week, which is what our taxi cabs are sending out."

Follow Rebecca Merrett on Twitter: @Rebecca_Merrett

Follow CIO Australia on Twitter and Like us on Facebook... Twitter: @CIO_Australia, Facebook: CIO Australia, or take part in the CIO conversation on LinkedIn: CIO Australia