How intelligent data platforms are powering smart cities

[A version of this post appears on the O’Reilly Radar.]

Smart cities and smart nations run on data.

According to a 2014 U.N. report, 54% of the world’s population resides in urban areas, with further urbanization projected to push that share up to 66% by the year 2050. This projected surge in population has encouraged local and national leaders throughout the world to rally around “smart cities” — a collection of digital and information technology initiatives designed to make urban areas more livable, agile, and sustainable.

Smart cities depend on a collection of enabling technologies that we’ve been highlighting at Strata + Hadoop World and in our publications: sensors, mobile computing, social media, high-speed communication networks, and intelligent data platforms. Early applications of smart city technologies are seen in transportation and logistics, local government services, utilities, health care, and education. Previous Strata + Hadoop World sessions have outlined the use of machine learning and big data technologies to understand and predict vehicular traffic and congestionpatterns, as well the use of wearables in large-scale health care data platforms.

As we put together the program for the upcoming Strata + Hadoop World in Singapore, we have been cognizant of the growing interest in our host country’s Smart Nation program. And more generally, we are mindful that large infrastructure investments throughout the Asia-Pacific region have engaged local leaders in smart city initiatives. For readers comfortable with large-scale streaming platforms, many of the key technologies for enabling smart cities will already be familiar:

Data collection and transport

In smart cities, Internet of Things and industrial Internet applications, proper instrumentation, and data collection depend on sensors, mobile devices, and high-speed communication networks. Much of the private infrastructure belongs to and is operated by large telecommunication companies, and many of the interesting early applications and platforms are originating from telcos and network equipment providers.

Data processing, storage, and real-time reports

As I noted in an earlier article, recent advances in distributed computing and hardware have produced high-throughput engines capable of handlingbounded and unbounded data processing workloads. Examples of this include cloud computing platforms (e.g., AWS, Google, Microsoft) andhomegrown data platforms comprised of popular open source components. At the most basic level, these data platforms provide near real-time reports (business intelligence) on massive data streams:

Real-time Data Fusion.

Intelligent data applications

Beyond simple counts and anomaly detection, the use of advanced techniques in machine learning and statistics opens up novel real-time applications (machine-to-machine) with no humans in the loop. Popular examples of such applications include systems that power environments like data centers, buildings and public spaces, and manufacturing (industrial Internet). Recognizing that future smart city applications will rely on disparate data — including event data (metrics from logs and time-series), unstructured data (images, audio, text), and geospatial data sources — we have planned sessions at Strata + Hadoop World Singapore that will cover advanced analytic techniques targeting these data types.

Smart city platforms represent some of the more exciting and impactful applications of real-time, intelligent big data systems. These platforms will confront many of the same challenges faced by applications in the commercial sector, including security, ethics, and governance. At Strata + Hadoop World Singapore, we’re looking forward to highlighting the intersection of communities and technologies that power our future cities.

Related content

Leave a Reply

%d bloggers like this: