Machine intelligence for content distribution, logistics, smarter cities, and more

[A version of this post appears on the O’Reilly Radar.]

The O’Reilly Data Show Podcast: Rhea Liu on technology trends in China.

In this episode of the Data Show, I spoke with Rhea Liu, analyst at China Tech Insights, a new research firm that is part of Tencent’s Online Media Group. If there’s one place where AI and machine learning are discussed even more than the San Francisco Bay Area, that would be China. Each time I go to China, there are new applications that weren’t widely available just the year before. This year, it was impossible to miss bike sharing, mobile payments seemed to be accepted everywhere, and people kept pointing out nascent applications of computer vision (facial recognition) to identity management and retail (unmanned stores).

I wanted to consult local market researchers to help make sense of some of the things I’ve been observing from afar. Liu and her colleagues have put out a series of interesting reports highlighting some of these important trends. They also have an annual report—Trends & Predictions for China’s Tech Industry in 2018—that Liu will discuss in her keynote and talk at Strata Data Singapore in December.

Here are some highlights from our conversation:
Continue reading “Machine intelligence for content distribution, logistics, smarter cities, and more”

Vehicle-to-vehicle communication networks can help fuel smart cities

[A version of this post appears on the O’Reilly Radar.]

The O’Reilly Data Show Podcast: Bruno Fernandez-Ruiz on the importance of building the ground control center of the future.

In this episode of the Data Show, I spoke with Bruno Fernandez-Ruiz, co-founder and CTO of Nexar. We first met when he was leading Yahoo! technical teams charged with delivering a variety of large-scale, real-time data products. His new company is helping build out critical infrastructure for the emerging transportation sector.

While some question whether V2X communication is necessary to get to fully autonomous vehicles, Nexar is already paving the way by demonstrating how a vehicle-to-vehicle (V2V) communication network can be built efficiently. As Fernandez-Ruiz points out, there are many applications for such a V2V network (safety being the most obvious one). I’m particularly fascinated by what such a network, and the accompanying data, opens up for future, smarter cities. As I pointed out in a post on continuous learning, simulations are an important component of training AI applications. It seems reasonable to expect that the data sets collected by V2V networks will be useful for smart city planners of the future.

Continue reading “Vehicle-to-vehicle communication networks can help fuel smart cities”

How intelligent data platforms are powering smart cities

[A version of this post appears on the O’Reilly Radar.]

Smart cities and smart nations run on data.

According to a 2014 U.N. report, 54% of the world’s population resides in urban areas, with further urbanization projected to push that share up to 66% by the year 2050. This projected surge in population has encouraged local and national leaders throughout the world to rally around “smart cities” — a collection of digital and information technology initiatives designed to make urban areas more livable, agile, and sustainable.

Smart cities depend on a collection of enabling technologies that we’ve been highlighting at Strata + Hadoop World and in our publications: sensors, mobile computing, social media, high-speed communication networks, and intelligent data platforms. Early applications of smart city technologies are seen in transportation and logistics, local government services, utilities, health care, and education. Previous Strata + Hadoop World sessions have outlined the use of machine learning and big data technologies to understand and predict vehicular traffic and congestionpatterns, as well the use of wearables in large-scale health care data platforms.

As we put together the program for the upcoming Strata + Hadoop World in Singapore, we have been cognizant of the growing interest in our host country’s Smart Nation program. And more generally, we are mindful that large infrastructure investments throughout the Asia-Pacific region have engaged local leaders in smart city initiatives. For readers comfortable with large-scale streaming platforms, many of the key technologies for enabling smart cities will already be familiar:

Data collection and transport

In smart cities, Internet of Things and industrial Internet applications, proper instrumentation, and data collection depend on sensors, mobile devices, and high-speed communication networks. Much of the private infrastructure belongs to and is operated by large telecommunication companies, and many of the interesting early applications and platforms are originating from telcos and network equipment providers.

Data processing, storage, and real-time reports

As I noted in an earlier article, recent advances in distributed computing and hardware have produced high-throughput engines capable of handlingbounded and unbounded data processing workloads. Examples of this include cloud computing platforms (e.g., AWS, Google, Microsoft) andhomegrown data platforms comprised of popular open source components. At the most basic level, these data platforms provide near real-time reports (business intelligence) on massive data streams:

Real-time Data Fusion.

Intelligent data applications

Beyond simple counts and anomaly detection, the use of advanced techniques in machine learning and statistics opens up novel real-time applications (machine-to-machine) with no humans in the loop. Popular examples of such applications include systems that power environments like data centers, buildings and public spaces, and manufacturing (industrial Internet). Recognizing that future smart city applications will rely on disparate data — including event data (metrics from logs and time-series), unstructured data (images, audio, text), and geospatial data sources — we have planned sessions at Strata + Hadoop World Singapore that will cover advanced analytic techniques targeting these data types.

Smart city platforms represent some of the more exciting and impactful applications of real-time, intelligent big data systems. These platforms will confront many of the same challenges faced by applications in the commercial sector, including security, ethics, and governance. At Strata + Hadoop World Singapore, we’re looking forward to highlighting the intersection of communities and technologies that power our future cities.

Related content