General-purpose platforms can come across as hammers in search of nails [A version of this post appears on the O’Reilly Data blog and Forbes.] As much as I love talking about general-purpose big data platforms and data science frameworks, I’m the first to admit that many of the interesting startups I talk to are focusedContinue reading “Verticalized Big Data solutions”
Author Archives: Ben Lorica
Advanced Analytics on Relational Data with Spark SQL
I’ll be hosting a webcast on Spark SQL featuring Michael Armbrust of Databricks: In this webcast, we’ll examine Spark SQL, a new Alpha component that is part of the Apache Spark 1.0 release. Spark SQL lets developers natively query data stored in both existing RDDs and external sources such as Apache Hive. A key featureContinue reading “Advanced Analytics on Relational Data with Spark SQL”
5 Fun Facts about HBase that you didn’t know
HBase has made inroads in companies across many industries and countries [A version of this post appears on the O’Reilly Data blog.] With HBaseCon right around the corner, I wanted to take stock of one of the more popular1 components in the Hadoop ecosystem. Over the last few years, many more companies have come toContinue reading “5 Fun Facts about HBase that you didn’t know”
Crowdsourcing Feature discovery
More than algorithms, companies gain access to models that incorporate ideas generated by teams of data scientists [A version of this post appears on the O’Reilly Data blog and Forbes.] Data scientists were among the earliest and most enthusiastic users of crowdsourcing services. Lukas Biewald noted in a recent talk that one of the reasonsContinue reading “Crowdsourcing Feature discovery”
Instrumenting collaboration tools used in data projects
Built-in audit trails can be useful for reproducing and debugging complex data analysis projects [A version of this post appears on the O’Reilly Data blog.] As I noted in a previous post, model building is just one component of the analytic lifecycle. Many analytic projects result in models that get deployed in production environments. Moreover,Continue reading “Instrumenting collaboration tools used in data projects”
2013 Revenue of some startup companies
The chart below is from Wikibon’s estimates1 of the 2013 revenue2 of some Big Data companies. Using d3 I drew a chart that shows 2013 revenue (in millions) from Big Data products and services, as well as the share of revenue derived from services, for a few select/startup companies: (Click HERE to enlarge) The BigContinue reading “2013 Revenue of some startup companies”
Interface Languages and Feature Discovery
It’s easier to “discover” features with tools that have broad coverage of the data science workflow [A version of this post appears on the O’Reilly Data blog and Forbes.] Here are a few more observations based on conversations I had during the just concluded Strata Santa Clara conference. Interface languages: Python, R, SQL (and Scala)Continue reading “Interface Languages and Feature Discovery”
Extending GraphLab to tables
The popular graph analytics framework extends its coverage of the data science workflow [A version of this post appears on the O’Reilly Data blog and Forbes.] GraphLab’s SFrame, an interesting and somewhat under-the-radar tool was unveiled1 at Strata Santa Clara. It is a disk-based, flat table representation that extends GraphLab to tabular data. With theContinue reading “Extending GraphLab to tables”
Bridging the gap between research and implementation
[A version of this post appears on the O’Reilly Data blog.] One of the most popular offerings at Strata Santa Clara was Hardcore Data Science day. Over the next few weeks we hope to profile some of the speakers who presented, and make the video of the talks available as a bundle. In the meantimeContinue reading “Bridging the gap between research and implementation”
Time-turner: Strata Santa Clara 2014, day 2
There are so many good talks happening at the same time that it’s impossible to not miss out on good sessions. But imagine I had a time-turner necklace and could actually “attend” 2 (maybe 3) sessions happening at the same time. Taking into account my current personal interests and tastes, here’s how my day wouldContinue reading “Time-turner: Strata Santa Clara 2014, day 2”