Semi-regular field notes from the world of data:
Apache Spark development community: Josh Rosen of Databricks recently built a tool for browsing pull requests. I like that it lets you scan each of the major components (Spark SQL, Streaming, MLlib, etc.). Now that Spark has become one of the most active open source projects in big data, tools like this make it easier for outsiders to follow what Spark developers are up to. [Full disclosure: I’m an advisor to Databricks, a startup commercializing Apache Spark.]
Treato: As many of you know, I’m a fan of domain specific big data platforms. During a trip to Israel last May, I met with the CEO of Treato, an interesting platform focused on health care. By analyzing unstructured text from big (social) sites and small patient support groups, the company hopes to understand patients concerns and problems (the company aspires to be the “voice of patients”). This requires integrating multiple data sources and health databases, and NLP tools tuned for extracting health experiences on the web. 70% of Treato’s millions of users come from North America.