Navigating the Future of Search
ChatGPT has revived interest in search. It revealed that a blend of artificial intelligence and a prompt-driven interface is exceptionally well-matched for search applications. As a result of ChatGPT, some well-funded search startups are emphasizing their use of large language models (LLM) and launching chatbot-like interfaces.
ChatGPT has been receiving a great deal of attention due to its ability to answer a broad range of questions fluently and in a comprehensive manner, thus outperforming previous public chatbots in usefulness. It should be noted that large language models were already used in next-generation search solutions before ChatGPT was announced. In the period prior to ChatGPT, new search engines and enterprise search providers were beginning to introduce hybrid search capabilities combining conventional keywords, text retrieval techniques, and “vector” or “neural” search techniques. Furthermore, some startups have been experimenting with search tools boosted by chatbots for several months.
A New Frontier in Enterprise Search
I anticipate that the upcoming generation of consumer search engines, including those developed by Google, Bing, and various startups, which feature chatbot interfaces, will have a significant impact on users’ expectations regarding their interactions with company websites. In order to engage with customers and users, companies will need to integrate artificial intelligence through foundation models and chatbot interfaces. ChatGPT is just the beginning; upcoming offerings will feature advanced multimodal models and deliver responses that integrate text, images, videos, and audio clips. Similar to the thesis we presented on decentralized custom models, the emergence of next-generation search and Q&A applications presents exciting prospects for Data and AI teams.
A traditional search platform is made up of various components such as extraction and indexing services, a knowledge base, user profiling, query processing, result display, result refinement, and history tracking. In the near future, enterprise search tools will incorporate foundation models that require continuous training, evaluation, and maintenance.
To maintain a competitive edge, protect privacy and comply with regulations, many companies are likely to seek greater control over their data. Furthermore, since the majority of foundation models and search providers solely provide API access, it is my belief that quite a few companies will choose to create their own custom models. This decision would not only facilitate accelerated innovation but also enhance privacy and compliance measures. While we are still in the early stages of experimentation, I have already encountered a few teams concerned about sending (proprietary) information to APIs.
Finally, search platforms that integrate foundation models require comprehensive machine learning pipelines, encompassing the entire process from extraction and indexing to retrieval, re-ranking, and calibration. These trends serve as a clear indicator of the critical role that Data and AI teams will play in guiding companies through the development, testing, evaluation, and maintenance of their search and Q&A applications – assets that will be of significant value in the future.
Data Exchange Podcast
1. Running Machine Learning Workloads On Any Cloud. Zongheng Yang offers a thorough and current synopsis of SkyPilot, a groundbreaking intercloud broker that views the cloud ecosystem as a cohesive and integrated entity rather than a collection of disparate, largely incompatible clouds. Given the difference in pricing and hardware offerings between regions, SkyPilot has also become an important tool for users who rely on a single cloud provider.
2. 2023 Trends in Data Engineering and Infrastructure. Data forms the backbone of artificial intelligence and machine learning and holds immense significance in the progression of these technologies. I explore the latest trends, advancements, and best practices in data engineering and data platforms with Jesse Anderson and Evan Chan.
Spotlight
1. GPT Index. Providing “knowledge” to Large Language Models (LLMs) is restricted by limits on prompt size and model weights. Enter GPT Index, a straightforward and adaptable solution that bridges the gap between your external data and the power of LLMs.
2. Scaling Vision Transformers to 22 Billion Parameters. Pre-trained vision backbones have improved computer vision tasks, just as pre-trained language models have improved natural language processing. Vision Transformers have been introduced for image and video modeling, but they haven’t yet been scaled to the same extent as language models. Google recently unveiled a vision Transformer model with a staggering 22 billion parameters in this paper. Their model shows improved performance with scale when applied to downstream tasks.
3. Guides, papers, and resources for prompt engineering. Prompt engineering refers to the method of creating and refining prompts to effectively utilize language models for a variety of applications. Top-notch prompt engineers conduct experiments, systematically record their findings, and refine their prompts to identify essential components.
4. Scaling Media Machine Learning at Netflix. Training machine learning models for media requires a lot of storage, network, and GPU resources. To solve this problem, Netflix created a large-scale GPU training cluster using Ray, which seamlessly enables multi-GPU and multi-node distributed training.
If you enjoyed this newsletter please support our work by encouraging your friends and colleagues to subscribe: