Kristian Aune

Head of Customer Success, Vespa

01 Oct 2019

In the August Vespa product update, we mentioned BM25 Rank Feature, Searchable Parent References, Tensor Summary Features, and Metrics Export. Largely developed by Yahoo engineers, Vespa is an open source big data processing and serving engine. It’s in use by many products, such as Yahoo News, Yahoo Sports, Yahoo Finance, and the Verizon Media Ad Platform. Thanks to feedback and contributions from the community, Vespa continues to grow.

This month, we’re excited to share the following updates with you:

Tensor Float Support

Tensors now supports float cell values, for example tensor<float>(key{}, x[100]). Using the 32 bits float type cuts memory footprint in half compared to the 64 bits double, and can increase ranking performance up to 30%. Vespa’s TensorFlow and ONNX integration now converts to float tensors for higher performance. Read more.

Reduced Memory Use for Text Attributes 

Attributes in Vespa are fields stored in columnar form in memory for access during ranking and grouping. From Vespa 7.102, the enum store used to hold attribute data uses a set of smaller buffers instead of one large. This typically cuts static memory usage by 5%, but more importantly reduces peak memory usage (during background compaction) by 30%.

Prometheus Monitoring Support

Integrating with the Prometheus open-source monitoring solution is now easy to do
using the new interface to Vespa metrics.
Read more.

Query Dispatch Integrated in Container

The Vespa query flow is optimized for multi-phase evaluation over a large set of search nodes. Since Vespa-7-109.10, the dispatch function is integrated into the Vespa Container process which simplifies the architecture with one less service to manage. Read more.

We welcome your contributions and feedback (tweet or email) about any of these new features or future improvements you’d like to request.

By Arnstein Ressem, Principal Software Systems Engineer, Verizon Media

When Vespa was open sourced in 2017 we looked for a continuous integration platform to build our source code on. We looked at several hosted solutions as well as Screwdriver – an open source CI/CD platform built by Yahoo/Verizon Media – that had just been open sourced in 2016. Another platform seemed the best fit for us at that point in time and we integrated with that.

Photo by Bill Oxford on Unsplash

The Vespa codebase is large with approximately 700 KLOC C++, 700 KLOC Java and more than 10k unit tests. For a given version of Vespa we build the complete codebase and version the artifacts with semantic versioning. We always build from the master branch and have no feature branches.

Compiling and testing this codebase is resource demanding and we soon realized that the default VMs that the provider had were not up to the task and took more than 2 hours to complete. This was a serious issue for the developers waiting for feedback on their pull requests. We ended up subscribing to a premium plan and did more caching of Maven artifacts and compiled C++ objects (ccache) to bring the build time just under one hour.

In the fall of 2020 we became aware of big changes in the selected CI/CD platform and we needed to migrate to something else. As part of this work we took another look at the open sourced version of Screwdriver as we knew that the project had significantly matured over the past years. Screwdriver is an open source build platform designed for Continuous Delivery that can easily be deployed on different IaaS providers and is currently an incubee in the Continuous Delivery Foundation.

Vespa pipeline on Screwdriver

The Vespa team got access to a hosted instance at cd.screwdriver.cd (invite only, but publicly readable with guest access). Working closely with Screwdriver we were able to reduce the build times for the master branch and pull requests from 50 minutes on the previous solution to 18 minutes. This result was obtained by using Screwdriver’s configurable resource management and fast build caches. We also appreciated the small set of requirements on container images allowing us to optimize the build image for our jobs.

Screwdriver integrated with pull request builds on GitHub

To further increase the developer feedback and productivity we decided to do some pull request analysis to check if only C++ or Java source code was touched. In those cases we could only build and test for the respective language. This brought the pull request build times from 18 down to 12 minutes for C++ and 8 minutes for Java. This allowed developers to have more issues discovered in pull requests without having to wait for a long time for the review and merge.

We are very happy with having the time spent on builds and pull requests reduced by 75% on average and this leads to better productivity and happier developers.