In the previous update,
we mentioned new Container Thread Pools and Feed Throughput improvements.
Subscribe to the mailing list to get these updates delivered to your inbox.
This month, we’re excited to share the following updates:
Tensor Performance Improvements
Vespa 7.319.17 and onwards includes new optimizations to
tensors with sparse dimensions.
We have implemented new memory structures to represent sparse and mixed tensors
and a new pipeline for evaluating tensor operations.
This has enabled applications to deploy new advanced ranking models using mixed tensors in production.
An example is a use case where end-to-end average latency went from 135ms to 13ms; a 10x speedup.
When measuring the latency of only mixed tensor operations, the speedup is 150x.
Latency improvement for basic sparse tensor operations is around 40%,
while more advanced sparse tensor operations have a speedup of up to 50x.
Vespa Container Apache ZooKeeper Integration
Vespa allows you to add custom Java components for query and document processing.
If this code needs a shared lock across servers in a cluster,
you can now configure a container cluster to run an embedded ZooKeeper cluster
and access it through an injected component.
pyvespa is a python library created to enable faster prototyping
and facilitate Machine Learning experiments for Vespa applications.
The library is under active development and ready for trial usage.
Please give it a try and help the Vespa team improve it through feedback and contributions.
To increase Vespa’s capacity for evaluating large models,
both in performance and model types supported,
Vespa has integrated ONNX Runtime.
This makes it easier to use both Vespa and ONNX, as there is no conversion.
See the blog post for details.
About Vespa: Largely developed by Yahoo engineers,
Vespa is an open source big data processing and serving engine.
It’s in use by many products, such as Yahoo News, Yahoo Sports, Yahoo Finance, and the Verizon Media Ad Platform.
Thanks to feedback and contributions from the community, Vespa continues to grow.