Martin Polden

Principal Vespa Engineer

22 May 2023

Photo by Shiro
hatori on Unsplash

For a long time
vespa-feed-client has been
the best option for feeding large sets of documents to Vespa efficiently. While
the client itself performs well, it depends on a Java runtime and its
installation method is rather cumbersome. Compared to Vespa CLI it also lacks
many ease-of-use features such as automatic configuration of authentication and
endpoint discovery.

Since our initial announcement of Vespa
CLI it has become the standard
interface for working with Vespa applications, both for self-hosted
installations and Vespa Cloud. However, document
feeding with Vespa CLI was initially limited to single-document operations,
using the vespa document command.

Having to juggle multiple tools while working with Vespa is obviously not ideal.
We therefore decided to implement a high performance feeding client inside Vespa
CLI, thus making it a universal client for Vespa.

Today we’re excited to announce this new feed client! See it in action in the
screencast below:

Performance

The new feed client is ready for most use-cases. If you’re already using
vespa-feed-client and want to switch to vespa feed, we recommend comparing
the feed performance of your particular document set before making the switch.
vespa feed outputs statistics on the same format as vespa-feed-client,
making comparison easy.

We’ve invested a lot of time into making vespa feed as performant as the old
client. In our performance tests, its current default configuration outperforms
the old client when feeding small- (10B) and medium-sized (1KB) documents, but
it still lags behind vespa-feed-client when feeding large (10KB+) documents.

Below you can see a throughput comparison (queries per second) of the two
clients when feeding two million documents at sizes 10B, 1KB and 10KB:

We’ll continue making performance improvements to the new client, so make sure
to keep your Vespa CLI installation up-to-date.

Future of the Java client

The introduction of vespa feed does not deprecate vespa-feed-client. If
you’re already using vespa-feed-client there is no immediate need to migrate
to the new client. vespa-feed-client provides both a Java library and a
command-line interface for that library, both of which will remain supported.

However, if you’d rather use Vespa CLI for all things Vespa and don’t depend on
vespa-feed-client as a Java library, we encourage you to try our new client.

Getting started

The new feed client is available in Vespa CLI as of version 8.164. See vespa
help feed for usage and the Vespa
documentation for further
details.

If you’re using Homebrew you can upgrade to the latest version using brew
upgrade vespa-cli or you can download the latest release from our GitHub
releases page.

New to Vespa CLI? Please see our quick start guides for self-hosted
Vespa or Vespa
Cloud.

Found a bug or have a feature request? Feel free to file a GitHub
issue. Need help with Vespa CLI
or Vespa in general? Drop by our community Slack
channel.