Internship at Vespa | Vespa Blog
Mathias Chunnoo
Ole-Magnus Vian Norum
Photo by Charles Deluvio on Unsplash
Over the course of the summer we the interns have gotten to explore the Vespa
engine and the workings of the company. At the start of our internship we got
an introduction to the company and the tools that the they used. To get
familiar with the vespa engine we went through the getting started tutorial,
where we made a news recommendation system.
During our internship we got to work on many different things, but
our two main projects were to use Vespa to implement two sample
applications for searching through the Vespa documentation. These two sample
apps were named
search-as-you-type
and
search-suggestions.
Search-as-you-type
The search-as-you-type application aims to implement an interface where results
are displayed live while the user is typing in the search bar. This requires
the search to both generate hits on incomplete words and to retrieve these hits
as close to instantly as possible. A substring search would fit the need for
incomplete word searches, however for large corpora, this would not meet the
performance requirement. Our solution instead uses n-grams (groups of n
characters) to simulate a substring-like search. The idea is to search for
n-grams and rank the hits where the n-grams are bunched up together higher than
the hits where the n-grams are spread throughout the document. After trying
various configurations we found that 3-grams in combination with Vespas
nativeRank fit our needs
very well. In addition we combined this with index search such that if the
search string consists of complete words the indexed search hits would rank
higher than the n-gram search hits.
schema doc {
field gram_content type string {
indexing: input content | index | summary
match {
gram
gram-size: 3
}
summary: dynamic
}
...
document doc {
...
field content type string {
indexing: index | summary
summary: dynamic
stemming: best
}
...
rank-profile weighted_doc_rank inherits default {
rank-properties {
$contentWeight: 10.0
$gramContentWeight: 1.0
}
first-phase {
expression {
query(contentWeight) * nativeRank(content)
+ query(gramContentWeight) * nativeRank(gram_content)
}
}
}
}
After the Vespa application was in place, we needed an actual search bar for
the search as you type to take place. This was implemented by incorporating a
simple static web page java server into the Vespa application and writing some
javascript to query the Vespa application every time a character was entered
into the search bar. In addition, a debounce-function was used to avoid race
condition due to simultaneous query requests.
Search-suggestion
The idea behind the search-suggestion application is to suggest possible search
terms to the user before they have typed out their whole query. In our
implementation these suggestions comes either from the document texts the users are
searching through or from previous searches performed by other users.
The first iteration of the search-suggestion application fed new search terms
by “put”, which thus resulted in storing multiples of the same terms. To
calculate relevance and single out terms, the generated hits where grouped and
counted. This was not a scalable solution as more data for the same search
terms would result in a linear increase in both storage space and process time
for each query. To solve this we switched to feeding by “update”. In other
words, adding unseen terms and incrementing their “query_count” variable when
processing previously seen terms.
[
{
"update": "id:term:term::example",
"create": true,
"fields": {
"term": {
"assign": "example"
},
"corpus_count": {
"assign": 181
},
"document_count": {
"assign": 40
}
}
},
{
"update": "id:term:term::example",
"create": true,
"fields": {
"term": { "assign": "example" },
"query_count": { "increment": 1 }
}
}
]
Since we were going to use queries written by users as suggestions, we had to
implement some form of moderation as to what could be suggested. To solve this
problem we made a list of allowed terms and used a document processor to filter
out any documents that contained terms not in the list. We chose to generate
the allowed-list by listing every word used in the document text. This made it
so that all relevant terms could be suggested, and things that could be seen as
offensive or otherwise irrelevant would not come up as suggestions, as they was
not contained in the document text and thus would be blocked by the document
processor and not fed.
For the first iteration of the search-suggestion, the application used Vespas
streaming search with prefix matching to search for documents with matching
prefixes. After a presentation of the application and some discussion it was
believed that streaming search would not be scalable as the number of
concurrent users increased. To test this belief we did a benchmark of the
application using fbench (benchmark results).
As suspected the performance of streaming search drastically decreased as
the number of concurrent users increased. We decided to change the application
to use index prefix search, and after a comparison benchmark test it was
confirmed that this implementation scaled much better than streaming search.
Like we did with the search-as-you-type application, we incorporated a static
web page java server into the Vespa application and wrote some javascript for
querying suggestions on every input and showing these suggestions in a dropdown
under the search bar.
We also took the two sample applications we made and integrated them in to an
already existing sample application
vespa-documentation-search
which is deployed on to Vespa Cloud. As of now this deployment is also used for
search suggestion on the Vespa documentation sites in the search bar.
AWS Lambda
One of the goals of the search-suggestion application was to favor searches
that where previously searched for. To accomplish this we decided to create a AWS
Lambda function which would read query logs and feed search term from these
back into the Vespa application. The reason for this was that the query logs
where stored in AWS S3 buckets and that this would make it possible to
continuously trigger the Lambda function and process future query logs. The
biggest problem we faced when writing this Lambda function was decompressing
the logs. Vespa stores its query logs compressed with zstd-compression and
finding a zstd-library usable in a AWS Lambda context was not straight forward.
Initially a lot of time was spent learning AWS SAM and deploying a docker image
to the Lambda function, as this would let us use native C++ libraries in the
Lambda. However, we later found a library which compiled to web assembly and
would let us decompress filed with just a Node.js Lambda function.
Other projects
While working on the main projects, we also got to work on other smaller side
projects. One of these side projects was to implement a visualization view of
proton’s memory usage. This view was
created using react and incorporated into the Vespa Console. We also got to
work with Vespa’s performance tests and moved some of the private performance
tests over to the public opensource repository by changing the document sets so
that they did not use private or sensitive data.
The experience at Vespa
At the start of the internship it felt a bit daunting to make our own sample
applications, given that the vespa engine was something completely new to us.
Even after the getting started tutorial many things were still unclear, but as we
started working on the sample applications, more and more things became clearer as
we had to dive in to the documentation and previously written code to be able to
make the applications from scratch.
Since there is so much you can do with Vespa it was hard at times to find the answers
to our questions in the documentation. Not necessarily because it did not exist, but
because we did not know what to search for to find the right documents. This led to us
some times using Vespas public github-repositories to find answers to our questions.
Even though we some times did not find the answers we were looking for, we never felt
lost as there were always someone ready to help us out when we got stuck.
During our internship at Vespa we got to learn a lot about the vespa search
engine and information retrieval in general. We have especially learned some
different methods of doing search with incomplete queries and query processing.
We have also gotten a feel for how it is to work in a software company through
daily stand-ups and presentations of our projects.
Working here have given us insight into the workflow and github etiquette of a
software company. The internship has given us experience with working in a
team of developers and how to collaborate through github effectively. We
have touched upon various technologies from writing user interfaces in React to
writing performance tests in Ruby. We have also gotten to work with and learn
about important platforms and services like Amazon Web Service and Docker, which
is commonly used in companies but not taught in schools.
Even after the internship there are still many things that we have not touched
upon or learned about, and we wish we could explore more. To go deeper in
to the lower levels of the code and the working of Vespa, and to learn more about
search from the experienced people working at Vespa. We grew attached to the
projects we worked on and wished we had more time to fine tune, and improve
them to get an even better search in the Vespa documentation.
We really enjoyed our stay here, with a nice staff who have an incredible
expertise regarding search and information retrieval, from whom we have learned a
lot. The experience at Vespa has been really pleasant and educational, and is some thing
that has and will benefit us in the future.
