Elastic Inference Service
Serverless Stack Self-Managed
The Elastic Inference Service (EIS) enables you to leverage AI-powered search as a service without deploying a model in your cluster. With EIS, you don't need to manage the infrastructure and resources required for machine learning inference by adding, configuring, and scaling machine learning nodes. Instead, you can use machine learning models for ingest, search, and chat independently of your Elasticsearch infrastructure.
Your Elastic deployment or project comes with a default
Elastic Managed LLMconnector. This connector is used in the AI Assistant, Attack Discovery, Automatic Import and Search Playground.You can use ELSER to perform semantic search as a service (ELSER on EIS). Stack Serverless
Requests through the Elastic Managed LLM are currently proxying to AWS Bedrock in AWS US regions, beginning with us-east-1.
The request routing does not restrict the location of your deployments.
ELSER requests are managed by Elastic's own EIS infrastructure and are also hosted in AWS US regions, beginning with us-east-1. All Elastic Cloud hosted deployments and serverless projects in any CSP and region can access the endpoint. As we expand the service to Azure and GCP and more regions, we will automatically route requests to the same CSP and closest region the Elaticsearch cluster is hosted on.
Serverless Stack
ELSER on EIS enables you to use the ELSER model on GPUs, without having to manage your own ML nodes. We expect significantly better performance for throughput and consistent search latency as compared to ML nodes, and will continue to benchmark, remove limitations and address concerns as we move towards General Availability.
You can now use semantic_text with the new ELSER endpoint on EIS. To learn how to use the .elser-2-elastic inference endpoint, refer to Using ELSER on EIS.
Semantic Search with semantic_text has a detailed tutorial on using the semantic_text field and using the ELSER endpoint on EIS instead of the default endpoint. This is a great way to get started and try the new endpoint.
While we do encourage experimentation, we do not recommend implementing production use cases on top of this feature while it is in Technical Preview.
There are no uptime guarantees during the Technical Preview. While Elastic will address issues promptly, the feature may be unavailable for extended periods.
Inference throughput via this endpoint is expected to exceed that of inference operations on an ML node. However, throughput and latency are not guaranteed. Performance may vary during the Technical Preview.
Batches are limited to a maximum of 16 documents. This is particularly relevant when using the _bulk API for data ingestion.
Rate limit for search and ingest is currently at 500 requests per minute. This allows you to ingest approximately 8000 documents per minute at 16 documents per request.
All models on EIS incur a charge per million tokens. The pricing details are at our Pricing page for the Elastic Managed LLM and ELSER.