﻿---
title: Custom Logs (Filestream) Package
description: In future releases it's expected to have an automated way to migrate the state. However, this is not possible at the moment. The current best option for...
url: https://www.elastic.co/docs/reference/integrations/filestream
---

# Custom Logs (Filestream) Package


|                                                                                                                                     |                                        |
|-------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------|
| **Version**Version                                                                                                                  | 1.2.0 ([View all](#changelog)View all) |
| **Subscription level**Subscription level[What's this?](https://www.elastic.co/subscriptions)What's this?                            | Basic                                  |
| **Level of support**Level of support[What's this?](https://www.elastic.co/docs/reference/integrations/level-of-support)What's this? | Elastic                                |
| **Ingestion method(s)**Ingestion method(s)                                                                                          | File                                   |

<warning>
  Migrating from the "Custom Logs (Deprecated)" to "Custom Logs
  (Filestream)" will cause files to be re-ingested because the state is not migrated.
</warning>

<important>
  The Filestream integration will only start ingesting files
  **when they are 1024 bytes in size or larger**. This can be adjusted by
  setting "Fingerprint length", however it will influence how files are
  identified. Refer to the
  [fingerprint](https://www.elastic.co/docs/reference/beats/filebeat/filebeat-input-filestream#filebeat-input-filestream-file-identity-fingerprint)
  documentation for more details.
</important>

In future releases it's expected to have an automated way to migrate the state. However, this is not possible at the moment.
The current best option for minimizing the data duplication while migrating to "Custom Logs (Filestream)" is to use the 'Ignore Older' or 'Exclude Files' options.
The `filestream` custom input is used to read lines from active log files. It is the
new, improved alternative to the `log` input. It comes with various improvements
to the existing input:
1. Checking of `close_*` options happens out of band. Thus, if an output is blocked,
   Elastic Agent can close the reader and avoid keeping too many files open.
2. The order of `parsers` is configurable. So it is possible to parse JSON lines and then
   aggregate the contents into a multiline event.
3. Some position updates and metadata changes no longer depend on the publishing pipeline.
   If the pipeline is blocked some changes are still applied to the registry.
4. Only the most recent updates are serialized to the registry. In contrast, the `log` input
   has to serialize the complete registry on each ACK from the outputs. This makes the registry updates
   much quicker with this input.
5. The input ensures that only offsets updates are written to the registry append only log.
   The `log` writes the complete file state.
6. Stale entries can be removed from the registry, even if there is no active input.
7. The fingerprint file identity is used by default.

More information can be found on the [Filestream documentation page](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html)
As Filestream configures a new input, configuring it to collect data
from a file that was previously collected by Custom Logs integration
will result in duplicate data. You may wish to configure
`ignore_older` or temporarily set `ignore_inactive: since_first_start`
to limit the amount of duplicate data collected.
If the Custom Logs integration is removed and the Custom Filestream
Logs is added in the same policy change, there risk of data being
missed between the last entry ingested by the Custom Logs and the
first one ingested by the Custom Filestream Logs.

## Changelog

<dropdown title="Changelog">


  | Version        | Details                                                                                                                                                                                                  | Kibana version(s)               |
  |----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------|
  | **1.2.0**1.2.0 | **Bug fix**Bug fix ([View pull request](https://github.com/elastic/integrations/pull/14351)View pull request)Add the native file identity                                                                | 8.15.0 or higher9.0.0 or higher |
  | **1.1.5**1.1.5 | **Bug fix**Bug fix ([View pull request](https://github.com/elastic/integrations/pull/14210)View pull request)Fix file identity options not getting applied correctly                                     | 8.15.0 or higher9.0.0 or higher |
  | **1.1.4**1.1.4 | **Enhancement**Enhancement ([View pull request](https://github.com/elastic/integrations/pull/14209)View pull request)Add warning about only ingesting files >= 1024 bytes                                | 8.15.0 or higher9.0.0 or higher |
  | **1.1.3**1.1.3 | **Enhancement**Enhancement ([View pull request](https://github.com/elastic/integrations/pull/13679)View pull request)Correct the readme                                                                  | 8.15.0 or higher9.0.0 or higher |
  | **1.1.2**1.1.2 | **Enhancement**Enhancement ([View pull request](https://github.com/elastic/integrations/pull/13670)View pull request)Add warning about migration from "Custom Logs (Legacy)" to"Custom Logs (Filestream" | 8.15.0 or higher9.0.0 or higher |
  | **1.1.1**1.1.1 | **Enhancement**Enhancement ([View pull request](https://github.com/elastic/integrations/pull/13666)View pull request)Rename the package                                                                  | 8.15.0 or higher9.0.0 or higher |
  | **1.1.0**1.1.0 | **Enhancement**Enhancement ([View pull request](https://github.com/elastic/integrations/pull/12880)View pull request)Add support for defining Conditions                                                 | 8.15.0 or higher9.0.0 or higher |
  | **1.0.1**1.0.1 | **Bug fix**Bug fix ([View pull request](https://github.com/elastic/integrations/pull/12861)View pull request)Resolving conflicts with past integration versions.                                         | 8.15.0 or higher9.0.0 or higher |
  | **1.0.0**1.0.0 | **Enhancement**Enhancement ([View pull request](https://github.com/elastic/integrations/pull/12845)View pull request)Make it GA                                                                          |                                 |
  | **0.1.0**0.1.0 | **Enhancement**Enhancement ([View pull request](https://github.com/elastic/integrations/pull/12837)View pull request)Make it available in 9.0                                                            |                                 |
  | **0.0.1**0.0.1 | **Enhancement**Enhancement ([View pull request](https://github.com/elastic/integrations/pull/11332)View pull request)Initial Release                                                                     |                                 |
</dropdown>
