Important: This documentation covers Yarn 1 (Classic).
For Yarn 2+ docs and migration guide, see yarnpkg.com.

Package detail

@sematext/logagent

sematext221Apache-2.03.3.1

Smart log parser written in Node

Logagent, monitoring, alerting, log management, sematext, logsene, log parser, logging, log shipper, elasticsearch, indexing, etl, logstash alternative, filebeat alternative, mqtt client, internet of things, IoT, mqtt broker, kafka client, graylog, Sematext Cloud, Sematext Enterprise, MySQL client, PostgreSQL client, Log aggregation, influxdb, Cassandra

readme

Build Status Deploy - read more

What is Logagent

Logagent is a modern, open-source, light-weight log shipper. It is like Filebeat and Logstash in one, without the JVM memory footprint. It comes with out of the box and extensible log parsing, on-disk buffering, secure transport, and bulk indexing to Elasticsearch, Sematext Logs, and other destinations. Its low memory footprint and low CPU overhead make it suitable for deploying on edge nodes and devices, while its ability to parse and structure logs makes it a great Logstash alternative.

Docker

Details about the the Logagent Docker image are described in the Docker Hub Readme

Installation

1) Install Node.js

Official Node.js downloads and instructions. E.g. for Debian/Ubuntu:

curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
sudo apt-get install -y nodejs
Install Logagent with npm
sudo npm i -g @sematext/logagent

2) Run logagent command line tool

logagent --help

3) Example: Index your log files in Elasticsearch

logagent -e http://localhost:9200 -i logs -g ‘/var/log/**/*.log

4) Optional: Install service & config

Install service for Logagent using systemd, upstart, launchd To quickly create a config file for indexing into Elasticsearch without having to edit it run something like this:

sudo logagent-setup -u http://localhost:9200 -i INDEX_NAME -g '/var/log/**/*.log'
# Logsene US: use -u https://logsene-receiver.sematext.com and your Logsene App Token as index name.
# Logsene EU: use -u https://logsene-receiver.eu.sematext.com and your Logsene App Token as index name.

Configuration

To configure different inputs, different event processing, or different outputs (e.g. your own Elasticsearch) edit /etc/sematext/logagent.conf, e.g.:

output:
  logsene:
    module: elasticsearch
    url: http://elasticsearch-server:9200
    index: logs

Then restart the service with sudo service logagent restart. Troubleshooting & Logs Logagent’s own logs:

  • Upstart: /var/log/upstart/logagent.log
  • Systemd: journalctl -u logagent
  • Launchd: /Library/Logs/logagent.log

Location of service scripts:

  • Upstart: /etc/init/logagent.conf
  • Systemd: /etc/systemd/system/logagent.service
  • Launchd: /Library/LaunchDaemons/com.sematext.logagent.plist

Start/stop service:

  • Upstart: service logagent stop/start
  • Systemd: systemctl stop/start logagent
  • Launchd: launchctl start/stop com.sematext.logagent

Documentation & Support

Development

  • Update to the last node version
  • From root folder node type: node ./bin/logagent -h
  • To test from root folder type: node test

changelog

Changelog

All notable changes to this project will be documented in this file. Dates are displayed in UTC.

Generated by auto-changelog.

3.3.1

9 October 2023

  • SC 17171 proxy settings is ignored #327
  • fix: remove moderate vulnerabilities 8ac1909
  • chore: update releaset-it library 1709c9a
  • fix: overide http and https proxies cli parameters from environment variables b379a14

3.3.0

25 May 2023

  • SC-16333 Extract fields from vercel syslog messages #325
  • Bump vm2 from 3.9.11 to 3.9.19 #324
  • Bump xml2js from 0.4.19 to 0.5.0 #321
  • Bump flat from 4.1.0 to 5.0.1 #319
  • Bump http-cache-semantics from 4.1.0 to 4.1.1 #320
  • Bump decode-uri-component from 0.2.0 to 0.2.2 #318
  • Bump minimatch from 3.0.4 to 3.0.5 #317
  • Bump qs from 6.5.2 to 6.5.3 #316
  • Update dependencies da86d3e
  • Update dependencies a530902
  • Eliminate duplicate message bbe315e

3.2.0

19 September 2022

  • Bump jose from 1.28.1 to 1.28.2 #315
  • Drop prettier-standard #314
  • Change prettier-standard dependency to patched repository #313
  • Migrate to Buffer.alloc() #310
  • Stop spamming logs #312
  • SC-13832 Update dependencies #311
  • Update dependencies b014a0e
  • Release 3.2.0 8c847a9

3.1.4

6 September 2022

  • Bump node-fetch from 2.6.1 to 2.6.7 #309
  • Bump moment-timezone from 0.5.31 to 0.5.35 #308
  • Bump moment from 2.29.2 to 2.29.4 #307
  • Release 3.1.4 56f5344

3.1.3

19 July 2022

  • Fix for debug flag ignore issue #306
  • Bump moment from 2.29.1 to 2.29.2 #302
  • Release 3.1.3 72f59a4

3.1.2

6 April 2022

  • Remove node-windows dependency due to an upstream conflict e34785c
  • Release 3.1.2 3b27507

3.1.1

1 April 2022

  • Bump requestretry from 3.1.0 to 7.0.0 #300
  • Bump node-fetch from 2.6.1 to 2.6.7 #298
  • Add missing dependencies 8afa40d
  • Add missing dependencies for Request and Graceful-fs 3fd5a39
  • Release 3.1.1 cc0f302

3.1.0

29 March 2022

  • Graceful-fs filesystem #301
  • Bump ajv from 6.11.0 to 6.12.6 #297
  • Revert "SC-12473 Comment code of container image digest (sha256)" #296
  • SC-12473 Comment code of container image digest (sha256) #295
  • Bump aws-sdk from 2.689.0 to 2.814.0 #290
  • [load-creds-from-ec2-role] #289
  • [load-creds-from-ec2-role] #288
  • Bump set-value from 3.0.2 to 4.0.1 #287
  • Update azure-event-hub.js #285
  • Bump path-parse from 1.0.6 to 1.0.7 #284
  • Update azure-event-hub.js #282
  • Update dependencies b8223bd
  • [load-creds-from-ec2-role] revert the changes 3fb3917
  • [load-creds-from-ec2-role] update aws elasticsearch output plugin to load credentials from IAM role attached to an EC2 instance 53c2858

3.0.72

2 July 2021

  • input-journald-upload: use removeFields from config, fix include/exclude test #277
  • input-journald-upload: use removeFields from config 6aadf20
  • Release 3.0.72 e3dbd62

3.0.71

1 July 2021

3.0.70

25 June 2021

  • Vercel receiver: added test case #280
  • Refactoring to use multiple Vercel client secrets #279
  • refactor: multiple client secrets 3c11a81
  • Release 3.0.70 94ec683
  • add another test case 877a228

3.0.69

22 June 2021

3.0.68

7 April 2021

  • updated rabbitmq patterns #276
  • hotfix: added object literal check for drop events filter 31de82f
  • added config for dropping event in k8s 2186471
  • Release 3.0.68 af1661e

3.0.67

23 February 2021

  • SC-10763: Added formatting for cached requests #274
  • SC-10763: added formatting for cached requests 7153bda
  • Release 3.0.67 39b041b

3.0.66

2 February 2021

  • Added check for token to be anonymized in error logs 41ec35d
  • Release 3.0.66 abd1d1d

3.0.65

20 January 2021

  • Release 3.0.65 2394f0d
  • edit sample syslog file 0ac8ecc
  • Merge pull request #272 from sematext/syslog-tags cc583dd

3.0.64

5 January 2021

  • Release 3.0.64 54f10b4
  • Merge pull request #271 from sematext/hotfix-systemd-journal-upload ab397ea
  • hotfix: added config.url and removed breaking toLowerCase 945dddc

3.0.63

29 December 2020

3.0.62

22 December 2020

  • Release 3.0.62 dda659c
  • Merge pull request #270 from sematext/sc-10519-journald-outputfilter-matchsource 65f6d7b
  • added sample config for testing journald matchsource 523d20d

3.0.61

9 December 2020

  • Release 3.0.61 410a3b0
  • Merge pull request #269 from sematext/sc-10353-ecs-improvements 135dfac
  • added gzip e687ee9

3.0.60

4 December 2020

  • initial setup for aws-ecs la receiver 100b342
  • added parsing for log.messages with log.sourceName f21bec4
  • Release 3.0.60 c101541

3.0.59

27 November 2020

  • Don't pull build images if not needed #265
  • Fix collections pattern for Solr audit logs #262
  • Release 3.0.59 458ae22
  • removed node 15 from travis d5ca8a4
  • Merge pull request #266 from sematext/sc-10318-heroku-log-parsing 9172cfd

3.0.58

13 October 2020

  • Fix #256 Write after stream end #261
  • Merge pull request #261 from sematext/sc-9545-fix-stream-err #256
  • cleaned up output files plugin ad6dcce
  • Linting cleanup 14ba81e
  • Linting... c51eaee

3.0.57

8 October 2020

  • hotfix: added rabbitmq_startup match and updated logsene-js version cef3512
  • Release 3.0.57 9ff35df

3.0.56

1 October 2020

  • logagent-setup: install service with custom config #260
  • Make default journald config easier to extend #259
  • Release 3.0.56 3a07f31
  • fixed bug (-l didn't work); also fixed doc for -j 0002092

3.0.55

25 September 2020

  • Release 3.0.55 88d63c4
  • Merge pull request #258 from sematext/sc-9544-journald-fix 19adfc9
  • SC-9544: added lowercase fields filter 163c598

3.0.54

23 September 2020

  • made colon optional for authentication messages #257
  • Added Apache Flink patterns f77586a
  • Release 3.0.54 6a1e15d
  • Audited deps e6d0486

3.0.53

2 September 2020

  • Release 3.0.53 4da5588
  • Merge pull request #254 from sematext/hotfix-k8s-enrichment-dropping-logs bef4577
  • Added log forwarding in initK8sClient 97e2fb6

3.0.52

21 August 2020

  • Release 3.0.52 e76ffe4
  • Merge pull request #253 from sematext/geo-refactor 1828440
  • removed maxmind code from geoip; added public ip fetching code to geoip 86aeb87

3.0.51

18 August 2020

  • logagent-setup can configure journalctl #252
  • Lowercase multiline journal fields as well #251
  • Release 3.0.51 67954f0

3.0.50

17 August 2020

3.0.49

17 August 2020

  • Release 3.0.49 f10f1db
  • SC-8299: Fix Github title and body parsing 6517da0

3.0.48

17 August 2020

3.0.47

17 August 2020

  • improvements to command input around QUERY_TIME #249
  • Release 3.0.47 9a50a45
  • Merge pull request #250 from sematext/sc-9192-fix-closed-prematurely 30198cb
  • linted code cd199f5

3.0.46

7 August 2020

  • Updated logsene-js logger #248
  • Release 3.0.46 04d4914
  • updated logsene-js and added better error handling 4cb42f7

3.0.45

4 August 2020

  • Output Filter for renaming fields #247
  • Expanded RabbitMQ patterns #246
  • added output filter for renaming fields 2a27e5b
  • Release 3.0.45 fd71ec5
  • lint c2f1c1d

3.0.44

24 July 2020

  • Use --network=host when building images #245
  • Release 3.0.44 ec169c9

3.0.43

23 July 2020

  • logagent-setup makes journald-upload receiver #239
  • Extend parsing Postgres logs #243
  • Release 3.0.43 f534a7e
  • Merge pull request #244 from sematext/sc-8931-k8s-containerd-hotfix f2d9921
  • added ibm cloud k8s example ds configs 1fe97f7

3.0.42

2 July 2020

  • Release 3.0.42 39ca523
  • Merge pull request #242 from sematext/sc-8687-cf-log-parsing 88de1ca
  • improvements for cf parsing 51f1b42

3.0.41

1 July 2020

  • Release 3.0.41 6231981
  • Merge pull request #241 from sematext/sc-8687-cf-log-parsing 6175d39
  • added additional patterns for Cloudfoundry 4f8797b

3.0.40

4 June 2020

  • refactored containerd input filter f7803d1
  • cleanup; added comments; removed consolelogs e68de73
  • fix parsing k8s filenames 913f89a

3.0.39

20 May 2020

  • Release 3.0.39 ace14e0
  • Merge pull request #235 from sematext/vercel-integration-updates 37532b4
  • added error parsing for vercel logs 079cee5

3.0.38

19 May 2020

  • hotfix: added check for assignee object 52df824
  • Release 3.0.38 c16b1cb

3.0.37

19 May 2020

  • Release 3.0.37 08ed70e
  • Merge pull request #234 from sematext/vercel-integration 207e86a
  • linted code with prettier 559fb23

3.0.36

7 May 2020

  • Release 3.0.36 99adca7
  • Merge pull request #231 from sematext/fix-docker-log-routing 97b2c1d
  • separated LOGS_RECEIVER_URL and LOGS_RECEIVER_URLS in the config; use singular for only one URL or the plural when configuring multiple endpoints 6a7f44e

3.0.35

1 May 2020

  • added error handler for getPodSpec 9a261a7
  • Release 3.0.35 12ce012
  • remove conditional for getPodSpec 4cf26af

3.0.34

30 April 2020

  • params and core parsing to Solr 7.x/8.x logs #225
  • Release 3.0.34 928f3bb
  • Merge pull request #227 from sematext/remove-container-host-hostname 259a740
  • Removed container.host.hostname from docker and k8s enrichment 5358f10

3.0.33

29 April 2020

  • Add support for building multi-arch images #226
  • edit dockerfiles to enable LA monitoring d762d64
  • params and core parsing to Solr 7.x/8.x logs 9ef4d12
  • Added core parsing for "generic" Solr logs dc31f0b

3.0.32

16 April 2020

  • Refactor commits field #223
  • Create graygelf in constructor once vs on every call to eventHandler #222
  • Sync #1
  • Gelf debug option #220
  • Edited commits field to be array instead of object to avoid 'Too many fields present in the index' issue 01500b8
  • Release 3.0.32 a89c663
  • Create graygelf in constructor once vs on every call to eventHandle 9a96990

3.0.31

7 April 2020

  • Fix for #191 - refactoring LOGS_ENABLED logic #217
  • added fix for whitelisting k8s pods 98921ca
  • refactored logic for checking the logs_enabled env vars a9c65f9
  • Release 3.0.31 0ef9444

3.0.30

3 April 2020

  • added dedicated user agent details field name f01c0d2
  • Release 3.0.30 f6e4976

3.0.29

3 April 2020

  • Fix for #213 - output filter for device detection and parsing of userAgent #214
  • Added patterns for Solr 7/8 and audit logs #216
  • Added patterns for Solr 7/8 and audit logs #215
  • changed device detector to output filter 0cd8c22
  • Explicitly defining field types for Solr patterns 756295e
  • added input filter for device detection and parsing of userAgent headers for web logs 6a44fb3

3.0.28

31 March 2020

  • updated run script to have override for logagent config file, to set endpoints and indices in the config file instead of using env vars 5d921f5
  • Release 3.0.28 38b87d7

3.0.27

30 March 2020

  • Release 3.0.27 09f8cbb
  • updated default value for MAX_CLIENT_SOCKETS to 5 02cd275
  • updated logsene-js to 1.1.68 58d2ae4

3.0.26

25 March 2020

3.0.25

16 March 2020

  • SC-7229: Refactor GitHub Webhook #211
  • added logs filter function; todo finish parser functions e5675b6
  • added formatting for release, create, and delete events 3ef16cd
  • added formatting for branch push event affef87

3.0.24

12 March 2020

  • SC-7229: refactored input, moved parsing code to filter, added region config value 9d43ad1
  • SC-7229: added basic structure for GitHub webhook events to use st logs filter and output ac027a2
  • es-output added debug option; cleaned up journald example config f743dbb

3.0.23

5 March 2020

  • SC-7164: Added token blacklisting for Github Webhook input #209
  • added token blacklisting to github webhook input and st events output a7cb2c2
  • refactored tokenblacklist to have less if nesting d6b4507
  • Release 3.0.23 8d4859a

3.0.22

4 March 2020

  • Code cleanup #208
  • Issue #206 #207
  • added dockerfile and run script for test config ead88f4
  • Release 3.0.22 07dc08e
  • hotfix: added missing env check for LOGS_TOKEN 3e3cbc0

3.0.21

28 February 2020

  • Make HTTP status for blacklisted tokens configurable #205
  • make HTTP status for blacklisted tokens configurable fd0f480
  • Release 3.0.21 3a703f1

3.0.20

27 February 2020

  • update pattern file in docker image 28826ee
  • add example for k8s annotations / log routing e8ee0af
  • Release 3.0.20 b7f7525

3.0.19

27 February 2020

  • Refactoring journald stream parser for multi-line fields #203
  • Add token blacklists to HTTP receivers #202
  • fix whitelisting pods via k8s annotations #201
  • Update date-fns to 2.9.0 #200
  • Edited release-it #199
  • refactoring journald stream parser for multi-line fields 64f4358
  • adjust date parser to date-fns@2.9.0 60e596b
  • fix whitelisting pods via k8s annotations and LOGSENE_ENABLE_DEFAULT=false #191 789ae3a

3.0.18

21 February 2020

  • Journald streaming mode #197
  • Update journald-upload-receiver.yaml #196
  • Add nodejs monitoring plugin #198
  • add streaming mode for jounrald plugin ef4835c
  • fix syslog facility value undefined, keep numeric values e56ca35
  • detect numeric fields in jounrald logs ce3457b

3.0.17

19 February 2020

3.0.16

19 February 2020

  • remove_console_log #194
  • add error handling for invalid tokens #193
  • add unix socket reader plugin, fix #176 #189
  • Merge pull request #189 from sematext/add_unix_socket_plugin #176
  • move journald formatting to journald-format output plugin d2aeccb
  • use Sematext common schema, lowercase field names 88c9921
  • update journald config examples a6463c2

3.0.15

17 February 2020

  • Fix GEOIP_FIELDS config settings #192
  • SC-7051: final touches for GitHub plugin #188
  • add role bindings in Kubernetes example #186
  • SC-7051: Added GitHub Webhook input plugin #185
  • add unix socket reader plugin, fix #176 #176
  • SC-7051: added create and delete events 6a3f201
  • add cli arg ---unixSocket, fix connection issue 976539e
  • fix_geop_fields a3d57e4

3.0.14

17 February 2020

  • Emit logagent stats #184
  • Add nodejs monitoring plugin to docker image #183
  • Azure eventhub plugin #182
  • Add new pipeline event #181
  • SC-7051: converted to es class syntax a0ffe82
  • SC-7051: working event ingestion for pull_requests 78000b1
  • SC-7051: functional refactor 3c833cd

3.0.13

6 February 2020

3.0.12

5 February 2020

3.0.11

31 January 2020

  • Make log routing for containers flexible #179
  • refactoring helper functions 5e08e06
  • refactoring helper functions 0f878d2
  • Release 3.0.11 d729ee9

3.0.10

31 January 2020

  • add v8-profiler-next as optional dependency d8c9126
  • Release 3.0.10 854cde6

3.0.9

31 January 2020

3.0.8

30 January 2020

3.0.7

30 January 2020

  • add swarm.service.name and swarm.stack.name fields #178
  • make log routing more flexible c697139
  • fix wrong log messages 3d7d0c1
  • make plugin configs aware of the config section name 2487f75

3.0.6

15 January 2020

  • Parse image to registry,name,tag,digest #177
  • fix optional digiest in image name 9ed56bb
  • optimize regex 818cd50
  • remove debug output 9c90b2d

3.0.5

26 December 2019

  • add azure-event-hub input plugin 3844b84
  • add eventPosition=now, partitions, closing all subscriptions 55d4f30
  • fix :port prefix from registry in imageName f1d0d1d

3.0.3

26 November 2019

  • fix #166, add elasticsearch.type config option #166
  • update comments in example config 4b7f016
  • Release 3.0.3 7c13979
  • update comments in example config 9242e11

3.0.2

18 September 2019

  • Release 3.0.2 d722314
  • fix 'Unhandled Rejection' in kuberntes-enrichment plugin cbb789c

3.0.1

18 September 2019

  • Release 3.0.1 353de94
  • fix 'Unhandled Rejection' in kuberntes-enrichment plugin 43ce783

3.0.0

16 September 2019

  • Add geoip plugin #171
  • Use Kubernetes annotations #170
  • Generate patterns file #169
  • cleanup code after review, use LRU cache for pod info 9c2493f
  • move to maxmind@3.1.2, use sync reader to open maxmind DB f07a8ab
  • remove geo-ip lookup from log parser c3b495a

2.0.184

30 July 2019

2.0.183

30 July 2019

2.0.182

30 July 2019

2.0.181

30 July 2019

  • add all plugins to example.yml 1d8afdd
  • Don't write to /etc/sematext/receivers.cfg 07397d3
  • Don't write to /etc/sematext/receivers.cfg 2bcdc01

2.0.180

3 July 2019

2.0.179

3 July 2019

  • Release 2.0.179 7f80fa0
  • make text replacement via remove-fields plugin optional 0217b5e

2.0.178

26 June 2019

  • Make LA_CONFIG env variable configurable 1a6f41e
  • Release 2.0.178 3fe188c
  • Setting LA_CONFIG env variable disables the creation of logagent.conf 32660fd

2.0.177

25 June 2019

  • add output filter: drop-events-filter b5f7498
  • Release 2.0.177 6e3a1a0

2.0.176

21 June 2019

  • update nexe build, remove old doc scripts f7353c9
  • Release 2.0.176 48a4918

2.0.175

21 June 2019

2.0.174

21 June 2019

2.0.173

21 June 2019

  • fix: UncaughtException:TypeError: cb is not a function 1fe8942
  • Release 2.0.173 142bda2
  • Disable GeoIP lookups by default 8e8d4b2

2.0.172

31 May 2019

  • Release 2.0.172 44d52a6
  • add healthcheck command to Dockerfile 016a1db

2.0.171

30 May 2019

  • Release 2.0.171 86d6fa4
  • add logagent binary assets 421815a
  • ignore dist directory in git and npm fb9b40d

2.0.170

30 May 2019

  • Release 2.0.170 7e26ffd
  • fix wrong releaseNotes key in package.json 3e050b0

2.0.169

30 May 2019

2.0.168

30 May 2019

2.0.167

30 May 2019

2.0.166

30 May 2019

  • add release-it changelog generation 020ce87
  • update http pattern solving #168 a962d2e
  • Release 2.0.166 26cf56e

2.0.165

24 May 2019

2.0.164

24 May 2019

2.0.163

23 May 2019

  • improve docker socket error handling 2eadeb4
  • add build commands npm run build-all for nexe compiled binaries ff14fbf
  • Release 2.0.163 fccd0eb

2.0.162

16 May 2019

  • adjust naming of container metadata to STA field names 015f36e
  • adjust field names to sematext/agent field names 8dc30d5
  • adjust field names to sematext/agent field names in patterns.yml 85d61f4

2.0.161

14 May 2019

  • Convert all values to strings when creating error messages #152
  • Dockerfile for aarch64 #153
  • Fix level->severity mapping #156
  • add man page for RH certification 1aec480
  • add LICENENSE file for RH certification e2d43e4
  • add kuberntes audit http input-plugin 7913ff2

2.0.160

3 May 2019

2.0.159

23 April 2019

  • Release 2.0.159 ecb2e0a
  • update dependencies for new tail-forever release 0b5484c
  • update dependencies for new tail-forever release 4005836

2.0.158

23 April 2019

  • update zookeeper patterns in docker image ecc5d2a
  • update zookeeper patterns 80a6b60
  • Release 2.0.158 e634931

2.0.157

15 April 2019

  • add dockerEvents input 0b9e5d3
  • add input-kubernetes-events a8ab59d
  • improve err handling, remove local test function 563271e

2.0.156

11 April 2019

  • add --journald port CLI argument 9548628
  • add JOUNRALD_UPLOAD_PORT option f2d0209
  • Release 2.0.156 61da405

2.0.155

11 April 2019

  • add filter options, field removal and tags for journald logs 1ced79f
  • Release 2.0.155 23cffe7

2.0.154

11 April 2019

  • add input-plugin to receive data from systemd-journal-upload.service dd22fdd
  • Release 2.0.154 b89304d

2.0.153

9 April 2019

  • Add ibm-cloud k8s daemonset example ad4bdef
  • add option to tag App containers with LOGS_TOKEN or LOGSENE_TOKEN (backwardw compatible) cd841c8
  • Release 2.0.153 75205a8

2.0.152

4 April 2019

2.0.151

4 April 2019

2.0.150

4 April 2019

2.0.149

2 April 2019

2.0.148

2 April 2019

  • Release 2.0.148 0352d25
  • move back to node 8, until some Node 10 related problesm are solved 6d15011

2.0.147

2 April 2019

  • update patterns, scan time for new files, use tailF from GH a0859ea
  • Release 2.0.147 a9e0f43

2.0.146

2 April 2019

2.0.145

1 April 2019

2.0.144

1 April 2019

  • improve log output for terminate reason c8ac778
  • Release 2.0.144 70277f7

2.0.143

1 April 2019

2.0.142

1 April 2019

  • add k8s/containerd support, file based metadata from /var/log/containers 85a140e
  • set originalLine: false in patterns.yml 8339763
  • Release 2.0.142 48d5661

2.0.141

12 March 2019

2.0.140

12 March 2019

2.0.139

12 March 2019

2.0.138

8 March 2019

  • add module name for syslog plugin 02c68c4
  • top level 'env' section on config to set nodejs process environment variables 7b70980
  • add PID to error and debug log output b9be280

2.0.137

8 March 2019

  • use proxy settings for pattern downloads 33a6734
  • fix error handling for pattern downloads 9f12d34
  • Add Dockerfile for aarch64 21e6c7a

2.0.136

8 February 2019

  • add RedHat support 428500a
  • add multi-line pattern for yandex/clickhouse e784727
  • adjust pattern file location 41fef26

2.0.135

18 January 2019

2.0.134

16 January 2019

  • optimize pattern for sematext/agent go logs d95bebe
  • Release 2.0.134 c149f72

2.0.133

16 January 2019

  • support parsing logs and GO panics from sematext/agent 52a357e
  • Release 2.0.133 ff2548a

2.0.132

14 January 2019

  • Fix confusing 'file does not exists' message in setup script fa41b8b
  • add solr patterns + don't change order of patterns 7fdbd09
  • Release 2.0.132 7027d86

2.0.131

17 December 2018

  • add more ClickHouse connect options ec4f54a
  • clickhouse connection options, fix unhandled Promise error d13e59b
  • Release 2.0.131 a1cabc9

2.0.130

13 December 2018

  • add ARM support, Dockerfile.arm d521613
  • update node versions to test on travis-ci 59ae99a
  • Make LOG_GLOB not mandatory 61a26c1

2.0.128

29 November 2018

  • fix TAGGING_LABELS feature, with useful defaults de2418a
  • Release 2.0.128 9b93b5c

2.0.127

28 November 2018

  • Fix Layout in Docker Hub: Convert HTML table to markdown table 4c6cb24
  • Reduce the DH readme to a minimum. Referring to /docs bd31fd8
  • remove rtail, because rtail is not maintained anymore and causes problems with npm audit c758010

2.0.126

22 November 2018

2.0.125

16 November 2018

  • self.terminate() on uncaught exceptions #146
  • add MQTT Broker authentication bd118ea
  • terminate on uncaught exceptions and rejections bcf7e5f
  • Release 2.0.125 0a4bc52

2.0.124

5 November 2018

  • add ClickHouse output plugin 9b98c66
  • support date pattern in AWS Elasticsearch index name 1a731d6
  • Release 2.0.124 0a90e09

2.0.123

30 October 2018

  • Add output prometheus-alertmanager #142
  • Adding new line character in ldjson format #141
  • Update README.md 3f118c3
  • Delay require error 93af1af
  • Fix tests for older node versions fc68dd5

2.0.122

23 October 2018

  • update readme with configuration parameters efa31a5
  • Update Readme f29ca4f
  • change LA_ARGUMETNS to LOGAGENT_ARGUMENTS e6e5ec1

2.0.121

1 October 2018

  • bugfix / http listener for influxdb did not start 84fb9ba
  • Release 2.0.121 239bd56
  • remove bitHound badge d31d8d4

2.0.120

26 September 2018

  • SC-2853 LA config improvements #139
  • SC-2853 Rename LOG_URL to LOGSENE_RECEIVER_URL ae3b832
  • performance improvements 5ec5a1a
  • bugfix, not collecting files via LOG_GLOB pattern e166dae

2.0.119

14 September 2018

  • add kubernetes daemonset example 34b21e5
  • fix log routing bug, dockerInspect info was not set in logContext 69ade0b
  • Release 2.0.119 57c1f5b

2.0.118

13 September 2018

  • trigger docker log collection via CLI instead of generating cfg file 50ba9e3
  • make docker socket configurable 7bc81cc
  • fix docker socket detection/LA_ARGUMENTS and readme updates 2c949c1

2.0.117

11 September 2018

  • changes for Logagent docker image 9004a77
  • set Docker Hostname from docker-info 89a6411
  • Release 2.0.117 b730162

2.0.116

10 September 2018

2.0.115

7 September 2018

2.0.114

7 September 2018

  • elasticsearch output plugin: function as config value #135 #136
  • add docker logs input and enrichment plugin 7d71272
  • update patterns.yml 98b1c43
  • elasticsearch output plugin: config value as function #135 a57c68a

2.0.113

19 July 2018

  • add example removing fields with SQL output filter 7117949
  • Release 2.0.113 b6a4ae1
  • adjust timeout for JSON parse 7cf7987

2.0.112

6 July 2018

  • Add a new pattern to match Traefik default accesslog in 'patterns.yml' #129
  • Release 2.0.112 40a93d6

2.0.111

4 July 2018

  • Add a new pattern to match Traefik default accesslog in 'patterns.yml' 703246a
  • Release 2.0.111 cca71d1
  • fix version of date-fns a103f43

2.0.110

26 April 2018

  • Fix typo #127
  • split anonymizer plugin into separate plugins 7e28913
  • add example for anonymizer plugin c865863
  • add truncate function for ipv4/v6 fields 61ef2bb

2.0.109

28 March 2018

  • Create Cassandra CQL query input plugin #122 #126
  • Start Cassandra input 3158e41
  • add connection pooling configuration 537f6d5
  • start cassandra input plugin 01a8228

2.0.108

7 March 2018

  • add example elasticsearch http input to MQTT output c8e6b94
  • fix JSON detection 3ff2228
  • improved error handling for empty input or invalid JSON d96360a

2.0.107

6 March 2018

  • fix dockerfile for DH cde4dc7
  • add output-influxdb plugin 0d88ccf
  • add bulk requests + other minor changes 33a5bbf

2.0.106

21 February 2018

  • fix #125, allow negative file postions, relative from EOF #125
  • fix #125 #125
  • Release 2.0.106 a654a2a
  • add options.tailStartPosition in config loader a780cf3
  • link patterns.yml in dockerhub dir 9f0f61e

2.0.105

1 February 2018

2.0.104

24 January 2018

  • support global AWS credentials 70ae126
  • print unhandled promise rejections 139f4d4
  • fix loading AWS credentials ac9707f

2.0.103

23 January 2018

  • remove nova sensor & apple-tracker plugin 1bb66bf
  • Release 2.0.103 7a5f210

2.0.102

21 January 2018

2.0.101

21 January 2018

  • Add Apple device tracker input plugin b0cfd7f
  • add novaSDS011-dust-sensor plugin 49bb60e
  • replace Mosca MQTT broker with Aedes MQTT lib, add websocket option b585419

2.0.99

16 January 2018

  • changed findToken argument from Array to String bc3c8c0
  • Release 2.0.99 bb1f85c

2.0.98

15 January 2018

  • remove double timestamp for error logs using consoleLogger.error function f383580
  • Make loading of ES plugin backward compatible, and add warning bd5f80c
  • Small changes to Elasticsearch slowlog parsing 7e1cb68

2.0.97

10 January 2018

2.0.96

5 January 2018

2.0.95

22 December 2017

  • Add GELF out plugin + move alias grok filter in input filter section #111
  • optimize patterns/remove JS transforms for webserver logs, set originalLine to false 93fa49c
  • cpu tuning, replacing moment.js with date-fns 0573785
  • rename addr field in hosts and add example c5cecf0

2.0.93

8 December 2017

  • Release 2.0.93 cb2dc02
  • fix debug condition in parser for json.transform 36320f6

2.0.92

8 December 2017

2.0.91

8 December 2017

  • fix #110 #110
  • add gelf output 4fc6e90
  • removed duplicated pattern/ better formatting, setting types 0f54d9c
  • add /ping /health and JSON parsing for message field 7d1a8a9

2.0.90

1 December 2017

2.0.89

28 November 2017

  • move graygelf (having native dependencies) to optionalDependencies section 541754b
  • Release 2.0.89 1c2ad46
  • Update cloudflare.yml ef949ad

2.0.88

23 November 2017

  • Add alias for grok filter #107
  • Add a new docker image and README updates #104
  • Fix typo #101
  • Add GELF Input Plugin #99
  • Implement Elasticsearch Index API #92
  • kafka plugins are external, removed local JS files 76731f4
  • apply 'standard-format -w' and fix eslint issues acf2782
  • implement dynamic response for N documents 88bc066

2.0.87

13 October 2017

2.0.86

13 October 2017

  • add Infux HTTP API input ae21ec4
  • fix HTTP return status for telegraf, add tags to separate object c6ee6fa
  • rename input-influx-http to input-influxdb-http 66418bf

2.0.85

6 October 2017

  • add example to fetch logs from cloudflare 0f2a1da
  • add and variables for command templates 5ec4a7a
  • Release 2.0.85 1ea89a9

2.0.84

21 September 2017

2.0.83

21 September 2017

  • add aws-elasticsearch.js skeleton 54bed7e
  • fix iitial bugs, add ES client tracing optiosn to config, log errors to stderr e730c45
  • add aws-elasticsearch example, support aws-config.json file e4e940e

2.0.82

21 September 2017

  • support template format for output-file plugin 14af15e
  • Release 2.0.82 aefc79a

2.0.81

20 September 2017

  • Release 2.0.81 1b7b696
  • rename ENABLE_JSON_ENRICHMENT -> JSON_ENRICHMENT_ENABLED d175e5a

2.0.80

20 September 2017

2.0.79

20 September 2017

2.0.78

20 September 2017

  • fix format of example config ce31618
  • add env var ENABLE_JSON_ENRICHMENT (default = true) 1349f22
  • Update output-files.yml 25e2751

2.0.77

19 September 2017

2.0.76

12 September 2017

2.0.75

12 September 2017

  • remove documentation -> sematext.com/docs/logagent 562add0
  • Update example.yml 9418c20
  • Cosmetics 1c52c06

2.0.74

22 August 2017

  • add Elasticsearch slow log pattern f67b799
  • Release 2.0.74 62c98db
  • fix start pos for tailing files adddb25

2.0.73

9 August 2017

  • Access Watch output filter #83
  • added possibility to choose different timezones and formats for query… #82
  • Use Kafka-node instead of kafka-native for in out and ssl support #81
  • added mssql-query plugin #80
  • Kafka input support with native #77
  • fix #72 #72
  • added mssql-query input plugin 54e49aa
  • remove wrong package file 11c0313
  • first working version, augment data form client_ip and user_agent 510e5a9

2.0.72

6 July 2017

  • add US/EU comment for setup call 8dfbb8d
  • Release 2.0.72 7581316
  • update dependency to logsene-js f1858a2

2.0.71

5 July 2017

  • postgres-query #36 #78
  • Add mulitple sql statements #76
  • Add date placeholder #74
  • add producer for try plugin 457311b
  • rename postgres-> postgresql, improved connection/error handling 520e1f5
  • improve connection handling open/close b1f7459

2.0.70

27 June 2017

2.0.69

27 June 2017

  • add mysql-query plugin #73
  • fix #72 #72
  • complete firt kafka input roundtrip 68a1895
  • add example for Prometheus to Elasticsearch 3b40620
  • add shhudown kafka endpoint b60dd84

2.0.68

23 June 2017

  • add options to setup-script for ES url (-u), glob pattern (-g) and index name (-i) 54a7c57
  • Release 2.0.68 2a9230a

2.0.66

8 June 2017

  • add token blacklist for cf/heroku receiver 2d162b7
  • Release 2.0.66 bc31a75

2.0.65

8 June 2017

2.0.64

6 June 2017

  • fix tailF, removing separator from constructor call e8665b9
  • Release 2.0.64 e7c80d4
  • Update elasticsearch-query-input.yml 8bf57a2

2.0.63

9 May 2017

  • fix datetime not matching on nginx #68
  • make default multi-line separator configurable 15873d7
  • Release 2.0.63 3cec3aa
  • use platform specific EOL bddbc2f

2.0.62

28 April 2017

2.0.61

28 April 2017

  • Re Issue #64 Logagent-setup fails to create init.d script #66
  • fix #64 #64
  • update docs 2485bc2
  • add slack-webhook output f7a3d09
  • add elasticsearch query input 7775b43

2.0.60

12 April 2017

2.0.59

20 March 2017

2.0.58

13 March 2017

  • Fix initial maxmind update #62
  • Release 2.0.58 746c105

2.0.57

7 March 2017

  • typo fix for word "example" #61
  • Release 2.0.56 2e485b6
  • Release 2.0.57 5c00383
  • reduce memory usage using --max-old-space-size=150 as default (500MB before) 5dbe012

2.0.56

16 February 2017

  • Release 2.0.56 c3323ea
  • fix 'depcrated os.tempdir()' message 10b7449
  • add precommit for standard/eslint e01abb3

2.0.55

10 February 2017

  • [WIP] add eslintrc config, fix various issues identified by eslint #60
  • Add support for time-based index patterns, prefer severity over level #59
  • add eslintrc config, fix various issues identified by eslint 9d65cc0
  • fix standard-format warnings bf99dee
  • support time-based elasticsearch indexes, prefer severity to level 0642c6e

2.0.54

9 February 2017

  • update examples and docs for elasticsearch output module 6063ae0
  • Release 2.0.54 40eef86

2.0.53

9 February 2017

  • assorted odds & ends #58
  • Bunyan log parsing #57
  • add test for bunyan log handling bd47ff0
  • fallback for filter name, initPugins -> initPlugins, outputFilterSections -> inputFilterSections, != -> !== 865f397
  • properly detect bunyan log messages, remove bunyan time when after setting @timestamp 2d98571

2.0.52

9 February 2017

2.0.51

9 February 2017

  • fix undefined sourceName when using command input #55
  • Release 2.0.51 bf1e21d
  • add node 7 remove node 5 f3fd375
  • remove node 0.10 from travis 32a6842

2.0.50

7 February 2017

  • fix null coordinates with GeoLite2 records cd9d863
  • Release 2.0.50 e7b5eb4

2.0.49

7 February 2017

2.0.48

7 February 2017

2.0.47

7 February 2017

2.0.46

6 February 2017

  • Revert "reset tail-forever every N minutes" d1c5285
  • reset tail-forever every N minutes ba8193a
  • fix --geoipEnabled flag 04f4491

2.0.45

22 January 2017

2.0.44

17 January 2017

  • add example to store BRO ids logs in Elasticsearch a54c1e1
  • update for Elasticsearch 5 timestamp format 56c575d
  • fix typo in bro-ids.yaml fef2651

2.0.43

2 January 2017

2.0.42

12 December 2016

2.0.41

8 December 2016

  • update example for journald 7bac233
  • add command stout/sterr as input plugin 'command' 35716c4
  • example for journald input 1d9c452

2.0.40

7 December 2016

  • add timerEvents to share one timer for all multi-line parsers f38c287
  • user timerEvent in multi-line parser 03dd407
  • Release 2.0.40 2a64224

2.0.39

6 December 2016

2.0.38

6 December 2016

2.0.37

6 December 2016

  • standard format, removed debug output d029e9f
  • add windows service installer dacbd09
  • Release 2.0.37 c3ae63f

2.0.36

5 December 2016

2.0.35

2 December 2016

  • Release 2.0.35 81a54b1
  • match against data.logSource and context.sourceName 02f2a41

2.0.34

2 December 2016

2.0.33

2 December 2016

2.0.32

2 December 2016

2.0.31

15 November 2016

2.0.30

15 November 2016

  • update package name in uninstall script 6353ab5
  • Release 2.0.30 7ecb3ce

2.0.29

11 November 2016

  • add httpOptions to elasticsearch output plugin 1a18279
  • Release 2.0.29 befe268

2.0.28

4 November 2016

  • output to multiple Elasticsearch servers, terminate after stdin.close and all logs shipped 3276989
  • Release 2.0.28 b29ddc3

2.0.27

2 November 2016

  • integrate tcp-input, grep filter, update sql filter, add docs da7a36e
  • fix pattern list by source bc4738e
  • update docs 75dfcf7

2.0.26

2 November 2016

2.0.25

24 October 2016

  • Release 2.0.25 1380ab1
  • add module alias table, e.g. to load sql output filter with a short alias 'sql' instead of ../lib/plugins/output-filter/sql.js 4fecd56

2.0.24

23 October 2016

2.0.23

17 October 2016

2.0.22

17 October 2016

2.0.21

17 October 2016

  • fix #45, rtail plugin not loaded #45
  • change error message 3ef3c0a
  • Release 2.0.21 8770870

2.0.20

7 October 2016

  • support filter/aggregate functions, using fibers 4d9e0c5
  • switch to enable JSON processing, JSON detection/parsing after pattern processing a6ef4aa
  • fix error when no settings for stdin plugin exist in config file 68ff4c1

2.0.19

4 October 2016

2.0.17

26 September 2016

2.0.16

23 September 2016

  • make sure patterns are matched from beginning -> better performance f253edf
  • revert a00d2bf33b0f6c34a57bf44e519ef6f2f1514c1b, load always default patterns 1c02f6c
  • don't load default patterns, when user configured pattern files a00d2bf

2.0.15

23 September 2016

2.0.14

19 September 2016

2.0.13

13 September 2016

  • add json post processing options (autohash, transform) in parser 1d5af28
  • example for json post processing b188f62
  • json post process example in comments 9655907

2.0.12

13 September 2016

2.0.11

13 September 2016

2.0.10

13 September 2016

2.0.9

13 September 2016

  • add putput plugin example 5476575
  • move install section to the top, reference to service installer and location of default config file bf97b4c
  • link to existing plugins 6c4d369

2.0.8

13 September 2016

2.0.7

13 September 2016

  • link from whats new to docs 9f7463b
  • add multiple pattern files, hot reload 346d2ad
  • Release 2.0.7 1899c61

2.0.6

13 September 2016

  • Release 2.0.6 5fe9a67
  • remove npm badge - not working for scoped packages e5a0a3d

2.0.5

12 September 2016

  • update docs a348714
  • fix cli without config file 6ad1a5f
  • load 3rd party modules first, e.g. for nodejs monitor too hook into http functions before heroku/cf server gets loaded 1117b43

2.0.4

8 September 2016

  • update cli options 3c58aa4
  • Release 2.0.4 74b7668
  • remove warning on parser errors, too verbose e.g. for empty lines 46026df

2.0.2

8 September 2016

  • load only required plugins/modules depending on config b2db631
  • don't terminate on stdin/ EOF when other input plugins are loaded 53d63e1
  • include pid in logs e02ab45

2.0.1

8 September 2016

  • refactoring, new dir structure, fix #35 #35
  • changes for #28, move to eventemitter2, started with plugin structure b52a22f
  • refactoring, split in modules, prep. for cli tests 36ce8da
  • code format, copyright statement a93ebde

1.2.62

31 May 2016

  • Fixed originalLine typo #8
  • fixed a typo in patterns.yml #6
  • structured logs dir b9451c2
  • initial version 3e5fe01
  • add source for mkdocs b1fe290

1.2.12

2 July 2016

  • Allow specifying the file pattern as $2 #10
  • omproved logging, add --help, moced to commander CLI option parser 02ebe51
  • improved log output, scanning files for glob patterns a249780
  • Store tail position and recover on restart. Incl. indode check to start at begin of file, when inode changed 79cf001

1.1.64

1 June 2016

  • add optional type for fields e.g. fieldname:string / fieldname:number c5078a5
  • version bump 51de36e

1.1.63

31 May 2016

  • add optional log messages for cf/heroku mode and stats af3c57b
  • version bump 7ee88ac