Anyone who’s ever dealt with system logs knows the feeling: staring at a seemingly endless stream of text, desperately searching for that one clue that solves your problem. Log files are essential for troubleshooting, but navigating them can be a frustrating experience.

Enter LNav, a powerful open-source log file navigator designed to make sense of the chaos. Here’s why you, as an AEM developer or administrator, should consider adding LNav to your toolkit:

Benefits of Using LNav with AEM and Dispatcher Logs: Link to heading

  • AEM-specific parsing: The custom format enables LNav to understand the structure of AEM and Dispatcher logs. This means you can filter and search based on fields like request methods, URLs, components, and requesters, making it easier to isolate AEM-related issues.
  • Real-time troubleshooting: Monitor your AEM and Dispatcher logs in real-time. Catch errors and identify potential problems as they occur, allowing for quicker intervention and resolution.
  • Advanced search capabilities: LNav’s powerful search functionality lets you pinpoint specific events within your logs. You can use regular expressions or keywords to target errors related to specific components, users, or functionalities.
  • Log file consolidation: LNav allows you to open and view multiple log files simultaneously. This is particularly useful when troubleshooting issues that span across AEM and Dispatcher logs.
  • Filter out the noise: With LNav’s filtering capabilities, you can focus on the information that matters. Filter out irrelevant messages and system logs, keeping your view focused on AEM and Dispatcher-specific events.
  • Custom format magic: By defining a custom format for AEM and Dispatcher logs, LNav can intelligently extract relevant fields like timestamps, log levels, components, and messages.
  • Targeted insights: This targeted parsing allows you to filter and search based on AEM-specific details, leading you straight to the information you need.

The LNav AEM/Dispatcher Custom Format: Link to heading

Here’s a sample LNav custom format specifically designed for AEM and Dispatcher logs. You can copy and paste this code into a file named aem_format.json and save it in your preferred location.

{
    "$schema": "https://lnav.org/schemas/format-v1.schema.json",
    "aem": {
        "description": "Format created by Abdul Munim - https://www.munim.net",
        "regex": {
            "default_request": {
                "pattern": "^(?<timestamp>\\d{2}\\.\\d{2}\\.\\d{4} \\d{2}:\\d{2}:\\d{2}\\.\\d{3}) \\*(?<level>\\w+?)\\* \\[(?<requester>[\\w\\-\\.]+) \\[(?<request_id>\\w+)\\] (?<request_method>\\w+) (?<request_url>.+?(?= HTTP)) (?<req_version>.+)\\] (?<executor>\\S+?) (?<body>.*)$"
            },
            "replicator": {
                "pattern": "^(?<timestamp>\\d{2}\\.\\d{2}\\.\\d{4} \\d{2}:\\d{2}:\\d{2}\\.\\d{3}) \\*(?<level>\\w+?)\\* \\[\\[(?<requester>[\\w\\-\\.\\:]+)\\] \\[(?<request_id>\\w+)\\] (?<request_method>\\w+) (?<request_url>.+?(?= HTTP)) (?<req_version>.+)\\] (?<executor>\\S+?) (?<body>.*)$"
            },
            "internal_services": {
                "pattern": "(?<timestamp>\\d{2}\\.\\d{2}\\.\\d{4} \\d{2}:\\d{2}:\\d{2}\\.\\d{3}) \\*(?<level>\\w+?)\\* \\[(?<requester>[^\\[\\]]+?)\\] (?<executor>\\S+?) (?<body>.*$)"
            }
        },
        "value": {
            "level": {
                "kind": "string",
                "identifier": true
            },
            "req_version": {
                "kind": "string",
                "identifier": true
            },
            "request_id": {
                "kind": "string"
            },
            "request_method": {
                "kind": "string",
                "identifier": true
            },
            "request_url": {
                "kind": "string",
                "identifier": true
            },
            "requester": {
                "kind": "string",
                "identifier": true
            },
            "executor": {
                "kind": "string",
                "identifier": true
            },
            "body": {
                "kind": "string"
            }
        },
        "level-field" : "level",
		"level" : {
			"error" : "ERROR",
			"warning" : "WARN",
			"debug" : "DEBUG",
			"info" : "INFO"
		},
        "timestamp-format": [
            "%d.%m.%Y %H:%M:%S.%L"
        ],
        "sample": [
            {
                "line": "09.06.2023 09:38:31.380 *ERROR* [3.252.47.23 [1686303509939] POST /bin/example/search HTTP/1.1] net.munim.example.aem.internal.services.search.v2.SearchService Error in tag : {}"
            },
            {
                "line": "09.06.2023 01:40:25.027 *INFO* [[0:0:0:0:0:0:0:1] [1686274825025] GET /etc/replication/agents.author/publish1northeurope.test.html HTTP/1.1] com.day.cq.replication.impl.ReplicatorImpl Setting up replication with options: ReplicationOptions{synchronous=true, revision='null', suppressStatusUpdate=true, suppressVersions=true, filter=com.day.cq.replication.AgentIdFilter@1641b893, aggregateHandler=null}"
            },
            {
                "line": "09.06.2023 00:00:00.022 *INFO* [sling-default-5-com.day.cq.dam.similaritysearch.internal.scheduler.PeriodicAutoTaggingJob.4517] com.day.cq.dam.similaritysearch.internal.scheduler.PeriodicAutoTaggingJob Smart Tags not configured. Ignoring periodic job."
            }
        ]
    }
}

Once you have saved the aem_format.json file, launch LNav and use the following command to install the custom format:

lnav -i aem_format.json

This command instructs LNav to use the definitions in aem_format.json for parsing your AEM and Dispatcher logs.

Now you can browse your logs by running:

lnav /path/to/your/file/or/directory

LNav Keyboard Navigation:

LNav offers a variety of keyboard shortcuts for efficient navigation and interaction. Here are some commonly used ones:

  • Up/Down arrow keys: Navigate through log entries.
  • / (forward slash): Open the search bar to filter or search for specific terms.
  • h: Display help information for available commands.
  • q: Quit LNav.

With the custom format and keyboard shortcuts at your disposal, you’ll be navigating AEM and Dispatcher logs with newfound ease!

There might be some learning curve with LNav, but once you get used to it, log navigation is going to be like a breeze. For more information and tutorial, please have a look at the LNav documentation.