Journalctl to View and Manipulate Systemd Logs

Journalctl-View-Manipulate-Systemd-Logs Using Journalctl to Display and Control Logs

What is Journalctl?

Sort of like systemctl, journalctl may be used to query the contents of the systemd services. Journal comprises of one or more binary files, journalctl is the standard way of reading messages from it.

​Example Journalctl Output

 Example of the output is below for journalctl without parameters

-- Logs begin at Sun 2018-04-15 16:02:40 PDT, end at Wed 2018-04-18 10:26:23 PDT. --
Apr 15 16:02:40 crypted kernel: Linux version 4.14.0 (root@crypted) (gcc version 6.3.0 20170516 (Debian 6.3.0-18)) #4 SMP Sat Feb 17 17:20:32 P
Apr 15 16:02:40 crypted kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-4.14.0 root=UUID=ccf86bf6-6269-4e44-861d-61304dcf09d9 ro nomodeset amd_i
Apr 15 16:02:40 crypted kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Apr 15 16:02:40 crypted kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Apr 15 16:02:40 crypted kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Apr 15 16:02:40 crypted kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
Apr 15 16:02:40 crypted kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
Apr 15 16:02:40 crypted kernel: e820: BIOS-provided physical RAM map:
Apr 15 16:02:40 crypted kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009d3ff] usable
Apr 15 16:02:40 crypted kernel: BIOS-e820: [mem 0x000000000009d400-0x000000000009ffff] reserved
...

Show Current Boot Messages

Show messages from a specific boot. This will add a match for "_BOOT_ID=" 

The argument may be empty, in which case logs for the current boot will be shown.

-- Logs begin at Sun 2018-04-15 16:02:40 PDT, end at Wed 2018-04-18 10:26:23 PDT. --
Apr 15 16:02:40 crypted kernel: Linux version 4.14.0 (root@crypted) (gcc version 6.3.0 20170516 (Debian 6.3.0-18)) #4 SMP Sat Feb 17 17:20:32 P
Apr 15 16:02:40 crypted kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-4.14.0 root=UUID=ccf86bf6-6269-4e44-861d-61304dcf09d9 ro nomodeset amd_i
Apr 15 16:02:40 crypted kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Apr 15 16:02:40 crypted kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Apr 15 16:02:40 crypted kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Apr 15 16:02:40 crypted kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
Apr 15 16:02:40 crypted kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
Apr 15 16:02:40 crypted kernel: e820: BIOS-provided physical RAM map:
Apr 15 16:02:40 crypted kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009d3ff] usable
Apr 15 16:02:40 crypted kernel: BIOS-e820: [mem 0x000000000009d400-0x000000000009ffff] reserved
...

Boot Messages using Ellipsize fields

Using -b and -l, -l meaning lower case "L". The default is to show full fields, allowing them to wrap or be truncated by the pager, if one is used. 

-- Logs begin at Sun 2018-04-15 16:02:40 PDT, end at Wed 2018-04-18 10:39:01 PDT. --
Apr 15 16:02:40 crypted kernel: Linux version 4.14.0 (root@crypted) (gcc version 6.3.0 20170516 (Debian 6.3.0-18)) #4 SMP Sat Feb 17 17:20:32 P
Apr 15 16:02:40 crypted kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-4.14.0 root=UUID=ccf86bf6-6269-4e44-861d-61304dcf09d9 ro nomodeset amd_i
Apr 15 16:02:40 crypted kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Apr 15 16:02:40 crypted kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Apr 15 16:02:40 crypted kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Apr 15 16:02:40 crypted kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
Apr 15 16:02:40 crypted kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
Apr 15 16:02:40 crypted kernel: e820: BIOS-provided physical RAM map:
Apr 15 16:02:40 crypted kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009d3ff] usable

Getting Boot ID to List

0 b52daff63e8f4ba0a7b48eb7ef7e19de Sun 2018-04-15 16:02:40 PDT—Wed 2018-04-18 10:40:01 PDT

List Journalctl Catalogs

0027229ca0644181a76c4e92458afa2e systemd: One or more messages could not be forwarded to syslog
1675d7f172174098b1108bf8c7dc8f5d systemd: DNSSEC validation failed
1dee0369c7fc4736b7099b38ecb46ee7 systemd: Mount point is not empty
24d8d4452573402496068381a6312df2 systemd: A virtual machine or container has been started
3354939424b4456d9802ca8333ed424a systemd: Session @SESSION_ID@ has been terminated
36db2dfa5a9045e1bd4af5f93e1cf057 systemd: DNSSEC mode has been turned off, as server doesn't support it
39f53479d3a045ac8e11786248231fbf systemd: Unit @UNIT@ has finished start-up
45f82f4aef7a4bbf942ce861d1f20990 systemd: Time zone change to @TIMEZONE@
4d4408cfd0d144859184d1e65d7c8a65 systemd: A DNSSEC trust anchor has been revoked
58432bd3bace477cb514b56381b8a758 systemd: A virtual machine or container has been terminated
5aadd8e954dc4b1a8c954d63fd9e1137 systemd: Core file was truncated to @SIZE_LIMIT@ bytes.
641257651c1b4ec9a8624d7a40a9e1e7 systemd: Process @EXECUTABLE@ could not be executed
6bbd95ee977941e497c48be27c254128 systemd: System sleep state @SLEEP@ entered
7b05ebc668384222baa8881179cfda54 systemd: Unit @UNIT@ has finished reloading its configuration
7d4958e842da4a758f6c1cdc7b36dcc5 systemd: Unit @UNIT@ has begun start-up
8811e6df2a8e40f58a94cea26f8ebf14 systemd: System sleep state @SLEEP@ left
8d45620c1a4348dbb17410da57c60c66 systemd: A new session @SESSION_ID@ has been created for user @USER_ID@
98268866d1d54a499c4e98921d93bc40 systemd: System shutdown initiated
9d1aaa27d60140bd96365438aad20286 systemd: Unit @UNIT@ has finished shutting down
a596d6fe7bfa4994828e72309e95d61e systemd: Messages from a service have been suppressed
b07a249cd024414a82dd00cd181378ff systemd: System start-up is now complete
be02cf6855d2428ba40df7e9d022f03d systemd: Unit @UNIT@ has failed
c7a787079b354eaaa9e77b371893cd27 systemd: Time change
d34d037fff1847e6ae669a370e694725 systemd: Unit @UNIT@ has begun reloading its configuration
d93fb3c9c24d451a97cea615ce59c00b systemd: The journal has been stopped
de5b426a63be47a7b6ac3eaac82e2f6f systemd: Unit @UNIT@ has begun shutting down
e7852bfe46784ed0accde04bc864c2d5 systemd: Seat @SEAT_ID@ has now been removed
e9bf28e6e834481bb6f48f548ad13606 systemd: Journal messages have been missed
ec387f577b844b8fa948f33cad9a75e6 systemd: Disk space used by the journal
f77379a8490b408bbe5f6940505a777b systemd: The journal has been started
fc2e22bc6ee647b6b90729ab34a250b1 systemd: Process @COREDUMP_PID@ (@COREDUMP_COMM@) dumped core

Display Journalctl Log Ranges

The below example gets logs since 3 hours ago on the current system 

-- Logs begin at Sun 2018-04-15 16:02:40 PDT, end at Wed 2018-04-18 11:07:31 PDT. --
Apr 18 08:09:00 crypted systemd[1]: Starting Clean php session files...
Apr 18 08:09:01 crypted systemd[1]: Started Clean php session files.
Apr 18 08:09:01 crypted CRON[88019]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 18 08:09:01 crypted CRON[88020]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessi
Apr 18 08:09:01 crypted CRON[88019]: pam_unix(cron:session): session closed for user root
Apr 18 08:10:01 crypted CRON[88150]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 18 08:10:01 crypted CRON[88151]: (root) CMD (/usr/local/maldetect/maldet --mkpubpaths >> /dev/null 2>&1)
Apr 18 08:10:02 crypted CRON[88150]: pam_unix(cron:session): session closed for user root
Apr 18 08:13:53 crypted gvfsd-network[106285]: Couldn't create directory monitor on smb://x-gnome-default-workgroup/. Error: Operation not supp
Apr 18 08:15:01 crypted CRON[88779]: pam_unix(cron:session): session opened for user root by (uid=0)

Display Journalctl Log by Time & Date

​Checking logs on 17th and 18th of April 2018 @ 00:00 until 01:00 would show like below example

-- Logs begin at Sun 2018-04-15 16:02:40 PDT, end at Wed 2018-04-18 11:17:01 PDT. --
Apr 17 00:00:02 crypted CRON[36541]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 17 00:00:02 crypted CRON[36542]: (root) CMD (/usr/local/maldetect/maldet --mkpubpaths >> /dev/null 2>&1)
Apr 17 00:00:02 crypted CRON[36541]: pam_unix(cron:session): session closed for user root
Apr 17 00:00:10 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:00:18 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:00:26 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:00:35 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:00:46 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:00:56 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:01:04 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:01:16 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:01:31 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:01:52 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:02:03 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:02:12 crypted dhclient[10877]: DHCPREQUEST of 192.168.1.15 on wlp4s0 to 192.168.1.1 port 67
Apr 17 00:02:24 crypted systemd[1]: Started Run anacron jobs.
Apr 17 00:02:24 crypted anacron[37036]: Anacron 2.3 started on 2018-04-17
Apr 17 00:02:24 crypted anacron[37036]: Will run job `cron.daily' in 5 min.
Apr 17 00:02:24 crypted anacron[37036]: Jobs will be executed sequentially

Journalctl Display Logs By Unit

Show messages for the specified systemd unit UNIT (such as a service unit), or for any of the units matched by PATTERN. If a pattern is specified, a list of unit names found in the journal is compared with the specified pattern and all that match are used. 

-- Logs begin at Sun 2018-04-15 16:02:40 PDT, end at Wed 2018-04-18 11:20:01 PDT. --
Apr 15 16:02:52 crypted systemd[1]: Starting Plex Media Server for Linux...
Apr 15 16:02:52 crypted systemd[1]: Started Plex Media Server for Linux.
Apr 15 16:03:37 crypted sh[1020]: Sun Apr 15 16:03:37 2018: INFO > resolvers : Adding Host direct to Interface
Apr 15 16:03:40 crypted sh[1020]: /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-ins/FMoviesPlus.bundle/Contents/L
Apr 15 16:03:40 crypted sh[1020]: InsecureRequestWarning)
Apr 15 16:03:40 crypted sh[1020]: Sun Apr 15 16:03:40 2018: INFO > resolvers : Adding Host gvideo to Interface
Apr 15 16:03:42 crypted sh[1020]: [16B blob data]

You could also specify multiple units from a single command line 


Showing Live Journalctl Messages

 Show only the most recent journal entries, and continuously print new entries as they are appended to the journal.


List Journalctl Logs Newest Entries

Reverse output so that the newest entries are displayed first. 

-- Logs begin at Sun 2018-04-15 16:02:40 PDT, end at Wed 2018-04-18 11:35:02 PDT. --
Apr 15 16:03:02 crypted systemd[1]: Started A high performance web server and a reverse proxy server.
Apr 15 16:02:52 crypted systemd[1]: Starting A high performance web server and a reverse proxy server...

Display Logs Today Only

Get logs produced today by using below command 

You could use both --since and --until like below pretty much just like -S and -U but it doesn't work with -S today, only --since today 


Journalctl Output Formats

Options below 

short - is the default and generates an output that is mostly identical to the formatting of classic syslog files, showing one line per journal entry.

short-full - is very similar, but shows timestamps in the format the --since= and --until= options accept. Unlike the timestamp information shown in short output mode this mode includes weekday, year and timezone information in the output, and is locale-independent.

short-iso - is very similar, but shows ISO 8601 wallclock timestamps.

short-precise - is very similar, but shows timestamps with full microsecond precision.

short-monotonic - is very similar, but shows monotonic timestamps instead of wallclock timestamps.

short-unix - is very similar, but shows seconds passed since January 1st 1970 UTC instead of wallclock timestamps ("UNIX time"). The time is shown with microsecond accuracy.

verbose - shows the full-structured entry items with all fields.

export - serializes the journal into a binary (but mostly text-based) stream suitable for backups and network transfer (see Journal
Export Format[1] for more information).

json - formats entries as JSON data structures, one per line (see Journal JSON Format[2] for more information).

json-pretty - formats entries as JSON data structures, but formats them in multiple lines in order to make them more readable by humans.

json-sse - formats entries as JSON data structures, but wraps them in a format suitable for Server-Sent Events[3].

cat - generates a very terse output, only showing the actual message of each journal entry with no metadata, not even a timestamp.


Display High Priority Journalctl Messages

List only 20 lines with priority "critical" on all logs in system 

List 20 lines with high priority "critical" on only 1 unit specified below

  • 0: emerg
  • 1: alert
  • 2: crit
  • 3: err
  • 4: warning
  • 5: notice
  • 6: info
  • 7: debu

Display Only Kernel Messages

Show only 10 lines from kernel only.  -k Show only kernel messages. This implies -b and adds the match "_TRANSPORT=kernel".

-- Logs begin at Wed 2018-04-11 09:12:19 PDT, end at Wed 2018-04-18 11:51:01 PDT. --
Apr 11 16:57:49 Linuxsecrets.com kernel: perf: interrupt took too long (3929 > 3921), lowering kernel.perf_event_max_sample_rate to 50750
Apr 15 10:51:34 Linuxsecrets.com kernel: loop: module loaded
Apr 15 10:51:37 Linuxsecrets.com kernel: bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
Apr 15 10:51:37 Linuxsecrets.com kernel: Bridge firewalling registered
Apr 15 10:51:37 Linuxsecrets.com kernel: Initializing XFRM netlink socket
Apr 15 10:51:37 Linuxsecrets.com kernel: Netfilter messages via NETLINK v0.30.
Apr 15 10:51:37 Linuxsecrets.com kernel: ctnetlink v0.93: registering with nfnetlink.
Apr 15 10:51:38 Linuxsecrets.com kernel: IPv6: ADDRCONF(NETDEV_UP): docker0: link is not ready
Apr 15 12:54:47 Linuxsecrets.com kernel: perf: interrupt took too long (4913 > 4911), lowering kernel.perf_event_max_sample_rate to 40500
Apr 15 13:04:12 Linuxsecrets.com kernel: perf: interrupt took too long (6153 > 6141), lowering kernel.perf_event_max_sample_rate to 32500

The user or service ID, below if nginx ID to display logs for nginx 

109

From ID:109 we can now get the logs 

To see which group IDs the systemd journal has entries for, you can type 

127
0
110
135

Display Kernel Logs from Previous Boot


Display Logs from D-Bus Executable


Truncating Logs by Size or Time

Delete logs if logs are more than 2G 

Delete journal logs if older then 2 years old 


Limiting Journal Expansion

Following commands can be used to limit journal log growth

  • SystemMaxUse=: Specifies the maximum disk space that can be used by the journal in persistent storage.
  • SystemKeepFree=: Specifies the amount of space that the journal should leave free when adding journal entries to persistent storage.
  • SystemMaxFileSize=: Controls how large individual journal files can grow to in persistent storage before being rotated.
  • RuntimeMaxUse=: Specifies the maximum disk space that can be used in volatile storage (within the /run filesystem).
  • RuntimeKeepFree=: Specifies the amount of space to be set aside for other uses when writing data to volatile storage (within the /run filesystem).
  • RuntimeMaxFileSize=: Specifies the amount of space that an individual journal file can take up in volatile storage (within the /run filesystem) before being rotated.

That covers the default syntax for system administration for logs using journalctl. Note: You can use all parameters in combination using journalctl [OPTIONS].   

If you found this article useful or any other on Linuxsecrets.com please donate belong.


Font size: +
Report Print
 

By accepting you will be accessing a service provided by a third-party external to https://www.linuxsecrets.com/