Logging – Part Four: Centralized Logging on ESXi Hosts

In vSphere 5 logging is standardised using Syslog for all logging, to handle log messages from the VMkernel, daemons, logger program and other programs and processes. For remote logging, log messages may be sent to a centralized logging system  and system panics can be sent to a remote dump collector. This allows for troubleshooting on the remote Syslog server due to the nature of log files being stored in memory if the ESXi host system is rebooted or crashes, the local log files are not available.

The VMkernel, daemons and programs will send log messages to the /dev/klog socket to which the vmsyslogd queries for incoming messages and on receipt the daemon is required to know what to do with the information.  This is handled by the configuration of the ESXi host system which can be modified using one of the following methods.

Configuring Syslog for an ESXi host using vSphere Web Client

1) Select the ESXi host system and browse to Manage > Settings > Advanced Settings

2) Modify one or more of the following syslog configuration options as below:

Name Value
Syslog.global.logDir Location to store local or remote syslog files
Syslog.global.logHost Destination host for remote log files
Syslog.global.logDirUnique A Boolean value to determine if a child directory for the ESXi host system is created
Syslog.global.defaultRotate Number of log files retainined on the local ESXi host system
Syslog.global.defaultSize Causes the log file to be rotated when it reaches the default size

 

Configuring Syslog for an ESXi host using the esxcli command line

1) Connect to an ESXi host system using a SSH client.

2) In order to modify the Syslog configuration using the esxcli system syslog namespace we can use one of the following set options:

--default-rotate=<long> Default number of rotated local logs to keep
--default-size=<long> Default size of local logs before rotation, in KiB
--default-timeout=<long> Default network retry timeout in seconds if a remote server fails to respond
--logdir=<str> The directory to output local logs to
--logdir-unique Place logs in a unique subdirectory of logdir, based on hostname
--loghost=<str> The remote host(s) to send logs to
--reset=<str> Reset values to default

For Example, if we are required to configure the size of the log file before rotation to be 2048 KiB, we can invoke the following:

esxcli system syslog config set --default-size=2048

3) In order to apply the configuration change the vmsyslogd is required to be restarted to load the changes into memory, as below:

esxcli system syslog reload

Installing and Configuring VMware Syslog Collector 

To configure a location for long term storage of log files we can install and configure a VMware Syslog Collector which on a Windows vCenter Server can be installed from the VMware vSphere installer media as follows:

1) Specify the install directory, repository location, size of the log file before rotation and the log rotation to keep and select Next.

Syslog-1

 

 

 

 

 

 

2) Select the installation type for the Syslog Collector and select Next. In this example we will be selecting a VMware vCenter Server installation.

Syslog-2

 

 

 

 

 

 

3) Specify the IP address, HTTP port and login credentials for the vCenter Server system and select Next.

Syslog-3

 

 

 

 

 

 

4) Specify the Syslog Collector port settings and select Next.

Syslog-4

 

 

 

 

 

 

5) Specify how the Syslog Collector should be identified on the network and select Next.

Syslog-5

 

 

 

 

 

 

6) Select Install.

Now that we have installed and configured the Syslog Collector we can now configure the ESXi host system to use a remote host. Firstly, we should confirm that the ESXi host system can communicate with the Syslog Collector as by default the Firewall service will enable outbound connections on TCP and UDP service ports 514.

We can confirm the connectivity on the ESXi host system by invoking the following command to determine if the host system can communicate with the remote host on TCP service port 514.

nc -z deanvc1.dean.local 514

Once we have verified a successful connection we can configure the syslog settings as follows:

esxcli system syslog config set --loghost=tcp://deanvc1.dean.local:514
esxcli system syslog reload

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s