In some environments NTP (Network Time Protocol) may be restricted to an internally managed server or limited to pre-determined public NTP servers. These definitions as set by the system administrator to meet company security guidelines or general policy. In most cases, all public NTP servers are allowed but there is no way to guarantee NTP service will be available to VIRL virtual machine when deployed. This guide will assist in finding the cause of common NTP sync errors and provide steps to mediate.
To troubleshoot NTP connectivity you will need to run commands directly on your VIRL server. Start two separate SSH connections and log in.
User name: virl
Set your timezone
- Using one of the SSH connections set your timezone
sudo dpkg-reconfigure tzdata
- Use the curser and tab keys to navigate and spacebar to select. [Enter] confirms your selection.
- Wait at least 2 minutes and then check for peers
sudo ntpq -p
- If after 5 minutes output still reports 'INIT' continue
Testing Connectivity to NTP Servers
For this test, we must stop the NTP service. If the service is not stopped, some commands will fail because the service will not release UDP port 123.
- Stop NTP service
sudo service ntp stop
- From second ssh session start a packet capture
sudo tcpdump -ni eth0 udp port 123
- From first ssh session, test access to default pool
sudo ntpdate -d pool.ntp.org
- Below is the expected result from the above command (viewed from first ssh session)
virl@virldev-5058:~$ sudo ntpdate -d pool.ntp.org 24 Mar 13:10:15 ntpdate: ntpdate firstname.lastname@example.org Wed Oct 5 12:35:26 UTC 2016 (1) Looking for host pool.ntp.org and service ntp host found : ha81.smatwebdesign.com transmit(22.214.171.124) receive(126.96.36.199) transmit(188.8.131.52) receive(184.108.40.206) … server 220.127.116.11, port 123 stratum 2, precision -23, leap 00, trust 000 refid [18.104.22.168], delay 0.06229, dispersion 0.00005 transmitted 4, in filter 4 reference time: dc7fd178.97ffc68f Fri, Mar 24 2017 13:10:16.593 originate timestamp: dc7fd17e.218f19ce Fri, Mar 24 2017 13:10:22.131 transmit timestamp: dc7fd17e.1c96472e Fri, Mar 24 2017 13:10:22.111 filter delay: 0.06270 0.06229 0.06233 0.06236 0.00000 0.00000 0.00000 0.00000 filter offset: 0.001138 0.001076 0.000975 0.001024 0.000000 0.000000 0.000000 0.000000 delay 0.06229, dispersion 0.00005 offset 0.001076 server 22.214.171.124, port 123 stratum 2, precision -20, leap 00, trust 000 refid [126.96.36.199], delay 0.03885, dispersion 0.00002 transmitted 4, in filter 4 reference time: dc7fcc15.313c88e2 Fri, Mar 24 2017 12:47:17.192 originate timestamp: dc7fd17e.522f59a0 Fri, Mar 24 2017 13:10:22.321 transmit timestamp: dc7fd17e.4fcaf481 Fri, Mar 24 2017 13:10:22.311 filter delay: 0.03917 0.03896 0.03893 0.03885 0.00000 0.00000 0.00000 0.00000 filter offset: 0.002846 0.002639 0.002692 0.002678 0.000000 0.000000 0.000000 0.000000 delay 0.03885, dispersion 0.00002 offset 0.002678
- Note that in the start of the output we see both 'transmit' and 'receive'; this is the expected result and clearly shows connectivity.
- Next output shows that no response from NTP servers. Until this behavior is corrected you will not be able to sync your date/time information.
virl@virldev-5058:~$ sudo ntpdate -d pool.ntp.org 2 Dec 14:08:16 ntpdate: ntpdate email@example.com Wed Oct 5 12:35:26 UTC 2016 (1) Looking for host pool.ntp.org and service ntp host found : ha81.smatwebdesign.com transmit(188.8.131.52) transmit(184.108.40.206) 220.127.116.11: Server dropped: no data server 18.104.22.168, port 123 stratum 0, precision 0, leap 00, trust 000 refid [22.214.171.124], delay 0.00000, dispersion 64.00000 transmitted 4, in filter 4 reference time: 00000000.00000000 Sun, Dec 31 1899 19:00:00.000 originate timestamp: 00000000.00000000 Sun, Dec 31 1899 19:00:00.000 transmit timestamp: dbec4526.57319f9f Fri, Dec 2 2016 14:08:22.340 filter delay: 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 filter offset: 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 delay 0.00000, dispersion 64.00000 offset 0.000000
- If your returned output is similar to above section, we recommend configuring another NTP provider (server) which may resolve the problem. Continue to section Resetting NTP Servers via UWM OR...
- Collect the output generated in the second ssh session and provide both outputs to your system administrator for assistance. If your network does not have a system administrator and you would like assistance via the forum, collect the output and start a new thread. Be sure to include details of your deployment and steps you have already taken.
Resetting NTP Servers via UWM
- Log into UWM web interface with admin user
- Default Credentials
- Username: uwmadmin
- Password: password
- Navigate to VIRL Server > System Configuration
- Change NTP Server from currently defined (default is: pool.ntp.org) to appropriate server as required by your network. If the default server is not working and there are no restrictions on NTP at your location, try any of the following public NTP servers:
- Do not edit any files via CLI. Just replace the NTP server defined in UWM with the name of your chosen provider. For example setting the NTP Server to time.nist.gov, allows VIRL to use all of the NIST time servers defined in their pool.
- Apply Setting and follow prompts
- Test connectivity as outlined in Testing Connectivity to NTP Servers above
Resetting NTP Servers via CLI (Not Recommended)
- Edit virl.ini to reflect desired NTP server pool
- If your environment uses a proxy server, edit virl.ini to reflect correct proxy server
- Edit /etc/ntp.conf to reflect desired NTP server pool (must match NTP server pool defined in virl.ini)
- Run following commands to update VIRL settings
sudo salt-call -l debug state.sls virl.vinstall
sudo vinstall salt
sudo salt-call -l debug state.sls virl.ntp