Discussion:
ntpd strange problem
Wojciech Puchar
2018-10-04 13:58:26 UTC
Permalink
what does message mean and how to fix it?

-----
Oct 4 15:56:46 <12.5> puchar ntpd[93928]: ntpd 4.2.8p12-a (1): Starting
Oct 4 15:56:46 <12.6> puchar ntpd[93928]: Command line: /usr/sbin/ntpd -g
-c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift
[***@puchar ~]# Oct 4 15:56:46 <12.6> puchar ntpd[93929]: proto:
precision = 0.078 usec (-24)
Oct 4 15:56:46 <12.3> puchar ntpd[93929]: restrict: ignoring line 6, mask
'::' unusable.
Oct 4 15:56:46 <12.6> puchar ntpd[93929]: Listen and drop on 0 v4wildcard
0.0.0.0:123
Oct 4 15:56:46 <12.6> puchar ntpd[93929]: Listen normally on 1 lo0
127.0.0.1:123
Oct 4 15:56:46 <12.6> puchar ntpd[93929]: Listen normally on 2 bridge0
10.0.1.1:123
Oct 4 15:56:46 <12.6> puchar ntpd[93929]: Listen normally on 3 bridge1
194.1.144.90:123
Oct 4 15:56:46 <12.6> puchar ntpd[93929]: Listen normally on 4 bridge1
194.1.144.91:123
Oct 4 15:56:46 <12.6> puchar ntpd[93929]: Listen normally on 5 tun4
10.0.230.1:123
Oct 4 15:56:46 <12.6> puchar ntpd[93929]: Listen normally on 6 tun0
10.0.224.1:123
Oct 4 15:56:46 <12.6> puchar ntpd[93929]: Listening on routing socket on
fd #27 for interface updates
Oct 4 15:56:46 <12.6> puchar ntpd[93929]: kernel reports TIME_ERROR:
0x2041: Clock Unsynchronized
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Peter Jeremy
2018-10-05 06:18:29 UTC
Permalink
Post by Wojciech Puchar
what does message mean and how to fix it?
-----
Oct 4 15:56:46 <12.5> puchar ntpd[93928]: ntpd 4.2.8p12-a (1): Starting
...
Post by Wojciech Puchar
0x2041: Clock Unsynchronized
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It will take 5-10 minutes for ntpd to synchronise to its upstream servers.
Until then, the clock will report that it's unsynchronised. If "ntptime"
is still reporting that it's unsynchronised after a long period, you will
need to do more troubleshooting.
--
Peter Jeremy
Wojciech Puchar
2018-10-05 09:24:42 UTC
Permalink
Post by Peter Jeremy
Post by Wojciech Puchar
0x2041: Clock Unsynchronized
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It will take 5-10 minutes for ntpd to synchronise to its upstream servers.
Until then, the clock will report that it's unsynchronised. If "ntptime"
is still reporting that it's unsynchronised after a long period, you will
need to do more troubleshooting.
yes the problem is that:

after starting ntpd time is exactly fine

ntpd_sync_on_start=YES
ntpd_enable=YES


but then while ntpd works, time seems to get out of sync to the extent of
normal RTC imprecission - in order of few seconds per day.

ntpd seems like not to update time properly.

restarting ntpd fixes time again.

what should i look at to find a source of problem
Matthew Seaman
2018-10-05 12:08:10 UTC
Permalink
Post by Wojciech Puchar
Post by Peter Jeremy
Post by Wojciech Puchar
0x2041: Clock Unsynchronized
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It will take 5-10 minutes for ntpd to synchronise to its upstream servers.
Until then, the clock will report that it's unsynchronised.  If "ntptime"
is still reporting that it's unsynchronised after a long period, you will
need to do more troubleshooting.
after starting ntpd time is exactly fine
ntpd_sync_on_start=YES
ntpd_enable=YES
but then while ntpd works, time seems to get out of sync to the extent
of normal RTC imprecission - in order of few seconds per day.
ntpd seems like not to update time properly.
restarting ntpd fixes time again.
what should i look at to find a source of problem
Is this a virtual machine or bare metal? VMs tend to have a lot more
problems with clock drift -- they lose time when the hypervisor switches
them off the CPU. Then if the clock gets more than a certain amount out
of sync, ntpd just gives up. You can add:

tinker panic 0

to ntpd.conf to make that less likely to happen.

If it's bare metal then it sounds like your system clock is running much
faster or slower than ntpd can manage to correct. That's a hardware
problem, but there are some tunings that may allow ntpd to cope. If you
enable a drift file (which I think is enabled by default):

driftfile /var/db/ntpd.drift

then ntpd will record how fast or slow the clock tends to run so it can
immediately account for that across restarts, rather than having to work
it out de-novo everytime ntpd gets restarted.

Cheers,

Matthew
Wojciech Puchar
2018-10-06 18:43:03 UTC
Permalink
Post by Matthew Seaman
Post by Wojciech Puchar
what should i look at to find a source of problem
Is this a virtual machine or bare metal? VMs tend to have a lot more
bare metal.
Post by Matthew Seaman
driftfile /var/db/ntpd.drift
then ntpd will record how fast or slow the clock tends to run so it can
immediately account for that across restarts, rather than having to work it
out de-novo everytime ntpd gets restarted.
# cat /var/db/ntpd.drift
35.586

what else can i do?
Ian Lepore
2018-10-06 19:45:51 UTC
Permalink
Post by Wojciech Puchar
Post by Wojciech Puchar
what should i look at to find a source of problem
Is this a virtual machine or bare metal?   VMs tend to have a lot
more 
bare metal.
driftfile /var/db/ntpd.drift
then ntpd will record how fast or slow the clock tends to run so it
can 
immediately account for that across restarts, rather than having to
work it 
out de-novo everytime ntpd gets restarted.
# cat /var/db/ntpd.drift
35.586
what else can i do?
The original output you posted showed the time as unsynchronized when
ntpd started up. That seems normal to me, it can't be synchronized
until ntpd has been running for a while. In a followup message you
seemed to say that it doesn't stay synchronized, but you haven't
provided much info about that.

What's in the logs when it become unsynchronized or when you notice the
time is several seconds off?

What does the output of ntptime show when the time is drifted off?

How about the output of "ntpq -p" and "ntpq -c rv"?

What's in your ntp.conf?

If you set sysctl kern.timecounter.stepwarnings=1, do you see any
warnings about the clock stepping as time drifts out of sync?

-- Ian
Wojciech Puchar
2018-10-08 11:19:18 UTC
Permalink
Post by Ian Lepore
Post by Wojciech Puchar
what else can i do?
The original output you posted showed the time as unsynchronized when
ntpd started up. That seems normal to me, it can't be synchronized
until ntpd has been running for a while. In a followup message you
seemed to say that it doesn't stay synchronized, but you haven't
provided much info about that.
What's in the logs when it become unsynchronized or when you notice the
time is several seconds off?
after starting ntpd - it seems to be no different than upsteream time
server.

After week or so it got out of sync in spite of ntpd still running.

Strange but today (4 days after i restarted ntpd) it seems to be in sync


# ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
LOCAL(0) .LOCL. 2 l 93h 64 0 0.000 0.000 0.000
*ntp.task.gda.pl 194.146.251.100 2 u 227 256 377 7.127 0.006 0.050

# ntpq -c rv
associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync, version="ntpd 4.2.8p12-a (1)", processor="amd64",
system="FreeBSD/11.2-STABLE", leap=00, stratum=3, precision=-24, rootdelay=13.877, rootdisp=31.084, refid=153.19.250.123,
reftime=df65bb67.6caa4c22 Mon, Oct 8 2018 13:17:59.424, clock=df65bb79.e7c2ec64 Mon, Oct 8 2018 13:18:17.905, peer=60901, tc=9,
mintc=3, offset=0.065124, frequency=35.388, sys_jitter=0.000000, clk_jitter=0.044, clk_wander=0.001

my ntp.conf

rlimit memlock 512
server ntp.task.gda.pl iburst maxpoll 9
restrict ntp.task.gda.pl nomodify nopeer noquery notrap
restrict -4 default ignore
restrict -6 default ignore
restrict 127.0.0.1 nomodify nopeer notrap
restrict 10.0.0.0 mask 255.0.0.0 nomodify nopeer notrap
server 127.127.1.0
fudge 127.127.1.0 stratum 2
Post by Ian Lepore
What does the output of ntptime show when the time is drifted off?
How about the output of "ntpq -p" and "ntpq -c rv"?
What's in your ntp.conf?
If you set sysctl kern.timecounter.stepwarnings=1, do you see any
warnings about the clock stepping as time drifts out of sync?
-- Ian
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
Peter Jeremy
2018-10-09 05:29:24 UTC
Permalink
Post by Wojciech Puchar
After week or so it got out of sync in spite of ntpd still running.
Strange but today (4 days after i restarted ntpd) it seems to be in sync
A couple of suggestions:
* (especially if you have an ADSL link): The NTP protocol assumes that
the path between the client and server are have symmetric timing. In
my experience, bulk uploads or downloads can cause significant path
delay asymmetries, which can confuse ntpd.
* You only have a single server and are therefore dependent on the
trustworthiness of that server. A few more servers could be useful.

You could try adding enabling some monitoring (eg loopstats and
peerstats) and see what is happening leading up to ntpd losing sync.
--
Peter Jeremy
Wojciech Puchar
2018-10-09 06:42:55 UTC
Permalink
Post by Peter Jeremy
Post by Wojciech Puchar
Strange but today (4 days after i restarted ntpd) it seems to be in sync
* (especially if you have an ADSL link): The NTP protocol assumes that
the path between the client and server are have symmetric timing. In
i have gigabit connectivity with low and stable delays. so it's not this.
Post by Peter Jeremy
my experience, bulk uploads or downloads can cause significant path
delay asymmetries, which can confuse ntpd.
* You only have a single server and are therefore dependent on the
trustworthiness of that server. A few more servers could be useful.
this server is quite (actually excellently) stable and i trust it.
Ian Lepore
2018-10-09 14:06:21 UTC
Permalink
Post by Wojciech Puchar
Post by Peter Jeremy
Post by Wojciech Puchar
Strange but today (4 days after i restarted ntpd) it seems to be in sync
* (especially if you have an ADSL link): The NTP protocol assumes that
the path between the client and server are have symmetric
timing.  In
i have gigabit connectivity with low and stable delays. so it's not this.
Post by Peter Jeremy
my experience, bulk uploads or downloads can cause significant path
delay asymmetries, which can confuse ntpd.
* You only have a single server and are therefore dependent on the
trustworthiness of that server.  A few more servers could be
useful.
this server is quite (actually excellently) stable and i trust it.
When I asked for various outputs, I meant "when the unsync problem is
happening"; those outputs are uninteresting when everything is working
right.

You should remove the config for the "local" clock from your ntp.conf.
Configuring a bogus local clock is NEVER the right thing to do, and
it's even worse to configure it at a high stratum number.

-- Ian
Wojciech Puchar
2018-10-10 11:10:38 UTC
Permalink
Post by Ian Lepore
Post by Wojciech Puchar
Post by Peter Jeremy
useful.
this server is quite (actually excellently) stable and i trust it.
When I asked for various outputs, I meant "when the unsync problem is
happening"; those outputs are uninteresting when everything is working
right.
so i will look back and tell you WHEN it will happen again.
Post by Ian Lepore
You should remove the config for the "local" clock from your ntp.conf.
Configuring a bogus local clock is NEVER the right thing to do, and
it's even worse to configure it at a high stratum number.
thank you. this is an error.

Loading...