New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Syslog-ng stops logging on console after system time is changed #2836
Comments
A similar issue was just posted on ServerFault. The report there says, that upgrading to 3.22 did not resolve the problem: |
I haven't dug into this issue, but it might be the consequence of how
https://www.freedesktop.org/software/systemd/man/sd_journal_next.html |
Hi @varunpandeyengg, I've gave it a try to see the problem, but didn't reproduce it so far. Can you help me list the exact steps you can trigger this issue, please? |
I think logging stops after a system restart. |
I managed to reproduce the issue with VirtualBox. Steps:
I will try to find the root cause now. :) |
I encountered this on an embedded device without a realtime clock. This is sort of a known issue in journald journalctl -f doesn't work if system clock is turned back after reboot, sd-journal: find a smart way to order log lines from different boots if neither has a useful wallclock time. Anyway, my workaround is to force journald and syslog-ng to use the old syslog interface.
Note: The downside to this approach is that jouranld forwards syslog messages in the old RFC3164 format so you will lose sub-second timestamp resolution. |
One possible solution would be to call |
hmm.. it seems the current boot_id is published by the kernel at /proc/sys/kernel/random/boot_id and this should be pretty simple, calling sd_journal_add_match(journal, "_BOOT_ID=<boot_id_value>") in fact this could make sense in a generic form, e.g. to allow specifying this and other similar filters in the systemd-journald() driver. Care to submit a patch? |
I took a look at your codebase and I'm not sure that I am familiar enough with it to put together a good patch. I think that ideally syslog-ng would:
Of note: you want to use I have attached some proof of concept code that I wrote against 3.24.1 that does "work" but doesn't read the end of the last boot or keep track of boot_id. |
This is probably the same issue: |
@TaborKelly I'd like to create a PR based on your patch. Can you paste a "Signed-off-by: " in this issue which I can then paste into the commit message. This is in-line with: https://developercertificate.org/ and our CONTRIBUTING guide. thanks |
@juhaszviktor can you perhaps remember why the systemd-journal() source driver has a mode where we don't link against libsystemd, rather it is dlopen()-ing that library and gets access to the various symbols using dlsym()? There are two layers of wrappers to perform this and I can't get why this was needed in the first place. This is the location what I have in mind:
When I tried to compile syslog-ng with this mode, it even errored out, so seemingly we are not exercising this compilation mode. I would get rid of the entire thing if there's no plausible explanation. Thanks. |
@bazsi, you absolutely have my permission to use my code in that patch in line with the Developer Certificate of Origin Version 1.1. Signed-off-by: Tabor Kelly taborkelly@gmail.com |
syslog-ng
Version of syslog-ng
Platform
Debian Stretch
Steps to reproduce
After changing the system time to an older timestamp, I am observing syslog-ng 3.8.1 on debian stretch stops forwarding logs to console. Here are the steps:
Configuration
The text was updated successfully, but these errors were encountered: