It’s with great pleasure that I announce that since last December I am working at Red Hat! I’ve already updated my Linkedin profile but forgot to blog about it 🙂 .
It’s not a secret to anyone that I always admired Red Hat and that I think it is one of the best places to work for people like me, who loves Open Source. So, I consider this a big step in my life and career!
I’m working in the Cloud Enablement team, which is responsible to bring Middleware products (JBoss, etc) to Red Hat’s PaaS, Openshift. That’s calledxPaaS. It’s a entire new world to me: Docker, Kubernetes, Openshift, JBoss, etc… and at the same time it’s a very exciting moment.
I’m still living in São Paulo, but working from home now, which gives me some opportunities 🙂 .
In other news, I’m now an Emeritus Member of the GNOME Foundation. This means that I’m quite a few time without substantial contributions to the project. Sad 🙁 . The good news is that working at Red Hat I’m closer than ever to contribute to Open Source projects! I hope to be back to the game soon 🙂
Under pressure, my system was generating lots of messages like this:
# systemctl -l status systemd-journald.service
Jun 24 13:47:23 localhost systemd-journal: Suppressed 15460 messages from /system.slice/...
This means the system is generating lots of messages and journal is configured to drop some of them. This is called rate limit, and is useful to not overload the logging system.
Sometimes, however, you just want to get all messages, or perhaps increase these limits. This can be achieved by setting the variables RateLimitInterval and RateLimitBurst inside the config file /etc/systemd/journald.conf.
Linux has a mechanism to avoid a DoS attack – with regard to logging – called rate limit. Every message logged by the kernel (including its modules), with printk(), is checked if it’s allowed to be actually printed through this mechanism.
The limits can be configured by tuning the files /proc/sys/kernel/printk_ratelimit and /proc/sys/kernel/printk_ratelimit_burst. In my machine, the values for these files are 5 and 10, respectively, meaning: It’s allowed 10 messages every 5 seconds. Exceeding this will make the kernel discard the message and print something like “ratelimit N: callbacks suppressed”.
However, the networking code in the kernel has its own limit configuration. They obey the same logic above, they use a different path just to allow independence from the generic logging functions. The files are: /proc/sys/net/core/message_cost and /proc/sys/net/core/message_burst. They are similar to their generic “parents” mentioned above.
The message_cost file contains the interval and message_burst contains the maximum number of messages allowed in that interval.
To disable this mechanism and allow every message to be logged, simply set the interval to 0:
# sysctl -w net.core.message_cost=0
Write “net.core.message_cost=0” to /etc/sysctl.d/some-file to make this change persistent to reboots.
This will make the message “net_ratelimit: N callbacks suppressed” go away. It’s up to you do disable this mechanism. Sometimes it’s just necessary, right?