How log2ram works.

log2ram

I investigated log2ram package mechanism.
You will find out how log2ram works after reading this post!

What is log2ram?

This is introduced in this page.

How To Write Log Files In RAM Using Log2ram In Linux - OSTechNix
This guide explains how to save or write log files in RAM using Log2ram in Linux operating systems.

log2ram changes the physical location of /var/log/ .
/var/log/ is HDD or microSD (if Raspberry Pi) .
log2ram makes /var/log/ to point RAM disk which is under /tmp/ .

What are the benefits?

Lifetime of storage can be extended because the counts to write storage by log can be decreased.

Raspberry Pi has /var/log/ on microSD.
Therefore, every time you write a log with syslog, etc., a write operation to microSD occurs.
(In fact the timing when physical writing occurs from kernel buffer to microSD depends on other conditions like “sync”)

So, logging affects the life of microSD.
The write limit for microSD depends the quality of microSD itself.
For example TLC microSD, is only about 1000 times!
In the worse case if you write the log too much, you may not be able to read any data in the microSD and Raspberry Pi does not work.

Types of microSDWrite limit (approximate)
SLC100,000 times
MLC3,000 times
TLC1000 times
Source: https://flashdba.com/2014/07/03/understanding-flash-slc-mlc-and-tlc/

Therefore, somebody wants to handle data that frequently writes such as logs on RAM.
With log2ram, the process of storing logs can be realized without changing anything!

log2ram itself is provided for debian as well.
The source code is available on github.

azlux/log2ram
ramlog like for systemd (Put log into a ram folder) - azlux/log2ram

File structure

Once installed, it will be deployed as following structure.

PathRole
/etc/log2ram.confConfiguration file
/etc/cron.daily/log2ramScripts for daily execution by cron
/etc/logrotate.d/log2ramScript for logrotate
/etc/systemd/system/log2ram.serviceService registration file
/usr/local/bin/log2ramMain logic file
/usr/local/bin/uninstall-log2ram.shUninstall script

Behavior – Startup –

/usr/local/bin/log2ram start works at startup.
/var/log/ is the default location for the target directory.
It can be changed with log2ram.conf.
The code that moves at start time is as follows:

start)
        IFS=';'
        for i in $PATH_DISK; do
            PATH_FIRST_PART=$( echo ${i%/*} )
            PATH_LAST_PART=$( echo ${i##/*/} )
            RAM_LOG=$i
            HDD_LOG=$PATH_FIRST_PART/hdd.$PATH_LAST_PART
            LOG2RAM_LOG="${RAM_LOG}/${LOG_NAME}"
            
            make_log_dir
            
            mount --bind $RAM_LOG/ $HDD_LOG/
            mount --make-private $HDD_LOG/
            wait_for $HDD_LOG

            if [ "$ZL2R" = true ]; then
                createZramLogDrive
                mount -t ext4 -o nosuid,noexec,noatime,nodev,user=log2ram /dev/zram${RAM_DEV} ${RAM_LOG}/
            else
                mount -t tmpfs -o nosuid,noexec,noatime,nodev,mode=0755,size=${SIZE} log2ram $RAM_LOG/
            fi
            wait_for $RAM_LOG
            syncFromDisk
        done
        ;;

PATH_DISK =”/var/log”.
Therefore, HDD_LOG=/var/hdd.log/.
Bind /var/log/ to /var/hdd.log/, and then branch by the value of ZL2R.

ZL2RBehavior
trueFormats /dev/zram0 in ext4 and mount to /var/log/
falseMount s/var/log/ as tmpfs

zram is a RAM disk with automatic compression/decompression mechanism provided by kernel. Finally, copy the contents of /var/hdd.log/ to /var/log/ with syncFromDisk and log2ram startup is finished.
After that, when you write a file to /var/log/below, it will be written to the created ext4 or tmpfs. The original /var/log/ is copied to /var/hdd.log/ .

Notes – Startup –

If there is any process to write to /var/log/ in parallel with the log2ram startup process, the log contents may be inconsistent depending on the timing.
In the table of below, the red character is syslog, and the blue character is the log2ram startup process.
In B, syslog writes to /var/log/, and then copies the contents
of /var/hdd.log// to /var/log/. If syslog/log2ram handles the same file name, it may not be what you expect.

Normal Case (A)Cases of inconsistency (B)
Writes something to /var/log/
Binds /var/log/ to /var/hdd.log/
/copy /var/hdd.log/ to /var/log/
Binds /var/log/ to /var/hdd.log/
Writes something to /var/log/
/copy /var/hdd.log/ to /var/log/

To avoid this kind of inconsistency please think about startup sequence of log2ram to start earlier than any other startup process that have possibility to write log file.

Behavior – Shutdown –

The behavior at the shutdown is as follows:

stop)
        IFS=';'
        for i in $PATH_DISK; do
            PATH_FIRST_PART=$( echo ${i%/*} )
            PATH_LAST_PART=$( echo ${i##/*/} )
            RAM_LOG=$i
            HDD_LOG=$PATH_FIRST_PART/hdd.$PATH_LAST_PART
            LOG2RAM_LOG="${RAM_LOG}/${LOG_NAME}"
            
            syncToDisk
            #ZRAM_LOG=$(awk '$2 == "/var/log" {print $1}' /proc/mounts)
            #ZRAM_LOG=$(echo ${ZRAM_LOG} | grep -o -E '+')[0-9]
            umount -l $RAM_LOG/
            umount -l $HDD_LOG/
            # Unsure as even with Root permision denied
            #echo ${ZRAM_LOG} > /sys/class/zram-control/hot_remove
        done
        ;;

The following code works in syncToDisk which is called in above shutdown.
The contents of /var/log/ are copied to /var/hdd.log/.

syncToDisk () {
    isSafe
    #INITIAL_STATE=$(remountRW)

    if [ "$USE_RSYNC" = true ]; then
        rsync -aXv --inplace --no-whole-file --delete-after $RAM_LOG/ $HDD_LOG/ 2>&1 | tee -a $LOG2RAM_LOG
    else
        cp -rfup $RAM_LOG/ -T $HDD_LOG/ 2>&1 | tee -a $LOG2RAM_LOG
    Fi
    #remountOriginal ${INITIAL_STATE}
}

The purpose of doing this is to keep logs on HDD/microSD.
At startup/var/log/ mounts on zram or tmpfs, or RAM disk. Therefore, if you do not write back to HDD/microSD, the log will deleted when you turn off computer.
log2ram copies /var/hdd.log/ to /var/log/ immediately after turning on computer.
/var/ itself is on HDD/microSD, so /var/hdd.log/is also HDD/microSD.
So if you copy log files to /var/hdd .log/, it will remain after shutdown.

Notes – Shutdown –

Above code works when the shutdown process is performed.
It is never called if the power supply is cut off.
– Unplug Raspberry Pi power cord
– Power failure
– Others

There is also log integrity issue during shutdown as well as startup.
To avoid it, let log2ram finish at the very end.

What is zram?

zram, formerly called compcache, is a Linux kernel module for creating a compressed block device in RAM, i.e. a RAM disk, , but with on-the-fly “disk” compression.

https://en.wikipedia.org/wiki/Zram

As you can see in above zram is RAM disk with on-the-fly compression/decompression function in the successor to the kernel module called compcache.
Compressing and retaining saves more RAM size than tmpfs, and allows application processes to handle without being aware of compression/decompression.

Some parameters, such as compression methods, can be set in the application.
log2ram sets in createZramLogDrive().

echo ${COMP_ALG} > /sys/block/zram${RAM_DEV}/comp_algorithm
echo ${LOG_DISK_SIZE} > /sys/block/zram${RAM_DEV}/disksize
echo ${SIZE} > /sys/block/zram${RAM_DEV}/mem_limit

What if we can use as personal use?

As also written in wikipedia, the main uses are as follows.
– A. Temporarily data location such as /tmp.
– B. Swap area.

To think about A, we may be able to use cache for browser.

Conclusion

How was it?

It may be useful with Raspberry Pi because it usually uses microSD as main storage.

If you are struggling to guarantee product lifetime due to microSD, log2ram is one of good solution!

Author
yatch

I am software engineer at foreign company in Japan.
My experience of embedded software is over 15 years.
I was in charge of designing of software which has embedded linux.
I love Raspberry Pi.
I earn 10000USD per year on the stock market.
I married Hong Konger.

Please follow me
log2ram
Please follow me
The World's Linux Journal

Comments

  1. LinuxNoob says:

    Thank you for the great article, Yatch!
    I have a question about the /var/hdd.log/

    I understand the importance of writing to /var/hdd.log/ in case of shutdown.

    1. Does log2ram write /var/log to /var/hdd.log/ daily?
    2. Does /var/log become completely mounted to the ram or is there a main copy that stay on the microsd?
    3. What happens to the files when the cron job for log2ram hits the task deadline?

    I am trying to understand the behavior.

    Thanks again!

    • yatch yatch says:

      Hello LinuxNoob!
      Thank you for being interested in this article!

      1. Does log2ram write /var/log to /var/hdd.log/ daily?
      -> Yes.
      We can find “log2ram-daily.service” file in log2ram package.

      This seems to restart log2ram mechanism once a day.
      – ExecStart=/bin/systemctl reload log2ram.service
      reload means stop and start.

      In “log2ram stop”, /var/log/ and /var/hdd.log/ are unmounted.
      When unmount is done, data in kernel memory is surely flashed to device.

      Be careful to confirm the timing when “log2ram-daily.service” runs because this kind of file is usually controlled by systemd or other daemon.
      You need to think about avoiding shutdown or power off when “log2ram-daily.service” runs.

      2. Does /var/log become completely mounted to the ram or is there a main copy that stay on the microsd?
      -> /var/log/ is mounted zram or tmpfs after “log2ram start”.
      So we can think /var/log/ is mounted to ram.

      3. What happens to the files when the cron job for log2ram hits the task deadline?
      -> Sorry I couldn’t get your point of “the task deadline”.
      I assumed your point is like below. Am I correct?
      “What happens to file when the cron job for log2ram and write operation to file by other process are done at the same time?”

      • LinuxNoob says:

        Thanks for the information, Yatch. I know understand more about the process.

        To your last point, I have log2ram process running on a pi with pihole. After running df -h command, I found out that log2ram is full at 100%. when the systemctl task for log2ram is executed, the log2ram stays full at 100%. I assumed that log2ram will reset to 0 after service execution.

        I am now thinking that entire /var/log stays in the ram even after flashing.

        is there any remedy beside deleting logs?

        • yatch yatch says:

          Hello LinuxNoob!

          log2ram-daily.service simply runs log2ram with “stop” and “start” command line argument.
          In “start”, log2ram calls “syncFromDisk” to synchronize files from /var/hdd.log/ to /var/log/.
          In “stop”, log2ram calls “syncToDisk” which synchronize from /var/log/ to /var/hdd.log/

          syncFromDisk/syncToDisk uses either rsync or cp.
          – rsync if rsync command is available.
          – cp if rsync command is not available.

          If you want to delete some files in /var/log/, I would suggest to install rsync because “–delete-after” is specified to use.

          “–delete-after” deletes files in destination directory which doesn’t exist in source directory.

          In case of using “cp” log2ram just adds/overwrites files to destination.
          – No files are removed.

          I hope this comment answers to you.

          If there is any unclear point, feel free to ask me.

          Thanks!

タイトルとURLをコピーしました