How log2ram works.

I investigated log2ram package mechanism.
You will find out how log2ram works after reading this post!

What is log2ram?

This is introduced in this page.

Just a moment...

www.ostechnix.com

log2ram changes the physical location of /var/log/ .
/var/log/ is HDD or microSD (if Raspberry Pi) .
log2ram makes /var/log/ to point RAM disk which is under /tmp/ .

What are the benefits?

Lifetime of storage can be extended because the counts to write storage by log can be decreased.

Raspberry Pi has /var/log/ on microSD.
Therefore, every time you write a log with syslog, etc., a write operation to microSD occurs.
(In fact the timing when physical writing occurs from kernel buffer to microSD depends on other conditions like “sync”)

So, logging affects the life of microSD.
The write limit for microSD depends the quality of microSD itself.
For example TLC microSD, is only about 1000 times!
In the worse case if you write the log too much, you may not be able to read any data in the microSD and Raspberry Pi does not work.

Types of microSD	Write limit (approximate)
SLC	100,000 times
MLC	3,000 times
TLC	1000 times

Therefore, somebody wants to handle data that frequently writes such as logs on RAM.
With log2ram, the process of storing logs can be realized without changing anything!

log2ram itself is provided for debian as well.
The source code is available on github.

GitHub - azlux/log2ram: ramlog like for systemd (Put log into a ram folder)

ramlog like for systemd (Put log into a ram folder) - GitHub - azlux/log2ram: ramlog like for systemd (Put log into a ra...

github.com

File structure

Once installed, it will be deployed as following structure.

Path	Role
/etc/log2ram.conf	Configuration file
/etc/cron.daily/log2ram	Scripts for daily execution by cron
/etc/logrotate.d/log2ram	Script for logrotate
/etc/systemd/system/log2ram.service	Service registration file
/usr/local/bin/log2ram	Main logic file
/usr/local/bin/uninstall-log2ram.sh	Uninstall script

Behavior – Startup –

/usr/local/bin/log2ram start works at startup.
/var/log/ is the default location for the target directory.
It can be changed with log2ram.conf.
The code that moves at start time is as follows:

start)
        IFS=';'
        for i in $PATH_DISK; do
            PATH_FIRST_PART=$( echo ${i%/*} )
            PATH_LAST_PART=$( echo ${i##/*/} )
            RAM_LOG=$i
            HDD_LOG=$PATH_FIRST_PART/hdd.$PATH_LAST_PART
            LOG2RAM_LOG="${RAM_LOG}/${LOG_NAME}"
            
            make_log_dir
            
            mount --bind $RAM_LOG/ $HDD_LOG/
            mount --make-private $HDD_LOG/
            wait_for $HDD_LOG

            if [ "$ZL2R" = true ]; then
                createZramLogDrive
                mount -t ext4 -o nosuid,noexec,noatime,nodev,user=log2ram /dev/zram${RAM_DEV} ${RAM_LOG}/
            else
                mount -t tmpfs -o nosuid,noexec,noatime,nodev,mode=0755,size=${SIZE} log2ram $RAM_LOG/
            fi
            wait_for $RAM_LOG
            syncFromDisk
        done
        ;;

PATH_DISK =”/var/log”.
Therefore, HDD_LOG=/var/hdd.log/.
Bind /var/log/ to /var/hdd.log/, and then branch by the value of ZL2R.

ZL2R	Behavior
true	Formats /dev/zram0 in ext4 and mount to /var/log/
false	Mount s/var/log/ as tmpfs

zram is a RAM disk with automatic compression/decompression mechanism provided by kernel. Finally, copy the contents of /var/hdd.log/ to /var/log/ with syncFromDisk and log2ram startup is finished.
After that, when you write a file to /var/log/below, it will be written to the created ext4 or tmpfs.　The original /var/log/ is copied to /var/hdd.log/ .

Notes – Startup –

If there is any process to write to /var/log/ in parallel with the log2ram startup process, the log contents may be inconsistent depending on the timing.
In the table of below, the red character is syslog, and the blue character is the log2ram startup process.
In B, syslog writes to /var/log/, and then copies the contents
of /var/hdd.log// to /var/log/. If syslog/log2ram handles the same file name, it may not be what you expect.

Normal Case (A)	Cases of inconsistency (B)
Writes something to /var/log/ Binds /var/log/ to /var/hdd.log/ /copy /var/hdd.log/ to /var/log/	Binds /var/log/ to /var/hdd.log/ Writes something to /var/log/ /copy /var/hdd.log/ to /var/log/

To avoid this kind of inconsistency please think about startup sequence of log2ram to start earlier than any other startup process that have possibility to write log file.

Behavior – Shutdown –

The behavior at the shutdown is as follows:

stop)
        IFS=';'
        for i in $PATH_DISK; do
            PATH_FIRST_PART=$( echo ${i%/*} )
            PATH_LAST_PART=$( echo ${i##/*/} )
            RAM_LOG=$i
            HDD_LOG=$PATH_FIRST_PART/hdd.$PATH_LAST_PART
            LOG2RAM_LOG="${RAM_LOG}/${LOG_NAME}"
            
            syncToDisk
            #ZRAM_LOG=$(awk '$2 == "/var/log" {print $1}' /proc/mounts)
            #ZRAM_LOG=$(echo ${ZRAM_LOG} | grep -o -E '+')[0-9]
            umount -l $RAM_LOG/
            umount -l $HDD_LOG/
            # Unsure as even with Root permision denied
            #echo ${ZRAM_LOG} > /sys/class/zram-control/hot_remove
        done
        ;;

The following code works in syncToDisk which is called in above shutdown.
The contents of /var/log/ are copied to /var/hdd.log/.

syncToDisk () {
    isSafe
    #INITIAL_STATE=$(remountRW)

    if [ "$USE_RSYNC" = true ]; then
        rsync -aXv --inplace --no-whole-file --delete-after $RAM_LOG/ $HDD_LOG/ 2>&1 | tee -a $LOG2RAM_LOG
    else
        cp -rfup $RAM_LOG/ -T $HDD_LOG/ 2>&1 | tee -a $LOG2RAM_LOG
    Fi
    #remountOriginal ${INITIAL_STATE}
}

The purpose of doing this is to keep logs on HDD/microSD.
At startup/var/log/ mounts on zram or tmpfs, or RAM disk. Therefore, if you do not write back to HDD/microSD, the log will deleted when you turn off computer.
log2ram copies /var/hdd.log/ to /var/log/ immediately after turning on computer.
/var/ itself is on HDD/microSD, so /var/hdd.log/is also HDD/microSD.
So if you copy log files to /var/hdd .log/, it will remain after shutdown.

Notes – Shutdown –

Above code works when the shutdown process is performed.
It is never called if the power supply is cut off.
– Unplug Raspberry Pi power cord
– Power failure
– Others

There is also log integrity issue during shutdown as well as startup.
To avoid it, let log2ram finish at the very end.

What is zram?

zram, formerly called compcache, is a Linux kernel module for creating a compressed block device in RAM, i.e. a RAM disk, , but with on-the-fly “disk” compression.

https://en.wikipedia.org/wiki/Zram

As you can see in above zram is RAM disk with on-the-fly compression/decompression function in the successor to the kernel module called compcache.
Compressing and retaining saves more RAM size than tmpfs, and allows application processes to handle without being aware of compression/decompression.

Some parameters, such as compression methods, can be set in the application.
log2ram sets in createZramLogDrive().

echo ${COMP_ALG} > /sys/block/zram${RAM_DEV}/comp_algorithm
echo ${LOG_DISK_SIZE} > /sys/block/zram${RAM_DEV}/disksize
echo ${SIZE} > /sys/block/zram${RAM_DEV}/mem_limit

What if we can use as personal use?

As also written in wikipedia, the main uses are as follows.
– A. Temporarily data location such as /tmp.
– B. Swap area.

To think about A, we may be able to use cache for browser.

Conclusion

How was it?

It may be useful with Raspberry Pi because it usually uses microSD as main storage.

If you are struggling to guarantee product lifetime due to microSD, log2ram is one of good solution!