SAP HANA 2.0 SPS05 rev 50

Please refer below SAP blog to get clear knowledge about SPS05

https://blogs.sap.com/2020/06/26/sap-hana-2.0-sps-05-installation-and-update-hands-on/

2083715 - Analyzing log volume full situations

HANA Log volume can be full due to various reasons

below are a few important reasons

1: The disk usage of log volumes only grows, if there are no more segment files available for overwriting/re-use ( having state FREE, see Database view  M_log_segments).

 Log segments are available for re-use when they have been successfully backed up (in log_mode = normal) and are not required for a database restart. Aim of this SAP Note is to help you identify the root cause, why log segments are not getting freed, and remain in state TRUNCATED (indicating that the log-segment has not yet been backed up successfully). In a log volume full situation, no free log-segments are available for re-use & no more new log segments can be allocated due to limited disk quota/size, thus the database cannot be started or stops accepting requests. Apart from bringing the system up again as soon as possible, the root cause needs to be investigated and resolved. Otherwise, you may run into a log volume full situation again soon.

2: SAP HANA Log Volume Full on Secondary or Tertiary Site with System Replication

sap note 20837

Recovery fails with "clear log volume because of recovery with clearlog"

ERROR : 

During a recovery from backup using "clear log" option, it fails with error:

recovery could not be completed, volume <volume number>, reached log position 0,
The error <exception 70029052: clear log volume because of recovery with clearlog: eror from remove all dir /hana/log/HDB/<mount>/<path>/ Not a directory(20)

Reason: 

During CLEAR LOG of the recovery, the database tries to remove this directory and fails due to permissions issue or if their is a symbolic link.

Solution

1: Move to directory  /hana/log/<sid>/<mount>/
2: Execute ls -l
3: Check permissions for the path are valid
4: If there exists symbolic link (such as one that may have been created as part of following the steps of SAP Note 1679938 - DiskFullEvent on Log Volume ) please remove that
Repeat the restore



https://rupeshschavan.blogspot.com/2020/06/diskfullevent-on-log-volume.html

DiskFullEvent on Log Volume

Error:

We may face issues while accessing the system from SAP GUI or from HANA Studio. From HANA studio error while be ' No connection to database".  In this case Log FS full is one of the major reasons. We can see in below snap.

Prerequisites

• Perform a backup before reclaiming space. 
• Move old log backup from Backup FS, ensure sufficient space available for log backup 

Reason: 

We faced issues because of Log backup FS ( File System) get full.

There are a few important statuses of log seg segments that we need to check

1: Free
2: writing
3: truncated


Check how many log segments are used.

hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i truncated | wc -l
105

Check from how many log segment log backup Running

 hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i writing | wc -l
1

Check from how many log segment log backup Free

hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i free | wc -l

0

Solution:

We have to way to resolve this issue. 

    

     1:   increase size of /hana/log

     2:   Create temp directory

1: Increase the size of /hana/log

if HANA DB is hosted on AWS or etc where resize is possible then resize it by minimum 2 GB. 

once size increases then to test whether archiving is running or not run below commands. if output size is decreasing it means log segment getting empty. you can also check the status by running "free" as shown below

 --  hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i truncated | wc -l

 --  hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i free | wc -l

Note: hdb00003  ( its directory for once of service log segment ). you can run command for each and every service to get more details knowledge. 

drwxr-x--- 3 hdbadm sapsys 4096 Mar 27 07:27 ..

drwxr-xr-- 2 hdbadm sapsys 4096 Jun 13 11:32 hdb00002.00004

drwxrwxr-x 8 hdbadm sapsys 4096 Jun 16 05:47 .

drwxr-xr-- 2 hdbadm sapsys 4096 Jun 16 05:47 hdb00004.00004

drwxr-xr-- 2 hdbadm sapsys 4096 Jun 16 05:57 hdb00003.00004

drwxr-x--- 2 hdbadm sapsys 4096 Jun 22 14:40 hdb00001

drwxr-x--x 2 hdbadm sapsys 4096 Jun 22 15:08 hdb00002

drwxr-x--x 2 hdbadm sapsys 4096 Jun 23 04:29 hdb00003

IMP: run reclaim command once truncated count 

Now see space at OS level

hdbadm@aws-s-hpz-200:/hana/log/HDB/mnt00001> df -h

Filesystem      Size  Used Avail Use% Mounted on

/dev/******  128G   14G  109G  11% /hana/log

/dev/******    492G  204G  266G  44% /backup

/dev/*******    49G   53M   47G   1% /sapmnt

/dev/*******    246G  193G   41G  83% /hana/shared

/dev/nvme5n1     49G  282M   47G   1% /usr/sap

/dev/nvme3n1    326G  221G   91G  71% /hana/data

2: Create a temp directory

We need to follow the below steps if we are unable to increase the size of FS /hana/log

1: stop HANA DB  

a: we need to stop HANA DB and Application 

/usr/sap/hostctrl/exe/sapcontrol -nr 00 -function StopSystem HDB   or HDB stop

b: create a Temporary Directory 

mkdir /hana/data/PRD/logs_temporary

/hana/data/PRD/logs_temporary

2: Move any one log volume to a new location ( min 2 BG space )

mv hdb00001 /hana/data/PRD/logs_temporary

3: Create a soft link from the original location to new location and start HANA DB

ln -s /hana/data/PRD/logs_temporary/hdb00001 /hana/log/PRD/mnt00001/hdb00001

note: run this command from "/hana/log/PRD/mnt00001/hdb00001"

4: Start HANA DB and Application

sapcontrol -nr 00 -function StartSystem HDB

5: Wait until log backups are performed.

7: Use the following SQL-Statement to clean up the log volume:

ALTER SYSTEM RECLAIM LOG;

8: Stop the database again:

/usr/sap/hostctrl/exe/sapcontrol -nr <instance_number> -function StopSystem HDB

9: Wait that the database on all hosts is stopped completely.

Remove the symbolic link:

rm -f /hana/log/HDB/mnt00001/hdb00003

10: Move the log volume back to its original location:

mv /hana/data/<SID>/logs_temporary/hdb00003 /hana/log/<SID>/mnt00001

11: Remove the temporary log folder:

rmdir /hana/data/<SID>/logs_temporary

12: Start the database

/usr/sap/hostctrl/exe/sapcontrol -nr 00 -function StartSystem HDB  or HDB start

note: check your instance number, in my case its 00

Few Important commands 

1: Check the status of the log segment

select b.host, b.service_name, a.state, count(*) from "PUBLIC"."M_LOG_SEGMENTS" a join "PUBLIC"."M_SERVICES" b on (a.host = b.host AND a.port = b.port) group by b.host, b.service_name, a.state

2:  Get details about the file that need to truncate, free or writing with full details

select top 1000 * from "SYS"."M_LOG_SEGMENTS" 

1679938 - DiskFullEvent on Log Volume

Why physical Separation of Data and Log Volumes require in HANA Database

There are different reasons but few important are like 

1: One reason for this recommendation is I/O performance: For SAP HANA, the I/O requirements for accessing the data volume are different from those for the log volume (see chapter “I/O Sizing”) so that, usually, separate file systems for data and log volume are needed to reach the I/O requirements for both volumes.

The log volume is accessed using a sequential I/O pattern while the data volume is accessed using a random pattern. Therefore, with separated volumes, better performance can be reached for the redo log writing.

2: The second reason is data safety: If the disks of one volume get damaged or data in one volume get corrupted then the other volume is not affected