Error:
We may face issues while accessing the system from SAP GUI or from HANA Studio. From HANA studio error while be ' No connection to database". In this case Log FS full is one of the major reasons. We can see in below snap.
Prerequisites
- Perform a backup before reclaiming space.
- Move old log backup from Backup FS, ensure sufficient space available for log backup
Reason:
We faced issues because of Log backup FS ( File System) get full.
There are a few important statuses of log seg segments that we need to check
1: Free
2: writing
3: truncated
Check how many log segments are used.
hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i truncated | wc -l
105
Check from how many log segment log backup Running
hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i writing | wc -l
1
Check from how many log segment log backup Free
hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i free | wc -l
0
Solution:
We have to way to resolve this issue.
1: increase size of /hana/log
2: Create temp directory
1: Increase the size of /hana/log
if HANA DB is hosted on AWS or etc where resize is possible then resize it by minimum 2 GB.
once size increases then to test whether archiving is running or not run below commands. if output size is decreasing it means log segment getting empty. you can also check the status by running "free" as shown below
-- hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i truncated | wc -l
-- hdblogdiag seglist /hana/log/HDB/mnt00001/hdb00003 | grep -i free | wc -l
Note: hdb00003 ( its directory for once of service log segment ). you can run command for each and every service to get more details knowledge.
drwxr-x--- 3 hdbadm sapsys 4096 Mar 27 07:27 ..
drwxr-xr-- 2 hdbadm sapsys 4096 Jun 13 11:32 hdb00002.00004
drwxrwxr-x 8 hdbadm sapsys 4096 Jun 16 05:47 .
drwxr-xr-- 2 hdbadm sapsys 4096 Jun 16 05:47 hdb00004.00004
drwxr-xr-- 2 hdbadm sapsys 4096 Jun 16 05:57 hdb00003.00004
drwxr-x--- 2 hdbadm sapsys 4096 Jun 22 14:40 hdb00001
drwxr-x--x 2 hdbadm sapsys 4096 Jun 22 15:08 hdb00002
drwxr-x--x 2 hdbadm sapsys 4096 Jun 23 04:29 hdb00003
IMP: run reclaim command once truncated count
Now see space at OS level
hdbadm@aws-s-hpz-200:/hana/log/HDB/mnt00001> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/****** 128G 14G 109G 11% /hana/log
/dev/****** 492G 204G 266G 44% /backup
/dev/******* 49G 53M 47G 1% /sapmnt
/dev/******* 246G 193G 41G 83% /hana/shared
/dev/nvme5n1 49G 282M 47G 1% /usr/sap
/dev/nvme3n1 326G 221G 91G 71% /hana/data
2: Create a temp directory
We need to follow the below steps if we are unable to increase the size of FS /hana/log
1: stop HANA DB
a: we need to stop HANA DB and Application
/usr/sap/hostctrl/exe/sapcontrol
-nr 00 -function StopSystem HDB or HDB stop
b: create a Temporary Directory
mkdir
/hana/data/PRD/logs_temporary
/hana/data/PRD/logs_temporary
2: Move any one log volume to a new location ( min
2 BG space )
mv
hdb00001 /hana/data/PRD/logs_temporary
3: Create a soft link from the original location to
new location and start HANA DB
ln -s
/hana/data/PRD/logs_temporary/hdb00001 /hana/log/PRD/mnt00001/hdb00001
note: run this command from "/hana/log/PRD/mnt00001/hdb00001"
4: Start HANA DB and Application
sapcontrol
-nr 00 -function StartSystem HDB
5: Wait until log backups are performed.
7: Use the following SQL-Statement to clean up the log volume:
ALTER SYSTEM RECLAIM LOG;
8: Stop the database again:
/usr/sap/hostctrl/exe/sapcontrol -nr <instance_number> -function StopSystem HDB
9: Wait that the database on all hosts is stopped completely.
Remove the symbolic link:
rm -f /hana/log/HDB/mnt00001/hdb00003
10: Move the log volume back to its original location:
mv /hana/data/<SID>/logs_temporary/hdb00003 /hana/log/<SID>/mnt00001
11: Remove the temporary log folder:
rmdir /hana/data/<SID>/logs_temporary
12: Start the database
/usr/sap/hostctrl/exe/sapcontrol -nr 00 -function StartSystem HDB or HDB start
note: check your instance number, in my case its 00
Few Important commands
1: Check the status of the log segment
2: Get details about the file that need to truncate, free or writing with full details
select top 1000 * from "SYS"."M_LOG_SEGMENTS"