About Me

My photo
I am Suresh Chinta, working on SAP HANA Cloud & SAP BTP Cloud/ AWS/Azure cloud consultant.I have experience in SAP Basis/Netweaver , S4HANA Cloud implementations / Support. I'm certified Microsoft Azure cloud & AWS professional. I have started this blog to share my knowledge with all those who are interested to learn & enhance their career.

Tuesday, October 27, 2020

 

HANA 1.0 upgrade scenarios

A lot of customers want to upgrade their HANA 1.0 databases toward to HANA 2.0. Usually this is not a big deal, if you have enough downtime. You can go straight on from HANA 1.0 >= 122.04 to any supported HANA 2.0 revision. The MDC conversion will take place automatically during the upgrade procedure. You need a revision >= 122.04 because there were some changes in the persistence layout which are needed for an upgrade to HANA 2.0 (see note 2372809). Best practice is to go for the latest HANA 1.0 revision and upgrade to the latest HANA 2.0 revision. But be careful if your HANA 1.0 SPS12 revision is newer than the HANA 2.0 revision it is not supported to upgrade (see note 1948334). Also check if your application is supported to run on HANA 2.0.

Another possible way (kudos to Nicholas Chang) is to create a backup on source DB and restore this one on the new target DB which is installed directly on HANA 2.0 with MDC. Here the MDC conversion will take place during the recovery procedure. But this can take a lot of time depending on the size and the underlying architecture, but is a valid option if your current hardware is not supported for HANA 2.0 or you want to change the location anyway. Details can be found in note 1642148.

 

But some customers are using HSR (HANA System Replication) and want to profit from the near zero downtime update (NZDU) feature with DBSL suspend (1913302 / 1984882). Another scenario is if you want to change the data center. For instance you want to change your hoster.
Here you want less downtime as possible. This means no upgrade steps between. This can be create some trouble because since HANA SPS01 the standard operation mode is MDC (see note 2423367). Does it work to sync from SDC (Single database container) to MDC? Just simple: not touching the old HANA instance and sync with HSR to the target HANA 2.0 revision. Is this possible?

There is some information in the documentation and notes which can be very confusing:


SAP HANA Tenant Databases Operations Guide
You can perform a near-zero downtime update of a single-container system to SAP HANA 2.0 SPS 01 in a system replication landscape. A single-container system will be automatically converted to a tenant database system during the update. Converting an SAP HANA system to a tenant database system is permanent and cannot be reversed.

Prerequisites

– The statistics server is not running as a separate server process (statisticsserver), but instead as an embedded service in the master index server. If this is not the case, migrate the statistics server to the embedded statistics service as described in SAP Note 1917938.
– The SAP HANA system has been installed with its server software on a shared file system (export options rw, no_root_squash).
– The SAP HANA system has been installed with the SAP HANA database lifecycle manager (HDBLCM).
– You are logged on as the system administrator user <sid>adm.

Perform one of the following procedures to update your system replication landscape to SAP HANA 2.0 SPS01:

Near-Zero Downtime Update (NZDU)

1. Update the secondary system using the SAP HANA database lifecycle manager. The migration to a tenant database system is triggered automatically.
2. Wait until the update has finished and all systems are in sync again. The replication will be possible in this situation although the primary is still a single-container system.
3. Perform a takeover to the updated secondary system. Only now the migration to a tenant database system is finalized for the secondary.
4. Update the primary system using the SAP HANA database lifecycle manager. The migration to a tenant database system is done automatically.
5. Wait until the update has finished and the primary system is up and running again.
6. Register this former primary system as the new secondary to the new primary (former secondary).


 

1999880 – FAQ: SAP HANA System Replication
Can I set up system replication between systems with different topologies?

The topology of primary and secondary site of a system replication scenario must be identical. As a consequence it isn’t possible to replicate from non-MDC to MDC (SAP Note 2101244) and vice versa, and it is also not possible to replicate from single-node to scale-out and vice versa.


2101244 – FAQ: SAP HANA Multitenant Database Containers (MDC)

Is it possible to convert SAP HANA 1.0 single container system to SAP HANA 1.0 MDC system with HANA System Replication?

No, conversion from single container to MDC with HANA System Replication is not supported. You would need to disable replication before starting the conversion else the conversion on secondary would fail with the below error.

SAP HANA Lifecycle Management – SAP HANA 1.00.122.17.1526900527

***************************************************************

10:03:43.088 – INFO: Start Date: 2018-10-26 – hdblcm

10:03:43.104 – INFO: Performing secondary system check

10:03:43.104 – ERR : The SAP HANA System cannot be converted to multitenant database containers, because it is a system replication site

10:03:43.105 – INFO: Summary of critical errors

10:03:43.104 – ERR : The SAP HANA System cannot be converted to multitenant database containers, because it is a system replication site

You would need to disable replication and need to perform the conversion on primary and all connected secondaries in case you have multitier replication.


Ok so there is some information that have to be sorted:
– it is not possible to sync from SDC to MDC
– it is possible to sync from MDC to MDC
– it is possible to sync HANA 1.0 revisions with HSR and upgrade the secondary to HANA 2.0 and sync back

 


 

In one of my projects we had a very special case. The old HANA 1.0 SDC instances should not be touched anymore and the target was on the IBM Power platform (PowerPC). This means SLES12/15 or RHEL >=7.2 on Little Endian and only HANA 2.0.

Scenario:
HANA 1.0 122.x SDC ====> HANA 2.0 >= SPS03 MDC

Do you see any direct supported scenario which is working and could solve this issue?

 


 

This is not supported by SAP. There is no direct way which is officially supported. So, why SAP creates such fundamental features like MDC which you have to use, and they are not compatible at all?

There are 3 possible solutions for this case:

1) HSR: HANA 1.0 SDC <=> HANA 2.0 SPS00 SDC

The easiest way which won’t touch the old source instance is to install a HANA 2.0 SPS00. Because SPS00 was the last official support package where you can choose between SDC and MDC. Just install it as SDC (interactive mode) und sync the HANA 1.0 with system replication.

But there is snag in it. You can only sync in a HANA which has the same revision or is newer. This means the last SPS00 revision was 2.02 and is not downloadable anymore (only with an OSS ticket). This means this solution is working from 122.04 through 122.10. If your source revision is above, you have to find another solution. So, if you follow the recommendation of SAP and upgrade your source revision to the latest one, you definitively can’t use it.

The most HANA 1.0 systems which I have seen are still on 122.05 or 122.08. Here the solution would be working.

2) HSR: HANA 1.0 MDC <=> HANA 2.0 MDC

This would be the easiest way, but you are touching the old source instance, because you have to convert it to MDC. This means an additional downtime. If your HANA 1.0 revision is newer than the latest HANA 2.0 one this solution won’t work.

For instance, the latest HANA 1.0 SPS12 revision is 122.22. The latest HANA 2.0 SPS02 is 24.07 and SPS03 Rev. 35. Currently it is not possible to upgrade or using HSR although your topology would allow it.

3) HSR: HANA 1.0 SDC <=> HANA 2.0 SPS01/02 SDC

HANA 2.0 SPS01/02 with SDC? Yes, this is not a typo. SAP integrated a backdoor if something won’t work with MDC. So, there is a hidden option/parameter:

./hdblcm --db_mode=single_container

This “dirty” option is still working though all revisions of SPS02. Since SPS03 they deactivated this one:

HANA 2.0 SPS02 Rev. 24.06

./hdblcm --action=install --ignore=check_signature_file --db_mode=single_container
SAP HANA Lifecycle Management - SAP HANA Database 2.00.024.06.1538035880
************************************************************************
Scanning software locations...
Detected components:
    SAP HANA Database (2.00.024.06.1538035880) in /int/software/sapcd/db/hana/hana_rev20_024_6/SAP_HANA_DATABASE/server
    SAP HANA AFL (incl.PAL,BFL,OFL,HIE) (2.00.024.0600.1538046631) in /int/software/sapcd/db/hana/hana_rev20_024_6/SAP_HANA_AFL/packages
    SAP HANA LCAPPS (2.00.024.0600.483443) in /int/software/sapcd/db/hana/hana_rev20_024_6/SAP_HANA_LCAPPS/packages
    SAP HANA Database Client (2.2.74.1536258378) in /int/software/sapcd/db/hana/hana_rev20_024_6/SAP_HANA_CLIENT_LINUX/client
    SAP HANA Smart Data Access (2.00.3.000.0) in /int/software/sapcd/db/hana/hana_rev20_024_6/SAP_HANA_SDA/packages

SAP HANA Database version '2.00.024.06.1538035880' will be installed.

 

HANA 2.0 SPS03 Rev. 35

./hdblcm --action=install --ignore=check_signature_file --db_mode=single_container
SAP HANA Lifecycle Management - SAP HANA Database 2.00.035.00.1545187853
************************************************************************
Scanning software locations...
Detected components:
    SAP HANA Database (2.00.035.00.1545187853) in /int/software/sapcd/db/hana/hana_rev20_035_0/SAP_HANA_DATABASE/server
    SAP HANA AFL (incl.PAL,BFL,OFL) (2.00.035.0000.1545200735) in /int/software/sapcd/db/hana/hana_rev20_035_0/SAP_HANA_AFL/packages
    SAP HANA LCAPPS (2.00.035.0000.485538) in /int/software/sapcd/db/hana/hana_rev20_035_0/SAP_HANA_LCAPPS/packages
    SAP HANA Database Client (2.3.130.1542919958) in /int/software/sapcd/db/hana/hana_rev20_035_0/SAP_HANA_CLIENT/client
    SAP HANA Smart Data Access (2.00.3.000.0) in /int/software/sapcd/db/hana/hana_rev20_035_0/SAP_HANA_SDA/packages

Configuration error:
  Checking command line parameter '--db_mode' failed.
    Value 'single_container' is invalid. Please use one of: multiple_containers

This means you can create an SDC instance and be able to use HSR. I would prefer this option because you can avoid a lot of known issue regarding HANA 2.0 SPS00 (option 1) during HSR and don’t have to touch the source instance like in option 2.

After your instances are in sync you can upgrade to the latest HANA 2.0 SPS03/04 revision incl. MDC conversion during the upgrade. The only disadvantage of this scenario => it is not officially supported by SAP. I have successfully tested it with some revisions of HANA 1.0 SPS12 and as target HANA 2.0 Rev. 24.06.

Upgrade HANA 2.0 SPS02 Rev. 24.06 SDC  to SPS03 Rev. 35 MDC:

Summary before execution:
=========================
SAP HANA Database
   Update Parameters
      SAP HANA System ID: AN1
      Remote Execution: ssh
      Update Execution Mode: optimized
      Database User Name: SYSTEM
   Software Components
      SAP HANA AFL (incl.PAL,BFL,OFL)
         Do not install
      SAP HANA LCAPPS
         Do not install
      SAP HANA Database
         Update from version 2.00.024.06.1538035880 to 2.00.035.00.1545187853
         Location: /int/software/sapcd/db/hana/hana_rev20_035_0/SAP_HANA_DATABASE/server
      SAP HANA Database Client
         Update from version 2.2.74.1536258378 to 2.3.130.1542919958
         Location: /int/software/sapcd/db/hana/hana_rev20_035_0/SAP_HANA_CLIENT/client
      SAP HANA Smart Data Access
         Do not install
Note: The upgrade of SAP HANA Database to this version will convert your system to multi-tenant database containers. This is a mandatory step. For more information, see SAP Note 2423367.

Do you want to continue? (y/n): y

 

For all customers and all upcoming migration and transition scenarios it would be great if SAP can create a supported solution, because if you have to convert the old system you have to adjust your concept and test an old HANA revision which you want to deactivate. I would appreciate any direct solution for this issue. In the next years a lot of customers will face such a situation. May be Ralf Czekalla or Craig Cmehil can address this topic.

Thursday, May 28, 2020

Reclaim size of volume “/hana/log” when it is full

In SAP HANA there are four main basepath parameters which you find in the ‘configuration’ tab in SAP HANA Studio:
  • Basepath_databackup -> for space management the recommendation is to point it to an external mount point. As an alternative, back it up to a local disk with sufficient space and move the databackups to the external mount point (performance of the backup will be faster than to an external mount point)
  • Basepath_datavolumes -> permanent location for data volumes, never delete any datavolume files on OS.
  • Basepath_logbackup -> automatically copies of log segment every 15 minutes or if log segment segment is full, so a lot of files get created quickly. Two important items, first point it to an external mount point. Second, use the script attached to the note at the end of this blog to identify which log backup files you can delete.
  • Basepath_logvolumes -> permanent location for log volumes, never delete any logvolume files on OS. Optionally use “ALTER SYSTEM RECLAIM LOG” is for cleaning up this directory.
“ALTER SYSTEM RECLAIM LOG” before and after SAP HANA SPS3:
Before SAP HANA SPS3 it was recommended to run this command manually after every backup to ensure disk space was reclaimed. The command physically removes the log segments that are no longer needed. With SPS3 log backup functionality was introduced which automates log segment reuse and can eliminate the need to run “ALTER SYSTEM RECLAIM LOG”.
In most SAP HANA instances, especially productive systems, the log_mode is set to ‘normal’ and enable_auto_log_backup set to ‘yes’. This means that log backups are created automatically when a log segment is full or a log segment is closed after exceeding the configured time threshold. The log backup allows the log segment to be reused for new log entries, which eliminates the need to run “ALTER SYSTEM RECLAIM LOG” under normal circumstances. So after SAP HANA SPS3 “ALTER SYSTEM RECLAIM LOG” should only be used in exceptional situations, for example if there is a problem with writing the log backup and you get an alert the log volume disk/path is close to full.
This note can be helpful to schedule backups: https://service.sap.com/sap/support/notes/1651055 – if you read the attachment to the note it gives you an option to identify which log backups can be to deleted. In SAP HANA there are four main basepath parameters which you find in the ‘configuration’ tab in SAP HANA Studio:
  • Basepath_databackup -> for space management the recommendation is to point it to an external mount point. As an alternative, back it up to a local disk with sufficient space and move the databackups to the external mount point (performance of the backup will be faster than to an external mount point)
  • Basepath_datavolumes -> permanent location for data volumes, never delete any datavolume files on OS.
  • Basepath_logbackup -> automatically copies of log segment every 15 minutes or if log segment segment is full, so a lot of files get created quickly. Two important items, first point it to an external mount point. Second, use the script attached to the note at the end of this blog to identify which log backup files you can delete.
  • Basepath_logvolumes -> permanent location for log volumes, never delete any logvolume files on OS. Optionally use “ALTER SYSTEM RECLAIM LOG” is for cleaning up this directory.
“ALTER SYSTEM RECLAIM LOG” before and after SAP HANA SPS3:
Before SAP HANA SPS3 it was recommended to run this command manually after every backup to ensure disk space was reclaimed. The command physically removes the log segments that are no longer needed. With SPS3 log backup functionality was introduced which automates log segment reuse and can eliminate the need to run “ALTER SYSTEM RECLAIM LOG”.
In most SAP HANA instances, especially productive systems, the log_mode is set to ‘normal’ and enable_auto_log_backup set to ‘yes’. This means that log backups are created automatically when a log segment is full or a log segment is closed after exceeding the configured time threshold. The log backup allows the log segment to be reused for new log entries, which eliminates the need to run “ALTER SYSTEM RECLAIM LOG” under normal circumstances. So after SAP HANA SPS3 “ALTER SYSTEM RECLAIM LOG” should only be used in exceptional situations, for example if there is a problem with writing the log backup and you get an alert the log volume disk/path is close to full.
This note can be helpful to schedule backups: https://service.sap.com/sap/support/notes/1651055 – if you read the attachment to the note it gives you an option to identify which log backups can be to deleted.

Wednesday, May 13, 2020

SAP SYSTEM STARTUP ISSUES & SOLUTIONS





SAP System startup Problems:

Two places you need to check: EventViewer (Application and System logs) and the SAP Management Console (MMC).
Event Viewer can provide useful information and it may help you pinpoint where the problem resides. The SAP MMC gives you the ability to visually see the system status (green, yellow or red lights), view the work processes status and view the developer traces, which are stored in the "work" directory. Example: /usr/sap/TST/DVEBMGS00/work.
For a central SAP instance to start successfully, both the message server and the dispatcher need to start. If one of them or both fail to start, users cannot log in to the system. The following scenarios will illustrate possible causes of why an SAP instance might not start and the reason of the message:
"DISPATCHER EMERGENCY SHUTDOWN ".
Developer Traces:
dev_disp Dispatcher developer trace
dev_ms Message Server developer trace
dev_wp0 Work process 0 developer trace
The "services" file, which contains TCP and UDP services and their respective port numbers. This plain-text configuration file is located under winnt/system32/drivers/etc.
Windows Task Manager (TASKMGR.exe), Event Viewer (EVENTVWR.exe).
Dispatcher Monitor (DPMON.exe), which is located under /usr/sap//sys/exe/run. Database logs.

1. Dispatcher does not start due to a port conflict
No work processes (disp+work.exe) exist in Task Manager.
Dispatcher shows status "stopped" in the SAP MMC.
Errors found in "dev_disp":
*** ERROR => NiIBind: service sapdp00 in use [nixxi.c 3936]
*** ERROR => NiIDgBind: NiBind (rc=-4) [LOG Q0I=> NiPBind: bind (10048: WSAEADDRINUSE: Address already in use) [ninti.c 1488]
nixxi.c 3505]
*** ERROR => DpCommInit: NiDgBind [dpxxdisp.c 7326]
*** DP_FATAL_ERROR => DpSapEnvInit: DpCommInit
*** DISPATCHER EMERGENCY SHUTDOWN ***
Problem Analysis
I highlighted the keywords in the error messages above: Address already in use Service sapdp00 in use The TCP port number assigned in the "services" file is being occupied by another application. Due to the conflict, the dispatcher shuts down.
Solution
If your server has a firewall client, disable it and attempt to start the SAP instance again.
If the instance starts successfully you can enable the client firewall back again.
If there is no firewall client at all, or if disabling it did not resolve the problem, edit the "services" file and check what port the appropriate "sapdp" is using.
If the instance number is 00, look for sapdp00. If the instance number is 01 look for sapdp01 and so on. You can use the following OS command to help you resolve port conflicts:
netstat -p TCP There are also utilities on the Internet that can help you list all the TCP and UDP ports a system is using.

2: Dispatcher dies due to a database connection problem
database connections.
No work processes
SAP MMC -> WP Table shows all processes as "ended".
Errors found in "dev_disp":
C setuser 'tst' failed -- connect terminated
C failed to establish conn. 0
M ***LOG R19=> tskh_init, db_connect (DB-Connect 000256) [thxxhead.c 1102]
M in_ThErrHandle: 1
M *** ERROR => tskh_init: db_connect (step 1, th_errno 13, action 3, level 1) [thxxhead.c 8437]
*** ERROR => W0 (pid 2460) died [dpxxdisp.c 11651]
*** ERROR => W1 (pid 2468) died [dpxxdisp.c 11651]
*** ERROR => W2 (pid 2476) died [dpxxdisp.c 11651]. . .
*** ERROR => W11 (pid 2552) died [dpxxdisp.c 11651]
*** ERROR => W12 (pid 2592) died [dpxxdisp.c 11651]
my types changed after wp death/restart 0xbf --> 0x80
*** DP_FATAL_ERROR => DpEnvCheck: no more work processes
*** DISPATCHER EMERGENCY SHUTDOWN ***
DpModState: change server state from STARTING to SHUTDOWN
Problem Analysis
A connection to the database could not be established because either the SQL login specified in parameter "dbs/mss/schema" is set incorrectly or the SQL login was deleted from the database server. This parameter needs to be set in the DEFAULT.pfl system profile (under /usr/sap//sys/profile). In the messages above, we see that the SQL login 'tst' is expected but it does not exist at the database level.
Solution
Set the entry to the appropriate database owner. If the system is based on Basis <= 4.6 or if the system was upgraded from 4.x to 4.7 the database owner should be "dbo". But, if the system was installed from scratch and it's based on the Web AS 6.x the database owner should match the SID name in lower case. Example: if the SID is TST then the database owner should be "tst". If the parameter is set correctly in the DEFAULT.pfl profile check at the database level if the SQL login exists. If it doesn't, create it and give it database ownership to the .

3: SAP does not start at all: no message server and no dispatcher
The message server and the dispatcher do not start at all in the SAP MMC. The following error when trying to view the developer traces within the SAP MMC: The network path was not found. No new developer traces written to disk (under the "work" directory.)
Problem Analysis
The network shares "saploc" and "sapmnt" do not exist. That explains the "network path not found" message when attempting to view the developer traces within the SAP MMC.
Solution
Re-create the "saploc" and "sapmnt" network shares. Both need to be created on the /usr/sap directory

4: Users get "No logon possible" messages

Work processes start but no logins are possible.
Users get the login screen but the system does not log them in. Instead, they get this error: No logon possible (no hw ID received by mssg server).
In the SAP MMC, the message server (msg_server.exe) shows status "stopped".
The dev_ms file reports these errors:
[Thr 2548] *** ERROR => MsCommInit: NiBufListen(sapmsTST) (rc=NIESERV_UNKNOWN) [msxxserv.c 8163]
[Thr 2548] *** ERROR => MsSInit: MsSCommInit [msxxserv.c 1561]
[Thr 2548] *** ERROR => main: MsSInit [msxxserv.c 5023]
[Thr 2548] ***LOG Q02=> MsSHalt, MSStop (Msg Server 2900) [msxxserv.c 5078]
Problem Analysis
Work processes were able to start but the message server was not. The reason is because the "services" file is missing the SAP System Message Port entry. Example: SAPmsTST 3600/tcp
Solution
Edit the "services" file and add the entry. Then, re-start the instance. Make sure you specify the appropriate TCP port (e.g. 3600) for the message server.

5: The message server starts but the dispatcher doesn't

The dispatcher shows status "stopped" in the SAP MMC.
The "dev_disp" file shows these errors:
***LOG Q0A=> NiIServToNo, service_unknown (sapdp00) [nixxi.c 2580]
*** ERROR => DpCommInit: NiDgBind [dpxxdisp.c 7326]
*** DP_FATAL_ERROR => DpSapEnvInit: DpCommInit
*** DISPATCHER EMERGENCY SHUTDOWN ***
Problem Analysis
The keyword in the messages above is "service unknown" followed by the entry name "sapdp00". The dispatcher entry "sapdp00" is missing in the "services" file. Example: sapdp00 3200/tcp
Solution
Add the necessary entry in the "services" file. Example: sapdp00 3200/tcp Then, re-start the instance.

6: Work processes die soon after they start

All work processes die right after the instance is started.
The SAP MMC shows work processes with status "ended".
Only one work process shows status "wait".
An ABAP dump saying "PXA_NO_SHARED_MEMORY" is generated as soon as a user logs in.
The SAP MMC Syslog shows the following error multiple times: "SAP-Basis System: Shared Memory for PXA buffer not available".
Problem Analysis
The instance profile contains misconfigured memory-related parameters. Most likely the "abap/buffersize" instance profile parameter is set to high.
Solution
Edit the instance system profile at the OS level under /usr/sap//sys/profile and lower the value assigned to "abap/buffersize". Then, restart the instance. Also, it's important to find out if any other memory parameter were changed. If not, the system should start once the adequate memory allocation has been set to the the "abap/buffersize" parameter.


Tuesday, April 28, 2020

SAP HANA System Down, HANA not starting


how to perform checks if the SAP HANA instance is not starting. At the end of this guide, there will be frequently asked questions and common problems that are encountered.

Checks to perform

The first thing that is needed to be determined, is if the SAP HANA database is running. To do this run:

ps -ef | grep hdb



If the HANA database is running the following processes will be present

hdbnameserver

hdbpreprocessor

hdbcompileserver

hdbindexserver

hdbstatisticsserver (this may not be present as of post SP7 this could be merged into the Indexserver)



Please ensure that processes are being ran by the correct <SID>adm user incase they have multiple HANA's running on the system

If you see the running processes then please review the System Hang section.



To see if the HANA database will start try via putty going to /usr/sap/<SID>/HDB<instance#> and running

HDB start



If this fails go to /usr/sap/<SID>/HDB<instance#>/exe here you can try and run the processes manually

usually you will only need to call ./hdbnameserver and then the ./hdbindexserver and continue with the

rest if it is successful, if it is successful the issue could be with hdbdaemon or sapstartsvr and you

will check the associated logs.





Check the HANA trace files in the following location /usr/sap/<SID>/HDB<instance#>/<server name>/trace

or create a full system dump by following SAP Note 1732157 - Collecting diagnosis information for SAP HANA



The order of checking the trace files should be first the daemon, nameserver, indexserver, compileserver, and

preprocessor (statistics server would not cause the system to stop starting).

However, after checking the indexserver, you should be able to see where the error lies



Issues and Reported Problems

Common issues that we can see are:

Disk Full Error

In this the trace file will contain the words 'rc=24 no space left on device errors' for this please

review SAP Note 2083715 - Analyzing log volume full situations



Corrupt Log Segments

The trace file will say something like cannot find or cannot read a log segment at a hexadecimal address, the only

resolution to a corrupt log segment is to do a recovery that does not involve that log segment



Missing Log Segments

In the trace we will see 'Cannot open file "/<path_to_missing_logsegment>/logsegment_000_XXXXXXXX.dat", rc=2: No such file or directory'

for this please review SAP Note 1788692 - Index Server crash due to missing LogSegment file



Authorization Issues

In the trace we will see the message 'not authorized' in the trace, in this scenario check as the <SID>adm

user and see if that user can make a file in the location specified in the trace to verify this. If you

cannot create the file run the chmod command on the folder to allow reading and writing (ie chmod 764)



Hardware issue

There is no generic line in the trace would point to hardware, but if the issue is OS related or a disk cannot mount please follow the

hardware portion of the survival guide



HANA Not Starting after a failed hdblcm rename (hdbrename)

When you try to start HANA it fails with "process hdbdaemon HDB Daemon not running". No daemon, nameserver, or indexserver trace is created which indicates that it hasn't even gotten to the point of trying to start the services.

SAP Note 2142432 - SAP HANA does not start after a failed attempt to rename the HANA SID







System Crash

An SAP incident will have to be made with a full system dump (SAP Note 1732157 - Collecting diagnosis information for SAP HANA)

HANA up but SAP system not starting:

Check if a connection is possible to the database by running

R3trans -d

this will end with a return code. RC <8 is a successful connection to the database but rc=12 would be a failure. 

Check the trans.log which is produced to see further details about why the abap side of the SAP system could not connect to the database.

Here are some examples of common issues when R3trans d results in r=12



Your HANA DB rev is SPS9 (rev 90 or higher) and you see something similar to what is listed below:
"

4 ETW000  [     dev trc,00000]  Database release is HDB 1.00.090.00.1413897729                            54  0.055046 4 ETW000  [dbhdbsql.cpp,00000]  *** ERROR => Using non supported HANA version: 1.00.090.00.1413897729 4 ETW000  [dbhdbsql.cpp,00000]  *** ERROR => Min. version for this release must be 1.00.62

"

 Please see SAP Note 1952701  - DBSL supports new SAP HANA SP9 version number





Timezone and DST issues:

The system may come up but have dumps of ZDATE_LARGE_TIME_DIFF

Follow the guidelines at: http://scn.sap.com/docs/DOC-58741  

SAP Note 1932132 - SAP HANA : Large time difference between application server and HANA database

SAP KBA  2137138 - Timezone name incorrect after DST switch

Related Documents

 For DST preparations: http://scn.sap.com/docs/DOC-58741







Related Videos:





Related SAP Notes/KBAs

SAP Note 1732157 - Collecting diagnosis information for SAP HANA

SAP Note 2083715 - Analyzing log volume full situations

SAP Note 1788692 - Index Server crash due to missing LogSegment file






If SAP instance is not getting start/up

How to check, If SAP instance is not getting start/up in Linux – disp+work dispatcher IGS Watchdog Gateway ICM



How to check, If SAP instance is not getting start/up in Linux – disp+work dispatcher IGS Watchdog Gateway ICM

we can check & analyze, if an sap instance is not getting start.

There are many root causes for that. That may be

Sap buffer memory allocation issue.

Shared memory  allocation issue.

Dispatcher work process is in struct/hang state

May be port issue, etc.

Root causes and analysis – SAP instance:

Memory Allocation issues :

If your maintaining the system sizing & files system properly as per standard sap guides & as per business process requirement. After that system will allocate some types buffer memories as default min values. But some times, as per system installation working process, the work processes had some more additional requires the shared segments or increment/decrements of abap buffer sizes.In this case, we can check the analysis by executing below sappfpar command with <SID>adm user at OS level.


>/usr/sap/<SID>/SYS/exe/run/sappfpar check pf=/usr/sap/<SID>/SYS/exe/run/<profile_name> nr=<instance nuber> name=<SID> | more
Here, profile name should like “<SID>_D<instance no>_Hostname“.
After executing the command, you will get the all buffer memory allocation report & requirement with errors & warnings. As per requirement change the profile parameters values & confirm by re-executing the same.

Dispatcher is stopped :

Most of the time sap instance is not getting boot because of the respective dispatcher is not in running state. We can easily check & confirm with the below command.
> sapcontrol -nr <instance number> -function GetProcessList



Description: instance

In cause, if suppose the network issue has occurred, then respective all services will down in the server. Then if you try to start the instance services manually, It could not be start & the dispatcher is in stopped state with Gray rather than GREEN & running status. Solution : 

Find out the stopped work processes id’s (pid) by executing above command once again. Then kill that all work process manually.
> kill -9 <WP ID>


Then start the sap_instance again and also check the dispatcher status.

Gateway/Dispatcher Ports issue :

Some times both instances ASCS, PASS are started but respective Dispatcher is not in running status. Because while booting, the respective gateway/dispatcher ports 33<nn>, 32<nn> are not in free with in the server. Those ports are already established in that server. So, you need find out & kill them manually by using below commands.
fuser port/tcp or >netstat -nap | grep 33/32  : to find listening ports>fuser -k port/tcp  : to kill the listening port.
Otherwise simple reboot the application server.


Once the Database is up and running, then it should be connect through the <SID>adm user from Application server. You can cross verify it by using below command. Here, R3trans should be finished with ‘0000’.
#sidadm> R3trans -d


Description: instance
If not, it may cause due to the dispatcher & gateway not working, you can cross verify from Step 2 again. You can also check the trans.log as like below,
>su – <sid>adm
>cat trans.log



Buffer instance IPC cleanup process :

You can cleanup the ipc buffer by executing the below commands at instance level.

Switch to <sid>adm user then run the below command

cleanipc <instance no> remove
OR

cleanipc all remove

Note : Still if you face any issue, please check the below log files, which are exist under the instance work directory. Take the action accordingly.
dev_disp
dev_icm
dev_rd
dev_w0, dev_w1