• <= Back to file list
• DESCRIPTION
  • run_cmd run_cmd(cmd => 'command', [runas => 'user', timeout => 60]);
  • run_cmd_retry run_cmd_retry(cmd => 'command', [retry => 3, timeout => 60, delay => 10]);
  • get_promoted_hostname() get_promoted_hostname();
  • deployment_cleanup
  • get_hana_topology Parses command output, returns hash of hashes containing values for each host.
  • is_hana_online is_hana_online([timeout => 120, wait_for_start => 'false']);
  • is_hana_resource_running is_hana_resource_running([quiet => 0]);
  • wait_hana_node_up wait_hana_node_up($my_instance, [timeout => 900, ssh_keepalive => 1]);
  • stop_hana stop_hana([timeout => $timeout, method => $method]);
  • start_hana start_hana([timeout => 60]);
  • cleanup_resource cleanup_resource([timeout => 60]);
  • record_takeover_diagnostics record_takeover_diagnostics();
  • is_local_primary_recovery_aborting_takeover
  • check_takeover check_takeover();
  • enable_replication enable_replication([site_name => 'site_a', hana_inn => '10' ]);
  • get_replication_info get_replication_info();
  • get_promoted_instance get_promoted_instance();
  • wait_for_sync wait_for_sync([timeout => $timeout]);
  • wait_for_pacemaker wait_for_pacemaker([timeout => $timeout]);
  • change_sbd_service_timeout $self->change_sbd_service_timeout(service_timeout => '30');
  • setup_sbd_delay_publiccloud $self->setup_sbd_delay_publiccloud();
  • sbd_delay_formula $self->sbd_delay_formula();
  • cloud_file_content_replace cloud_file_content_replace(filename => $filename, search_pattern => $search_pattern, replace_with => $replace_with);
  • create_instance_data
  • deployment_name
  • create_ansible_playbook_list
  • azure_fencing_agents_playbook_args
  • get_hana_site_names
  • create_hana_vars_section
  • display_full_status
  • list_cluster_nodes
  • get_hana_database_status
  • is_hana_database_online
  • is_primary_node_online
  • pacemaker_version
  • get_online_string
  • saphanasr_showAttr_version
  • wait_for_cluster
  • wait_for_zypper
  • check_zypper_ref
  • wait_for_idle

lib/sles4sap/publiccloud.pm

DESCRIPTION

Package with common methods and default or constant  values for sles4sap tests in the cloud

run_cmd run_cmd(cmd => 'command', [runas => 'user', timeout => 60]);

Runs a command C<cmd> via ssh in the given VM and log the output.
All commands are executed through C<sudo>.
If 'runas' defined, command will be executed as specified user,
otherwise it will be executed as root.

cmd - command string to be executed remotely

timeout - command execution timeout

title - used in record_info

runas - pre-pend the command with su to execute it as specific user

ssh_keepalive - optional: enable SSH keepalive options to detect hung connections faster

... - pass through all other arguments supported by ssh_script_run

run_cmd_retry run_cmd_retry(cmd => 'command', [retry => 3, timeout => 60, delay => 10]);

Runs a command C<cmd> via ssh in the given VM and log the output.
If command fails or C<timeout> is reached, then the command will be
retried for C<retry> times, with C<delay> seconds of interval in between.

cmd - command string to be executed remotely

timeout - command execution timeout

retry - number of retry attempts

delay - idle time between retry attempts

ssh_keepalive - optional: enable SSH keepalive options and send ETX on failure

... - pass through all other arguments supported by run_cmd

get_promoted_hostname() get_promoted_hostname();

Checks and returns hostname of HANA promoted node according to crm shell output.

deployment_cleanup

Clean up Network peering and qesap deployment
This method does not internally die and try to execute
terraform destroy in any case.
Return 0 if no internal error.

cleanup_called - flag to indicate cleanup status

ansible_present - flag to indicate ansible has need executed as part of the deployment

get_hana_topology Parses command output, returns hash of hashes containing values for each host.

is_hana_online is_hana_online([timeout => 120, wait_for_start => 'false']);

Check if hana DB is online.

wait_for_start - Define 'wait_for_start' to wait for DB to start.

timeout - timeout for the wait of the online state

is_hana_resource_running is_hana_resource_running([quiet => 0]);

Checks if resource msl_SAPHanaCtl_* is running on given node.

quiet - if set, returns the value without recording info (default: 0)

wait_hana_node_up wait_hana_node_up($my_instance, [timeout => 900, ssh_keepalive => 1]);

Waits until 'is-system-running' returns successfully on the target instance.

instance - the instance the test needs to wait for

timeout - how much time to wait for before aborting

ssh_keepalive - optional: enable SSH keepalive options

stop_hana stop_hana([timeout => $timeout, method => $method]);

Stops HANA database using default or specified method.
"stop" - stops database using "HDB stop" command.
"kill" - kills database processes using "HDB -kill" command.
"crash" - crashes entire OS using "/proc/sysrq-trigger" method.

method - Allow to specify a specific stop method

timeout - only used for stop and kill

online_string - mandatory string used to match node state in topology output; '4' for pacemaker >= 2.1.7 (lss score), 'online' for older versions. Use $self-get_online_string()> to compute this value.

start_hana start_hana([timeout => 60]);

Start HANA DB using "HDB start" command

cleanup_resource cleanup_resource([timeout => 60]);

Cleanup resource 'msl_SAPHanaCtl_*', wait for DB start automatically.

timeout - timeout for waiting resource to start

record_takeover_diagnostics record_takeover_diagnostics();

Collect cluster and HANA system replication diagnostics for takeover failures.

is_local_primary_recovery_aborting_takeover

Returns 1 when the cluster appears to have restarted HANA as primary on the
local node while takeover to the peer has not completed yet.

Requires the database to be online again. During a normal slow takeover the
local node can stay Promoted with SR mode PRIMARY while the database is still
offline; that state must not be treated as recovery.

check_takeover check_takeover();

Checks takeover status and waits for finish until successful or reaches timeout.
Waits for cluster topology to show takeover first, then verifies local system
replication left PRIMARY mode. Dies with diagnostics on failure.

enable_replication enable_replication([site_name => 'site_a', hana_inn => '10' ]);

Re-enables replication on the previously fenced database node. Database must be offline.
Dies if the the given <site name> doesn't correspond with the SAPHanaSR topology local site name
Optionaly the 'hana_innstance_id' could be provided for better reliability

site_name - local HANA site name of the site to be registered

hana_inn - optional HANA instance number to be registered

get_replication_info get_replication_info();

Parses "hdbnsutil -sr_state" command output.
Returns hash of found values converted to lowercase and replaces spaces to underscores.

get_promoted_instance get_promoted_instance();

Retrieves hostname from currently promoted (Master) database and returns instance data from $self->{instances}.

wait_for_sync wait_for_sync([timeout => $timeout]);

Wait for replica site to sync data with primary.
Checks "SAPHanaSR-showAttr" output and ensures replica site has "sync_state" "SOK && PRIM" and no SFAIL.
 Continue after expected output matched N times continually to make sure cluster is synced.

Expected conditions:
 - Both primary and replica must be online.
 - primary must have sync_state 'PRIM'
 - primary must have clone_state 'PROMOTED'
 - replica must have sync_state 'SOK'  - this means data is in sync
 - replica must have clone_state 'DEMOTED'
 - site order does not matter

timeout - timeout for waiting sync state

online_string - mandatory string used to match node state in topology output; '4' for pacemaker >= 2.1.7 (lss score), 'online' for older versions. Use $self-get_online_string()> to compute this value.

wait_for_pacemaker wait_for_pacemaker([timeout => $timeout]);

Checks status of pacemaker via systemd 'is-active' command an waits for startup.

timeout - timeout for waiting for pacemaker service

change_sbd_service_timeout $self->change_sbd_service_timeout(service_timeout => '30');

Overrides timeout for sbd systemd service to a value provided by argument.
This is done by creating or changing file "/etc/systemd/system/sbd.service.d/sbd_delay_start.conf"

service_timeout - value for the TimeoutSec setting

setup_sbd_delay_publiccloud $self->setup_sbd_delay_publiccloud();

Set (activate or deactivate) SBD_DELAY_START setting in /etc/sysconfig/sbd.
Delay is used in case of cluster VM joining cluster too quickly after fencing operation.
For more information check sbd man page.

Setting is changed via OpenQA parameter: HA_SBD_START_DELAY
Possible values:
"no" - do not set and turn off SBD delay time
"yes" - sets default SBD value which is calculated from a formula
"<number of seconds>" - sets specific delay in seconds

Returns integer representing wait time.

sbd_delay_formula $self->sbd_delay_formula();

return calculated sbd delay

cloud_file_content_replace cloud_file_content_replace(filename => $filename, search_pattern => $search_pattern, replace_with => $replace_with);

Replaces file content direct on PC SUT. Similar to lib/utils.pm file_content_replace()

filename - file location

search_pattern - search pattern

replace_with - string to replace

create_instance_data

Create and populate a list of publiccloud::instance and publiccloud::provider compatible
class instances.

provider - Instance of PC object "provider", the one usually created by provider_factory()

deployment_name

Return a string to be used as value for the deployment_name variable
in the qe-sap-deployment.

create_ansible_playbook_list

Detects HANA/HA scenario from function arguments and returns a list of ansible playbooks to include
in the "ansible: create:" section of config.yaml file.

ha_enabled - Enable the installation of HANA and the cluster configuration

registration - select registration mode, possible values are * registercloudguest (default) * suseconnect * noreg skip scheduling of register.yaml at all

scc_code - registration code

ltss - name and reg_code for LTSS extension to register. This argument is a two element comma separated list string. Like: 'SLES-LTSS-Extended-Security/12.5/x86_64,123456789' First string before the comma has to be a valid SCC extension name, later used by Ansible as argument for SUSEConnect or registercloudguest argument. Second string has to be valid registration code for the particular LTSS extension.

fencing - select fencing mechanism

fence_type - select Azure native fencing mechanism. Only two accepted values 'spn' or 'msi'. This argument is only applicable to Azure. (optional)

spn_application_id - Application ID for the SPN Azure native fencing agent.This argument is only applicable to Azure configured with native fencing of type SPN. (optional)

spn_application_password - password for the SPN Azure native fencing agent.This argument is only applicable to Azure configured with native fencing of type SPN. (optional)

ptf_files - list of PTF files (optional)

ptf_token - SAS token to access the PTF files (optional)

ptf_account - name of the account for the ptf container (optional)

ptf_container - name of the container for PTF files (optional)

ibsm_ip - IP address of the IBS Mirror (optional)

download_hostname - hostname associated to the IBS Mirror IP address on each hana node (optional)

repos - comma separated list of repos (optional)

azure_fencing_agents_playbook_args

azure_fencing_agents_playbook_args(
    fence_type => 'spn'
    spn_application_id=>$spn_application_id,
    spn_application_password=>$spn_application_password
);

Collects data and creates string of arguments that can be supplied to playbook.

FENCE_TYPE - Azure native fence type (spn|msi)

SPN_APPLICATION_ID - application ID that allows API access for STONITH device

SPN_APPLICATION_PASSWORD - password provided for application ID above

get_hana_site_names

Get primary and secondary site name.
This information is both needed to configure the qe-sap-deployment
and later in the test when calling `hdbnsutil -sr_register`.
This function mostly read the information from job settings
HANA_PRIMARY_SITE and HANA_SECONDARY_SITE, so main reason to have
it behind a function is to have coherent defaults.

create_hana_vars_section

Detects HANA/HA scenario from openQA variables and create
data later used for "ansible: hana_vars:" section in config.yaml file.

display_full_status

$self->display_full_status()

Displays most useful debugging info about cluster status, database status within a 'record_info' test entry.

list_cluster_nodes

$self->list_cluster_nodes()

Returns list of hostnames that are part of a cluster using crm shell command from one of the cluster nodes.

get_hana_database_status

Run a query to the hana database, parses "hdbsql" command output and check if the connection still is alive.
Returns 1 if the response from hana database is online, 0 otherwise

password_db - password

instance_id - instance id

is_hana_database_online

Setup a timeout and check the hana database status is offline and there is not connection.
Requires several consecutive offline readings before treating the database as offline.
If the connection still is online run a wait and try again to get the status.
Returns 1 if the output of the hana database is online, 0 means that hana database is offline

timeout - default 900

total_consecutive_passes - consecutive offline polls required - default 5

is_primary_node_online

Check if primary node in a hana cluster is offline.
Returns if primary node status is offline with 0 and 1 online

timeout - wait time for PRIMARY mode to disappear, default 600

cmd_timeout - timeout for each systemReplicationStatus.py call, default 300

pacemaker_version

Returns the pacemaker version

retries - (OPTIONAL) amount of retries to attempt - default 0

sleep_time - (OPTIONAL) seconds to sleep between retries - default 5

get_online_string

$self->get_online_string()

Returns the string used to match the online/active node state in C<check_hana_topology>.
For pacemaker >= 2.1.7 this is C<'4'> (lss score); for older versions it is C<'online'>.
Use this helper wherever C<online_string> must be computed from the cluster's pacemaker version.

saphanasr_showAttr_version

Returns the SAPHanaSR-showattr version

wait_for_cluster

Verifies that nodes are online, resources are started and DB is in sync

wait_time - time to wait before retry in seconds, default 10

max_retries - maximum number of retries, default 7

online_string - mandatory string used to match node state in topology output; '4' for pacemaker >= 2.1.7 (lss score), 'online' for older versions. Use $self-get_online_string()> to compute this value.

wait_for_zypper

The function attempts to run 'zypper ref' to check for a lock. If Zypper is locked, it waits for a specified delay before retrying.
Returns normally if Zypper is not locked or dies after a maximum number of retries if Zypper remains locked.

$instance - The instance object on which the Zypper command is executed. This object must have the ssh_script_run method implemented.

max_retries - The maximum number of times the function will retry checking if Zypper is locked. Default is 10.

retry_delay - The number of seconds to wait between retries. Default is 20 seconds.

timeout - The number of seconds to wait before aborting zypper ref ssh_script_run =item runas - If 'runas' defined, command will be executed as specified user, otherwise it will be executed as cloudadmin.

check_zypper_ref

The function attempts to run 'zypper ref' and check for errors related to repositories.

$instance - The instance object on which the Zypper command is executed. This object must have the ssh_script_run method implemented.

runas - If 'runas' defined, command will be executed as specified user, otherwise it will be executed as cloudadmin.

wait_for_idle

The function wraps the `cs_wait_for_idle` command, and restarts in case of timeout (once, this
time fatal) after displaying cluster information.

$timeout - The timeout (in seconds) for the command.