qe-sap-deployment test lib
Copyright 2022 SUSE LLC
SPDX-License-Identifier: FSFAP
QE SAP <qe-sap@suse.de>
Package with common methods and default or constant values for qe-sap-deployment
Returns a hash containing file paths for configuration files
Create all needed folders
Scans yaml configuration for '%OPENQA_VARIABLE%' placeholders and
searches for values in OpenQA defined variables.
Returns hash with openqa variable key/value pairs.
Writes "ansible" section into yaml configuration file.
$args{ansible_section} defines section(key) name.
$args{section_content} defines content of names section.
Example:
@playbook_list = ("pre-cluster.yaml", "cluster_sbd_prep.yaml");
qesap_create_ansible_section(ansible_section=>'create', section_content=>\@playbook_list);
Run a command within the Python virtualenv
created by qesap_pip_install
Return is only valid if failok = 1
Return string of the python to use
Return string of the pip to use
Install all Python requirements of the qe-sap-deployment
in a dedicated virtual environment
Install all Ansible requirements of the qe-sap-deployment
qesap_upload_logs([failok=1])
Collect and upload logs present in @log_files.
Get the qe-sap-deployment code
Keep in mind that to allow qe-sap-deployment to use roles from this repo,
your config.yaml has to have a specific setting ansible::roles_path.
Replaces yaml configuration file variables with parameters
defined by OpenQA test code, yaml template or yaml schedule.
Openqa variables need to be added as a hash
with key/value pair inside %run_args{openqa_variables}.
Example:
my %variables;
$variables{HANA_SAR} = get_required_var("HANA_SAR");
$variables{HANA_CLIENT_SAR} = get_required_var("HANA_CLIENT_SAR");
qesap_yaml_replace(openqa_variables=>\%variables);
qesap_execute(cmd => $qesap_script_cmd [, verbose => 1, cmd_options => $cmd_options] );
cmd_options - allows to append additional qesap.py commands arguments
like "qesap.py terraform -d"
Example:
qesap_execute(cmd => 'terraform', cmd_options => '-d') will result in:
qesap.py terraform -d
Execute qesap glue script commands. Check project documentation for available options:
https://github.com/SUSE/qe-sap-deployment
Test only returns execution result, failure has to be handled by calling method.
qesap_execute(cmd => $qesap_script_cmd [, verbose => 1, cmd_options => $cmd_options] );
cmd_options - allows to append additional qesap.py commands arguments
like "qesap.py terraform -d"
Example:
qesap_execute(cmd => 'terraform', cmd_options => '-d') will result in:
qesap.py terraform -d
Execute qesap glue script commands. Check project documentation for available options:
https://github.com/SUSE/qe-sap-deployment
Test only returns execution result, failure has to be handled by calling method.
Search for a string in the Ansible log file.
Returns 1 if the string is found in the log file, 0 otherwise.
Return the path of the generated inventory
Get the number of cluster nodes from the inventory.yaml
Get the cluster nodes' names from the inventory.yaml
Return the path used by the qesap script as -chdir argument for terraform
It is useful if test would like to call terraform
Return the path where sap-linuxlab/community.sles-for-sap
has been installed
qesap_prepare_env(variables=>{dict with variables}, provider => 'aws');
Prepare terraform environment.
- creates file structures
- pulls git repository
- external configuration files
- installs pip requirements and OS packages
- generates configuration files with qesap script
For variables example see 'qesap_yaml_replace'
Returns only result, failure handling has to be done by calling method.
Download the playbook from the test code repo
that is on the worker within the running JompHost.
Use Ansible to run a command remotely on some or all
the hosts from the inventory.yaml
Use Ansible to run a command remotely and get the stdout.
Command could be executed with elevated privileges
qesap_ansible_script_output_file(cmd => 'crm status', provider => 'aws', host => 'vmhana01', root => 1);
It uses playbook data/sles4sap/script_output.yaml
1. ansible-playbook runs the playbook
2. the playbook executes the command remotely and redirects the output to file, both remotely
3. qesap_ansible_fetch_file downloads the file locally
4. the file is read and stored to be returned to the caller
Return is the local full path of the file containing the output of the
remotely executed command.
Return the output of a command executed on the remote machine via Ansible.
Use Ansible to fetch a file from remote.
Command could be executed with elevated privileges
qesap_ansible_fetch_file(provider => 'aws', host => 'vmhana01', root => 1);
It uses playbook data/sles4sap/fetch_file.yaml
1. ansible-playbook run the playbook
3. the playbook download the file locally
4. the file is read and stored to be returned to the caller
Return the local path of the downloaded file.
Creates a AWS credentials file as required by QE-SAP Terraform deployment code.
Creates a AWS configuration file in ~/.aws
as required by the QE-SAP Terraform & Ansible deployment code.
Return a list of the public IP addresses of the systems
deployed by qe-sap-deployment, as reported by C<terraform output>.
Needs to run after C<qesap_execute(cmd => 'terraform');> call.
Probe specified port on the remote host each 5sec till response.
Return -1 in case of timeout
Return total time of retry loop in case of pass.
Run crm report on a host and upload the resulting tarball to openqa
List of commands to collect logs from a deployed cluster
Collect logs from a deployed cluster
Compose the deployment name. It always has the JobId
Return a list of subnets. Return a single subnet for each region.
Get the vpc_id of a given instance in the cluster.
This function looks for the cluster using the aws describe-instances
and filtering by terraform deployment_name value, that qe-sap-deployment
is kind to use as tag for each resource.
Example:
qesap_aws_get_transit_gateway_vpc_attachment(name => 'SOMETHING')
Result internally in aws cli to be called like
aws ec2 describe-transit-gateway-attachments --filter='Name=tag:Name,Values=SOMETHING
Only one filter mode is supported at any time.
Returns a HASH reference to the decoded JSON returned by the AWS command or undef on failure.
Call create-transit-gateway-vpc-attachment and
wait until Transit Gateway Attachment is available.
Return 1 (true) if properly managed to create the transit-gateway-vpc-attachment
Return 0 (false) if create-transit-gateway-vpc-attachment fails or
the gateway does not become active before the timeout
Call delete-transit-gateway-vpc-attachment and
wait until Transit Gateway Attachment is deleted.
Return 1 (true) if properly managed to delete the transit-gateway-vpc-attachment
Return 0 (false) if delete-transit-gateway-vpc-attachment fails or
the gateway does not become inactive before the timeout
Generic function to compose a aws cli command with:
- `aws ec2` something
- use both `filter` and `query`
- has text output
Return the Transient Gateway ID of the IBS Mirror
Get the VPC tag Workspace
Get the Routing table: searching Routing Table with external connection
and get the Workspace tag
Create a pair of network peering between
the two provided deployments.
Return 1 (true) if the overall peering procedure completes successfully
Adds a 'ip -> name' pair in the end of /etc/hosts in the hosts
Downloads assets required for re-using infrastructure from previously exported test.
qesap_import_instances(<$test_id>)
Downloads assets required for re-using infrastructure from previously exported test.
qesap_export_instances()
Get whether a specified job is still running or not.
In cases of ambiguous responses, they are considered to be in `running` state.
Return the output of az network vnet list
Query and return the resource group used by the qe-sap-deployment
Calculate the vnet and subnet address ranges. The format is 10.ip2.ip3.0/21 and /24 respectively. ip2 and ip3 are calculated using the slot number as seed.
Create a pair of network peering between
the two provided deployments.
Delete a single peering one way
Delete all the network peering between the two provided deployments.
Search for all network peering related to both:
- resource group related to the current job
- the provided resource group.
Returns the peering name or
empty string if a peering doesn't exist
Get active peering for Azure jobs
Delete leftover peering for Azure jobs that finished without cleaning up
Gets the native fencing type (spn/msi)
qesap_az_setup_native_fencing_permissions(vmname=>$vm_name,
resource_group=>$resource_group);
Sets up managed identity (MSI) by enabling system assigned identity and
role 'Virtual Machine Contributor'
qesap_az_enable_system_assigned_identity($vm_name, $resource_group);
Enables 'System assigned identity' for specified VM.
Returns 'systemAssignedIdentity' ID.
qesap_az_assign_role( assignee=>$assignee, role=>$role, resource_group=>$resource_group )
Assigns defined role to 'assignee' (user, vm, etc...) using subscription id.
assignee - UUID for the resource (VM in this case)
role - role to be assigned
resource_group - resource group resource belongs to
qesap_az_get_tenant_id( subscription_id=>$subscription_id )
Returns tenant ID related to the specified subscription ID.
subscription_id - valid azure subscription
qesap_az_validate_uuid_pattern( uuid_string=>$uuid_string )
Function checks input string against uuid pattern
which is commonly used as an identifier for azure resources.
returns uuid (true) on match, 0 (false) on mismatch.
Generate a SAS URI token for a storage container of choice
Return the token string
Call `az vm boot-diagnostics json` for each running VM in the resource group associated to this openQA job
Return a list of diagnostic file paths on the JumpHost
qesap_terraform_clean_up_retry()
Perform terraform destroy and catch and ignore any error.
This method is mostly useful when doing cleanup before retry in case of
Ansible failed on 'Timed out waiting for last boot time check'
qesap_terrafom_ansible_deploy_retry( error_log=>$error_log )
error_log - ansible error log file name
Retry to deploy terraform + ansible
Return 0: we manage the failure properly
Return 1: something went wrong or we do not know what to do with the failure