Dovetailed Technologies

5. Co:Z Launcher Examples

This chapter contains common examples ("recipes") for using the Co:Z Launcher. These examples assume that you have installed and configured the Co:Z toolkit on your z/OS and target systems. Additionally, these examples rely on Dataset Pipes commands. See the Co:Z Dataset Pipes User's Guide for the command reference.

5.1 Execute commands on a target server

This is a simple example of how to use the Co:Z Launcher to run commands on a remote Linux server.

//COZCB1  JOB (),'COZ'
//STEP1   EXEC PROC=COZPROC,
//        ARGS='myuid@linux1.myco.com'
//STDIN   DD *                        
# This is input to the remote shell
echo "We are running on: " `uname -sr`
// 
    
  • The userid and hostname (myuid@linux1.myco.com) are given as a parameter to the COZPROC stored procedure, but all other configuration options are taken from the installation defaults.

  • The Co:Z Launcher will start an SSH connection to the remote server as user "myuid".

  • Since SSH is unable in a batch job to prompt for a password, it will use a private key associated with the current z/OS user to login to the target server.

  • The default program to launch on the target server is the user's "default shell", which happens to be "bash".

  • Input to the remote shell is redirected from the job's //STDIN DD.

  • Output from the remote shell is redirected to //STDOUT DD and //STDERR DD in the launching jobstep. By default these are defined in COZPROC to go to SYSOUT spool files.

  • Output from the remote shell is redirected to //STDOUT DD and //STDERR DD in the launching jobstep. By default these are defined in COZPROC to go to SYSOUT spool files.

In this example, the following output will be written to the //STDOUT DD:

We are running on:  Linux 2.6.15-27-k7
      

The exit code from the remote program (bash) will be adopted as the return code for the batch job step; in this case: "0". Log messages from the Co:Z Toolkit are written to //SYSOUT DD:

fromdsn(DD:STDIN)[N]: 2 records/160 bytes read; 75 bytes written in 0 milliseconds.       
todsn(DD:STDERR)[N]: 0 bytes read; 0 records/0 bytes written in 0.072 seconds (0.000 Bytes/Sec
todsn(DD:STDOUT)[N]: 39 bytes read; 1 records/38 bytes written in 0.074 seconds (527.027 Bytes
CoZLauncher[N]: myuid@linux1.myco.com target command '<default shell>' ended with RC=0       
      

5.2 Launch remote shell that reads a PDS member

In this example we use the Co:Z Launcher to send commands to a target Linux server which reads a PDS member from the launching z/OS system.

//COZCB2  JOB (),'COZ' 
//STEP1   EXEC PROC=COZPROC, 
//        ARGS='myuid@linux1.myco.com' 
//STDIN   DD * 
fromdsn '//sys1.maclib(acb)' | grep BLKSIZE 
// 
    
  • Input to the remote shell is redirected from the job step's //STDIN DD, which in this example has a single line.

  • The fromdsn command, running on the target server, establishes a connection with the launching z/OS job. This connection is used to read a PDS member.

  • The single quotes are required so that the Linux shell does not interpret the parentheses as meta characters.

  • The fromdsn command converts the records in the dataset to a stream of bytes that is written to stdout. By default the data will be converted to a text file using the target platform's codepage and line separator.

  • The data is piped ('|') by the shell into the Unix grep command which writes matching lines to stdout.

  • Output from the remote shell is redirected to //STDOUT DD and //STDERR DD in the launching jobstep. By default, these are defined in COZPROC to go to SYSOUT spool files.

In this example, the following output will be written to the //STDOUT DD:

&BLKSIZE=0,&LRECL=0,&BUFSP=0,                           -00001600
BLKSIZE=&BLKSIZE,LRECL=&LRECL,                          -01700000
BLKSIZE=&BLKSIZE,LRECL=&LRECL,                          -02406800
      

5.3 Offload PGP encryption of MVS Datasets

In this example we use the Co:Z Launcher to send commands to a Linux server which reads data from an //INPUT DD in the launching job step and writes PGP encrypted output data to //OUTPUT DD.

  
//COZCB3  JOB (),'COZ' 
//STEP1   EXEC PROC=COZPROC, 
//        ARGS='myuid@linux1.myco.com' 
//STDIN   DD * 
fromdsn -l rdw -k //DD:INPUT   \ 
  | gpg -r key-1 --batch --output=- --encrypt=- \ 
  | todsn -b //DD:OUTPUT 
/* 
//INPUT DD DISP=SHR,DSN=KIRK.CLEARTEXT.DATA 
//OUTPUT DD DSN=KIRK.ENCRYPT,DISP=(NEW,PASS), 
//          SPACE=(CYL,(1,1),RLSE), 
//          DCB=(RECFM=U,BLKSIZE=4096) 
    
  • Input to the remote shell is redirected from the job step's //STDIN DD, which in this example contain three commands chained together with Unix pipes.

  • The fromdsn command, running on the target server, establishes a connection with the launching z/OS job. This connection is used to read from the //INPUT DD. The -l rdw option is used so that 4 byte RDWs are used as record separators. This option also disables any default codepage translation. The -k option disables any trimming of trailing pad (space) characters from the end of records. The result is that the fromdsn simply writes RDW-prefixed records, as-is, to stdout.

  • The output from fromdsn is piped into the Linux gpg command which PGP-encrypts the data stream. The "-encrypt=-" option causes gpg to read input from stdin (the output pipe from fromdsn).

  • The "output=-" option causes gpg to write its encrypted output to stdout, which is piped ('|') into a todsn command.

  • The todsn command, running on the Linux server, tunnels back into the launching jobstep and writes the encrypted data stream to DD:OUTPUT, which in the example goes to new cataloged MVS dataset. The "-b" option causes fromdsn to write the records in binary, with no record separators, in effect filling each record to its maximum size, which is set by the DD card in this case to be 4096 bytes.

  • Output log messages from the Co:Z Launcher, the Co:Z Agent (running on the target server), and the Dataset Pipes utilities fromdsn and todsn are written to //SYSOUT DD.

In this example, the following log message will be written:

fromdsn(DD:STDIN)[N]: 3 records/240 bytes read; 106 bytes written in 0 milliseconds.                       
fromdsn(DD:INPUT)[N]: 78 records/6240 bytes read; 6552 bytes written in 0 milliseconds.                    
todsn(DD:OUTPUT)[N]: 2034 bytes read; 2 records/2034 bytes written in 0.038 seconds (52.272 KByt
todsn(DD:STDOUT)[N]: 0 bytes read; 0 records/0 bytes written in 0.708 seconds (0.000 Bytes/sec).           
todsn(DD:STDERR)[N]: 0 bytes read; 0 records/0 bytes written in 0.706 seconds (0.000 Bytes/sec).           
CoZLauncher[N]: myuid@linux1.myco.com target command '<default shell>' ended with RC=0
      

Note that the data is encrypted during transfer automatically by the SSH tunnel used by Co:Z to communicate between the target server and the launching batch job. Also note that the file is never stored on disk on the target server.

Decrypting is just as easy:

fromdsn -b  //DD:INPUT   \
 | gpg -r key-1 --batch --output=- --decrypt=- \
 | todsn -l rdw  //DD:OUTPUT 
      

5.4 Use a Linux server as a secure gateway to information on the Internet

In this example the Co:Z Launcher is used to run a script on a server which downloads a tab-delimited file from the Internet and converts selected fields to SQL statements which are written to a temporary MVS dataset. A second step in the job runs the DB2 batch SPUFI utility to load the data to a DB2 table.

//COZCB4  JOB (),'COZ'
//********************************************************************** 
//* STEP1: Launch a remote script to ftp download a tab-delimited 
//* text file.  Use selected columns to generate DB2 INSERT statments 
//* which are written to an MVS temporary dataset. 
//* 
//STEP1   EXEC PROC=COZPROC,ARGS='cozcb4@dmz1.myco.com' 
//STDIN   DD * 
wget -O- ftp://ftp.visi.com/users/juan/ContactingCongress.db.txt | 
awk -F "\t" -v sq="'" '{ 
  if (NR == 1)  #skip header/empty table 
    print "DELETE FROM CONGRESS;" 
  else { 
    print "INSERT INTO CONGRESS VALUES(" 
    print sq $1 sq ", " 
    print sq $2 sq ", " 
    print sq $4 sq ", " 
    print sq $5 sq ", " 
    print sq $3 sq 
    print ");" 
  } 
'} 
//STDOUT DD DSN=&&SPUFIN,DISP=(NEW,PASS),SPACE=(CYL,(2,1)), 
//       DCB=(RECFM=FB,LRECL=80) 
//********************************************************************** 
/* STEP2: Run DB2 "SPUFI" in batch to execute the insert statements 
/* to reload a DB2 table 
/* 
//STEP2    EXEC PGM=IKJEFT01,DYNAMNBR=20,COND=(0,NE) 
//SYSTSPRT DD SYSOUT=* 
//SYSPRINT DD SYSOUT=* 
//SYSTSIN  DD * 
DSN SYSTEM(DBS1) 
RUN PROGRAM(DSNTEP2) PLAN(DSNTEP71) LIB('DB2V810.RUNLIB.LOAD') 
END 
//SYSIN    DD DSN=&&SPUFIN,DISP=(OLD,DELETE) 
// 
    
  • //STEP1.STDIN DD contains command input to the remote shell. The wget command is used to download a file from the internet using the ftp protocol, piping the output into the awk command.

  • The awk command script, running on the Linux target server, is used to reformat selected columns from the tab-delimited file into SQL INSERT commands, which are written to stdout.

  • //STEP1.STDOUT DD is overridden to point to a temporary MVS dataset, which is passed to STEP2.

  • The second step runs the DB2 batch SPUFI utility to execute the SQL, thereby loading the CONGRESS table.

In environments where the z/OS mainframe is not connected to the Internet, the target server may be deployed in a DMZ which is only accessible in one direction from the z/OS host using SSH. Only the target process started by the Co:Z launcher (and its children) have access to redirected I/O resources in the launching Co:Z job step.

This example also demonstrates how open source Linux tools (wget, awk) may be used to access and transform data for use within the z/OS environment.

5.5 Offload processing of SMF data to a Linux system

In this example the Co:Z Launcher is used to offload processing of z/OS SMF data to a Linux system.

//USERC4  JOB (),'DOVETAIL',MSGCLASS=H,NOTIFY=&SYSUID 
//PROCLIB JCLLIB ORDER='USER.COZ.SAMPJCL' 
//* 
//*********************************************************************
//* 
//* This Co:Z example dumps SMF records to a temporary dataset then
//* remotely processes the records.  The sample program smfp.c 
//* reports statistics on the number and types of SMF records
//* processed.  It can easily be modified to perform other processing,
//* or replaced by a SAS program.
//* 
//* Tailor the proc and job for your installation: 
//* 1.) Modify the Job card per your installation's requirements 
//* 2.) Modify the PROCLIB card to point to this PDS, or wherever 
//*     the COZPROC procedure has been installed. 
//* 3.) Modify the SMF dataset names for your installation.
//* 4.) Compile the remote SMF processing program "smfp.c" on the 
//*     target system:
//*         gcc -o smfp smfp.c.  
//*     Ensure that the executable (smfc) is in the path.
//*
//* When executed, smfp will write a report to stdout similar to the
//* following:
//*
//*                         ------ Length ------
//*Type    Count     Pct    Min     Max      Avg
//*   2        1       0     14      14       14
//*   3        1       0     14      14       14
//*   4        1       0    291     291      291
//*   5        1       0    142     142      142
//*  20        3       0     87      88       87
//*  30      129       2    394    1337      928
//*  42     6304      97    192   19768      467
//*  80        5       0    351     376      360
//*  89        2       0    362    4018     2190
//* 177        1       0     47      47       47
//*
//*6448 SMF records processed
//*
//*********************************************************************
//*
//DUMPSMF  EXEC PGM=IFASMFDP
//SYSPRINT DD   SYSOUT=*                                          
//SMFDATA  DD   DISP=SHR,DSN=SYS1.SMF.DATA                      
//SMFOUT   DD   SYSOUT=*                                          
//OUTDD    DD   DSN=&&SMFUNLD,DISP=(NEW,PASS),
//         UNIT=SYSDA,SPACE=(CYL,(20,20))                                  
//ADUPRINT DD   SYSOUT=*                                          
//SYSIN    DD   *                                                 
    INDD(SMFDATA,OPTIONS(DUMP))                                   
    OUTDD(OUTDD,TYPE(000:255))                                    
/*                                                                
//RUNCOZ EXEC PROC=COZPROC,ARGS='user@linux1.myco.com'
//SMFUNLD  DD   DSN=&&SMFUNLD,DISP=(OLD,DELETE,DELETE) 
//STDIN    DD   * 
fromdsn -b -l rdw //DD:SMFUNLD | smfp
//
    
  • In most installations, this job would be started as a started task by a IEFU29 SMF dump exit, passing the name of the SMF dataset to dump as an argument to the proc.

  • Step DUMPSMF unloads the SMF dataset to a temporary dataset.

  • The RUNCOZ step runs the Co:Z Launcher to launch a shell on the target Linux system.

  • STDIN input to the Co:Z launcher runs the fromdsn command on the Linux target system, which reaches back into the launching z/OS job to read the contents of DD SMFUNLD. The -b and -l rdw switches cause the SMF data to be read in binary, without translation, and each SMF record to be prefixed by a 4-byte RDW. fromdsn supports several RDW layouts, including -ibmrdw and -mfrdw (for MicroFocus output)

  • Source code for the smfp sample program is available on the downloads page.

Copyright© 2009-2017 Dovetailed Technologies, LLC. All rights reserved.
Co:Z® is a registered trademark of Dovetailed Technologies, LLC.