Using “Expect” in UNIX Script

There is a request to create script. The idea is to send some commands to the routers (around 16 routers in total) from remote Unix server and save the outputs to file.

The problem is, this routers haven’t had capability yet for password less setup by using key exchange which is more secure.

There were several options, but all of them are using password “hardcoded”, meaning the password of the router must be written inside the script.

Finally, “expect” is chose, since it is already installed in the remote Unix server. Moreover, according to rule in the company, any additional software package can not be installed unless there is approval.

Below are codes. Let’s name it as getoutput and make it to be executable.

#!/usr/local/bin/expect -f

set uu "<username>"
set pp "<password>"

set hh [lindex $argv 0]

spawn ssh -oStrictHostKeyChecking=no -oCheckHostIP=no $uu@$hh

expect "*password*"
send -- "$pp\r"
expect "*>*"
send -- "<1st command >"
send -- "\r"
expect "*>*"
send -- "<2nd command>"
send -- "\r"
expect "*>*"
send -- "<3rd command>"
send -- "\r"
send -- "<4th command>"
send -- "\r"

# done
expect eof

Then loop above script to be executed for all routers.

#!/bin/bash
for router in `cat routers_ip_file`; do getoutput $router > $router.outputfile.log; done

Note:

There are StrictHostKeyChecking and CheckHostIP parameters whose value as “no” inside above script. Actually the purpose is to avoid host checking prompt, if the ssh session is performed for the first time to destination host.

Step by Step: SSH Tunneling with Putty

Supposed there is server A (UNIX based), where you have firewall opened to do SSH connection from your PC directly. And there is another server (server B), where it’s connected from Server A with SSH but you have no direct SSH connection from your PC to server B (firewall is not opened from your PC to server B) .

This article will try to give tips, to make direct SSH connection possible with SSH tunneling from your PC toward server B by using Putty tool.

Note: as pre-requisite to do SSH tunneling, please set AllowTcpForwarding parameter with value equal Yes inside sshd_config fileof Server A, then restart ssh service

server-A> grep -i tcp /etc/ssh/sshd_config
AllowTcpForwarding yes
server-A> svcadm restart ssh

Method#1- Using Dynamic Port

First, connect to Server A and setup the tunnel using dynamic port

Establish SSH connection to server A using Putty. Once connected, open change settings menu (right click on top panel and choose change settings). There is category box on left side, go to Connection > SSH > then click Tunnels. Put any available local port on Source port text box (in this example, 4000 is chosen), tick Dynamic radio button, after that click Add then Apply.

Second, connect to Server B using tunnel that already set

Prepare new putty session to connect to Server B, after putting the IP and port on session part, before clicking on Open button, go to Connection > then Proxy. Tick SOCKS 5 radio button on proxy type, then on proxy hostname, fill localhost and port 4000. Click open, then now you can connect to Server B directly by using server A as proxy to make a tunnel.

Method#2- Put IP and port of another server explicitly

First, connect to Server A and setup the tunnel using specific IP and port of Server B

Establish SSH connection to server A using Putty. Once connected, open change settings menu (right click on top panel and choose change settings). There is category box on left side, go to Connection > SSH > then click Tunnels. Put any available local port on Source port text box (in this example, 4001 is chosen). And on Destination, fill explicitly the IP and port of server B with format IP:port, in this example, value is 10.10.10.10:22), tick Local radio button, after that click Add then Apply.

Second, connect to Server B using tunnel that already set

Prepare new putty session to connect to Server B, on Hostname (or IP address), fill with localhost and port, is 4001. Click open, then connection to Server B can be done directly.

How to Resolve TAR error: “filename is greater than 100” in UNIX

For instance, we want to tar one file called: adkfhasdkfhkjbasdfkusdbnwqpadwlfpwemrnfuabfasdfyasdfbasdflasdfdyfdfafdlsasdflkjasdflkasdfkljasdflkj.xml

Normal tar processing, will give an error as below:

# tar -cf result.tar adkfhasdkfhkjbasdfkusdbnwqpadwlfpwemrnfuabfasdfyasdfbasdflasdfdyfdfafdlsasdflkjasdflkasdfkljasdflkj.xml
tar: adkfhasdkfhkjbasdfkusdbnwqpadwlfpwemrnfuabfasdfyasdfbasdflasdfdyfdfafdlsasdflkjasdflkasdfkljasdflkj.xml: filename is greater than 100

To tackle such above issue, please run below command:

#  tar cEvf - adkfhasdkfhkjbasdfkusdbnwqpadwlfpwemrnfuabfasdfyasdfbasdflasdfdyfdfafdlsasdflkjasdflkasdfkljasdflkj.xml > result.tar

There is another case, for example, a folder (named: TESTFOLDER) containing lot of files whose filename’s length is more than 100 chars. The purpose is to tar and then compress it directly. Executing following command will help to deal with it:

#  tar cEvf - TESTFOLDER | gzip > result.tar.gz

How to Troubleshoot “Not Working SSH Password Less Setup”

Usually, ssh password less setup that i have done many times before were working fine. But, one time it was not, albeit same instructions were followed. Similar hardware, similar operating system. Interesting!

Finally, after spent some time, i got it resolved. Here are summarized notes of my experience for additional guidance to troubleshoot. Hopefully it will be also useful for anybody having similar issue.

1# Run ssh with verbose mode from source server toward destination server having problem to see for any error or useful message as clue

debug1: Failed to acquire GSS-API credentials for any mechanisms (No credentials were supplied, or the credentials were unavailable or inaccessible
Unknown code 0
)
debug1: Next authentication method: publickey
debug1: Trying private key: /.ssh/identity
debug1: Trying public key: /.ssh/id_rsa
debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
debug1: Trying public key: /.ssh/id_dsa
debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
debug1: Next authentication method: keyboard-interactive
Password:

2# Do the same (ssh with verbose mode), but now toward destination server having no issue with password less

debug1: Next authentication method: publickey
debug1: Trying private key: /.ssh/identity
debug1: Trying public key: /.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 149 lastkey 80aaf80 hint 1
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey)
debug1: SSH receive window size: 198560 B
debug1: channel 0: new [client-session]
debug1: send channel open 0

3# By comparing both logs, found that, it seems destination server does not accept key from source server. From log in point 2#, can be seen Server accepts key and Authentication succeeded. But it’s not in point 1#

4#  I thought, there it was due to something like mismatch key, then tried to re-setup password less between source and destination, but unfortunately it was not helping.

5# Double checked permission for .ssh directory and authorized_keys file, both were fine (.ssh is already 700 and authorized_keys is already 600). In this case, there should be no issue.

6# And last thing i forgot was to check parent directory of .ssh directory (which is home folder of the user). I came to know, the main problem was it! I used root user for this setup, and the permission for / was belong to other user instead of root.

So, for my case, the problem was incorrect permission for parent folder of .ssh

Permission of / before change

bash-3.2# cd /
bash-3.2# ls -la | head
total 9456
drwxr-xr-x  48 webservd     webservd        1536 Feb 22 14:28 .

Got it correct

bash-3.2#chown root:root .

After change

bash-3.2# ls -la | head
total 9456
drwxr-xr-x  48 root     root        1536 Feb 22 14:28 .

Note: Above was tried on Unix Solaris 10 operating system.

Replacing Many Similar Strings at the Same Time by Using VI Text Editor

VI is most common text editor in Unix or Linux based system. It is one of my favorites text editor, especially while working on developing shell script or editing simple text based file.

Sometimes it’s required to have a short cut way to replace many similar strings at the same time. Below steps will help to do the task:

  1. Open the script/text file with vi text editor
  2. Press “Esc” button
  3. Type command with following usage, then press “Enter
:<begin of line number>,<end of line number> s/<string replaced>/<string replacing>/g

For instance, we want to replace string “apple” with string “mango”. Here are two scenarios to show the implementation of 3rd step as mentioned above.

First, Replacing between from 2nd until 10th line

:2,10 s/apple/mango/g

Second, Replacing all

:1,$ s/apple/mango/g

How to Setup SSH Password Less in UNIX

Let us say, there are two servers, server-A and server-B. The objective is to setup password less from server-A toward server-B using RSA Key.

Here are step by step to follow:

Step#1: Create .ssh directory in server-A if it’s not exist.

server-A# mkdir ~/.ssh

Step#2: Go to inside .ssh directory

server-A# cd ~/.ssh

Step#3: Generate both RSA private and public keys

server-A# ssh-keygen -t rsa

Step#4: Transfer generated RSA public key from server-A to server-B

server-A# scp ~/.ssh/id_rsa.pub <server-B>:/<home>/id_rsa.server-A.pub

Step#5: Go to server-B

server-A# ssh <server-B>
Password:

Step#6: Create .ssh directory in server-B if it’s not exist.

server-B# mkdir .ssh

Step#7: Change directory permission to be readable, writable and executable by the owner

server-B# chmod 700 .ssh

Step#8: Append transferred RSA public key of server-A to authorized_keys file inside .ssh directory of server-B

server-B# cat id_rsa.server-A.pub >> .ssh/authorized_keys

Step#9: Change authorized_keys file as following

server-B# chmod 644 .ssh/authorized_keys

Step#10: Logoff from server-A and test RSA password less setup from server-A toward server-B

server-B# exit
server-A# ssh <server-B>

Note:

1- Above steps were tried between two Unix Solaris based Servers.

2- If DSA Key is preferred, just replace ‘rsa’ with ‘dsa’ in above steps