Evoking and detaching bash scripts from current shell -
i'm have tad bit of difficulty developing bash based deployment scripts pipeline want run on openstack vm. there 4 scripts in total:
head_node.sh - launches vm , attaches appropriate disk storage vm. once that's completed, runs scripts (2 , 3) sequentially passing command through ssh vm.
install.sh - vm-side, installs of appropriate software needed pipeline.
run.sh - vm-side, mounts storage on vm , downloads raw data object storage. runs final script, detaching process shell created ssh using
nohup ./pipeline.sh &
. reason want detach shell next portion largely compute , may take days finish. therefore, user shouldn't have keep shell open long , should run in background.pipeline.sh - vm-side, loop iterates through list of files, , sequential runs commands on , intermediate files. result analysed staged object storage. vm tells head node kill it.
now i'm running problem nohup. if launch pipeline.sh script (i.e. without nohup) , keep attached shell, runs smoothly. however, if detach script, errors out after first command in first iteration of loop. thinking wrong way? what's correct way this?
so how looks:
$./head_node.sh
head_node.sh
#!/bin/bash ... launched vm etc ssh $vm_ip './install.sh' ssh $vm_ip './run.sh' exit 0
install.sh - omitted - not important problem
run.sh
#!/bin/bash ... mounts storage downloads appropriate files nohup ./pipeline.sh > log & exit 0
pipeline.sh
#!/bin/bash f in $(find . -name '*ext') process1 $f process2 $f ... done ... stage files object storage, unmount disks, additional cleanups ssh $head_node 'nova delete $vm_hash' exit 0
since i'm evoking run.sh script ssh instance, subprocesses launched script (namely pipeline.sh) not detach shell , error out on termination of ssh instance evoking run.sh. pipeline.sh script can detached calling head node, e.g., nohup ssh $vm_ip './pipeline.sh' &, keep session alive until end of pipeline.
Comments
Post a Comment