SCG software and scripts
The data sources
The original GWR software running under MS-Windows on brimer creates
ASCII files in a directory structure
The user-id that grants full access to this file tree is SG Administrator - with a
blankspace!
Access from Cygwin on brimer
In order to provide access from Cygwin's directory C:\cygwin = /cygdrive/c , also Cygwin login must
specify 'SG Administrator' as the
user. However, the blank is disturbing in unix.
Therefore there is a symbolic link named SGA in
/home
In /home/SCG/ there are
symbolic links to the annual directories yyyyo054
Login-shell is bash:
When SG Administrator
starts the tcsh, `SGA´ is used in the shell variables $home and $user
Study .tcshrc !
For faster evaluation of the exec path, a tcsh-shell script minipath is available:
B[SGA]~> source minipath
with
set path = ( . /home/SGA/bin
/home/HGS/bin /home/SGA/perlproc /usr/local/bin /usr/bin /bin )
source minipath must be
used when programs running under Cygwin are to be started or ended.
The only programs running on brimer under Cygwin are sh, csh and
perl scripts. And one Fortran program:
JDC -> jdc-e
It is most convenient to edit csh and perl scripts on another
machine and copy them to brimer.
Main part: Description of software
Everything here is running on unix platforms, be it Linux Mint or
Cygwin.
There is little you have to consider when porting a script from
Linux to Cygwin or vice versa.
We run perl version 5.14 sub-versions 2 or later
On the different machines the user is always hgs (linux) or HGS
(MS-Windows, linux).
The home directory and user declarations are quite similar.
The login shell recommended to run the SCG software is tcsh
Passwordless ssh login is used throughout.
Login:
on a machine, here holt, we typically have
.ssh:
authorized_keys brimer-HGS ecmwf
id_dsa
id_ecdsa
id_rsa known_hosts utter
barre
brimer-SGA elder id_dsa.pub id_ecdsa.pub
id_rsa.pub ore yggdrasil
.ssh/brimer-HGS:
id_dsa.pub id_ecdsa.pub id_rsa.pub
.ssh/brimer-SGA:
id_ecdsa.pub id_rsa.pub
.ssh/elder:
id_ecdsa.pub id_rsa.pub
.ssh/ore:
id_dsa.pub id_ecdsa.pub id_rsa.pub
.ssh/utter:
id_dsa.pub id_ecdsa.pub id_rsa.pub
Thus, after creating a security key (ssh-keygen) on one of the other
machines, renew with
cd .ssh
scp userid@machine:.ssh/'id_*.pub'
machine[-userid]/
cat */id_*.pub >! authorized_keys
Scripts
There are plenty of scripts,
those that you call to start a process
those that are called from within other scripts (see cron-jobs below)
and there are a few work-horse programs written in Fortran-77 (see below)
Scripts come in a few variants:
sh - the standard shell
(advantage: piping both STDOUT and STDERR)
csh - the C-shell (advantage:
array-like shell variables)
perl - pattern recognition and
regular-expression language (advantage: a lot of importable modules,
socket programming, maths...)
awk - a poor-man's perl
tcsh - terminal C-shell, some scripts to be
invoked with the command
source
Scripts to consider in the first case are those running under cron
and those for start and stop of sending/receiving the UPD data and
its "broker"
Scripts usually document themselves presenting a man-page-like text
upon specifying a lone option -h :
script-name -h
A few Fortran programs do that too. However, the really complicated
and versatile programs that are controlled by command line options
have usually a manual page.
This document presents a list of those programs that have manual
pages. Most of them are found on
http://holt.oso.chalmers.se/hgs/hgs.man/
http://holt.oso.chalmers.se/hgs/hgs.man/SCG/
Fortran programs,
processing
`(-h)´ indicates that the program documents itself when called with
option -h
expfitbm
- nonlinear fit of exponentials (gravimeter drift terms)
polmotm - convert polar
motion to gravity effect (-h)
sasm02 - interactive
but primitive: filter assembly (-h)
sasm03
- prediction error filters, power spectra with interactive
graphics
sasem03 - power spectra,
variant that runs on utter and elder. On elder the name is sasm03
sasm031
- power spectra, batch, under construction.
sasm06
- cross-spectral analysis, Wiener filters
tslist
- (read on at TSL-family)
- a lot of time series processing, including tsf-edit
command control
urtapt
- tide analysis
urtapm
- merged multi-campaign AG-SCG analysis
urtipgt
- theoretical tide generator
Fortran programs, utilities
jdc
- date converter with a pipe feature
axlim - rounding axis range,
generates GMT psxy options (-h)
and from the TSL-family
Fortran programs may
- be controlled by instructions from an ins-file, e.g.
sasm03 @ sasm03.ins
:SEGMENT
where the ins-file has alternative code segments,
flagged like e.g. SEGMENT>
- or take command line options, e.g.
tslist signal.ts -E
addnoise.tse,FRACTAL -E pdgram.tse,P -NFrqh -w signal+noise.spf
Platform elements:
Compilers: Absoft Pro-Fortran (Linux), gfortran (Cygwin)
Libraries: IMSL (distributed by Absoft, Linux), Numerical Recipes,
Pgplot with own add-ons (interactive), GMT (postscript), ImageMagick
GMT add-ons:
GMT is a collection of postscript producing routines to be called
from within shell scripts.
I have added: addlegend, pslegend, ps2png
Time series plot script: tslg is simple to get something on paper;
in default mode: tslg <tsfile> +
Start/stop UDP streaming on brimer
Start cygwin
. sx
In the new xterm window:
tcsh
source minipath
cd bctemp
broadcast-G1 &
broadcast-G1 - a sh script. It
starts two non-interactive xterms, one running
rolling ->
/home/SGA/perlproc/rolling.pl
udp-brc-file -> /home/SGA/perlproc/udp-brc-file.pl
the other.
Keep the xterm window alive
To stop:
source KX
Start/stop UDP stream handling on utter/elder
source START invokes the following scripts:
SCG-accu-agent.pl -
takes UDP-stream, accumulates ascii file.
file-monitor-agent.pl - takes
SCG-accu-agent's ascii files and assembles 1-h ascii files.
Calls
fill-monitor-gaps.pl for ascii-bin conversion;
takes care of 1-h file integrity
tide-press-plot
- perl script (redundant?)
11-mems
- csh script. Plots the MEM Power
Spectrum and sends the graphics to holt
To do:
source minipath
cd perlproc/SCG
source START
STOP RFMT
Start/stop UDP stream handling and html pages content on holt
source START implies cd ~/www/SCG , the working dir where all the
monitor files are located. These are:
SCG-html-agent.pl - udp to ascii
data file and monitor-plot.html with monitor-plot.png whstles and
bells
retro-agent.pl -
historical monitor-plot html pages on demand
eqwatch.pl
- USGS notifications to small ps / png insets into
monitor-plot.html
cd perlproc
source START
cd ~/perlproc
STOP EQW
STOP RCV RETRO
cron jobs
On Jan 22, 2016, the cron job table was:
# m h dom mon
dow command
30 01 * * *
/home/hgs/wx/ARC/get-oso-weather
> /home/hgs/wx/ARC/get-oso-weather.log 2>&1
55 01 * * *
/home/hgs/wx/MAREO/get-oso-mareograph
> /home/hgs/wx/MAREO/get-oso-mareograph.log 2>&1
50 06 * * *
/home/hgs/wx/WELL/get-well-level
> /home/hgs/wx/WELL/get-well-level.log 2>&1
50 14 * * *
/home/hgs/wx/WELL/get-well-level
> /home/hgs/wx/WELL/get-well-level.log 2>&1
#56 01 * * *
/home/hgs/wx/BUBBLER/get-oso-bubbler -d 1h -S 2 >
/home/hgs/wx/BUBBLER/get-oso-bubbler.log 2>&1
40 * * * *
/home/hgs/wx/TEMP/get-monutemps
> /home/hgs/wx/TEMP/get-monutemps.log 2>&1
41 * * * *
/home/hgs/wx/TEMP/plot/plot-monutemps
> /home/hgs/wx/TEMP/plot/plot-monutemps.log 2>&1
45 07 * * * /home/hgs/SMHI/getsealevel
-g r3g.txt >>
/home/hgs/SMHI/crontouch 2>&1
45 19 * * * /home/hgs/SMHI/getsealevel
-g r3g.txt >>
/home/hgs/SMHI/crontouch 2>&1
25 07 * * *
/home/hgs/TD/get-tide-data
> /home/hgs/TD/logs/get-tide.log 2>&1
50 07 * * *
/home/hgs/TD/tide-press-plot -DD
-U >
/home/hgs/TD/logs/tide-press-plot.log 2>&1
40 07 * * *
/home/hgs/TD/yesterdays-G1-plot
> /home/hgs/TD/logs/yesterdays-G1-plot.log 2>&1
05 15 * * *
/home/hgs/TD/tide-press-plot -DD
-UF >
/home/hgs/TD/logs/tide-press-plot.log 2>&1
07 15 * * *
/home/hgs/TD/actual-memsp-vs-time
> /home/hgs/TD/logs/actual-memsp-vs-time.log 2>&1
45 23 * * *
/home/hgs/TD/actual-memsp-vs-time
> /home/hgs/TD/logs/actual-memsp-vs-time.log 2>&1
30 06 1 * * /home/hgs/TD/GGP-send-data
-F
> /home/hgs/TD/logs/GGP-send-data.log 2>&1
00 21 3 * *
/home/hgs/TD/pt4brimer
> /home/hgs/TD/logs/pt4brimer.log 2>&1
00 22 * * 5 /home/hgs/TD/get-atmacs
-w
> /home/hgs/TD/logs/get-atmacs-w.log 2>&1
15 22 * * *
/home/hgs/SMHI/auto-tgg
> /home/hgs/SMHI/auto-tgg.log 2>&1
00 08 2 * * /home/hgs/TD/monthly-600s
-NP
R
> /home/hgs/TD/logs/monthly-600s.log 2>&1
10 03 * * *
/home/hgs/TD/tilt-control-monitor -pwr -PZ >
/home/hgs/TD/logs/tilt-control-monitor.log 2>&1
12 03 * * *
/home/hgs/TD/tilt-control-monitor -bal -PZ >>
/home/hgs/TD/logs/tilt-control-monitor.log 2>&1
10 08 * * *
/home/hgs/TD/longt-gbres
> /home/hgs/TD/logs/longt-gbres.log 2>&1
10 22 * * 5 /home/hgs/TD/ts4openend
GBR-PMR-ATR-TG-TAXP > /home/hgs/TD/logs/ts4openend.log
2>&1
00 07 * * *
/home/hgs/wx/WELL/plot-rr2wl-lastmonth
> /home/hgs/wx/WELL/plot-rr2wl-lastmonth.log 2>&1
35 23 * * *
/home/hgs/SMHI/plot-tgg
> /home/hgs/SMHI/plot-tgg.log 2>&1
58 23 * * *
/home/hgs/Seism/USGS/ens-daily-upd
> /home/hgs/TD/logs/ens-daily-upd.log 2>&1
58 11 * * *
/home/hgs/Seism/USGS/ens-daily-upd
> /home/hgs/TD/logs/ens-daily-upd.log 2>&1
#00 08 * * *
/home/hgs/TD/daily-fo-pdg.sh -r -25/55/5 -e Coquimbo
-A >
/home/hgs/TD/logs/daily-fo-pdg.log 2>&1
10 08 * * *
/home/hgs/TD/daily-fo-pdg.sh -r AUTO -floor -25 -d -30 -FW -e
last-30-days -A -remember > /home/hgs/TD/logs/dail\
y-rolling-fo-pdg.log 2>&1
All of these programs show a manual page upon option -h
The nitty-gritty
Small scripts, often used:
calc, beak, sb, helptext (based on awk),
Tough bits:
Time series files: Binary, home-grown structure, need a collection
of read and write subroutines.
There are two variants, unlabeled single-channel ts-files, and
multi-channel mc-files.
(Name extensions .ts and .mc are tentative yet no guarantee as to
the kind. Try tslql <file>).
The simplest way to print them out in ascii is
tslist <ts-file> -C3
tslist <mc-file> `tslql -L <mc-file>` -C3
.bye