SCG software and scripts

The data sources

The original GWR software running under MS-Windows on brimer creates ASCII files in a directory structure

The user-id that grants full access to this file tree is SG Administrator - with a blankspace!

Access from Cygwin on brimer

In order to provide access from Cygwin's directory C:\cygwin = /cygdrive/c , also Cygwin login must specify 'SG Administrator' as the user. However, the blank is disturbing in unix.
Therefore there is a symbolic link named SGA in /home
In /home/SCG/ there are symbolic links to the annual directories yyyyo054
Login-shell is bash:

When  SG Administrator  starts the tcsh,  `SGA´ is used in the shell variables $home and $user 
Study .tcshrc !

For faster evaluation of the exec path, a tcsh-shell script minipath is available:
B[SGA]~> source minipath
set path = ( . /home/SGA/bin /home/HGS/bin /home/SGA/perlproc /usr/local/bin /usr/bin /bin )
source minipath  must be used when programs running under Cygwin are to be started or ended.
The only programs running on brimer under Cygwin are sh, csh and perl scripts. And one Fortran program: JDC -> jdc-e 
It is most convenient to edit csh and perl scripts on another machine and copy them to brimer.

Main part: Description of software

Everything here is running on unix platforms, be it Linux Mint or Cygwin.
There is little you have to consider when porting a script from Linux to Cygwin or vice versa.
We run perl version 5.14 sub-versions 2 or later

On the different machines the user is always hgs (linux) or HGS (MS-Windows, linux).
The home directory and user declarations are quite similar.
The login shell recommended to run the SCG software is tcsh
Passwordless ssh login is used throughout.

on a machine, here holt, we typically have
authorized_keys  brimer-HGS  ecmwf  id_dsa      id_ecdsa      id_rsa      known_hosts  utter
barre            brimer-SGA  elder  ore    yggdrasil





Thus, after creating a security key (ssh-keygen) on one of the other machines, renew with
cd .ssh
scp userid@machine:.ssh/'id_*.pub' machine[-userid]/
cat */id_*.pub >! authorized_keys


There are plenty of scripts,
those that you call to start a process
those that are called from within other scripts (see cron-jobs below)

and there are a few work-horse programs written in Fortran-77 (see below)

Scripts come in a few variants:
sh        - the standard shell (advantage: piping both STDOUT and STDERR)
csh      - the C-shell (advantage: array-like shell variables)
perl     - pattern recognition and regular-expression language (advantage: a lot of importable modules, socket programming, maths...)
awk     - a poor-man's perl
tcsh     - terminal C-shell, some scripts to be invoked with the command  source

Scripts to consider in the first case are those running under cron
and those for start and stop of sending/receiving the UPD data and its "broker"

Scripts usually document themselves presenting a man-page-like text upon specifying a lone option -h :
script-name -h

A few Fortran programs do that too. However, the really complicated and versatile programs that are controlled by command line options have usually a manual page.

This document presents a list of those programs that have manual pages. Most of them are found on

Fortran programs, processing

`(-h)´ indicates that the program documents itself when called with option -h

expfitbm - nonlinear fit of exponentials (gravimeter drift terms)
polmotm  - convert polar motion to gravity effect (-h)
sasm02   - interactive but primitive: filter assembly (-h)
sasm03   - prediction error filters, power spectra with interactive graphics
sasem03  - power spectra, variant that runs on utter and elder. On elder the name is sasm03
sasm031  - power spectra, batch, under construction.
sasm06   - cross-spectral analysis, Wiener filters
tslist   - (read on at TSL-family) - a lot of time series processing, including tsf-edit command control
urtapt   - tide analysis
urtapm   - merged multi-campaign AG-SCG analysis
urtipgt  - theoretical tide generator

Fortran programs, utilities

jdc   - date converter with a pipe feature
axlim - rounding axis range, generates GMT psxy options (-h)
and from the TSL-family

Fortran programs may
 - be controlled by instructions from an ins-file, e.g.
    sasm03 @ sasm03.ins :SEGMENT
   where the ins-file has alternative code segments, flagged like e.g.   SEGMENT>

- or take command line options, e.g.
     tslist signal.ts -E addnoise.tse,FRACTAL -E pdgram.tse,P -NFrqh -w signal+noise.spf

Platform elements:

Compilers: Absoft Pro-Fortran (Linux), gfortran (Cygwin)
Libraries: IMSL (distributed by Absoft, Linux), Numerical Recipes, Pgplot with own add-ons (interactive), GMT (postscript), ImageMagick

GMT add-ons:

GMT is a collection of postscript producing routines to be called from within shell scripts.
I have added: addlegend, pslegend, ps2png
Time series plot script: tslg is simple to get something on paper; in default mode: tslg <tsfile> +

Start/stop UDP streaming on brimer

Start cygwin
. sx
In the new xterm window:
source minipath
cd bctemp
broadcast-G1 &

broadcast-G1 - a sh script. It starts two non-interactive xterms, one running
  rolling      -> /home/SGA/perlproc/
  udp-brc-file -> /home/SGA/perlproc/
the other.
Keep the xterm window alive
To stop:
source KX

Start/stop UDP stream handling on utter/elder

source START invokes the following scripts:     - takes UDP-stream, accumulates ascii file. - takes SCG-accu-agent's ascii files and assembles 1-h ascii files.
                       Calls for ascii-bin conversion;
                       takes care of 1-h file integrity
tide-press-plot       - perl script (redundant?)
11-mems               - csh script. Plots the MEM Power Spectrum and sends the graphics to holt

To do:
source minipath
cd perlproc/SCG
source START


Start/stop UDP stream handling and html pages content on holt

source START implies cd ~/www/SCG , the working dir where all the monitor files are located. These are: - udp to ascii data file and monitor-plot.html with monitor-plot.png whstles and bells    - historical monitor-plot html pages on demand        - USGS notifications to small ps / png insets into monitor-plot.html
cd perlproc
source START

cd ~/perlproc

cron jobs

On Jan 22, 2016, the cron job table was:
# m h  dom mon dow   command
30 01 * * * /home/hgs/wx/ARC/get-oso-weather            > /home/hgs/wx/ARC/get-oso-weather.log 2>&1
55 01 * * * /home/hgs/wx/MAREO/get-oso-mareograph       > /home/hgs/wx/MAREO/get-oso-mareograph.log 2>&1
50 06 * * * /home/hgs/wx/WELL/get-well-level            > /home/hgs/wx/WELL/get-well-level.log 2>&1
50 14 * * * /home/hgs/wx/WELL/get-well-level            > /home/hgs/wx/WELL/get-well-level.log 2>&1
#56 01 * * * /home/hgs/wx/BUBBLER/get-oso-bubbler -d 1h -S 2 > /home/hgs/wx/BUBBLER/get-oso-bubbler.log 2>&1
40  * * * * /home/hgs/wx/TEMP/get-monutemps             > /home/hgs/wx/TEMP/get-monutemps.log 2>&1
41  * * * * /home/hgs/wx/TEMP/plot/plot-monutemps       > /home/hgs/wx/TEMP/plot/plot-monutemps.log 2>&1
45 07 * * * /home/hgs/SMHI/getsealevel -g r3g.txt      >> /home/hgs/SMHI/crontouch 2>&1
45 19 * * * /home/hgs/SMHI/getsealevel -g r3g.txt      >> /home/hgs/SMHI/crontouch 2>&1
25 07 * * * /home/hgs/TD/get-tide-data                  > /home/hgs/TD/logs/get-tide.log 2>&1
50 07 * * * /home/hgs/TD/tide-press-plot -DD -U         > /home/hgs/TD/logs/tide-press-plot.log 2>&1
40 07 * * * /home/hgs/TD/yesterdays-G1-plot             > /home/hgs/TD/logs/yesterdays-G1-plot.log  2>&1
05 15 * * * /home/hgs/TD/tide-press-plot -DD -UF        > /home/hgs/TD/logs/tide-press-plot.log 2>&1
07 15 * * * /home/hgs/TD/actual-memsp-vs-time           > /home/hgs/TD/logs/actual-memsp-vs-time.log 2>&1
45 23 * * * /home/hgs/TD/actual-memsp-vs-time           > /home/hgs/TD/logs/actual-memsp-vs-time.log 2>&1
30 06 1 * * /home/hgs/TD/GGP-send-data -F               > /home/hgs/TD/logs/GGP-send-data.log 2>&1
00 21 3 * * /home/hgs/TD/pt4brimer                      > /home/hgs/TD/logs/pt4brimer.log 2>&1
00 22 * * 5 /home/hgs/TD/get-atmacs -w                  > /home/hgs/TD/logs/get-atmacs-w.log 2>&1
15 22 * * * /home/hgs/SMHI/auto-tgg                     > /home/hgs/SMHI/auto-tgg.log 2>&1
00 08 2 * * /home/hgs/TD/monthly-600s -NP R             > /home/hgs/TD/logs/monthly-600s.log  2>&1
10 03 * * * /home/hgs/TD/tilt-control-monitor -pwr -PZ  > /home/hgs/TD/logs/tilt-control-monitor.log 2>&1
12 03 * * * /home/hgs/TD/tilt-control-monitor -bal -PZ >> /home/hgs/TD/logs/tilt-control-monitor.log 2>&1
10 08 * * * /home/hgs/TD/longt-gbres                    > /home/hgs/TD/logs/longt-gbres.log 2>&1
10 22 * * 5 /home/hgs/TD/ts4openend GBR-PMR-ATR-TG-TAXP > /home/hgs/TD/logs/ts4openend.log 2>&1
00 07 * * * /home/hgs/wx/WELL/plot-rr2wl-lastmonth      > /home/hgs/wx/WELL/plot-rr2wl-lastmonth.log 2>&1
35 23 * * * /home/hgs/SMHI/plot-tgg                     > /home/hgs/SMHI/plot-tgg.log 2>&1
58 23 * * * /home/hgs/Seism/USGS/ens-daily-upd          > /home/hgs/TD/logs/ens-daily-upd.log 2>&1
58 11 * * * /home/hgs/Seism/USGS/ens-daily-upd          > /home/hgs/TD/logs/ens-daily-upd.log 2>&1
#00 08 * * * /home/hgs/TD/ -r -25/55/5   -e Coquimbo -A     > /home/hgs/TD/logs/daily-fo-pdg.log 2>&1
10 08 * * * /home/hgs/TD/ -r AUTO -floor -25 -d -30 -FW -e last-30-days -A -remember > /home/hgs/TD/logs/dail\
y-rolling-fo-pdg.log 2>&1
All of these programs show a manual page upon option -h

The nitty-gritty

Small scripts, often used:

calc, beak, sb, helptext (based on awk),

Tough bits:

Time series files: Binary, home-grown structure, need a collection of read and write subroutines.
There are two variants, unlabeled single-channel ts-files, and multi-channel mc-files.
(Name extensions .ts and .mc are tentative yet no guarantee as to the kind. Try tslql <file>).
The simplest way to print them out in ascii is
tslist <ts-file> -C3
tslist <mc-file> `tslql -L <mc-file>` -C3