Super-script
~/TD/a/Allcamps/do-project-statistics
#_____ This is an example ______#
# setup for an analysis variant #
#
#
setenv VAR -OO-ndr-fsg-H
#
setenv DVAR
-ochy
#
setenv DVAR -ochy1d
#
setenv UVAR $VAR$DVAR
#
setenv OVAR
-ov0
#
#################################
# The comments here and those in the script differ a
bit.
# Wonder about data files produced under this roof? Look here.
Prepare
cd ~
. .4tides
. .4ag
g C
. .4recent
Do
Pre-requisit: You have campaign condumps, a merged condump,
a feature table, and a jd-file for analysis variant $VAR ,
and a drift solution designated $DVAR
i.e. you have passed the stage where urtapm and urtapt
produce solutions
(includes preparation stages make-ts4VAR and
make-tse4urtapm ).
Depending on $VAR you'll need a collecting directory, also
used for some intermediary result files:
setenv ODIR tmp$UVAR
Stage "Campaigns"
You will have obtained a residual with one or the other project
being far from the ensemble mean. We are going to compute partial
derivatives of the residuals w.r.t. offsets one campaign at a time.
Originally we use the instrument-specific offsets from the
international comparison campaigns, which may encompass a sequence
of campaigns at OSO. The script that alters the offsets at this
processing stage is tmp/cvaroffsets2env (a backup
original should be kept in safe/ ). The offsets from the
international comparisons are in safe/offsets2env.orig .
The process is controlled by a source script do-offset-variations
.
do-offset-variations :
With switch -c it enacts
campaign-wise processing. It involves a loop running
urtapm and urtapt for each
offset. The sequence number creates a file name mark
$OVAR, here -cv0 (no variation,
reference case), and -cv1 ... -cv14 for the
campaign nudged.
With switch -s it varies only selected campaigns,
however together. The file name mark will be -ovs
Without a switch, all campaigns formally depending on
the ICC's will be varied one-by-one. The file name mark will be
-ov0 ... -ov9.
The scripts doing the variations are
-s : svaroffsets2env
-c : cvaroffsets2env
default: varoffsets2env
There are copies in subdirs. The one going to be used
must be placed/symlinked in ~/TD/a/Allcamps/tmp/
The originals are in ~/TD/a/Allcamps/safe/?varoffsets2env
At every pass of the loop, a tp-ts-file is created,
tmp/meters${OVAR}.ts
set do_prep=1
set do_solve=1
. do-offset-variations -c
plot-project-statistics D -c
plot-project-statistics . -c $campaigns
This creates files with project-means and
mean-differences-from-cv0 in $ODIR
The output that that project-statistics creates from tsex
$odir/wstats.tse,S
contain MEAN, WMEAN and WRMS
Stage "Adjustment"
We collect the multi-campaign mean and RMS differences as follows:
touch $odir/campstats-diff-cv.smry
foreach i ( `N 0 $#campaigns` )
setenv OVAR -cv$i
project-statistics I -c >>
$odir/campstats-diff-cv.smry
end
and sort them by decreasing RMS:
#================================================================================
sort -r -n -k6
$odir/campstats-diff-cv.smry >! $odir/campstats-diff-cv.sort
#================================================================================
Finally we collect the g-rate results from the urtapt -cv
protocol files:
awk
'/SSCG/&&/year/{s=FILENAME;sub(/.*-cv/,"-cv",s);sub(/\.prl/,"",s);
\
printf "%-5s %8.2f +- %8.2f\n",
s,$6,$8}' \
o/scg-cal-merged$UVAR-cv*.prl >!
$ODIR/rates-cv.dat
Now, using the differences w.r.t. the reference case, a number of
campaigns can be adjusted simultaneously to lower the residual. In
the example following we use up to five campaigns in descending
order of misfit RMS.
superc-pp @ superc-pp.ins :FSG >!
logs/superc-pp$UVAR.log
. do-offset-variations -s 5
plot-project-statistics D -s 5
plot-project-statistics . -s 5 $campaigns
# So far the super-script
Aftermath:
To list the original and the adjusted offsets,
rm -f tmp/coffs.dat
touch tmp/coffs.dat
foreach c ( $campaigns )
eval echo \$YOFFS_$c \>\> tmp/coffs.dat
end
foreach i ( `N 1 5` )
awk '/>ADD/{printf "%8.2f
:: %s\n",$3,$9}' logs/varoffsets2env-sv$i.log |\
paste -d' ' tmp/coffs.dat
- >!
$ODIR/adjusted-campaign-offsets-sv$i.rsl
end
paste -d' '
$ODIR/adjusted-campaign-offsets-sv1.rsl
$ODIR/adjusted-campaign-offsets-sv5.rsl
10.0 10.00 :: 200907a 10.0
10.00 :: 200907a
17.0 17.00 :: 200911a 17.0
17.00 :: 200911a
17.0 17.00 :: 201004a 17.0
17.00 :: 201004a
47.0 47.00 :: 201006a 47.0
70.70 :: 201006a
47.0 47.00 :: 201009a 47.0
47.00 :: 201009a
47.0 47.00 :: 201106a 47.0
47.00 :: 201106a
11.0 11.00 :: 201106b 11.0
-2.79 :: 201106b
11.0 11.00 :: 201106c 11.0
-2.79 :: 201106c
22.0 22.00 :: 201304a 22.0
36.05 :: 201304a
22.0 22.00 :: 201405a 22.0
23.92 :: 201405a
52.0 52.00 :: 201405b 52.0
52.00 :: 201405b
52.0 52.00 :: 201502b 52.0
52.00 :: 201502b
25.3 25.30 :: 201505a 25.3
25.30 :: 201505a
25.3 -20.58 :: 201606a 25.3
-3.70 :: 201606a
10.0 10.00 :: 201707a 10.0
10.00 :: 201707a
awk '{if($2!=$1){print}}'
$ODIR/adjusted-campaign-offsets-sv6.rsl
47.0 65.99 ::
201006a
11.0 -1.06 :: 201106b
11.0 -1.06 :: 201106c
22.0 35.31 :: 201304a
22.0 24.79 :: 201405a
52.0 74.78 :: 201502b
25.3 -0.05 :: 201606a
To list the g-rate results - remember we have a rate bias in the
tide analysis, so we add it here:
set ratebias=`awk '/<ASlope>>>
Sum/{print $4}' ~/Ttide/SCG/logs/urtap-openend-lpt-ochy1d.log`
awk -v rb=$ratebias
'/SSCG/&&/year/{s=FILENAME;sub(/.*-sv/,"-sv",s);sub(/\.prl/,"",s);
\
printf "%-5s %8.2f +- %8.2f\n",
s,$6+rb,$8}' \
o/scg-cal-merged$UVAR-sv*.prl >!
$ODIR/adjusted-rates-sv.dat
-sv0 -10.99
+- 0.13
-sv1 -2.45
+- 0.13
-sv2 -2.54
+- 0.13
-sv3 -3.04
+- 0.13
-sv4 -2.94
+- 0.13
-sv5 -2.94
+- 0.13
Files, columns,
permissible values, results to verify
Under plot-project-statistics D, .rsl files, e.g. (
OVAR = -cv9 , odir = tmp-OO-ndr-H-ochy )
which calls project-statistics W
$odir/project-stats$OVAR.rsl -
statistics evaluated on the basis of $ODIR/ra.mc
Mean
WMean
WRMS
%RMS
%XSQ %NVS
18.72532
18.65941 1.211 ::
Onsala_AS_20090702a.drop.txt FG5-233
42.18837
41.95518 1200 AS
18.55008
18.42779 2.192 ::
Onsala_AA_20090703a.drop.txt FG5-233
77.76180
75.79336 1196 AA
...
where WRMS = R`%XSQ %RVS /` and %RVS
= sqrt(%NVS), so that WRMS is the 1-sigma uncertainty of the
mean.
$odir/project-stats-diff$OVAR.rsl
- difference of $OVAR w.r.t. -cv0
Mean
WMean
WRMS
%RMS %XSQ %NVS
1.27643
1.2532 1.2110 ::
Onsala_AS_20090702a.drop.txt
FG5-233 42.18837 41.95518 1200 AS
-0.34683
-0.3536 2.1920 ::
Onsala_AA_20090703a.drop.txt
FG5-233 77.76180 75.79336 1196 AA
and, under project-statistics
O
$odir/project-coffs$OVAR.rsl -
applied offsets = official values and one variation
(here, for -cv9 the extra offset is not visible since it's for
campaign 201707a,
check with tmp/cvaroffsets2env )
10.00000
:: Onsala_AS_20090702a.drop.txt FG5-233 AS
10.00000 ::
Onsala_AA_20090703a.drop.txt FG5-233 AA
Preparing the input to superc-pp :
set rsl=$odir/campstats-diff
What project-statistics I writes to
STDIN is the summary of the statistics internally saved in
$rsl$OVAR.dat
$rsl$OVAR.dat
Mean
WMean
WRMS
%RMS
1.27643
1.25323 1.21100 ::
Onsala_AS_20090702a.drop.txt
FG5-233 42.18837
-0.34683
-0.35364 2.19200 ::
Onsala_AA_20090703a.drop.txt
FG5-233 77.76180
An awk-script sums this up according to
set means = ( `awk '{su+=$1;sw+=$2;n++} END{print
su/n,sw/n}' ${rsl}$OVAR.dat` )
i.e. the means of the project means, unweighted and weighted,
respectively,
awk -v su=$means[1] -v sw=$means[2] -v s=$OVAR \
'{ru+=($1-su)**2;rw+=($1-sw)**2;n++} \
END{printf
"Summary: %-5s
%12.3f%10.3f%10.3f%10.3f\n",s,su,sw,sqrt(ru/n),sqrt(rw/n)}'
${rsl}$OVAR.dat
e.g. for OVAR = -cv9
Mean WMean
Sdev(M) Sdev(WM)
Summary:
-cv9
0.841 0.821
4.076 4.074
.bye