urtap merging and solution processing super-script
urtap-merge-pp
and utilities
stats-of-merged make-residual-tpmc
plot-tptight
tpindex-for-campaign tpindex-for-project
ts2tpmc
There are four alternatives to treat the drift of the SCG, and two
analysis methods to account for them in the merged solutions
DVAR = -expf -dsyn -dwwf -ochy - drift
variants
VAR = -O
-O-sdr
- analysis variants (the `-O´ stands for orientation offsets by
instrument)
-expf - solution from expfitbm
-dsyn - solution from standard open-end tide analysis
-dwwf - solution from extended open-end
tide analysis
-ochy - solution from extended tide
analysis with nontidal ocean and hydrology
-O - subtracting the drift series -
urtapm-big-O-ndr.ins urtap-merged-O-ndr.ins
and symlinks without `-ndr´
-O-sdr - solving a drift coefficient -
urtapm-big-O-sdr.ins
urtap-merged-O-sdr.ins
Drift
solutions must have been prepared in advance.
expfitbm determines the exponential decay parameters.
They are used in the standard and extended tide solutions. They
are long-term stable
unless a new event has to be accounted for. However, expfitbm
determines also the
long-term linear and offset terms, so that re-running expfitbm is
advisable from time to time.
urtap-merge-pp branches off at make-drift-tsf
in the processing
of SG-AG-data
It comprises the following stages:
1. make-drift-tsf
2. mc4campaigns
3. make-urtap-ins
4. urtapt on one campaign at a time, optionally with post-fit
RMS-based reweighting
5. urtapm merging step for a specific combination of analysis
and drift variants.
6. Message announcing that urtapt can be run on the merged
problem.
urtapm and what to do next (under construction)
Ad 4:
In the first series of campaign solutions, if a job has failed, a
common reason is a project that contains too little data (e.g.
many outliers get edited away). To remove such a project, update
the associated jump file. The log file will inform which column,
say n, of the design matrix that has no signal.
The incriminating slope and offset definition is group n-1
(the first column contains the SCG-series).
Rerun with directive U
Stage 4 implies re-iteration for outliers.
The files campaign.weights and campaign.var :
The first solution produces weight factors, one for each campaign,
specifying how much the AG uncertainties can be reduced to arrive
at a normalized chi^2 of unity. The average (RMS) a-priori
uncertainty of the merged files is computed and added as the last
line of campaign.weights
...
%W 201606a -dwwf 3.4368E-01
%W 201606a -dsyn 3.4368E-01
%W 201606a -expf 3.4368E-01
%W 201606a -201606a 3.4368E-01
%E all -expf 220.379
%E all -dsyn 220.379
%E all -dwwf 220.379
The sum-square a-priori uncertainty of each campaign is written
to campaign.var
Sum sig^2 N
...
%V 201606a -dwwf 0.26494E+05 53225
%V 201606a -dsyn 0.26494E+05 53225
%V 201606a -expf 0.26494E+05 53225
Reweighting makes it slightly more complicated to compute
a-priori offsets for campaigns or projects.
Read on here.
Ad 5:
The list of input .cdmp-files is created manually as is the whole
.ins-file. Check that the
list contains the recent .cdmp-files from stage 4.
You can introduce qualified a-priori offsets. Read
on here.
A-priori offsets
for campaigns or projects
You will probably introduce an offset in experimenting with
solutions after you have obtained the first urtap-merged result.
Thus, the feature table can be used to find out the indexes.
Campaign-wide offsets can be introduced in a simple way: This
procedure invokes make-urtapm-tse which prepares a section YOFFS.
setenv YOFFS_201004a dg/w
where dg is the offset in nm/s^2 and w the
effective mean uncertainty of the campaign. Use the calculator
script
setenv YOFFS_<campaign>
`eff-yoffs <campaign> dg`
Project-wide offsets can/should be added in this
block manually.
eff-yoffs -p <project> dg
will return the tse-command.
$ eff-yoffs -p Onsala_AC_20160628 20
ADD 0.357521 From #129785 To
#139276 ; 201606a Onsala_AC_20160628
"Qualified" offsets
If you suspect that projects are systematically offset, here's an
example how to go about.
Assume you have run both urtapm and urtapt on the merged problem.
Run
stats-of-merged DE
and check the results (by default written to o/yoffs4prjs.tse).
Example: (under construction)
ADD F`3.9188 * ...`
From #000000 To #001183 :: Onsala_AS_20090630a FG5-233
ADD F`0.7266 * ...` From #001184 To #002383
:: Onsala_AS_20090702a FG5-233
ADD F`-3.2576 * ...` From #002384 To #003569
:: Onsala_AA_20090703a FG5-233
...
The offset values to ADD are the differences Median minus Mean
in nm/s^2
You suspect that these differences are big and significant.
Add a line
${CONT_YOFFS:[; ]
in the block headed by TSF EDIT YOFFS in urtapm$VAR.tse
and activate the offset section using
setenv CONT_YOFFS "CONT ?? R o/yoffs4prjs.tse"
Then run utapm and urtapt again.
USAGE:
urtap-merge-pp
[DIRECTIVES] [options] [<campaign>
<campaign> ...]
ALIAS:
ts4merge-superscript
PURPOSE:
Prepare observation mc-files
Prepare and exec urtap-mrg.ins
for different alternatives concerning drift;
iterates until number of outliers is zero,.
Prepare urtapm jobs.
Exec urtapm for a specified drift alternative,
Default campaigns (unless option -c is used):
The
campaigns in file campaigns.lst
Drift alternatives:
DVAR label
drift-file
-dsyn SGD+DSYN from d/synth-drift-simp.ts
-dwwf SGD+DWWF from d/synth-drift-extd.ts
-expf SGD+EXPF from d/synth-drift-expfit.ts
-ochy SGD+OCHY from d/synth-drift-ochy.ts
Keep OQUILTUR and M directives to separate jobs
DIRECTIVES:
O -
Observations. Prepare campaign mc-files with
extra columns for drift alternatives.
Os - Skip the
drift-file renewal.
Q -
Inspect the mc-files for their columns content (labels)
I -
Produce urtap-mrg.ins files. The precise file names are
found in ins4mrg.lst .
L -
Re-produce ins4mrg.lst .
T -
Create symlinks for outlier tse-files
U - Exec
urtapt for all ins-files in ins4mrg.lst
Uw - add a
reweighting step so the input to
urtapm is more
balanced. All drops in a campaign
are up- or
down-weighted by the same amount.
R - Show
results. See option -show
M - Run
the merging job. Needs options -v and -d
Checks integrity
of pre-processing jobs.
Mi - Exit after
integrity check.
OPTIONS:
-show <searchstring> - With
directive R , shows protocol lines
containing the string. Default is " SCG "
-s <var>[,<var>...]
- Under directives OQUILTUR , select the drift
variants
for which the perparatory jobs are issue; any of
expf dsyn dwwf ochy
All four are the default.
-c <listfile>
- A file with the names of the campaigns instead of
specifying campaign names on the command line.
Uses pre-existing files urtap-*-mrg.ins
OBS! Do not use with directive I ,
since the ins-files must keep a unique
setup of slope features. All campaigns
must be included ( source ~/.4ag )
Best to first specify all campaign names on the
command line with directive I and then narrow
the range of campaigns by editing the list file.
-i <ins-file>
- Take the campaigns from an ins-file for urtapm
e.g. urtapm-big-O-ndr.ins
This option takes precedence over -c
-v <var>
- Required under M : job variant.
O for distinguishing orientation
offsets
by instrument.
Implies drift subtraction.
The ins-file for the merged solution
will be urtap-merged-O-ndr.ins
O-sdr to solve a drift admittance coefficient
instead of subtracting the drift.
The ins-file for the merged solution
will be urtap-merged-O-sdr.ins
-d <dvar>
- Required under M : drift variant
dsyn dwwf expf ochy
Both -v and -d are mercyful as to a
leading `-´ in the argument.
The ins-file for urtapm is urtapm-big-$var.ins
-sel <str>
- Select urtap ins-files (fgrep str ins4mrg.lst)
EXAMPLES:
urtap-merge-pp OQILTUR $campaigns
urtap-merge-pp OQILTUR -s ochy $campaigns
urtap-merge-pp M -v O-sdr -d dsyn