urtap merging and solution processing super-script
There are four alternatives to treat the drift of the SCG, and two
analysis methods to account for them in the merged solutions
DVAR = -expf -dsyn -dwwf -ochy - drift
VAR = -O
- analysis variants (the `-O´ stands for orientation offsets by
-expf - solution from expfitbm
-dsyn - solution from standard open-end tide analysis
-dwwf - solution from extended open-end
-ochy - solution from extended tide
analysis with nontidal ocean and hydrology
-O - subtracting the drift series -
and symlinks without `-ndr´
-O-sdr - solving a drift coefficient -
solutions must have been prepared in advance.
expfitbm determines the exponential decay parameters.
They are used in the standard and extended tide solutions. They
are long-term stable
unless a new event has to be accounted for. However, expfitbm
determines also the
long-term linear and offset terms, so that re-running expfitbm is
advisable from time to time.
urtap-merge-pp branches off at make-drift-tsf
in the processing
It comprises the following stages:
urtapm and what to do next (under construction)
4. urtapt on one campaign at a time, optionally with post-fit
5. urtapm merging step for a specific combination of analysis
and drift variants.
6. Message announcing that urtapt can be run on the merged
In the first series of campaign solutions, if a job has failed, a
common reason is a project that contains too little data (e.g.
many outliers get edited away). To remove such a project, update
the associated jump file. The log file will inform which column,
say n, of the design matrix that has no signal.
The incriminating slope and offset definition is group n-1
(the first column contains the SCG-series).
Rerun with directive U
Stage 4 implies re-iteration for outliers.
The files campaign.weights and campaign.var :
The first solution produces weight factors, one for each campaign,
specifying how much the AG uncertainties can be reduced to arrive
at a normalized chi^2 of unity. The average (RMS) a-priori
uncertainty of the merged files is computed and added as the last
line of campaign.weights
The sum-square a-priori uncertainty of each campaign is written
%W 201606a -dwwf 3.4368E-01
%W 201606a -dsyn 3.4368E-01
%W 201606a -expf 3.4368E-01
%W 201606a -201606a 3.4368E-01
%E all -expf 220.379
%E all -dsyn 220.379
%E all -dwwf 220.379
Sum sig^2 N
Reweighting makes it slightly more complicated to compute
a-priori offsets for campaigns or projects.
%V 201606a -dwwf 0.26494E+05 53225
%V 201606a -dsyn 0.26494E+05 53225
%V 201606a -expf 0.26494E+05 53225
Read on here.
The list of input .cdmp-files is created manually as is the whole
.ins-file. Check that the
list contains the recent .cdmp-files from stage 4.
You can introduce qualified a-priori offsets. Read
for campaigns or projects
You will probably introduce an offset in experimenting with
solutions after you have obtained the first urtap-merged result.
Thus, the feature table can be used to find out the indexes.
Campaign-wide offsets can be introduced in a simple way: This
procedure invokes make-urtapm-tse which prepares a section YOFFS.
setenv YOFFS_201004a dg/w
where dg is the offset in nm/s^2 and w the
effective mean uncertainty of the campaign. Use the calculator
`eff-yoffs <campaign> dg`
Project-wide offsets can/should be added in this
eff-yoffs -p <project> dg
will return the tse-command.
$ eff-yoffs -p Onsala_AC_20160628 20
ADD 0.357521 From #129785 To
#139276 ; 201606a Onsala_AC_20160628
If you suspect that projects are systematically offset, here's an
example how to go about.
Assume you have run both urtapm and urtapt on the merged problem.
and check the results (by default written to o/yoffs4prjs.tse).
Example: (under construction)
ADD F`3.9188 * ...`
From #000000 To #001183 :: Onsala_AS_20090630a FG5-233
The offset values to ADD are the differences Median minus Mean
ADD F`0.7266 * ...` From #001184 To #002383
:: Onsala_AS_20090702a FG5-233
ADD F`-3.2576 * ...` From #002384 To #003569
:: Onsala_AA_20090703a FG5-233
You suspect that these differences are big and significant.
Add a line
in the block headed by TSF EDIT YOFFS in urtapm$VAR.tse
and activate the offset section using
setenv CONT_YOFFS "CONT ?? R o/yoffs4prjs.tse"
Then run utapm and urtapt again.
[DIRECTIVES] [options] [<campaign>
Prepare observation mc-files
Prepare and exec urtap-mrg.ins
for different alternatives concerning drift;
iterates until number of outliers is zero,.
Prepare urtapm jobs.
Exec urtapm for a specified drift alternative,
Default campaigns (unless option -c is used):
campaigns in file campaigns.lst
-dsyn SGD+DSYN from d/synth-drift-simp.ts
-dwwf SGD+DWWF from d/synth-drift-extd.ts
-expf SGD+EXPF from d/synth-drift-expfit.ts
-ochy SGD+OCHY from d/synth-drift-ochy.ts
Keep OQUILTUR and M directives to separate jobs
Observations. Prepare campaign mc-files with
extra columns for drift alternatives.
Os - Skip the
Inspect the mc-files for their columns content (labels)
Produce urtap-mrg.ins files. The precise file names are
found in ins4mrg.lst .
Re-produce ins4mrg.lst .
Create symlinks for outlier tse-files
U - Exec
urtapt for all ins-files in ins4mrg.lst
Uw - add a
reweighting step so the input to
urtapm is more
balanced. All drops in a campaign
are up- or
down-weighted by the same amount.
R - Show
results. See option -show
M - Run
the merging job. Needs options -v and -d
of pre-processing jobs.
Mi - Exit after
-show <searchstring> - With
directive R , shows protocol lines
containing the string. Default is " SCG "
- Under directives OQUILTUR , select the drift
for which the perparatory jobs are issue; any of
expf dsyn dwwf ochy
All four are the default.
- A file with the names of the campaigns instead of
specifying campaign names on the command line.
Uses pre-existing files urtap-*-mrg.ins
OBS! Do not use with directive I ,
since the ins-files must keep a unique
setup of slope features. All campaigns
must be included ( source ~/.4ag )
Best to first specify all campaign names on the
command line with directive I and then narrow
the range of campaigns by editing the list file.
- Take the campaigns from an ins-file for urtapm
This option takes precedence over -c
- Required under M : job variant.
O for distinguishing orientation
Implies drift subtraction.
The ins-file for the merged solution
will be urtap-merged-O-ndr.ins
O-sdr to solve a drift admittance coefficient
instead of subtracting the drift.
The ins-file for the merged solution
will be urtap-merged-O-sdr.ins
- Required under M : drift variant
dsyn dwwf expf ochy
Both -v and -d are mercyful as to a
leading `-´ in the argument.
The ins-file for urtapm is urtapm-big-$var.ins
- Select urtap ins-files (fgrep str ins4mrg.lst)
urtap-merge-pp OQILTUR $campaigns
urtap-merge-pp OQILTUR -s ochy $campaigns
urtap-merge-pp M -v O-sdr -d dsyn