HOW.TO-AG-and-SCG-at-drop-level

How to analyze AG campaigns at the drop level, and calibrate the SCG

Analysis and reduction of microseismic perturbations is subject of another document

Recipes under HOW TO are for a tcsh environment. In particular we use at command prompt level...
foreach loops (for loops in bash)
setenv parameters (export in bash)
arrays of parameters and strings

Activate shell settings for AG:

cd ~
. .4ag

Utilities in ~/bin

Date conversion:

Utility	converts	example
`procag2kal`	day-of-year to month-day	`procag ... \| procag2kal`
`procag2jd`	day-of-year to Julian	`procag ... \| procag2jd`
`procag2h`	dates to hours after epoch	`procag ... \| procag2h -D 2014,05,28`
`tsf-kal2dy`	dates to decimal years	`cat *.tsf \| tsf-kal2dy`

Directory structure

TD/a/
TD/a/bin/ contains symlinks to everything needed that is not in the ~/bin

Table 1 - Directories and content. Process flow proceeds from left (acquired zip-files) to right (final results)
Important lst-files are shown in bold.

Top level = `TD/a/`	`IfE-2009-11/` `IfE-2014-05/` `Recalc2014/` ... Original data	`Alldrops/` Drop files	`Allsets/` Set and project files	`Allsmp/` Processing files	`Allcamps/` Analysis and Result files
Directories	result of unzip per campaign as needed				`PS/` plot `d/` data `tmp/` `o/` output `tsfo/` final `msets/` own sets.txt per project `msets/d/` raw for concatenation`msets/b/` bundled`t/` urtapm dumps
Symlinks		symlinks to original data sources, `*.drop.txt`	symlinks to original data sources, `.set.txt,.project.txt`		`prl2tse -> ~/Ttide/SCG/+campaigns.lst -> ../Allsmp/+`
Data files	`.tar, .zip`			`.drop.smp.*-drift.tsf`
Control files				`*.drop.smp.tseremdc.tse wmean.tse`	`urtap-.insagreduct.tseallties.tse autoset.tsediff-.tsesynth-*.tseweed.tsemsets/msets-<campaign>.tse`
File catalogs and lists			`project-files.lst`	`files.lst bundles.lst1)``campaigns.lst` `dropfiles.lst`	`cdumps.lst` (temporary)`files.lst` (temporary)`campaigns.lst -> ../Allsmp/+ features-MJD.lst 2)``meters.lst``mc-files.lst``orientations.lst` `project-dates.lst equivalent-sets.lst 3)` `drop-symlinks.lst4)smp-symlinks.lst 4)`
Programs All programs should either reside in `~/TD/a/bin` or should have symlinks there. At least their path should be in the $path shell variable. There are a few exceptions: `scgts4ag`	`add-campaign`		`procag.pl` (symlink in `~/bin`)	`agsmp2mc)make-drift-tsf) make-s-and-j-tsfmc4campaignspf-of-project*)`please no symlink in `bin/`	`analyse-mcbundlecal-wmeancalplotcalplot-sopevaluate-b+s evaluate-tp-by-projectfinal-analysisfinal-results4plot list-featureslist-project-dates MJD-tp make-mc-files-listmake-outlier-mcmake-own-sets make-scg-rmsmake-urtap-ins` `pf-of-project plot-cal-resultsplot-final-resultsplot-project plot-all-projects plot-all-sets plot-merged-residuals plot-tpprl2tse -> ~/Ttide/SCG/+procag.pl` (symlink in `~/bin`) `produce-agproject-from-toproject-timesproject-times2tseps-add-platformredtpslope-bars4plotthe-campaign-of the-dropfiles the-drop.smp-files the-meter-of the-orientation-ofthe-platform-ofthe-project-filestp-index-of-project tplist tplist-project tp2tsfvary-bias-and-slopes`

¹)bundles.lstBundles are concatenated own-set files, ready with headers

²)
awk '/^[/=]/{print $0}' urtap-*-mrg.ins
³)equivalent.lstEquivalences between a bundled set file and the covered AG original set-files.

⁴)
drop-symlinks.lst and smp-symlinks.lst
Used in tp-index-of-project to find aliasses of project file names given a project name .
------------------ www/4me/ www/4me/ag2014png-collections

Definitions

Table 2 - Definition of terms. Contents in directories and files

Part 1 What is / are...?
drop files `.drop.txt`	They result from analysis of each single mass drop, comprise all drops within a project.
set files `.set.txt`	time and measurements. One set comprises / averages over a series of drops, typically 50
project files `.project.txt`	Parameters and information
a project	a succession of sets No move of instrument within a project
a campaign	several projects within a time period of days to weeks. The original data should be put into a directory under `~/TD/a` , e.g. `~/TD/a/IfE-2014-05` Campaign names should be a common substring in all file names, e.g. 201502b The association with drop, set and project files is defined in a file named`campaigns.lst` If the original .txt-file names run month names instead if numbers, it is advisable to create symlinks with dates in the form `YYYYMMDD`. There is some flexibility with the rest of the file name, b ut not too much should be expected. A functioning file name in campaign `200907a` may e.g. be `Onsala_AC_20090706b_1.drop.txt`
platform and orientation codes	Due to limitations in urtap, the least-squares program, the platform cannot have more than two characters. In order to signify bias terms with a four-letter word, one character is needed for orientation and one to enumerate the list, `AAN3` . The slope terms are not depending on platform nor orientation, so that the are named `SLO``n`

Part 2 File name extensions...	What's in the files?	ASCII Binary
`txt`	after `project.` , `set.` , `drop.` , original result files from g-software	A
`tse`	instructions for tsf-edit, a subroutine e.g. in tslist	A
`tsi`	instructions for tsf-edit without a leader and a trailing `END`, biding their time for import into a complete tse-file. Use `wraptse` if the instructions form a complete set.	A
`ts` `mc`	binary time series files, one component (column) multi-column	B
`tsf`	time series files, irregular sampling intervals possible `yyyy mm dd hhh mm ss fff data data ... :: comment`	A
`prl`	Result sheet from tide least-squares program urtapt	A
`ins`	instruction files for (almost all of my) Fortran main programs	A
`drop.smp`	tsf-files from drop files, regularized sampling interval `yyyy mm dd hhh mm ss fff jul` `:``SCG-obs``::``AG-obs AG-sigma`	A

Frequently used programs and utilities:
GMT for plotting
tslist (tsl) with tsfedittslq tslqltsf-kal2dytslgjdcurtapturtap-outlitprl2tse

In scripts: ~/bin beak fromto ecq sb helptext calc decopt
In plotting scripts: ~/bin axlim axlim4ts addlegend pslegend ps-add-platform ps2png

Languages
Fortran-77 (Absoft 11.5)
perlawksedcshsh

HOW TO

Most programs show usage information upon option -h

The name model for mc-files containing AG,VAL AG,SIG and SCG,VAL columns can be given with templates:

template meaning ( Search string in project file ) %D - start date %S - sub-Campaign, the last character of <campaign> %C - Campaign name %I - instrument ID ( Meter S/N: ) %P - platform ID ( ^Name: ) %O - Orientation first character, N or S ( toupper() ~ "ORIENT" ) %N - number of concatenated projects + `P' %M - input file name substring between last `_' and `.drop.smp' %L - input file name last char before `.drop.smp' %Z - last character of the project file name (less .project.txt)
For extracting data from drop and set files, use
procag.pl (alias procag)

The purpose of the processing operations in this chapter is

Setting up the files of the AG-campaign in the appropriate directories and the meta information in .lst files.
Extracting SCG-readings synchronous with the AG
Reducing the SCG drift
Preparing the input (data and instructions) for least-squares analysis (using urtap); includes setting up bias+slope events
Examining the output of the least-squares fit
The results comprise: SCG-calibration factor; AG-biases and drift-slopes for platforms and orientations

Begin here: Copy project, set and drop files into a new subdirectory under ~/TD/a/

mkdir ~/TD/a/LM-201606
cp <files> ~/TD/a/LM-201606cd ~/TD/a
add-campaign 201606a FG5-233 LM-201606 source<file>.csh# according to the report

!!! You'll get a number of lst-files in a/Allcamps/ marked with the campaign name.
!!! They are to be added to the respective campaigns lists, see the pink box below.
!!!Error: File project-dates-<campaign>.lst won't be made unless there are .smp-files
!!! Instead, after scgts4ag ,

cd /TD/aproject-times <yyyymm> >! Allcamps/project-dates-<campaign>.lstproject-times -J <yyyymn> >>Allcamps/project-dates-<campaign>.lst

!!! end

cd ~/TD/a/Allsmp
pprocag<drop-files># e.g. pprocag ../Alldrops/Onsala_??_2018*.drop.txt source run_procag.csh

First a short way to sample SCG data and output SCG + AG ascii files
For one project only

cd ~/TD/a/Allsmp
cp files.lst files.lst-bup#n

For creation of .drop.smp and .drop.smp.tse -files,

cd ~/TD
scgts4ag -new <campaign>-O a/Allsmp -sigma -n -S %.smp.tse a/Alldrops/*A?_201304*.drop.txt

and this is for GGP-filter corrected files:

lpf-unbessel -N -GO -C <campaign>cp tmp/G1_g_<yymm>*-1s.ts lpfu/ scgts4ag -new<campaign>-O a/Allsmp-s -SCG lpfu/G1_g_%%%%%%-1s.ts -E NONE -L NONE-f-sigma -n -S %.smp.tse \ `the-dropfiles -D<campaign>`

OBS! There seem to be offsets from one G1_g_ file to the next. We must adjust them.
lpf-unbessel has been built out to this end (2017-07-17). Test the result using
check-overlap-dc lpfu/G1_g_201707*-1s.ts

and binary mc-files containing the data from SCG and AG (skip the `-%P´ if you have more than one platform)

agsmp2mc -O a/Allsmp -o AGSG_201304a_%N-%P-%I-%O a/Allsmp/*A?_2013042*.drop.smp

(if there are multiple campaigns that match the filename expression, add the campaign's id-letter, a or b or ... ) like here:

agsmp2mc -O a/Allsmp -o AGSG_201405a_%N-%P-%I-%O a/Allsmp/*A?_201405*a.drop.smp

If you have created lpf-unbessel'ed files, use instead

agsmp2mc -O a/Allsmp-s -C 201502b -DY-o AGSG_201502b_s_%N-%P-%I-%O \
a/Allsmp-s/A?_20150204*.drop.smp \ a/Allsmp-s/A?_20150205*.drop.smp \ a/Allsmp-s/A?_20150206*.drop.smp \ a/Allsmp-s/A?_20150207*.drop.smp \ a/Allsmp-s/A?_20150209*.drop.smp \ a/Allsmp-s/A?_2015021*.drop.smp # obs! drop.smp-files must be in temporal order # You may use `the-dropfiles -S 201502b`

If the drift series has been created in a subdir, like hereDSYN, use the -DD option:

agsmp2mc -O a/Allsmp-s -C 201707a -DY -DD a/Allcamps/d/DSYN/ -dt LIST -o AGSG_201707a_%N-%P-%I-%O `the-dropfiles -S 201707a`

and you're ready for calibration with urtapt. The instruction file urtap-cal201304a.ins would contain
21 ^ a/Allsmp/AGSG_201304a_2P-AA-FG5-233-N-5s.mcYou might like to create symlinks in Allcamps/d

ln -s ../../Allsmp-s/AGSG_201707a_2P-AA-FG5-233-S-5s.mc AGSG_201707a_FG5-233-5s.mc

If you want to explore the impact of low-pass filtering the SCG, generate parallel mc-files, like here for 201106a:

agsmp2mc -O a/Allsmp-s -C 201106a -DY -DD a/Allcamps/d/DSYN/-dt 1 -o AGSG_201106a_s_%N-%P-%I-%O `the-dropfiles -S 201106a`
agsmp2mc -O a/Allsmp -C 201106a -DY -DD a/Allcamps/d/DSYN/-dt 1 -o AGSG_201106a_s_%N-%P-%I-%O `the-dropfiles -S -da/Allsmp 201106a`

but wait!
Below we will use a procedure that carries out this step for one or more campaigns in one stroke,

mc4campaigns

and warrant consistent file names using the .lst -catalogues.

      OBS! FILE NAME CONVENTIONS
      In the .mc-file names, the campaign is surrounded by underscores

      If there are file names with month names instead of numbers, then create symlinks ina/Allsmp/

cd a/Allsmp/
ln -s AA_2009Nov03a.drop.smp AA_20091103a.drop.smp

etc.
Since we love to go automatic, we should concatenate drop.smp files by campaigns.
So we need a recipe to combine files and get the respective campaign parameters.

If you need a new catalogue, usescgts4agto go through all files
(you can delete the catalogues except files-new.lst in a/Allcamps/d/ afterwards)

cd ~/TD
scgts4ag -O a/Allcamps/d/ -fl files-new.lst-sigma -t -n -S - a/Alldrops/*.drop.txt

collects SCG 1s-data in synch with the AG drop files.; and a file catalogue, files.lst
-t will prohibit another sampling.

Cataloging files and projects

This box explains the hand-work to be done for cataloging. If you have used the add-campaign procedure
check that the lst-files you have obtained are o.k.

cd ~/TD/a/Allsmp

Make a copy of the files.lst (a/Allsmp)
Update campaigns.lst (a/Allsmp)and edit in lines of the following kind

#C 201004a AGSG_201004a_%I AS_2010Apr17a.drop.smp AS_2010Apr17b.drop.smp ...
^-- Campaign ^--- mc-filename stub ^----input files...

all those files you want to combine for a campaign.
You may have help from the file a/Allcamps/files.lst Once the campaign's drop files are processed, add the information to

a/Allsmp/dropfiles.lst

in temporal order; example:

201606a Onsala_AC_20160627a.drop201606a Onsala_AC_20160628a.drop201606a Onsala_AA_20160629a.drop201606a Onsala_AA_20160701a.drop201606a Onsala_AA_20160702a.drop201606a Onsala_AA_20160703a.drop

For all campaigns:

set campaigns = ( `awk '/^#C /{print $2}' ../Allcamps/campaigns.lst | sort | uniq` )

Edit this as required.

Then in a/Allcamps
add orientations to the file orientations.lst
Sometimes a one-liner can help:

cd ~/TD/a/Allsets
awk '/Nord/{print "%O",FILENAME,"north"}' A?_2011061*.project.txt
awk '/Sued/{print "%O",FILENAME,"south"}' A?_2011061*.project.txt

and in meters.lst (a/Allcamps) #Mrecords and %P records

Finally, update project-dates.lst (a/Allcamps/ , manually editing in the output of

cd ~/TD/a/Allsmp
project-times 2011061
project-times -J 2011061

Then, heading for mc4campaigns, again in a/Allsmp we are going to produce drift files.
Doing that on the basis of parameters solved in our croned SG jobs ( ~/TD/ts4openend ) can be risky. First check that the tsf-edit segments and commands in make-drift-tsf are o.k., and check the result with urtap's ph04 files.

For the *SYNTH variant you need to produce timely tsf-edit commands EXP, ADD, and SLO in /home/hgs/TD/drift.tse from a decent recent urtapt analysis.
Example:

cd ~/Ttide/SCG
prl2tse -S+ o/g090615-OPNEND-1h-atmw.prl >! /home/hgs/TD/DSYNTH-drift.tsi setenv DSYNTH_TSI/home/hgs/TD/DSYNTH-drift.tsi

Where the CONT command is used, check that the file has been renewed recently, and that the urtap solution was successful.
The models for drift synthesis that are relevant here areDSYNTHand DWWFSYNTH . There are other solutions (OCHY using hydrology and nontidal ocean in the analysis; EXPFIT solving exponential decay parameter) that are either lagging in time or need to be re-executed manually.

cd ~/TD/a/Allsmp-s make-drift-tsf [-DY] -Odest-dir/ *<yyyymm>*.drop.smp.tse

alt.

make-drift-tsf [-DY] -CY -Odest-dir/ *<yyyymm>*.drop.smp.tse

alt.

make-drift-tsf -DY -CY -O ../Allcamps/d/ `the-dropfiles -x .smp.tse<campaign>`

most actual (three drift solutions in one AGSG*.mc -file):
How many hours do you need?

set nhours=`tsld2p +N yyyy,mm,dd ~/Ttide/SCG/o/g090615-OPNEND-1h-lap.ra.ts`

Make drift files, three variants:

tslist _$nhours -B2009,06,15 -r1 -Emake-drift-tsf,DWWFSYNTH -I -o~/TD/a/Allcamps/d/synth-drift-extd.tstslist _$nhours -B2009,06,15 -r1 -Emake-drift-tsf,EXPF -I -o ~/TD/a/Allcamps/d/synth-drift-expfit.ts

of which this one ...

cd ~/Ttide/SCG prl2tse -S+ o/g090615-OPNEND-1h-atm.prl > ! ~/TD/a/Allcamps/d/g090615-OPNEND-1h-atm-drft.tsi setenv DSYNTH_TSI ~/TD/a/Allcamps/d/g090615-OPNEND-1h-atm-drft.tsi cd~/TD/a/Allsmp-s tslist _$nhours -B2009,06,15 -r1 -Emake-drift-tsf,DSYNTH -I -o~/TD/a/Allcamps/d/synth-drift-simp.ts

...uses the latest weekly solution with local barometer.

We now produce ascii-files synchronized at drop times:

make-drift-tsf -DY -from synth-drift-extd.ts -CY -O ../Allcamps/d/DWWF/ `the-dropfiles -d ../Allsmp-s-x .smp.tse <campaign>`
make-drift-tsf -DY -from synth-drift-simp.ts -CY -O ../Allcamps/d/DSYN/ `the-dropfiles -d ../Allsmp-s-x .smp.tse <campaign>`make-drift-tsf -DY -from synth-drift-expfit.ts -CY -O ../Allcamps/d/EXPF/ `the-dropfiles -d ../Allsmp-s-x .smp.tse<campaign>`

which will create samples of the SCG drift model in synch with the drop.smp-files
one file for every drop.txt -file. In the case above both AG's of the parallel campaigns are included.
The directory for the drift.ts -files is ~/TD/a/Allcamps/d (hard-wired).
Destination directory can be ./ but is usually ../Allcamps/d/ Files produced: *-drift.tsfTo get a list of drop.smp.tse files from./dropfiles.lst, issue (or exec within ` ` )

the-dropfiles -x .smp.tse <campaign>

Prefer to add the check-up option -CY whether the old syn-drift.ts file is long enough.
If not,

cd ~/Ttide/SCGtslist _#n -B2009,6,15 -r1 -Emake-drift-tsf,SYNTH -I -o ~/TD/a/Allcamps/d/synth-drift-new.ts

and copy to~/TD/a/Allcamps/d/synth-drift.tsif the file is o.k.
Replace SYNTH with DSYNTH or DWWFSYNTH to base drift on the most recent open-end solution of the weekly SCG analyses, standard or extended solution, respectively.

All of that and much of the following can be carried out with a super-script, urtap-merge-pp

Then, again in a/Allsmp

mc4campaigns -DY

which will run a full series of agsmp2mc , one for each#Cline in campaigns.lst .
The filename expressions determine which drop sets will be combined for each individual calibration.
The mc-files are (as long as the hard-wired parameters of 2014-08-11 are intact) stored in ../Allcamps/d/ .
To reprocess one campaign,

mc4campaigns -DY <DVAR>[-DD ../Allcamps/d/DSYN/ ]-C <campaign>

where the -DY option asks to use the "synthetic" drift files (from TSF EDIT-recipes inside the make-drift-tsfscript).

And this is for GGP-filter corrected files:

scgts4ag -GO -new 201405a -f -O a/Allsmp-s -ilp -sigma -n -S %.smp.tse a/Alldrops/*A?_201405*a.drop.txt cd a/Allsmp mc4campaigns -DY -smpdir ../Allsmp-s -dt 5 -C 201405a

Most actual:

mc4campaigns -DY DWWF -smpdir ../Allsmp-s -dt 5 -DD ../Allcamps/d/DWWF/ -C<campaign>mc4campaigns -U -DY EXPF -smpdir ../Allsmp-s -dt 5 -DD ../Allcamps/d/EXPF/ -C<campaign>mc4campaigns -U -DY DSYN -smpdir ../Allsmp-s -dt 5 -DD ../Allcamps/d/DSYN/ -C<campaign>

The latter one is most probable to extend through a campaign, since its creation is most independent of acquired model data.

OBS! In campaign 201006a, see urtap-$c.ins, there is only one "="-command spanning the whole
project period (there is only one project!).

cd ~/TD/a/Allcamps
cp mc-files.lst mc-files.lst-bup#n
make-mc-files-list -o mc-files.lst $campaigns

and manually update mc-files.lst

There is a potential (and common) problem with combining projects:
The exact epoch of each drop file. The help text of mc4campaigns says:
A file ./agsmp2mc.problems is created with information on
eventually low retrieval quotas in the sampling process.
In such a case, analyse whether sampling schemes make jumps
(worst case if e.g. a 10 - 20 - 30 seconds scheme jumps to a 05 - 15 - 25 scheme).
The doubt margin of readfuf/readfmd cannot cope with this.
The best way is to force 1/2 of the auto-detected sampling interval upon the output mc-file:
In a/Allcamps/campaigns.lst add a line
#D <campaign> <dt>
decide on a common divisor of the sampling times (in 200907a this would be 2 seconds), and

mc4campaigns -dt p-DY -C <campaign>

or, if Allcamps/campaigns.lst holds the information in a#D -record

mc4campaigns -dt LIST-DY -C <campaign>

However, if the mc-data is to be combined with seismometer records where missing-records and precision to the second matter,
a series of mc-files by-project should be generated. The -s option in mc4campaigns can be employed.
More on this theme, in application the the campaign in February 2015 is here.
Check that Allcamps/campaigns.lst is consistent with the imposed sampling intervals!

Then we assemble urtap.ins files for each campaign, using the <campaign>-jump.dat files and a template inside the script

make-urtap-ins -C <campaign>-with-drift -outliers-from-raw

or, to fetch the mc-file name from mc-files.lst rather than from the jump-file(s), like in campaign 201502b,

make-urtap-ins -C<campaign>-mc-from-list -with-drift-var SYNT -outliers-from-raw -include =

where we also specify a drift signal (SYNT) and allow only bias parameters in the signal model.
Manually update the list files meters.lst and orientations.lst (this may help:)

ls *$campaign[n]*.drop.smp | awk '{print "%O",$1}'

Then you can run

urtapt @ urtap-<campaign>.ins

or if you dare

foreach i ( urtap*.ins )
urtap-outlit $i
end

Branch here to urtapm.html for a multi-campaign analysis

The sequel here continues with single campaigns

Now you can analyse the bias-and-slopes results, and the calibration result.
The weighted mean calibration factor from all campaigns:

cal-wmean [-d]

where the -d option makes cal-wmean more talkative.

Bias and slope results:

evaluate-b+s <campaign>
evaluate-b+s -b <campaign>

Then you can collect the outlier data (for showing them on thecalplot)

make-outlier-mc `awk '/^#C /{print $2}' campaigns.lst`
make-outlier-mc $campaigns

producing e.g. o/AGSG_200907a_FG5-233-10s-del.mc for campaign 200907a

CUTOUT:
Next step is plotting with calplot .
There is a simpler utility too:

plot-cal-results [-scg-rms] [-with-outliers] <campaign>

But let's continue with calplot . The bias-and-slope solution that
was produced by urtap will be plotted at all times of the campaign.
i.e. also through the pauses. In order to suppress it there,

project-times2tse -do-cutout -o o/cutout $campaigns

Have a look at the tse-files produced; use setenv to set variables to appropriate
values before you continue:

setenv CUTOUT o/cutout
foreach c ( $campaigns )
tslist o/scg-cal-$c.ph01.ts -E o/cutout-$c.tse,CUTOUT -I
end

Finally, you can plot

setenv PNGOUT ~/www/4me/ag2014/
calplot -sj -cal -del d/AGSG_200907a_FG5-233-10s.mc

Extra settings are needed for e.g. 201004a

setenv WEEDOUT 3000
set c=201004a
calplot +LEG C:8.8,4.0 -sj REMDC -pfs -cal -del d/`the-mc-file-for $c`

Examining the bias-and-slope files:
Explore the uncertainty range of biases and slopes:

prl2tse -XB 1 -S o/scg-cal-201405a.prl BPS0 >! synth.tse
prl2tse -XB -1 -S o/scg-cal-201405a.prl BMS0 >> synth.tse
prl2tse -XS 1 -S o/scg-cal-201405a.prl B0SP >> synth.tse
prl2tse -XS -1 -S o/scg-cal-201405a.prl B0SM >> synth.tse
prl2tse -S o/scg-cal-201405a.prl B0S0 >> synth.tse

or by a script:

vary-bias-and-slopes $campaigns

Everything from the above starting with urtap in a one-size-fits-all approach can be done with

foreach c ( $campaigns )
analyse-mc -S -C -PP $c
end

(sorry, can analyse-mc with plotting only one campaign at a time)
____________________________________________________________________________________

Plotting with plot-cal-results
Plot includes:
    AG samples with error bars minus SCG
    optionally, outliers can be included (option -with-outliers)
    bias+slopes
    SCG long-term residual
    microseismic RMS from SNSN Seismograph
    ... from SCG (optional)
If seismograph data is not available, use the option -scg-rms
or try produce with ~/Seismo/gcf/long-time-movrms

long-time-movrms -h

shows all necessary steps.

Next project:
CREATE ANCILLARY DATA FOR DIRECT CORRECTION OF AG-DATA WITH SCG DATA
Concept: AG = AG_raw - AG_jumps&slopes - ( Filter{SCG_G|V, deci.tse,F30} - SCG-drift ) * Cal_new/Cal_old

Cal_new = weighted mean of all cal campaigns
Cal_old = value in ~/TD/t/urtap.trs
Subdirectory for the example: tsfo/ under Allcamps/

Reducing Drop-data to Set-data:
Example: As a demonstration of principle, do one campaign,
simply the measurements, don't care what, just as an exercise

tslist d/AGSG_200907a_FG5-233-10s.mc -LA -LA -E autoset.tse,A -C3

with autoset.tse

TSF EDIT A
OPEN 31 < set.ts
AUTOSLMEAN U=6 ->W G=5.
; = use weights, print results ASCII to protocol, gap width=5*dt,
; then copy the result into the W-array and output:
DUMPW U=31
END

Another example, subtracting the SCG:
(that this works o.k. is owned to the fact that subtracting a series involves work storage W while the AG-series and its weights are imported into X)

setenv FUNIT 11 setenv OPEN22 '; ' setenv FNSUBLABEL 'S+V' setenv FACSUB 774.16 tslist d/AGSG_201405b_FG5-220-4P-05s.mc -LA -LA -keep-open \ -E subts.tse,ADDVL -E autoset.tse,A -C3 -qqq -w tmp/A-S-201405b-autoset.tsf

and converting dates to hours from midnight of day 1:

awk '{print ($7-56805)*24,substr($0,32)}' tmp/A-S-201405b-autoset.tsf

Again, we have a useful script for going through all campaigns:

produce-ag -results -weighted-means -with-slopes $campaigns
prodcue-ag -results $campaigns

which amounts to the same using -results defaults.
using autoset.tse (by default).
ask produce-ag -h what else it can produce

Depending upon the task for produce-ag, a specific block
in autoset.tse is loaded. The commands specify a.o. the
output subdir for the results. autoset.tse is responsible for
creating the columns in the final ...$campaign.set.mc files

foreach c ( $campaigns )
final-analysis -add 380. -S -I tsfo $c
end
final-results4plot -I tsfo > tsfo/final-results.tbl
plot-final-results

Produce own set-files, reduced as far as we can go (no biases though)

make-mc-files-list $campaigns

setenv REPORT msets/msets
setenv MSETSADDG 380.

foreach c ( $campaigns )
   project-times2tse -S -C 'AUTOSLMEAN U=32' -C+ '${FFMT:[+TSF]} ->W ${WOPT} G=${GOPT:[1000.]} ' -o msets/msets $c
end
setenv DELCOMMAND RJC
setenv DELTHRESH 200.

foreach c ( $campaigns )
    produce-ag -own-sets msets/ -weighted-means -with-slopes -use-gaplength 5 $c
end
set c = 200911a
produce-ag -own-sets msets/ -weighted-means -with-slopes -use-gaplength 2 $c

foreach c ( $campaigns )
foreach m ( AA AC AS )
    echo M=$m C=$c
    bundle -by-camp $c -by-monu $m
end
end
bundle -by-monu AC -by-instr FG5-233
bundle -by-monu AA -by-instr FG5-233
bundle -by-monu AA -by-instr FG5-220
bundle -by-monu AC -by-instr FG5-220

procag commands with their and our sets:
Their sets are usually more than one per campaign, while our covers all sets on the same monument with the same instrument during campaign

procag.pl -o -u -s -r -A -c Gravity -f "%5.2f " $our
procag.pl -u -s -r -A -c G_ures -f "%5.2f " $theirs

See plot-project how it's used
agsetwmean commands, examples (results in uGal!)
own sets:

agsetwmean.pl -q -o -R 0.0 -w msets/b/own-AC-FG5-233.set.txt
agsetwmean.pl -q -o -R 0.0 -w msets/b/own-AA-FG5-220.set.txt

their sets:

agsetwmean.pl -q -u -r -w -A ../Allsets/Onsala_AC*.set.txt
agsetwmean.pl -q -u -r -w -A ../Allsets/AA*.set.txt

plot-project -g 2 msets/b/own-AA-201405a.set.txt

The project plots suggest that we need to account for orientation biases. The respective ADD commands have to be issued inside produce-ag
and for that to happen the orientations have to be quoted so that they appear in the urtap results. So back to the urtap-step. The responsible
script is agsmp2mc See and revise the script code at "NEW Sep 5 2014:"

Take North as the reference orientation
Use = commands for monuments different from AA and for AA if orientation is South.
Use / commands for each project.
Mark the bias events with platform and orientation, not JMP
Mark the slope events with SLO1...

After bundle you can...

plot-all-sets -M 233 -P AA

__________________________________

.bye