tslist { filename | - | generating option } [options]
tsl [+a] { filename | - | generating option } [options]
Purpose:
|
Environment: TSLIST_LEAPS nonemptyto recognise leap seconds. The -TLS option will toggle the leap second setting. TSLIST_MSGLEVEL #mlvl a value less than 5 will cause printing diagnostics to STDOUT, usually in the form <a-name>s> message text Level 0 is like debug, although the -DBG option might be even more informative, especially concerning the main program. TSLIST_SHM filename a file name in the RAM-disk shared memory as an alternative to hard-disk files. For input use `@´ as the first argument, output -o @ |
|
tsl +l directory to replace $HOME/tslist-logs +L logfile name of the log-file instead of the auto-generated. |
"Multi-column" (labeled) files:
If you need to quickly bind several binary time-series files into
one multi-component ("mc") file,
consider ts2mc as a model to code up a
similar script.
A label is internally a string of 16 characters. It may
consist of two parts; then, the first part starts at the first
position, the second part ends at position 16, and the centre is
filled with underscores. Example GRAV_________VAL
For retrieval, a simpler expression can be given (see
options -l... -L...)
For generation, a script ~/bin/label is available.
Labels are generated on output and written in the binary files.
For listing the labels of an mc-file, a script ~/bin/tslql is
available.
Input data:
Maximum length = 1,000,000 (c.f. PARAMETER (NDIM=) in
~/sas/p/mt/tslist.f
Reserved file units: 21 (input), 31 (output, optional), 4 (TSF EDIT, optional)
With the -1c option up to 10,000,000 samples
can be handled; this decreases though if the -E option is
used.
Specification of options:
Compulsory code appears in boldface,
occasionaly for emphasis of interpunction etc. underlined,
like ,
User-supplied parameters etc. in italics,
Names of data arrays in bold italics
like X
s - single
character or numerical value,
code - a string,
#v -
(occasionally) a numerical value
[ ] - optional
argument. Also, at the right margin: a
default value
{ | | ... } - choose one
-# -% -0 -@ -A -AU -A> -A>GWR
-B -BN -Ban -Bn -C -Cts
-D -DBG -DM -E -F -H -I
-J -L -M -MA -MB -ML -MM -MQ
-MSYNC -MSYNCON -MT -MV -Ml -N -O
-OEA -OEP -OP -PRI -Q -R -RTB -RTP
-S -SQ -T -TLS -TMC -TPW
-U -V -Y
-a -brk -c -cv -d -dc -e -f
-g -h -i -j -k -keep-open -l -m
-mjd2dy -msglvl -n -ncd -o -p -q -r
-ra -s -sD -sR -sl -sx -t -u
-v -w -y -z
Option -
meaning
[default]
_____________________________________________________________________________________
FILE NAME | STDIN | DATA GENERATING OPTIONS
filename - a binary
file or an ascii file for time series input.
FAST DATA
TRANSFER FROM ONE tslist TO THE NEXT
Since binary data cannot be piped through the `|´ mechanism,
a fast alternative may use /dev/shm/filename
tslist ... -o @filename ; tslist
@filename ...
A fixed file name can be predefined in the environment:
setenv TSLIST_SHM filename
tslist ... -o @ ; tslist
@
MORE THAN
ONE BINARY FILE? USE
tslist filename1 -#
filename2 ...
OR
tslist
filename1 -@
filename2 ...
OR
tslist
`tss2mc filename1 filename2 ...` instead of
the filename
Creates a multi-column work area in memory, columns in the
order of the files
Please read tss2mc -h
NEED TO
APPEND BINARY FILES? USE
tslist-app
-o out.ts -I + filename1
filename2 ...
or with tsf-edit:
tslist ... -E tse-file,trg
NEED TO READ
ASCII DATA, MULTIPLE COLUMNS OR MULTIPLE FILES?
With
date-stamps and one data column at a time: a range of options
are
available. If samples of the signal are spread along rows,
you must code your own special tsf2bin-ts or tsf2ascii scripts.
-
- (minus-sign) Input ASCII from STDIN.
netCDF-multicolumn files (we assume a certain structure, has to
be documented,
we've only just begun on 2019-01-15)
tslist -ncd[opt]label
file [file ...]
The files are appended to each other. If there is no more option
after
the last file name, add a placeholder `-´
-ncd[o]lbl - lbl
is the search string for the target column. The search function
is
index(nc-label(i),lbl(:l)).gt.0 , 0 < i <= Ncolumns
where l is the position of the last nonblank character.
Options o, one character: Any of
? - * to show the column titles of the nc-file
* : ignore case in search of lbl
Don't forget quotation marks when file name globbing is in
effect!
DATA GENERATING OPTIONS
REQUIRE SPECIFICATION OF EPOCH (-B) AND SAMPLING INTERVAL
(-r)
_n[,v[,c]]
- Generate a file of length n
with value v in
column c.
[0.0d0,1]
_n[,v[,c]] +c2,v2 [+c3,v3 ... ]
v can be the character `M´ to create a column
with n MRS's
(OBS! swapped order between `_´-generation code and
`+´-parameters)
_ - Generate
a minimum "file" with one sample,
can be useful to test other options including tsf-edit
functions.
-B<today> and -r1.0 are set by
default.
+n[,s][,t] - Instead of `+´ you can code `%´
(accepted by tslg, tslq and axlim4ts)
Generate a random series. Use -B -rdt and -0
to set epoch, rate and start. Use -Svalue
to set amplitude.
(CAN WE DO COLUMNS 2...
LIKE UNDER `_´ ?)
if s is not given, a
series of zeroes is produced. Else...
s is the random seed, t controls the distribution
s must be a negative
number
t = U for a
uniform distribution
t = [m,]G for
a Gaussian distribution with Markov parameter m.
t = b,F for a fractal distribution, spectral power P(f) ~ f
-b
t = b,c,X for extended fractal, spectral
power P(f) ~ 1/(c + f b)
n specifies the number
of samples.
OBS! In noise types G F
and X the seed is always converted to a negative
number.
-I
- print protocol information only.
Else: protocol and time-series. To
strip protocol,
pipe output through fgrep -v '>'
or output time-series to an ascii file
using option -w
-IQ - test section quality (Tidal_Filter, RMS --
doubtful)
-IHDopt - report
Hausdorff dimension (doubtful -- a better description is
needed)
-t - Suppress printing of the GMT
option string (no -R or
-B)
The -t option should be given early
on the line so that
the report from an eventual -Bepoch option gets muted.
-tS
- Capital-S will print items without delimiters.
sopt = t[x|y]
- requests a suitable string for the GMT-grid/tick option (-b)
sopt = r[x|y]
- dito for the GMT-range option (-R)
x resp. y requests
the information for the x resp. y axis only.
dopt
= dymin/ymax:
dopt
= Ddummy/dummy/ymin/ymax:
- suggest a minimum
range that the actual data may extend.
The / may be
replaced with a comma. We don't have this for the
time range yet, sorry!
This option
must be delimited with a colon at the end.
topt = minx,maxx[/miny,maxy][fopt][bopt]
- where the numbers
control the minimum and maximum number of
labeled ticks
on each axis.
fopt = ,f:nx,ny
or ,fx:nx or ,fy:ny
- where the
numbers nx and ny indicate fixed amounts
of unlabeled tick intervals
(INTERVALS!)
bopt = ,by:by
- where
by is a range-margin, e.g. by:0.2 to extend the
min-max
ordinate range on each side by 20%. This is useful to get
generous plot boundaries.
String bopt must be last. A bx option is not implemented
(yet)
-t implies -I and -qq (stop before copy to STDOUT, quiet)
MANY
DOUBTFUL RESULTS - NEEDS IMPROVEMENT. EXAMPLE(o.k):
$ tslist +59,2,U -I
-td-5/5:,by:0.1 -rs1 -n1
-R0.000000/60.000000/-6.00000/6.00000
-Ba10.00f1.000/a2.000f0.1000
-qs[s...] - query
information
-qb -qe -qa -qbe
- first (s = b
or a) and last (s = e or a) date.
-qB ... - s = B
s = E s = A like above, a record with
tslist options to
limit the time scope of input is written,
however:
({ -Byyyy,mm,dd -Hehh,mm,ss,ff } { -Uyyyy,mm,dd,hh,mm,ss,ff })
Add a c to produce
the cutting version of the -B and -H
options:
-qAc produces -Bcyyyy,mm,dd -Hcehh,mm,ss,ff -Uyyyy,mm,dd,hh,mm,ss,ff
If the apply-early option -He is not desired, use
-qBl or -qAl .
Thus you can input a series with the date limits of another
series:
tslist file2.ts `tslist file1.ts -I
-qAc | grep -e '^-'` ...
-qm[y,Y] - query
minimum and maximum and their times of occurrence.
Extend on the specified range y,Y
[1.d40,-1.d40]
-qsg[#c1,#c2,...]
- show gaps for columns c
[1]
-qc - query comment part of label if there is one (mc-data only)
-qq -qqq - quiet; suppress
normal messages in protocol
(not really a query, rather an anti-query).
Messages from a range of subroutines will not be suppressed.
In that case, pipe STDOUT through fgrep -v '>'
-qq leaves the gap records (>) in the output.
If -qq[q] is specified early on the parameter
line, will suppress
messages from subroutine set_epoch too.
-L
- List all labels in a multi-column file.
-IHD[i,j,k]
- investigate Hausdorff-dimension. The starting point in the
time series is stepped through a do-loop DO io=i,j,k
[1,1,1]
-1c
- For time series with more than 1M samples, this option makes
the entire multichannel memory available for one time series.
Mnemonic 1c = one-column. This option must be given first,
since it affords new defaults for array sizes and truncation
points.
WARNING!!! Multicolumn options will fail to work correctly.
No safeguards have been built in yet.
-c#c - If no other
option (directly or implicitly) sets a maximum
column number, use this option.
With
ASCII input, see there.
-D[I][fmt]
- remove DC-level. The value is printed under format fmt.
The I option requests the DC-information only for
print;
actual removal occurs only if I is avoided.
Reading mc-files, the DC-level is removed only
from column 1. To remove it from all columns,
-D[I]C#c1,#c2,...
- remove DC-level from channels c1,c2...
-D[I]A[fmt]
- remove DC-level from all
columns.
[1p,d13.4]
-Y[L]FRN#s,m,#a[,#s,m,#a...]
- add fractal noise with parameter
m.
-Y[L]FRX#s,m,#a,#c[,#s,m,#a,#c...]
- add extended fractal
noise with parameters m and c.
+{u|g|f|x}[#c:]#a,#s[,#p]
- Add noise to column c (default 1)
with amplitude a and seed s.
u
- uniform
g - Gaussian
f
- fractal with power-law parameter p
-R[s]#p,t0,dt
- Resample. p is interpolation polynomial order,
...............t0
is time-shift [h], dt is new time step [h].
Use -Rs with time specifications in seconds.
-E[#c1,#c2...:][ ][tsefile][,target[]option]]
[1:tslist.tse,TSLIST]
- TsfEdit commands are read from file (unit 4).
Up to 10 -E options can
be issued. They are applied left-to-right.
Default tsefile = previous file
target and option: cf ~/sas/p/tsfedit.f
c1,c2...: (right-delimited by
`:´) specify the
data columns
on which the processing
is to act; default=1.
Caution if the file name or target or option contains a colon:
In that case a delimiting colon must be given (-E:),
even if defaulting to c=1 is intended:
-E:filename,AB:CD should work (if
there's no bug).
See example (24).
-Alistfile[,| ][tsefile][,target[]option]]
- for appending many files with filtering and decimation
(or anything else you'd like to do with TsfEdit)
See the example in ~/sas/casc.tse
.
Peculiarities:
Only one label = one MC column can be processed at a time.
The -Llabel declaration must be valid for the entire
set of files.
-Mm
- define missing record
symbol for internal use and binary output,
num.value
m . Also resets the test accuracy
to 1.0
[internal]
-MTa
- define test accuracy, num.value
a
-MT must appear after -M.
-MSYNC
On multiple columns, delete all rows that carry one or more
missing records.
-MSYNCON like MSYNC,
plus contract
to close the gaps.
THEREBY THE TIME INFORMATION BECOMES INVALID.
-Mother - see print options
-m[o]
- mend (repair) with option o. Option 'L' requests linear interpolation.
Else: substitute
DC-level
[' ']
-S[/]s[,s...] - multiply time series column j with scale value s(j).
A zero value for the 2nd etc. s defaults to
|s(1)| !!! !!!
(useful for measurement in column 1 and uncertainty in column
2)
/ - divide by s (else multiply with s)
(before
TSF EDIT operations).
[1,0,0...]
-SO[/]s[,s...] - like -S, but after TSF EDIT and -U operations).
[1,0,0...]
-SQ
- take square root of time series (MRS at negative values)
-SR[/]s[,s..] - scale time
series to unit-RMS deviation, further divide by s
/ - multiply
(sorry!!!)
[1,0,0...]
-SM[/]s[,s..] - map time
series to interval [-1,+1], then multiply with s
/ - divide
[1,0,0...]
-SOM[/]s[,s..] -
like -SM, but after TSF
EDIT and
-U operations).
[1,0,0...]
NB: We don't have -SOR (yet) Use tsfedit! !!!
Some "row maths" (more maths: see Label
maths)
-RMG#c1,c2[,...]=#ct
- Gap filling. Requires -tie+X
[no defaults]
Suggested: -RMG1,2=1 -tie+X {-o1 file.ts
| -p1 -w file.tsf}
-RML[o:]#s1,#c1[,#s2,#c2...]=#ct
- Linear arithmetics.
Xct = Sum(si Xci)
[no defaults]
-RMM[o][:#c1,#c2][=#ct]
-RMD[o][:#c1,#c2][=#ct]
- Multiply, resp. divide, column c1 with c2 and
store result in ct [1,2,1]
Option o :
e
- apply early with respect to Tsf Edit and other row
maths
[e]
l
- apply late, right before output (verify!)
o - option to apply early `e´ or late `l´ w.r.t. tsfedit [e]
Notice how decoding of the option
differs between -RML and -RMM/-RMD !
-RMQoptions[:#c1,#c2..,#cm][=#ct]
- square-sum arithmetics. Sums comprise columns #c1..#cm
Result is stored in column #ct.
Options:
m - divide by the
number of columns
s - take the
square-root
r - remove DC-levels before squaring
e - call time_range reading a hard-wired
file tslist_trms
(one line) - needs to be built out
v - set to
missing-record symbol outside the time range.
The result is copied into column #ct
k - The number of active columns will be
kept
(else it is set to #ct)
Defaults:
none for options, none useful for c1..m
but #ct defaults
to one column of extension
A 3-D vector norm would be evaluated using
-RMQr:1,2,3=1 or -RMQr:1,2,3=4
in conjunction with multicolumn input e.g. -L'X|V' -L'Y|V'
-L'Z|V'
-RMPoptions[:#c1][=#ct]
- Transform from X Y to Polar R Phi
columns c1 and c1+1 to ct and ct+1. Options:
r - remove DC-levels
d - Phi in
degrees
[radians]
-RMXoptions[:#c1][=#ct][,A=a]
- Transform from Polar R, Phi - a
to X
Y
[0.0]
columns c1 and c1+1 to ct and ct+1. Options:
d - Phi in
degrees
[radians]
-cv#v,#w[,#c]
- Prints records only if the values in column c are
within an
interval [ v-w , v+w ] Default
column is 1.
Date cutting
-Be - If
none of the following sub-options, is used,
set the beginning at the epoch's midnight.
NOTE: This option appears necessary for -n
output time tags.
Should work as a default (i.e. when qcut and qfix are both
.false.)
in order to distinguish -n from -nr such that
under
-n the first tag is at the time of the first record
while under
-nr the first tag would be zero.
Why we appear to need -Be is not really clear. Use -Be as a
quick-fix.
-B[a]t[c|f][s]#t
- Cut
samples worth duration t from start,
-B[a]n
s = h m s
or f specify in what units, hours (default), minutes,
seconds or milliseconds, respectively. Adjust
epoch right before
cutting date (a). (useful if time
series has significant start lag).
-B[a]n is synonymous with -B[a]t. ( - really?)
These two options do not affect binary
file output!
-B[a][c|f]date
- set beginning epoch. Default=first date on file.
Set (= "fix" against) epoch (f), cut at epoch (c).
Adjust epoch always before the cutting date (a) (useful
if
time series has significant start lag).
Default = no fix, no cut.
Format for date is yyyy,mm,dd
E.g. tslist
d/g1-090810-1s.ts -qbe -BHc2009,08,10,18:12:12 -C3
E.g. tslist
d/g1-090810-1s.ts -qbe -BHc2009,08,10,18.02 -C3
| Note: If you intend to
call tsfedit to work on the series such that cutting effects
at the start occur, do not use the -BHc
version of the option. It will first shift the origin to a day boundary, then call tsfedit, and then shift the remainder---which may have been cut off. Rather use -Bcyyyy,mm,dd specifying one day earlier, and -Hv with a value v > 24 h In this way you will preserve a lead-in data segment. |
-H
is similar to -B but restricts scope to hour part.
Epoch setting
is not dealt with; that must be accomplished during file inoput
or by epoch setting using -Byyyy,mm,dd
.
An e sub-option requests that the cut is performed after
tsfedit
processing is complete.
-H[a][t][{c|C}|f][e|l]h - fix (f)
or cut (c) against number h of hours from epoch
n
(h may be specified as a decimal
number or in the
form h[,m[,s[,f]]] with legal delimiters
, - : )
or cut at h samples
from epoch (n).
Adjust epoch right before cutting date (a).
c|C - Without -B
but with -HC : If a record starts at hour t
you can
specify an hour h >= 24 and the output will start
from
max(h-24,t).
That is useful for sets of records which alternately
start
e.g. at midnight or a few hours earlier, epoch one day
earlier.
e|l
- Apply early (e) or late (l), meaning before or
after tsfedit,
(-E option). Default is cut and late.
[0]
-UEu
- Truncation to perform before TsfEdit
[at output]
Beware a
few exceptions the sub-options of -U are
identical.
-Uu
- Truncation of various
kinds, sub-option u see
below.
Default is to output all data of course.
-Uh#n
- truncate to achieve duration n
hours.
-Ud#n -Um#n -Us#n
- like -Uh#n , to
alternatively specify days, minutes, seconds.
This (H, h, d, m, s) works cumulatively.
OBS! If -Uh or its variants above
are specified, it takes precedence
over the -U's shown below.
-Un[v]#n - truncate after n samples from
start (n)
or n valid samples total (nv).
If
n < 0, truncate at n samples before the end (n)
with
v, does not work under -UE
In context with -BN#m ,
print until and including record nr. n.
For
consistency the options ... (true?)
-UT#n - #n
is the final sample number,
-UN#n - #n is
the amount of samples,
are available. Note that -BN and -UN -UT prohibit binary
output.
-Un+#n - truncate after n samples from
start, to be used with -j+ or -j@
-Ul[v]#n - remove n samples (l) or valid samples (lv)
from the end.
(What's the diff to -Un-n ?)
-Uon -
truncate in output loop (reads and processes all data
before output), number of samples
-Uy,m,d[,h]- truncate at this
date + hour
(h may be specified as floating-point or in the form h[,m[,s[,f]]],
legal delimiters are , - : )
-tie[o] - in MC
processing, align columns 2... with the leading column 1
(the master column) such that the time stamps
apply throughout.
The
alignment occurs after tsf-edit.
o -
Option string
+M0
+DC - fill with zero or DC-level where data is
missing [MRS]
+L
+X - cut to the original length or
extend to the length
of the master; default: keep
as much data as
possible
[keep]
The default, untied, is deceiving! As a preliminary attempt
to improve, tie is set without option if time-dated
records
are output (-C and -J , but not with the default, Decimal
year).
-untie With -N (and?),
you have to specify -tie later on the command line
with
-C, -J and ..., -untie must be specified lster.
-sc[o] - An early
instance of tying might become necessary before
tsf-edit. Inside tsf-edit this could have adverse effects when
the tied columns have long segments of MRS,
esp. with REPAIR.
The options to -tie and -sc are identically applied.
One
specification suffices.
Should need arise we'll make them
separate.
PRINT OPTIONS
-ws[a][:#n] [f]
- One-column output, not time-tagged, -wsa for append.
f is a
filename
[STDOUT]
#n is the column to
output
[1]
Unit 8 is used if a file name is specified.
The -F -q.. and -I
options are significant.
-w[a][':text']]
f
- Default time-series printing is to STDOUT.
Use -w to redirect to
file f, -wa
for append.
OBS! Append uses file unit 98 for
temporary storage.
The -q.. and -I options
are significant.
:text] -
Prepend every output line with this text. Useful to
create a
SAMPLE file (combine with -C4 , see
example).
[none]
-qq
- quiet (suppress normal messages on protocoll)
-qqq
- Quiet, -qq and
suppress gap markers (>)
The -qqq option affects
the output under pption -w
-zo#v[,#v...]
- The missing-record symbol on output for each column
-zo#v
- The missing-record symbol on output for all columns
Default is the binary
input's MRS unless redefined by -M
-j[s]#n - Jump; skip the first n values on output if n > 0
s - j : With -jj , skip the first n-1 values. -C[{R|X|Y}][k][,D:fmt]][,F:fmt]]
- print date in Calendar form,
k
hour-minute-second depth(khms
. Default from -k
[0]
R or X
Print reduced resp. full Julian date.
Y Print decimal year
[modified Julian date]
,D:fmt fmt
is the format for the calendar date
[i4,2i3.2,1x,3i3.2,i4.3]
,F:fmt
fmt is the format for
the Julian date
[f13.6]
-N[code] - Output time
record is an ordinal number 1 2 3... Format code
may indicate integer or floating-point real, depending on -n option.
-Ni will print integers with the default
format
[i10]
-N floating
point
[f13.6]
-nFrq[{s|m|h|d}]
- If TsfEdit produced a periodogram or a MEM spectrum,
a frequency tag is produced that replaces the time tag in ascii
output.
The tag
can be in a measurement unit or dimensionless (-nNyq)
The range of dimensioned units is
s - Hz | m - mHz
| h - cyc/h | d - cyc/d
If you read in a binary spectrum file that was output with
tslist
... -o spectrum.sps, you can use -nh to print the
frequency
tag in
the same unit as when it was produced. You can also use
-nFrqu with the unit u of choice.
-nNyq - The frequency tag
divides the dimensionless Nyquist range 0 ... 0.5
-n and -nr ... in the
following are equivalent; the difference being that
time-tags are relative to the epoch or to the start of output,
respectively.
(If the first time-tag
isn't 0, the reason is in variable ncutb)
-n[hms][a] - Time record = j dt + a in hours (h), minutes (m) or seconds (s)
from epoch + t0, add a. Specify e for a to obtain time from
epoch.
Example:
tslist d/A2_9_120104-1s.ts -Nf16.8 -BHc2012,1,3,12 -ns -00
(Sets epoch 1d before file start, forces t0 = 0 at epoch 2012,1,3,12
thus first record is shown as 43200s after epoch, but actually
belongs to 2012,1,4 - so it's an example how to fake a time
shift
with the -00 option.)
-ni[b] - time record = array
index, yields 1+b
2+b
... n+b
[-1]
If -Bnj has been given,
yields j+1
j+2
... n
If -Ulvj has been given, yields n-j-m+1 n-j-m+2 ... n
(m is the number
of missing samples in that segment)
-nff
- Multiply output time record under -N with factor f
-n/f
- Divide ...
-n/N
- Divide by the number of samples; yields 0.0 .. 1.0 step
1/(N-1)
-n[[s]b][{/|*|x}[s]f][sa]]
-nm[[s]b][{/|*|x}[s]f][sa]]
-ns[[s]b][{/|*|x}[s]f][sa]]
-ns+t0
You may specify the symbols t0 for b and dt for f in order to
add the time of the first sample from the epoch and
the sampling
rate, respectively. It will be invalid to specify t0 for a
Example: tslist
o/scg-cal.ra.ts -Ff8.3 -Nf12.0 -n'-1*20' -C3 -qqq
2010 04 19 19 00
00
0. -36.393
2010 04 19 19 00
20 3600.
-29.299
2010 04 19 19 00
40 7200.
-48.745
2010 04 19 19 01
00 10800.
-18.703
2010 04 19 19 01
20 14400.
-45.400
2010 04 19 19 01
40 18000.
-51.634
2010 04 19 19 02
00 21600.
-30.810
2010 04 19 19
02 20 25200.
-38.973
OBS! The -TT option will
once more transform the time record.
-i[fmt]:[y]-
Output time record is t0+n*dt (from file)
in hours
Add y to
time.
[0.0d0]
Use the -TT option for further scaling and offsetting.
-J[fmt] - time
record is Mod.Julian date. fmt format
code.
[f13.6]
-JR[fmt] - time
record is Red.Julian date. fmt format
code.
[f13.6]
-JX[fmt] - Exact (long)
Julian dates; they start at
noon.
[f15.6]
N.B.: The Julian date default comprises day and hour.
If you only have the day (with or without decimal places)
you must use -J with the -k0 option.
-{h|m|s}fe[fmt]
- hours or minutes rep. seconds from epoch
[f13.6]
-Llabel [ -Llabel ... ] Input labeled columns; see
section on input options.
If you process n labels you've got to output n
columns,
so you'll need to define n numerical fields, e.g. -Fnf10.3
[10G12.6]
-FA[e][w] - Let the program recommend a format.
The maximum field width
for a plain floating point value is w
[8]
Specify e if you like
exponent symbol e
[d]
If the program suggests an exponential form, the
number of post-dot places will be max(1,w-4)
Exponential forms are always shifted using 1P.
-FA can advantageously
be combined with -qqq -w
/dev/null
The last (and prospectively only) line output will then read
e.g.
<Main-->>>
suggest FMT: -Ff8.3
-FA is a
new feature (May 2010) not really well tested.
Subroutine: numeric_range in ~/math/afor/p/numrange.f
Limitation: analyses column 1 only.
-V[o][v1,v2,...]
- print
numerical value v only
at valid records . Options have been
deprecated.
o = I - print this value only at invalid
records Use -MM
instead
o = R - print zero at invalid, value v at valid records
v1,v2,... by default
taken from -Y
-MA[v]
-
include
the
missing
record
symbols
in
the printout.
v = replacement value.
By default, the internal symbolic value is used.
(To change the symbolic values for GGS output, use -zos1,s2,...
Explained under -p )
-MA with -Vv will print zero at invalid
and v at valid records,
N.B. -MA must then be specified without a value.
-MQ | -MS - print a row
only when all columns contain a valid record.
[-TMC applies]
-MM[v,vi] -
print at invalid records the value vv = vv
+ vi
[0.0,0.0]
where vv is reset to v at each valid sample.
The output format comprises the date part as usual.
The ordinates are suppressed. Instead, messages are
produced reading date
BEGIN FILE, END FILE and
BEGIN
BREAK (-MB) or BEGIN
VALID (-MV)
followed
by a line that lists the ending date and the
length
of
the block.
(The
flag is empty when regular records resume).
...............See
example (15). -MA,
-MB, and -MV are mutually exclusive.
-TMC#c
- test for missing-record symbols in output column #c.
Suppress to print the line if
yes.
[1]
-pc:1,2,3... print c columns, permutation
1..10 of input sequence
(Requires BIN reading from multicolumn files).
-zos1,s2,... The missing-record symbols for the
output columns in the
order as they are printed, e.g. -p2:5,1
-zo99.99,-99.99 [internal]
(-zo is currently
implemented for GGS records (-TGGS) only)
-Q
- Fail if an error is encountered, else continue.
(Not fully implemented yet, but works with -E)
Special Output Formats and operations
-TGGS[fmt]
- Write GGS files (needs two columns of data) Fixed format.
fmt is the format for the numeric fields
[f10.6,f10.4]
Should be appended to a pre-existing header using -wa
see Example 18.
How to read GGS files, see Example 19.
-TGGP[fmt]
- Write GGP files. Fixed format. Not really complete yet, gaps
formatted o.k.???
fmt is the format for the numeric fields
[f10.6,f10.4]
Should be appended to a pre-existing header using -wa
-TPD#d[,#c]
- Place gap records `>´ based on jumps of data in column c
[1]
(X(c)t - X(c)t-1)
> d or < 0
Useful for tp-mc files when plotting output.
(other strategies need coding; search for qxgap in tslist.f)
-TPW[fmt] - Write out ASCII
in tp-format, i.e. first '>', then
Julian date and
value in two lines, where the first line's Julian is dt/2
early
and the
second line's dt/2 late. Can be processed later with -RTP
-TPS[u]#d[,opt]
- TP-MC-file
processing in conjunction with Absolute gravity):
Auto-detect sets in a campaign-merged file, and calculate set
means.
If only
-LM -Ldata are loaded, calculate simple averages
If
-LM -Ldata -Lsigma are loaded,
calculate weighted means and
stdev of means. See subroutine tp_setmean in ~/sas/p/tps.f
u - time unit, s h or d for seconds,
hours (default) or days.
d - length of gaps for detection of sets (projects or
even campaigns)
opt - +BE
- print two lines with MJD begin and end for each set
and a
division record `>´
+BX - print four lines forming a box around
the mean with
width ±σ, and a division
record `>´
+F[R] - Full extent of column. With R
the columns with the
means and sigmas are moved
to columns 2 and 3,
without they remain in two
new columns.
+RMS
- (not with +BX) calculate RMS instead of mean.
S=#s - (appended to opt): for
scaling the input sigmas with s
!!! Do not interfer with selection of output columns
using -p and the like!
!!! Careful to use TSF EDIT in this context! TPS
occurs before TSF EDIT.
-TAPL
- special output format Air Pressure Loading, 3-column mc-file.
Example:
.................tslist
apls/OLKI_apl.ts -LRADI -LEAST -LNORTH -TAPL
-TEPH site - special output format EPHEDISP, 3-column mc-file.
-TC[D][{A|W}][:#i,#j]][a]
-
To
Cartesian.
Converts
from
polar
(amplitude and phase)
to X Y. Angle a is
subtracted from the angle in the file.
On output, the columns i j can be understood as
east and north
components.
D - angles in the file (and
a) are given in
degrees
[radian]
A - Azimuth is reckoned clockwise from north.
W
- windrose definition; transforms wind direction into
velocity components. Azimuth is reckoned clockwise from
north.
:#i,#j] -
amplitude from column i
and phase from column j
[1,2]
Example: Columns containing
listed with option -TCDA results in1 1.000 0.000
2 1.000 45.000
3 1.000 90.000
4 1.000 135.000
5 1.000 180.000
(command used: tslist windrose-tst.mc -LR -LA -Ni -n -F2f10.3 )1 0.000 1.000
2 0.707 0.707
3 1.000 0.000
4 0.707 -0.707
5 0.000 -1.000
-TROT[R]a[:ic1,ic2]
- rotate with angle a, column ic1 is x-, ic2 is
y-component [1,2]
R
- angle is in units of
radians
[degrees]
-TRF[{D|F}]lon,lat
-TRF[{D|F}]X,Y,Z
- special case: If you retrieve a 3-column mc X,Y,Z-file, the
output is
transformed to R,E,N at the indicated location
(lat,lon) or (X,Y,Z) (which is decided upon the number of
commas in the string
N.B. you
must retrieve the columns in the order X,Y,Z, e.g.
.................tslist
ib.ts
-LX -LY -L'Z],Rwd$' -F3d12.4 -TRF-71.,42.4 -TAPL
Instead of numbers,
inject the -TRFlon,lat using xyz2lola: e.g.
-TRF`xyz2lola
VLBI99-sites.xyz ONSA`
With -TRFD input 3 more columns with
uncertainties (sqrt of the
diagonal elements of the
variance-covariance matrix)
With -TRFF input 6 more
columns with the full
variance-covariance
information (in the order SIG11 22 33 12 13 23)
-TRF[{D|F}]Ilon,lat
-TRF[{D|F}]IX,Y,Z
- the inverse of the above. Try this:
tslist
WW/ONSALA60_apl.ts -LR -LE -LN -F3f10.5 \
-TRFI`~/gps/bin/ssc2lola en , ONSA`
-TRF[D][I]Tlon,lat - in both cases, the trailing T option initiates a test:
input = 3 rows, [1,0,0] [0,1,0] and [0,0,1]
instead
of real data. However, the mc-file must be read.
Absolute Gravimetry -
merged multi-campaign data: MJD in the first column
prepare:
rm -f <mc-file>
tslist <jd-file> -I
O:MJD <mc-file> #
Modified Julian dates
tslist <ts-file> -I O:<label>
<mc-file> # data
tslist
<dw-file> -I
O:<label> <mc-file>
# weights
The following options
may be useful in processing:
-mjd2dy[#c] - Convert MJD in column c to decimal year. Default c=1
-brk[:#c,]#w[{d|h|m|s}]
- Generate break indicators in printed output based on
abs diff between successive samples > w
If the MJD column is the indicated series, w can be
given
in units of days (d) ... seconds (s).
-TPAM[{W|I}[R]][#w]
- with application to Absolute Gravimetry with FG5's and
g-software
drop files: Automatic detection of sets and averaging.
Occurs after tsfedit.
Assumes MJD, data and weights in columns 1, 2 and 3,
respectively.
On output, the series are replaced as follows:
MJD with central time of set start and end,
data with (weighted) mean,
weights with (weighted) RMS.
w - The
inter-set gap width in seconds. Default is 60.
W|I
- Apply weights by multiplication (W) or division (I).
Default assumption is that weights haven't been input.
R - Undo weights (the data might already be weighted)
Alternatives: ASCII and Binary
Options -o -O imply suppressing
ASCII output to STDOUT (as if -I had been given).
If this is undesired, add option -p after the file
name argument.
-w[a] filename
- ASCII output to a file on unit 8.
The
print options above apply.
-wa for append
Binary output to a file on unit 31. The following
modifications are possible:
-OEA
- option to suppress writing of leading
missing-records, will adjust the epoch data
written in the file header.
-o[t][c] filename
- binary output file. Overwrite.
-ot implies trimming off leading MRS, does not
adjust the epoch unless -OEA has been
given.
If filename = % the input file name is
used (!)
The column to be output can
be specified by c.
Default c = 1
-O[T][c][:label[,comment]] filename
- binary output file. Append
a "column".
-OT implies trimming off leading MRS
Adding a comment to the label is
deprecated
Labelled output implies
append.
The label option is utilised for locating
file sections in multi-component files
(MC-files).
For multicolumn output in one sweep, this form
can be repeated, delimiter='|'
or + as follows:
Coding `=´ for label copies the
input label of column c
tslist
d/PMG090615-OPNEND-1h.ts -LP+I -LP+X -I -O1:=+2:= tmp/tmp.mc
-O[c]:label:
filename
- The colon will be replaced by the comment part
of the label that was read last from the input file
Labels are normally 16 characters long; words are separated by
`_´.
`_´ can be used
to fill the label; this is done automatically.
A two-word label can be generated using e.g.
`label a,b`
Returns a______________b
Characters `+´ and `|´ serve as delimiters; they
cannot be used in label text.
The use of
comments appended to labels is deprecated at the moment.
-ofilename (no blank) is obsolete and non-functional since March 2004.
filename given as `%´ indicates that output
is to the input file.
That may lead to adverse results.
Careful when adding columns to the same file, especially with
identical label!
Use tslql to check that labels are unique, else you must
repeat the -L option
to get to the desired label.
Other label utilities: ~/bin/olab
We distinguish between ASCII input and binary input.
A column in a binary MC-file is retrieved by means of a label.
First general input options, then binary input, then ASCII input in a separate chapter.
-ffilename - A way to specify a file name
late on the line.
-f filename will also work.
-sxn - shift the time series by n samples. Additive to ...
-sR[c]#v - adjust
origin time to the next integer v seconds (c =
s)
or minutes (c = m) or hours (c = h).
E.g. -sRs5.0 will adjust t0 to the next integer
5 second instance.
-dc[m] - duplicate
the last column m times [1]
-dc:[c1[,c2...]
- extend column range: copy these columns
-s[f][s]#d
- shift the time series (absolute or relative);
i.e. the origin time (t0)
is set to d ( -
t0 ) and the samples
up to this date are shifted out.
f = f - shift with respect to
time of first sample (subtract t0)
f = o
- shift time on output only. Affetcs ASCII and BIN output.
else: shift absolute with the indicated amount.
Positive values will shift the series to earlier times.
s = t
d is given as h[,m[,s[,ms]]]
s = d
d is given in days
s = h d is given in hours
s = m d is given in minutes
s = s
d is given in
seconds
else d is given in
amount of samples (integer)
The shift in units of time is cumulative.
Example:
-sfd5 -st3,10,10
shifts the time series ahead by
5 days
plus 3h 10m 10 s relative to t0
(one specification of the f-option is sufficient.)
This is particularly useful when series must be aligned
to the same t0
or the particluar series has that many
missing records at the beginning.
-0[s]#v
- trim start time of input to v days (s=d)
... v seconds (s=s) from epoch,
s may be s m h or d .
(only if v is later than the file's
first record - (restriction will be relieved)).
Works with filling multiple columns (as far
as the mentioned restriction allows).
N.B.: -B.c...
cuts on output, -0 on input.
tslist fileA
[-LlabelA1 -LlabelA2 ... ] [ -# fileB
... ] [ -#L fileC -LlabelC1 ...]
-@ is an alternative to -#
Coding labels and a few tools are explained here.
-# file
- open another input file, read into the next available column.
-@ file
Label parameters, if given, will pertain to the file specified
closest to the left (fileA
in the example above)
-keep-open - Keep
the (last) input file open for tsf_edit to be able to work with
it.
-#Mf,n,o file
- Do 'label math' with this file, see -ML below.
-@ is an alternative to -#
-#L file
-Llabel1 -Llabel2 ...
open another input MC-file and retrieve the columns according to
the specified labels. Labels? Read on below, option -l.
-@ is an alternative to -#
The maximum amount of columns
must not exceed 10;
an unlabeled file is counted as a labeled.
A range of single-column files can only be processed in a
meaningful way
if their starting times, lengths, and sampling intervals are
identical.
You might be able to trim them with option -0
or with -tie or -sc
-%
- open
binary file with CONVERT='BIG_ENDIAN'
Default is LITTLE_ENDIAN.
-llabel - Process MC-data
(multi-component (a.k.a. multi-column)
with
labeled sections).
label specifies a search string.
Specify -l' ' to obtain a list of the file's
labels.
Load more than one file section by specifying a series of
location strings in the order of the columns to appear on the
output.
In this case the output format must be extended. If
n columns are to be
retrieved and printed, -Fformatcode
where formatcode is
Fortran format code for n
columns, e.g.
nF10.4
To load all labels:
set labels = ( `tslql -L mc-file` )
tslist mc-file $labels ...
A leading, unique substring suffices. Thus,
-lR will find RADIAL. A construct stra|strb
alt. stra+strb
can be used to create short, unique location strings. Thus,
-l'N|S' will find the
segment labelled NORTH__SIGMA
(obs quotes
to avoid
shell's interpretation of |).
A trailing ]Rwd may be
useful if more than one segment is to
be printed; e.g. -lE -lR]Rwd
-F2F10.3 to print the segments
labeled EAST and RADIAL, when RADIAL is first in the file.
More
explanations here and in
~/sas/p/tss.f,
integer function loc_getts_mc
The second column may often be used for data uncertainty.
The scale parameter (-Sscale) applies to the
first,
second and following columns; all other editing etc. is
carried out only on the first column.
-LLABEL - like -llabel
; however, the file labels are converted to
upper case before they are compared with the search LABEL.
-L'*' -L'*],Rwd' - This sequence sends
a copy of a file into two columns. (Beware to code '*'
or
set noglob). This works even with unlabeled single-segment
files:
tslist o/g100202-OPNEND-1h.ph01.ts -L'*' -L'*],Rwd' -Un3
-LL/label - "Let", permit failure
to locate a label; the next MC-section of the file will be
loaded and the length and time information copied from the
previous column.
(Cannot be applied to the first label
search (yet). Use -L'*' instead)
There is an important convention with ASCII input..
Concerning duplicate records. By default we assume that records
that appear later on the input are more credible than earlier
ones.
If this is not the case, specify the O- option after the format
specifier under -g. See there.
-c#n
- The number of columns to be
input
[1]
See example.
-A[>target]] - ascii
data.
[binary]
Target string (cf /sas/readfmd.f parameter
trg).
[none]
-A'>BRX]'
Special file format: BRX - old tidal sub-day records.
-A'>SAC]'
Special file format: SAC - ASCII-converted seismic SAC
file
-A'>GCF]' Special
file format: GCF - ASCII-converted Guralp seismic
data
-A'>GWR]' Special file format
and simplified reading (no checks!) of
output from perl script ~/TD/xchan-unit
OBS! The
'>' is mandatory now, -A: is deprecated. -A (no target)
still o.k.
New option -Afile... see -E
Change made on 2013-04-22
-at[]o] - initial
move to a file location by search-target t
[none]
Reading will start after this line unless option includes
B
o
-
option
string
to
qloc_in_file.
Default
'AN'
means:
['AN']
target anywhere on the line, max. one rewind.
Specify 'ANB' if backspace is desired.
-AU[target]] - irregular
ascii data; employs the abscissa-and-ordinate input
routine Read_Fm_U followed by a call to a regularisation
algorithm.
Use the following options to specify modifiers for the
regularisation:
...............Gaussian
interpolation:
. -uwwidth
- width of sample search in units of dt
[1]
-ussigma - Gaussian bell
width in units of dt.
[1]
...............Spread
("extirpolation", Numerical Recipes, Press et al., Chap. 13.8)
-umnumber - number of samples to
be
affected
[5]
-um
selects the spread method; the Gaussian method is the default.
Use the -r option to define dt. Use the -0 option to
define t0.
The -U option can be used to specify the desired
length of the data
series.
[none]
-sD[date]
- "sine diem", the file has no year-month-day information.
The string date is to be added before the time column.
A dummy character field must be reserved in the -g'(format)',
and hour-minute-second-depth (-k option) must at least be 1:
e.g. -g'(a1,t1,f10.1,f10.4)'
-k1
[2000,1,1]
...............See
example
12
-sn
- skip the first n
records
[0]
-sln
- keep the last n
records
[0]
-s[f]s - no effect in
ascii-input. Use -B !
-ecode -
end-of-file target string.
-gu'>'[#v] - binary files: change
missing records to value v.
[0.0]
-g -gn
- in the following, the form -gn will
suppress warnings about sample
times
that were in doubt and/or values that have been
overwritten
-g[n]tsf[:c]
- A shorthand to input(!) column number c from one of
our most common
ascii formats, -gi4,2i3,1x,3i3,tj,e12.0
where j = {27,28,29} + 3n + 12(c-1)
[c =
1]
Note that -k#n must precede if n=3 isn't
suitable!
[n = 3]
-g[n]code['>'[#v]]
-g[n]'(code)[],opt[,opt...][:#c,][>[#v]]'
- format code and eventually options for input
['(i4,3i3,f10.0)']
Use single-quotes to prevent the shell to interpolate.
#c - Fill column
nr. c
[1]
Special conventions exist for BRX data
files. (obsolete)
For the normal case (reading with read_fm_d) the following
applies:
code
signifies
a fortran format expression. If a leading
left parenthesis is missing, code will be enclosed by a
pair of
parentheses automatically. With -c#n the format
must provide for
the ordinates to be input.
Additional options can be given after the closing parenthesis.
The format expression must comprise:
the date and time part and the sampled
parameter part.
The read_fm_d convention is that each record contains one of
each
part (date can be supplied once for all from the call
line, see
the -sd option, time can be skipped, -k0).
The date part can signify a string (subsequently
converted using
the -d option), a floating point number (Julian date,
decimal year) or three integer values; the latter is the
default.
The time part may be zero to four integer fields deep
depending
upon whether hour, minute, second and fractional
seconds are
to be read from the file (see -k# option; the
default assumption
is that the hour field is to be input.
opt
Some options concerning the reading and the interpretation of
the time record can be passed via code after the
closing ')',
cf proc_f_on_rec
/ urtapu13.f
Y2K: Your year values might have
two digits only. Then you can append
],Y2K
after the closing parenthesis. You can specify a turn-over year
],Y2K=y
Example: -g'(i2,2i3)],Y2K=10'
with which you can read dates from 1911 to 2010
JULIAN? read_fm_d analyses the
format string. If it finds an f format
descriptor before any i, it will assume Julian days.
Modifications
of Julian date can be indicated after the closing parenthesis.
Also, if the time record consists of decimal years, this has to
be declared after the closing parenthesis. Consult read_fuf.
,O - This option means
"overstoring = false", i.e. records of the same date
that occur later on the input stream do not replace records
already
input.
The default is "overstoring = true". To re-instate specify
,O+
N.B. In tslist floating-point time
records are always to be associated
with the Fortran format descriptor 'F'.
With input operations all
three decimal floating-point descriptors ( 'F' 'E' 'D' ) appear
to give identical results. I might miss some subtilities here.
See example 11 below.
Free-format input is possible. Format code='*'
Variants: Julian days or decimal years: '*f)' resp
'*f)Y=1992]'
...............alpha-months,
'*a[#n][,delim])'
where the width of the alphameric field can be given (#n)
and optionally a delimiter that separates the date part from
the time and sampled parameter parts. The search for the
delimiter starts in that case after the n'th column.
There will be examples one pretty day.
-dcode - date
format for converted string (months 3-letter code
converted to numeric form), after conversion.
Free-format input is allowed, code='*'
['(i4,i3,i2)']
-TZ#h - The ITZ
(time-zone) parameter. Floating-point values can
be
specified.
[0]
-TZs#s - Start
time offset in seconds, will be subtracted from what
read_fuf returns in t0. Will be added to -TZ#h
[0.0]
-TLS
- Exact leap-second
processing
[no]
-k#k
-
read_fm_d parameter k = `khms´ indicating
hour minute second frac.sec. record
depth
(1..4)
[1]
-y[#y]
- For input=decimal years, -g'(f...,)'
needed. Implies -k0 .
-k#k must be issued after -y if nonzero `khms´ is
desired.
y
(integer) sets the reference year for input and
print [2000]
-v
- Time duplicates are dropped. Default is that they replace the
sample previously read in.
-r[u]#dt - dt
- sampling interval).
u - units are days (-rd), hours (-r
or -rh),
seconds (-rs) or minutes
(-rm).
[0.0d0]
-0[x|r][u]#h - ( `zero´ !) Set start
time
h - (first sample time offset)
[0.0d0]
This is the lag of the first sample after the
epoch at midnight.
u - see explanations under -r
-0x[u]#h - Set file
start at exactly h hours (seconds) after epoch .
Else: round starting time to specified value, still detect
file start from first record date.
If the first file record hasn't the earliest date, a suitable
starting date must be defined using -By,m,d and
the -0x option
must be used.
If -0... is used, -r... must also be
given.
In conjuntion with binary input:
-0r[u]#h - round the file time
(after epoch) to h hours
(or seconds)
[1.0d0 (hour)]
-0[u]#h
- set the file time to this hour (or
second)
[0.0d0]
-z#v[,#t] - ASCII-input missing-record symbol. [-99999.0d0,1.0d0]
T_POINT DATA (a.k.a.
tp-format)
This data form is defined as a 2- (or more-) column structure with
one of the columns (usually the first)
running a floating-point MJD. The following options are available:
Extracting one sample:
tp-index-of-project -L urtapm-big-AUNO.log
$projects[21]
16051 FG5-220 AS N
tslist o/scg-cal-merged-AUNO.ra.mc -LM -LR -j16051
-Uo1 -Ft1,f12.6,1x,f10.2 -qqq
55304.9556715 90.14
tslist
o/scg-cal-merged-AUNO.ra.mc -LM -LR -j16051 -Uo1
-Ft1,f12.6,1x,f10.2 -qqq -Etp.tse,Y | fgrep -v '<'
2010.2948825
90.14
Also consider tplist
Extracting a lot by MJD
-RTP[v][:#c] file
- reads in free-format a two-column ASCII file with MJD and
data.
Such files can be made with tslist, option -TPW
v
- mark unset values as missing
#c -
fill into column c
[2]
A
t-point series must have been read first as a support array.
Places the data in column 2 by linear interpolation on the
support array.
Example evaluate-tp.tsj (from ~/TD/a/Allcamps )
This input for instance generates boxcars.55015.389120 18.87
55016.352199 18.87
>
55016.395822 21.65
55016.526609 21.65
>
tslist o/scg-cal-merged.jd.ts -RTP evaluate-tp.tsj -N -n1 -F't1,f13.6,f10.2,5h ]' \Note the format, overwriting the integer time column.
-I -O1:'MJD|2:DCLVL' o/scg-cal-merged.dc.mc
The -cv option can be used to limit the extent of printing (it does not affect>
55015.389005 0.00
55015.389120 18.87
55015.389236 18.87
55015.389352 18.87
55015.389468 18.87
55015.389583 18.87
55015.389699 18.87
55015.389815 18.87
binary output!) See Catching values.
See ~/TD/a/Allcamps/td.tse for instructions to convert MJD to decimal years
and other t_point operations.
Desirable extensions:
1) mask the tp-column so that only valid samples are shown.
Example for creating a file for -RTP input later:tslist d/g090615-OPNEND-1h.mc -TPWf16.7 -BHc2018,10,1 -U2018,10,2 -w file.tpf
STATION/INSTRUMENT/CHANNEL year month day seg x1 x2 x3 ... xq
where seg is a segment number indicating
the first or the second half of the day,
and x1 ... xq are 12 samples. Divisions
into segments of length 6 or 8 are also possible.
The reading routine that is used for this data type is
read_brx (readbrx.f)
The ambition with read_brx is that the
record part up to and including seg is
handeled
with hard-wired programming according to a fixed number of
options, and the parameter sample
format can be specified at run-time.
Options:
(for clarity the stipulated characters are
underligned. Also see Example (7) below.
Note: in the unix shell the )
character must be quoted.)
-ABRX - switch to this mode
-g=[numfmt]:type')'
numfmt is the format code applying to one parameter sample.
|
|
numfmt default | |
| JA |
Joint Archive for Sea Level | F5.0 |
| HY |
HYCON, 4 sub-daily records |
|
| default |
Bruxelles, incl. the so-called jump-cards |
|
GRAV_________VALThe string to label an output column can conveniently be produced with the label script, e.g.
tslist ... -O:`label GRAV,VAL` output.mcReading a column, a label can be coded explicitly
tslist gravity.mc -LGRAV_________VALor if the first (or only) column is uniquely identifyable with a short version of the string
tslist gravity.mc -L`label GRAV,VAL`
tslist gravity.mc -L'G|V'
tslist gravity.mc -LG+V
tslist gravity.mc -LGmight suffice.
tslist gravity.mc -L'*' -L'*' -L'*'For inspection what labels are there you can use tslql -h
tslist gravity.mc -LG+S -LG+V]Rwdif the second column in the file GRAV_________SIG is to be read into the first memory column for processing.
(1) You have formatted data, decimal years (*.ep):
tslist $DDIR/ONSA.gd.ep -g'(1x,f11.6,t13,f10.3)Y:1992' -k0 -C1 -Ff12.2 -012 -r24
the last two options (-0time
and -rrate) may be
necessary if some of the first three
samples are missing. And to set
the reference exact.
The -C1
option requests printing of calendar form with width +
1, i.e.
year month day hour
. This might be useful to check that the date conversion is
accurate enough.
(1') Making mc-files out of data like this:
rm ONSA_gd.mc
tslist
$DDIR/ONSA.gd.ep -g'(1x,f11.6,t13,f10.3)Y:1992' -k0 -I -012
-r24 \
-O:RAD_ONSA_____VAL ONSA_gd.mc
tslist
$DDIR/ONSA.gd.ep -g'(1x,f11.6,t33,f10.3)Y:1992' -k0 -I -012
-r24 \
-O:EAS_ONSA_____VAL ONSA_gd.mc
tslist
$DDIR/ONSA.gd.ep -g'(1x,f11.6,t53,f10.3)Y:1992' -k0 -I -012
-r24 \
-O:NOR_ONSA_____VAL ONSA_gd.mc
tslist
$DDIR/ONSA.gd.ep -g'(1x,f11.6,t23,f10.3)Y:1992' -k0 -I -012
-r24 \
-O:RAD_ONSA_____SIG ONSA_gd.mc
tslist
$DDIR/ONSA.gd.ep -g'(1x,f11.6,t43,f10.3)Y:1992' -k0 -I -012
-r24 \
-O:EAS_ONSA_____SIG ONSA_gd.mc
tslist
$DDIR/ONSA.gd.ep -g'(1x,f11.6,t63,f10.3)Y:1992' -k0 -I -012
-r24 \
-O:NOR_ONSA_____SIG ONSA_gd.mc
To retrieve one of
these segments,
tslist ONSA_gd.ts
-L'EAS|VAL' -Ff12.2 -C1
(1") Modified Julian dates:
50083.00 .00499
.00004 .00015
50083.25 .00451
.00009 -.00008
50083.50 .00513
.00021 -.00044
50083.75 .00466
-.00024 -.00066
50084.00 .00407
-.00052 -.00045
Reduced
Julian Date: specify ,RJD:
(2)
You have formatted data
(2')
Slightly more complicated:
You have formatted data, yet (nearly-)free format input of
records like
1997 sep 17 14 00 00 1.23456
tslist data.tsf -A -k3 -g'*a11)' -J -Etest.tse,TEST]
while test.tse may look like
TSF EDIT TEST
DELETE At 1997 09 17 15 0 0 0
END
(3)
You have not really a time-series, rather a set of
positional data (a vector); records like
1
6.4843E-01 .0000E+00
2
8.2342E-01 .0000E+00
3
7.4881E-01 .0000E+00
tslist UMEA-res.dat -g'(f6.0,6x,f10.0)' -r24 -J -k0 -00. -TZ-12
(4) You
have an undated time series; time record is in decimal hours
from a certain epoch.
22.0000
.5395 10.9354
-3.5522
23.0000
-1.2805
6.8930 -1.8882
24.0000
-2.7594
1.3339 .2259
25.0000
-3.5621
-4.3536 2.2552
26.0000
-3.5251
-8.7694 3.6889
Let the epoch be 1998-02-02 as an example.
awk '{print "1998 02 02",$0}' gel-stra.dat | tslist - -g'(i4,2i3,2x,2f12.0)DH+]' -C
gives:
1998 02 02 22
50846.916667 .53950
1998 02 02 23
50846.958333 -1.28050
1998 02 03 00
50847.000000 -2.75940
1998 02 03 01
50847.041667 -3.56210
1998 02 03 02
50847.083333 -3.52510
Next on the wish-list: linear transformation of a general floating record to get floating point hours
(4') You know the year and you have Day-of-Year records (e.g. DOY=154)
154073003 1015.3
154073008 1015.4
154080003 1015.7
154080008 1015.6
... (file name test.tsf)
We'll utilize that read_fm_d switches to DOY-mode if the month is specified as zero.
awk '{print
"1993 00",$0}' test.tsf |\
tslist -
-A -g'(i4,i3,i4,2i2,2x,f10.1)' -k2 -00. -r1. -Ff10.1 -C1
(5) One-minute
data without time tags, to be decimated to one-hour data.
Missing samples are symbolized
by 999999. The interesting data is in
column 2.
(/geo/hgs/SIVS/data - Iceland vol-strain gauge)
awk
'{r=NR-int(NR/60)*60; \
printf
"1997 10 01 %3d %3d %6.1f\n ",\
int(NR/60),r,
$2}' 1997-10-min |\
tslist -
-g'(i5,2i3,2i4,f10.0)' -k2 -oska-1h.ts -I \
-z999999.0
-rs60. -Em2h.tse,M]
where the file m2h.tse contains
TSF EDIT M
DECI 60 S=121
END
(6) GPS PPP time series
The problem is here that there
are some records at precise hour whereas the ones we'd use are
issued at 5 minutes after the
hour. We first need a suitable start time.
cal2sec 1997 1
1 0 0 0
gives -94651200.0
There were a couple of bad records. This one removes all records starting with `0.0u':
awk -v
s=-94651200.0 \
'\!/^0.0u.*/{if (NR>2)
{printf "1997 01 01 %15.6f %s\n",($1-s)/3600,$0}}' \
wtide.WES2_15 >! wtide.wes2
After this transfer, a binary MC file can be made:
rm -f
Westford.mc
tslist
wtide.wes2 -g'(i4,2i3,f16.6,t74,f13.0)DH+]' \
-E,WES2_R] -r1. -k2 -I -O:RADIAL_______VAL Westford.mc
tslist
wtide.wes2 -g'(i4,2i3,f16.6,t90,f10.0)DH+]' \
-r1. -k2 -I -O:RADIAL_______SIG Westford.mc
with editing file tslist.tse
TSF EDIT WES2_R
REMDC
DEL > 0.1 ALL
END
TSF EDIT WES2_N
REMDC
SCALE 111.0e3
DEL > 0.1 ALL
END
TSF EDIT WES2_E
REMDC
SCALE 150978.
DEL > 0.1 ALL
END
(7) Joint Archive for Sea Level (JASL) time series
tslist h275a95.dat -ABRX -g=':JA)' -FF10.3 -S0.001
notice carefully the
use of the BRX read routine (~hgs/sas/p/readbrx.f) and how the
format group
JA (for Joint Archive)
is entered. Result will be in metres.
(8) NOAA/NOS tide gauge time series:
tslist
boston-tgg-obs.dat -A'<B>Station]' \
-s1
-g'(13x,i4,4(1x,i2),f8.0)' \
-k2 -C2 -I -oboston-tgg-obs.ts
(9)
Use with GMT:
Try first
tslist
$TAPDIR/ONSA.ra.wr.ts -t2,5/4,8/f:12,5
-R1992./2000./-8.000/4.000
-Ba2.000f.1667/a2.000f.4000
Then apply in
psxy:
tslist
$TAPDIR/ONSA.ra.wr.ts -qq |\
psxy
`tslist $TAPDIR/ONSA.ra.wr.ts -t2,5/4,8/f:12,5`\
-JX6/3 -M -W2/0 -K -O >> $psout
To insert axis text, do
.........bobt=`tslist
$TAPDIR/ONSA.ra.wr.ts -tt2,5/4,8/f:12,5`
Example:
.........bobt=`echo
${bobt}":Y-axis label:"WeSn | sed 's|/|:X-axis label:/|'`
...........tslist $TAPDIR/ONSA.ra.wr.ts -qq |\
.............psxy `tslist
$TAPDIR/ONSA.ra.wr.ts -tr2,5/4,8/f:12,5`
\
...............$bobt -JX6/3 -M
-W2/0 -K -O >> $psout
or, which is equivalent,
.........tslist
$TAPDIR/ONSA.ra.wr.ts -qq |\
...........psxy `tslist
$TAPDIR/ONSA.ra.wr.ts -tt2,5/4,8/f:12,5 |\
.............sed 's|/|:Y-axis
label:/|; s|$|:X-axis label:WsEn|'` \
............. .-JX6/3 -M -W2/0
-K -O >> $psout
(10) A typical air pressure effect - time series reading:
fgrep TOKYO /home/hgs/Oload/smhi/keystone-apl/local.pressure
|\
tslist - -A -g'(t6,a9,1x,i2,t31,f10.0)' -d'(i4,i3,i2)' -k1
-Ff12.4
(11) A
typical Lubo-file from matlab, time record specifies days
after some given date
Example
is
for days after 1993 01 01
2.3100000e+02 1.3349500e-01
1.3349500e-02
2.3150000e+02 1.1365900e-01
1.1365900e-02
JDC
-d -m 1993 01 01
48998.0
tslist SUND_rs.dat -A -g'(2f16.0)J+48998' \
-r24.
-k0 -mL -o SUND_rs_24.ts
Second
example prepares a data set with time record in Lubo's
decimal years (365.0
days) in the same format as SUND_wet.dat above:
awk
'{printf "%16.7e%16.7e%16.7e\n",($1-1993)*365,$2,$3}' \
/wvr2/SanFra/Matlab/SUND_wet.dat > !
/Bifrost/hgs/wet/SUND_wet.dat
(12) Year-month-day
part is missing
Assume time is meant to mean 6
minutes intervals
awk 'BEGIN{for(i=0;i<11;i++){printf "%10.1f %10.4f\n",
i,1+i/10}}' |\
tslist - -A -g'(a1,t1,2f10.0)DH+0,10.,SD=1999-1-1]' -k1 -C2
or
... |\
tslist - -A -g'(a1,t1,2f10.0)DH+0,10.' -C2 -sD1999-1-1
result of awk:
0.0 1.0000
1.0 1.1000
2.0 1.2000
3.0 1.3000
4.0 1.4000
5.0 1.5000
6.0 1.6000
7.0 1.7000
8.0 1.8000
9.0 1.9000
10.0 2.0000
Output data:
1999 01 01 00 00
51179.000000 1.00000
1999 01 01 00 06
51179.004167 1.10000
1999 01 01 00 12
51179.008333 1.20000
1999 01 01 00 18
51179.012500 1.30000
1999 01 01 00 24
51179.016667 1.40000
1999 01 01 00 30
51179.020833 1.50000
1999 01 01 00 36
51179.025000 1.60000
1999 01 01 00 42
51179.029167 1.70000
1999 01 01 00 48
51179.033333 1.80000
1999 01 01 00 54
51179.037500 1.90000
1999 01 01 01 00
51179.041667 2.00000
Explanation:
'DH+0,10.0'
added to format means: decimal hour, offset 0, scale 1/10 =
0.1 h = 6 min
A dummy
date string is read from the file and replaced by the base
date. Therefore
-k1
-g'(a1,t1,...)' Observe that -k1 is the default.
-sd1991-1-1
introduces the base date.
... | tslist - -A -g'(a1,t1,2f10.0)' -sd -C2
takes
elegantly care of that file, using the defaults: date 2000,1,1
and
time
increment one hour.
(13) Irregularly spaced data:
cd
~/g/Bohotide/Data/
tslist
$fn -AUcm] -g'(6i2,f10.0)' -k3 -Ff10.3 -TZ1 -C3 -01. -r2.
-uw2. -I -o ../$sect.ts
Input
data:
setenv
fn
KI0023B_1.mio
setenv
sect KI0023B_1
m
~/g/Bohotide/Data/KI0023B_1.mio
l 16
;REPORT KI0023B_1 generated
01/01/24 15:13:55
...
cu MD31 kPa
cm time 980212000000
010101000000
980216100833 4270.541
980216110705 4267.679
980216120000 4267.472
980216123328 4267.472
980216130631 4267.472
980216140000 4267.27
980216160000 4266.861
980216180000 4267.472
...
(14) Moving-average smoothing
setenv MA_LENGTH 11
# for example
tslist file.ts
-E'~/gps/movave.tse,MA]'
tslist SESHAN25_apl.ts -MB
-C1
...
1997 09 01
00
50692.000000 BEGIN FILE
2000 10 01
00
51818.000000 BEGIN BREAK
2001 01 01
00
51910.000000
2001 12 31
18
52274.750000 END FILE
(16)
Noises
tslist +1440,-13,0.9,G -rs60 -00
-B1998,12,6
generates a Gauss-Markov time series, one sample for every
minute
on Santa Claus day 1998, with seed 13. Markov parameter 0.9
meaning
x(t)=x(t-1)*m + Gauss
tslist +1440,-13,0.9,G -rs60 -00
-B1998,12,6 -YRND-21,0.1,G
adds Gaussian white noise with amplitude 0.1 to the first series
tslist +1440,-13,0.9,G -rs60 -00 -B1998,12,6 -YGMN-21,0.4,0.1
adds Gauss-Markov noise with Markov parameter 0.4 to the first
series.
tslist +1440,-13,1.5,F -rs60 -00
-B1998,12,6
generates a fractal noise time series with an ~ f -1.5
power spectrum.
(17) Label gymnastics
Make a string consisting of the labels of an mc-file to re-use for output to a second mc-file
set olabs=`tslist o/MC090622-30s.mc -L | awk
'/LocGMc/{i++;printf "%s|",i":"substr($3,7)}'`
Make an array of retrieving labels to get all available columns into memory
set labels= ( `tslist o/MC090622-30s.mc -L | awk '/LocGMc/{print
"-L"substr($3,7)}'` )
Then, e.g. the following tslist commands will synchronize all
columns with respect to valid data,
contract the gaps, and produce a file (the time information in
which will be deliberately wrong)
tslist o/MC090622-30s.mc $labels -F1p,5d12.4 -MSYNCON -C3 -I
-o$olabs tst.mc
rm
tmp.mc
tslist file ... -O:`label GRAV,VAL` tmp.mc
tslist file ... -O:`label BARO,VAL` tmp.mc
tslist
tmp.mc
-TGGS -L'G|V' -L'B|V' -qqq -wa GGS/OS100300.GGS
77777777
0.000000
20100301 000000 -0.2060873 982.4010
20100301 000001 -0.2054710 982.3980
20100301 000002 -0.2060370 982.4010
20100301 000003 -0.2082918 982.4020
20100301 000004 -0.2102593 982.3980
20100301 000005 -0.2095818 982.4010
...
tslist OS100300.GGS
-g'(i4,2i2,i3,2i2,f12.0),WOX:1,777777]' -k3 -A77777777]
-e99999999 -I -rs1.
The
format for accessing the barometer column: Replace f12.0
with 12x,f11.0
WOX:1,77777777]
is used to skip all records that have a gap signal
in column 1 -
- tslist manages gaps anyway.
-A77777777] is
used to poition the file, which occurs in MAIN before
READ_FM_D is called.
-e99999999 defines
the EOF-mark. Without this definition the procedure
decodes the mark as a date.
It's stubborn and stupid, I know.
The options
suitable for the files we send to Potsdam are lengthy. Do it like
this:
set ggp = ( -a77777777
-g'(i4,2i2,i3,i2,t16,f10.0),EOF:99999999],WO:88888888,WO:77777777'
-k2 -rs60 )
tslist ~/www/4GGP/GGP/OS130900.GGP $ggp -I -o tmp/ggp.ts
(21) A sequence of periodograms
foreach i ( `fromto -f %2.2i 0 23` )
tslist d/G1_gar_100628-1s.mc -qqq -L'G|R' -N1p,e12.4 \
-nFrq -sfh${i}.0 -F1p,e12.4 -Epdgram.tse,1H \
-w o/G1_gar_100628-1s.${i}.pdg
end
with pdgram.tse
TSF EDIT 1H
FILTER 0 1 ' '
1.0 -1.0
REMDC
PERIODOGRAM U,/N,dB To #3599
END
In the same file there is a HANN-window
version of the above. Notice that the tse-file is simple
and invariable.
We could subtract the barometric effects with
-w without the -p option always outputs all columns - it's the first column that is pressure-reduced.tslist d/G1_gar_100628-1s.mc -qqq -L'G|R' -ML3.156,1,a:'B|V],Rwd' \
-N1p,e12.4 -nf2.777777777777778E-004 -sfh${i}.0 \
-F1p,2e12.4 -E1,2:pdgram.tse,1H -w o/G1_gar_100628-1s.${i}.pdg
Use the internally generated frequency
interval. For verification, find out the frequency
interval with
tslist
d/G1_gar_100628-1s.mc -qqq -Epdgram.tse,1H -N1p,e12.4
-ni-1 -F1p,e12.4 -I |\
awk '/Spectrum\:/&&/FNy/{print
$9}'
2.777777777777778E-004
The
time field option can also be coded as -n/3600 , something we could have
guessed.
To make a nice plot,
consider
but edit it to fit your requirements. An appropriate xyz2grd command is printed as the bottom line of rastersp.rastersp -x0,0.5,100 -y-100,100,100 -w o/G1_gar_100628-1s.ras.pdg \
cd plot
-R o/G1_gar_100628-1s.??.pdg
source rastered-pdg.env
(22)
ASCII file with a number
of columns, input in one job.
The example is for a file
with three data columns.
tslist data.tsf -gi4,3i3,f10.0 -k1 -r1 -p3 -E2:input.tse,I2 -E3:input.tse,I3(annoying feature: we have to anticipate the 1-h sampling time: -r1 )
TSF EDIT I2(23) Creating a tse-file for simultaneous sampling
OPEN 41 R data.tsf
41,'TOP',-99999.d0,'(i4,3i3,10x,f10.0)',1,0,'N'
CLOSE 41
END
TSF EDIT I3
OPEN 41 R data.tsf
41,'TOP',-99999.d0,'(i4,3i3,20x,f10.0)',1,0,'N'
CLOSE 41
END
echo "OPEN 31 B sampled.dat" >! rms-smp.tsi
echo "SAMPLE U=31" >> rms-smp.tsi
tslist rms.tmp -qqq -C4 -wa:'SAMPLE At]' rms-smp.tsi
wraptse -o rms-smp.tse SAMPLE rms-smp.tsi
tslist seismo.ts -I -E rms.tse,RMS -E rms-smp.tse,SAMPLE -o seismo-rms.ts
...
and the output is found in ascii file sampled.dat .
In the example the instructions in rms.tse prepare
a moving-RMS series.
File rms.tmp contains moving-RMS values from
another source. Both sets are to be compared at
simultaneous times.
(24) Updating
the mareograph series for tide-press-plot
setenv BUBB_RJCLEVEL 200.
setenv APPENDFILE
/home/hgs/wx/MAREO/OSBBLM2_2015-1h.ts
setenv APPENDAMP 100.d0
tslist
/home/hgs/wx/BUBBLER/OSBBLM-2013-OPNEND-1h.ts -qq -S100 -Bc$bd
-I \
-E /home/hgs/wx/BUBBLER/repair.tse,BUBB \
-E /home/hgs/TD/append.tse,A \
-E /home/hgs/wx/BUBBLER/gapfill.tse,RCM \
-o /home/hgs/TD/SCG_MONITOR/tgg-oso.ts
Note that the
level units are centimeters.
(25) Values and uncertainties input in one
stroke:
tslist o/g.atm.b4.tsf -c2 -g'i4,2(1x,i2),2f11.0' -k0 -r24. -I \
-O1:`label B4,VAL`+2:`label B4,SIG` o/g.atm.b4.mc
where o/g.atm.b4.tsf is
2018-02-15 5.6900 0.8800
2018-02-16 5.8100 0.8700
2018-02-23 5.7600 0.8800
