gcf2ts
gcf2ts-hourly
gcf2ts-likeg
cal-zacc10Hz-SCG.html

USAGE

  (1)       gcf2ts [options] -d duration yyyy mm dd hh mm ss

  (2)       gcf2ts [options] files.gcf ...
 
  (3)       gcf2ts [-U] files.tar.bz2 ...                     

 PURPOSE:

    Converts gcf files to BIN ts-files, one to three components, via sac-format.  It should be called from the
    directory level above the gcf-collections ( ~/Seismo/gcf above 3u93z2 etc. )
    Supposed to work over day boundaries as we use gcfconv.log to find out which sac-files have been generated.
    The main feature of gcf2ts is the application of  a seismometer transfer model in the spectrum domain.
    Currently however, only the Guralp CMG3T is coded up. This task is carried out by tslist/tsfedit with
    commands in ~/Seismo/gcf/guralp.tse
   
    This script can decimate the output to 10 or 1 Sps ("Hz") using predefined schemes under the options
    -10Hz or -1Hz, respectively
    Additional tsf-edit operations can optionally be invoked. If you have your own ideas for decimation, use
    -100Hz

    One ts- (alternately mc-) file is produced for each call. The parameter of the -l option controls the length
    of the series; a generous overlap should be considered if e.g. hourly files are to be concatenated.

    A super-script exists that runs the routine case of a sequence of hourly batches,
    gcf2ts-hourly
    and one to simulate the GGP filter so that the result is comparable to the SG's 1-s gravity series,
    gcf2ts-likeg -h, e.g.
    gcf2ts-likeg -n 24 G2016.032/GCF.2016:032:00:00:00.3U93Z2-ARECT-10Hz.ts
    See also HOW.TO-make-gcf2ts-like-SG.html
 
    Method (1) is by date and duration.
 
    Method (2) by specific gcf-files; the signal from these files is concatenated to produce one mc-file.
               Careful with file size: Either the size of the output becomes large, or the filtering operation
               (in paprticular to get 1Hz) will take long time.
 
    Method (3) is not fool-proof, since it's not clear at this moment what kinds of files are tarred.
               With -U , only  bunzip2 and tar -xfv  will be carried out, so next time you can
               use the other methods to process the files.   
 
    The gcfconv program creates the sac-files in subdirectories    G#yyyy.#doy/   (year, day of year).
    An additional layer of subdirectories can be specified. The ts-files are store in the same directory as the sac-files.
 
    Levels are adjusted using hard-wired 
      set y = ( 10500 17000 83300 )
    Remove DC-levels after concatenation of resulting time series!

    In order to output a series that covers one hour without end effects, two or more 100 Sps files are appended
    before decimation. The appending operation is again critical to end effects, so by default cross-fading is
    invoked internally:
     setenv TSAPP_ROPT XF=length/2,length
    If you want to change this, the -XF option must be given after the -l option.
    Using  gcf2ts-hourly :

     scpu -z 'yyyymmdd*.'
     gcf2ts -U files ...
     gcf2ts-hourly -calibrated 3u93z2/yyyymmdd_hhmmz.gcf 24

     for one day of 10 Sps Z-acceleration data; implies the default option set to gcf2ts
        -E guralp.tse,CMGT3AR -10Hz -c z -V AUTO -l 7200 

     If there's a problem already in the gcf-data, the strict length imposed by gcf2ts-hourly will cause additional breaks.
     In that case the gcf2ts process for the incriminated hour must be run manually. gcf2ts is so kind as to produce a file
     with the required setenv-commands. Source it
     source gcf2ts-hourly.env
     and run gcf2ts according to
     fgrep 'EXEC: gcf2ts' gcf2ts-hourly.log
     however with the -n option to extend the length of the output series,

    e.g. for a 10Hz series that had a 33-s long gap starting at 22:00:00 :
setenv ONASCALEZ 1.0
setenv GCF_HOURLY 1
setenv ONASCALEZ 27.504
setenv GURALP_BL +BL
setenv LPUF 0.075
setenv LPLEN 96
     due to    source gcf2ts-hourly.env 
     and then
     gcf2ts -E guralp.tse,CMGT3AR -10Hz -c z -V AUTO -l 7200 -n 42000 \
            3u93z2/20180626_2100z.gcf 3u93z2/20180626_2200z.gcf |\
        tee gcf2ts-man.log

 OPTIONS:

    -files       - show the actual sac files and stop

    -x[c][p] sac-file
                 - add an additional sac-file (option can be repeated)
                   c specifies component e n z
                   p = b or e for placing it first resp. last on the list.
                   Useful for preventing a mid-nightly gap. 

    -test        - Will go through one component's file set and stop.
                   The tmp??.ts files will not be removed.
                  
    -check       - like -test, but does not execute the tslist-app stage.

    -keep        - extend the collecting mc-file, i.e. don't remove before writing.
  
    -log logfile - Collects info about the binary files that have been generated.
                   Default: gcf2ts-files.lst

    -lst         - Writes the names of the files produced into ./gcf2ts.lst

    -c [e][n][z] - Process these components. Default = enz 
 
    -d duration  - duration, specified as hh:mm
                   Instead of file names, specify start time(at least to the hour).
                   Default is: file names specified on the command line.
 
    -l length    - length in seconds for resulting .sac files. Default: 600

   -XF early,xfl - Cross-fading parameters. Default is early,xfl = 5×length,10×length
                   where xfl is the length of cross-fading and early shifts the
                   fading interval back in time (integers signifying indexes):
                   -XF 0,0           switches cross-fading effectively off
                   -XF 0,1000        yields appending with over-writing.

    -n nout      - number of samples in the resulting .ts files
 
    -R           - use REPAIR before decimation.
 
     -1Hz [DIFF]
    -10Hz [DIFF] - Decimate to 1 or 10 or 100 samples per second, optionally with a DIFF filter.
                   The tsf-edit commands are found at the end of in this very script.
                   Default = 100 Hz i.e. as is.
                   This option should be given early on the line, but see -E
                   Optionally calibrate with environment parameter GCF_CALFACTOR
 
   -100Hz [DIFF] - Apply the transfer function.
                   Optionally calibrate with environment parameter GCF_CALFACTOR.

    -cal factor  - Internally, setenv GCF_CALFACTOR factor. A reasonable value is 30.86
                   OBS! With -Eguralp.tse,CMGT3VR or -Eguralp-10Hz.tse,CMGT3VR
                          or -Eguralp.tse,CMGT3AR or -Eguralp-10Hz.tse,CMGT3AR
                   A pre-scaling 0.01 and ONASCALEZ are applied (default = 1.0) 

    -I icat      - input directories are named icat${c}2 (c = e n z ). Default is 3u93
 
    -O odir      - extra output subdirectory layer(s). Default = ./  Will be created if necessary.
 
    -E tse,trg   - (cumulative) tsf-edit file (tse) and target (trg).
                   For rectification of the seismometer's transfer function, specify
                   `-E guralp.tse,<TARGET>´ 
                   choosing the target that applies, e.g. CMGT3VR for velocity
                   or CMGT3AR for acceleration.
                   It matters what option, -nHz or -E, is specified first. If decimation
                   takes place before rectification, the operation might lead to long
                   lead times of the first valid sample in the result. 
                   If you are converting to 10 Sps or 1 Sps and specify -10Hz before -E,
                   use the commands from guralp-10Hz.tse .

                   Default is `RAW', no transfer function is applied.
                   `RAW' is a placeholder. It will not show up in the
                   arguments to tslist. If other tsf-edit files are
                   to be invoked, `-E RAW' must expressly be given. 
 
    -M[A[B bdir]] - produce labelled mc-file. Default: three ts-files. With A , append new columns.
                   With AB , backup the existing file to directory bdir
      
 
    -V label     - If you change the spectral characteristic of the signal, indicate it in the
                   label-part-2 of the MC-file, e.g. ARECT for "Acceleration rectified"
                   Default: VRAW (velocity raw)
                   (If three ts-files are produced, the file name will carry the label in front of
                   the sampling frq. unless it's the default label)
    -V AUTO      - in combination with -E tse-file,trg :
                   If the tse-file contains a record ;:trg: set typlbl = SPEC
                   the label will be set to `label $C,SPEC` where $C is upper-case of component. 
                   (at present only one command per `;:trg:´ can be specified).  

 TEMP FILES:

    tmp.asc        - sac2asc output
    gcf-files.lst  - the list of gcf files (with -d)
    gcfconv.log    - STDOUT from gcfconv, somewhat reduced.

 REQUIRED PROGRAMS AND SCRIPTS:

    ~/Seism/bin/gcfconv
    ~/Seism/bin/sac2asc
    ~/bin/tslist  (~/sas/p/mt/tslist.f http:.../hgs/hgs.man/tslist.html)
    ~/bin/tslist-app
    ~/bin/jdc     (~/sas/p/mt/jdc.f    http:.../hgs/hgs.man/jdc.html)
    ~/bin/beak

 OUTPUT:

    Binary. The names of the files generated are written to a summary file:
       $logf, default gcf2ts-files.lst  set with option -log
       You must maintain/backup this file yourself!

    Added to filename:
              either ${c}2-${samplingrate}.ts  (c = E N Z)
        or  ENZ2-${samplingrate}.mc  (with -M)
       labelled ${c}___________VRAW  (c = E N Z)
 

 EXAMPLES:

   scpu 20150207. ; gcf2ts -U 3u93?2/20150201.?.tar.bz2

copies and untars the files of the day


   gcf2ts -xze G2015.037/GCF.2015:037:00:00:00.3U93Z2.sac -E guralp.tse,CMGT3AR -10Hz -c z -l 7200 3u93z2/2015020'5_*.gcf'

with decimation after transformation to acceleration and adding the first file of the following day.
 

   gcf2ts -E guralp.tse,CMGT3AR -10Hz -c z -l 7200 3u93z2/20150219_*.gcf

very often used version.

   foreach f ( 3u93z2/20120904_*.gcf 3u93z2/20120905_*.gcf )
      gcf2ts -E guralp.tse,CMGT3AR -10Hz -c z -l 7200 -n 36000 -V ZACC $f
   end
        creates 48 ts-files 36000 samples long with 10 Smp/s rectified Z acceleration. Last file:
      G2012.249/GCF.2012:249:23:00:00.3531Z2-ZACC-10Hz.ts.


   gcf2ts -1Hz -M -l 7200 -d 2:00 2014 04 12 20
 
        creates a 2-hours long mc-file, 1-Hz rate. tslq says:
    <Main-->>> File begin: 2014 04 12  12 00 05 MDJ=56759
    <Main-->>> File end:   2014 04 12  14 00 55 MDJ=56759
        sac-files are also 2 hours long (100Hz !).

 
    gcf2ts -10Hz -M -l 7200 -d 01:10 2014 06 10 02
       
        Creates a 1-hour 10-Hz mc-file with a neat overlap for appending the next hour.
                  To run it in a loop:
 
    foreach d ( `jdc -Ah1. -, -m -k3 -L0,120 -C 2014 06 10` )
      gcf2ts -10Hz -M -l 7200 -d 01:10 `echo $d | sed 's/,/ /g'`
    end
        NB: The -m option is important, else you get a broken pipe.
 

    foreach d ( `jdc -Ah1. -, -m -k3 -L0,264 -DD 2014 146` )
       gcf2ts -E guralp.tse,CMGT3AR -10Hz -M -V ARECT -l 7200 -d 01:10 `echo $d | sed 's/,/ /g'`
    end
 
       for spectrally rectifying the seismometer's transfer function and computing acceleration.

gcf2ts-hourly

 USAGE:
        gcf2ts-hourly [options] [+ gcf2ts-options ++] <file> #n
 
 PURPOSE:
        Calls gcf2ts repeatedly for n batches of one hour length.
        Default n = 1.
        Creates Z-velocities or accelerations with rate 10 Sps.
        Must be issued from the directory one level above the one
        that contains the gcf-files.
 
        Default MARK = AUTO, see gcf2ts
 
        <file> is the first gcf-file of the series,
        e.g. 3u93z2/20210511_1100z.gcf
         The system issues gcf2ts commands for all files
        in the time window, but will not execute the command
        if the first file of each hourly section does not exist.
 
        The default set of options to gcf2ts is
 
          -E guralp.tse,CMGT3AR -10Hz -c z -mark AUTO -l 7200
 
        If changes are necessary, the whole set must
        be specified between a leading lone `+´ and `++´
        See here
  
 OPTIONS:
 
  -GO             - skip the prompts. However, if LPUF, LPLEN and GURALP_BL have not
                    been set in the environment, prompting will resume.

  -T              - test. Echo critical commands, don't exec.

  -show           - Run gcf2ts with the -show option
                    (will show output file names and tse-options;
                    no exec).

  -show-log       - Interactively: show the log of gcf2ts and pause.

  -us             - save ZSPF-pppp.mc files in every stride of the loop
                    by renaming them to ZSPF-$n-pppp.mc
                    OBS! Will remove previously existing files ZSPF-????.mc
  -usa <ADDM>     - add this mark after the auto-number $n
  -usd <subdir>   - move the mc-files to this sub-directory

  -mark <MARK>    - default is AUTO for resulting file name
                    marks, which will result in a mark -ZACC
                    to which `-10Hz.ts´ will be added.
                    `-<MARK>-10Hz.ts´ replaces `.sac´

  -calibrated     - scale output with a hard-wired factor
                    (30.86  compatible with BSFFC 0.0174533)
                    (27.504 compatible with BSFFC 0.0177) * current default!
                    or specify your own scaling factor:
  -cal #factor   

  -no-bandlim     - don't apply the band-limiting filter
 
  -sps #n         - samples per second in output (default=10)
 
  -tse-target <t> - the target in guralp.tse (default=CMGT3AR)
 
 ENVIRONMENT:
 
   LPUF   corner frequency in units of the Nyquist 50 Hz.
   LPLEN  filter length
                 - low-pass filter parameters (guralp.tse,CMGT3AR)
                   In the case of 10Hz (example 2 below) , if these
                   parameters have not been set, LPUF 0.075 and
                   LPLEN 96 will be set internally; the program will
                   pause for confirmation. The mechanism may not work
                   if other schemes are invoked (depends on the
                   options -E or -tse-target)
   GURALP_BL +BL or empty (band-limiting or not)

   ONASCALEZ amplitude factor    
 
 EXAMPLES:
 
  (1)   gcf2ts-hourly 3u93z2/20110611_1200z.gcf 5
 
  (2)   gcf2ts-hourly -calibrated \
                    + -E guralp.tse,CMGT3AR -10Hz -c z -mark AUTO -l 7200 \
                   ++ 3u93z2/20140601_0000z.gcf 24
 
gcf2ts-likeg
 
 USAGE:
          gcf2ts-likeg [-1Hz|-10Hz] [-F] [-U] [-n hours] [-l length] \
                       <GCF-dir>/<10-Hz-{mc|ts}-file-name-root>
 
 PURPOSE:
          WARNING! The resulting signals are spiky at 12 min intervals = 7200 samples at 10-Hz;
                   improvement is badly needed.
 
          Creates 1 or 10 Sps Z-acceleration filtered like the SCG
          Works also with unlabeled BIN-files
 
          The <10-Hz-{mc|ts}-file-name-root> argument will be completed
          according to the options. The name root is
          GCF.<yyyy>:<doy>:<hh>:<mm>:<ss> e.g.  GCF.2022:045:06:00:00
          A complete file name can also be specified - if you can anticipate it.
 
 OPTIONS:
 
  -10Hz           - create 10Hz-files, don't collect
                    Default is simple resampling 10 -> 1 Sps (resmp.tse,S10)
 
  -1Hz            - create 1Hz-files from 10Hz using a low-pass filter
                    (resmp.tse,10).
                    OBS: check whether filter-start cut-away is o.k.!
                    There is a command  `setenv CUTBEG #length'  in this script.
                    Default is simple resampling (resmp.tse,S10)

  -U | -E         - renew files, the gcf2ts-files .sac and .mc
  -F              - ... fetch .gcf files from the archive first
                    -U from Utter (default) -E from Elder
  -R                renew the *.ts-files only. Don't specify -U -E nor -F then.
 
  -n nh           - process the first file and up to nh gcf-files
 
  -o <filename>   - the 1Hz collected TS output file
                    Default = tmp/likeg-1Hz.ts
 

  -l length       - Output number of samples.
                    Option must be given after -1Hz or -10Hz
                    Default: 3630 (1 Sps) 36300 (10 Sps)
 
  -c [e][n][z]    - Components. Default is z
 
 FILES:
 
  files.out will contain the names of the output BIN-files
            These files can be collected and decimated ...
 
  logs/gcf2ts-likeg.log logs the output of the gcf2ts commands.
            gcfconv writes trivial messages to STDERR.
 
  
 EXAMPLES:
 
  gcf2ts-likeg -10Hz -U -n 3 -l 42000  G2012.249/GCF.2012:249:00:00:00.3531Z2-ARECT-10Hz.ts
 
  gcf2ts-likeg -10Hz -c enz -n 48      G2012.248/GCF.2012:248:00:00:00.3531ENZ2-10Hz.mc

  gcf2ts-likeg -c enz -n 24            G2016.032/GCF.2016:032:00:00:00.3U93ENZ2-10Hz.mc
 
  gcf2ts-likeg -U -1Hz -n 10 G2022.045/GCF.2022:045:00:00:00
 
  foreach d ( `fromto -f "%3.3i " 33 51` )
  gcf2ts-likeg -1Hz -F -U -n 24 -c z GCF2022.$d/GCF.2022:${d}:00:00:00
  end
 
 PROBLEM: The last file of a day will be shorter if the next day hasen't been processed.
          To cure the gap you can make a special run for hour 23 later:
          For the last example above:

   foreach d ( `fromto -f "%3.3i " 33 51` )
      gcf2ts-likeg -1Hz -R -n 1 -c z G2022.$d/GCF.2022:${d}:23:00:00
   end
 
 CONCATENATING 1Hz:
   foreach d ( `fromto -f "%3.3i " 33 51` )
      set ymd=`jdc -DD -fs 2022 $d`
      tslist-app -I -o GCF2022/GCF.$ymd-ZACG-1Hz.ts + G2022.$d/GCF.2022:${d}:??:00:00.3U93Z2-ZACG-1Hz.ts
   end

.bye