Differences between revisions 2 and 5 (spanning 3 versions)
Revision 2 as of 2009-11-04 05:00:12
Size: 3808
Editor: l4-m0
Comment:
Revision 5 as of 2013-05-01 09:59:02
Size: 4057
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:

== Quick Summary of WSO Data Reduction Process ==

 . Data files for each observation are transferred daily by the WSO Shutdown program to /data/scans/rawdata/transfer
 . Analysis must be done in order day by day using '''~wso/src/reductions/WTelred'''. Use '''/home/wso/scripts/checkwso''' to see which files are available.
 . If things get done out of order then the final data sets must be altered to remove the offending data and the reductions then re-run in proper order. This is tedious and requires care.
 .
  . Meanfld's are appended daily to ''/wso/scans/mf/rawmf''
   . At month end the MFs are evaluated using '''~wso/src/reductions/MF.monthly''' and good ones ultimately added individually to ''mf.ds'' and daily averages in ''mf''.
  . Magscans are put into ''/wso/scans/level0/sn#### aux0/sn####'' and ''ns0/sn####'' and appended to ''level1, level1.aux'', and ''level1.ns''.
   . After immediate evaluation and trouble bit setting using '''settrouble''', a call to '''Dolev3''' extends ''/wso/scans/level3'' and updates daily synoptic maps and coefficients in ''/anonftp/wso'' and files on the WSO web site in ''/web/wso/htdocs''
   . Monthly evaluation is done before making final synoptic maps and daily plots for SGD
Line 29: Line 41:
=== Quick Summary of WSO Data Reduction Process ===
Line 31: Line 42:
 . Data files for each observation are transferred daily by the WSO Shutdown program to /data/scans/rawdata/transfer
 . Analysis must be done in order day by day using ''~wso/src/reductions/WTelred''. Use ''/home/wso/scripts/checkwso'' to see which files are available.
  . Meanfld's are appended daily to '''/wso/scans/mf/rawmf'''
   . At month end the MFs are evaluated using ''~wso/src/reductions/MF.monthly'' and good ones ultimately added individually to '''mf.ds''' and daily averages in '''mf'''.
  . Magscans are put into /wso/scans/level0/sn#### aux0/sn#### and ns0/sn#### and appended to level1, level1.aux, and level1.ns
   . After immediate evaluation and trouble bit setting using '''settrouble''', '''Dolev3''' extends /wso/scans/level3 and updates daily synoptic maps and coefficients in /anonftp/wso and on the web site /web/wso/htdocs
   . Monthly evaluation is done before making final synoptic maps and daily plots for SGD
Line 39: Line 43:
Return to [:WSOReductions:WSO Reductions] Return to [[WSOReductions|WSO Reductions]]

Fixing Problems with Daily Reductions

Please try these things only if you know what you are doing. It takes considerable effort to recover the WSO daily reductions when they proceed out of order or need to be redone.

Quick Summary of WSO Data Reduction Process

  • Data files for each observation are transferred daily by the WSO Shutdown program to /data/scans/rawdata/transfer
  • Analysis must be done in order day by day using ~wso/src/reductions/WTelred. Use /home/wso/scripts/checkwso to see which files are available.

  • If things get done out of order then the final data sets must be altered to remove the offending data and the reductions then re-run in proper order. This is tedious and requires care.
    • Meanfld's are appended daily to /wso/scans/mf/rawmf

      • At month end the MFs are evaluated using ~wso/src/reductions/MF.monthly and good ones ultimately added individually to mf.ds and daily averages in mf.

    • Magscans are put into /wso/scans/level0/sn#### aux0/sn#### and ns0/sn#### and appended to level1, level1.aux, and level1.ns.

      • After immediate evaluation and trouble bit setting using settrouble, a call to Dolev3 extends /wso/scans/level3 and updates daily synoptic maps and coefficients in /anonftp/wso and files on the WSO web site in /web/wso/htdocs

      • Monthly evaluation is done before making final synoptic maps and daily plots for SGD

WSO Daily Reduction Fixes

Fixing Magnetograms

  • WRedoMAG

    • WRedoMAG requires 3 parameters giving the Day, Serial_number, and Filename. There is an optional 4th parameter, "-a" The output of the program will be captured and added to a reduction script at /data/scans/reduction.scripts/YYYY_Day.MAGredo.script

    • WRedoMAG re-reduces magnetogram Serial_number that has already been reduced. Data are found in the directory /data/scans/reduced/Day in the file Filename

    • Example: WRedoMAG Oct28 9906 1028:1755_MG01 redoes the reduction of magnetogram 9906 found on /data/scans/reduced/Oct28/1028:1755_MG01

  • WRedoMag.timefix2 does the same, but does not automatically capture a record of the reduction script.

Fixing Level 3 Remapped Magnetograms

  • pokel3

    • pokel3 sets values in a single level 3 magnetogram to missing in a rectangle specified by xmin, xmax, ymin, & ymax

  • pokel3_runs

    • pokel3_runs is a script that calls pokel3 to set to missing a range of values in a particular level 3 re-mapped magnetogram to eliminate bad pixels. It is generally used when a partial magnetogram must be included in the synoptic chart, but the interpolation/extrapolation fails leaving bad values in the level 3 magnetogram. pokel3_runs has a record of all adjusted level 3 magnetogram pixels since it was started in 1992 to keep a record of corrected scans. New lines are added when needed and lines already run are commented out of the script.

      • - x is 0 at right edge; y is 0 at bottom of print3 - x is 22 at left edge; y is 29 at top of print3
  • print3 -n sn=SN will provide a print-out and some information about a level 3 magnetogram

  • plotl3 SN (in ~wso/scans/scripts) will make a plot of level 3 magnetogram SN.

Fixing Meanfield Reductions

  • WRedoMF

    • WRedoMF requires 2 arguments, the DAY and the MFNUMBER of the first target Meanfield.

    • WRedoMF re-reduces all of the MF's in the appropriate directory in the same order as usual, but instead of adding to the end of rawmf.ds, it starts with the specified MFNUMBER. It searches first for files in /data/scans/rawdata/DAY and if that directory does not exist it looks in /data/scans/reduced/DAY. It does not make a record of the re-reduction, but it should.

    • Example: WRedoMF Oct28 67062 would redo the MF reduction for the MF files in /data/scans/reduced/Oct28 starting with 67062.


Return to WSO Reductions

WSO_Wiki: FixDailyReductions (last edited 2013-05-01 09:59:02 by localhost)