Instant Plot: FMB SAXS/WAXS data viewing and reduction

Before you start- add path to .bashrc

You only need to do this once- you're setting up a shortcut for yourself.

1. Connect to NoMachine and open a terminal window.
2. Make sure you're in your home directory: "pwd" should return "/home/your_userid"
   1. if you're somewhere else, typing "cd ~" should return you to your home area
3. Enter > pluma .bashrc (or open .bashrc in your favorite text editor).
   1. This should open a text editor window withe the line "# .bashrc" at the top
4. Below the section

   # Source global definitions
   if [ -f /etc/bashrc ]; then
   . /etc/bashrc
   fi

   
   Add the lines:
   

   ### InstantPlot ###

   alias activate_msnc="source /nfs/chess/opt/miniconda3_msnc/bin/activate; conda activate saxswaxs-viewer"

   export PATH="/nfs/chess/user/chess_id3b/git_repos/saxswaxs-viewer:$PATH"

5. Save the file and close the text editor.

Connect to NoMachine and set up terminal

• Connect to NoMachine, lnx201.classe.cornell.edu

• Open a terminal window

• In the terminal window, you need to run a qrsh command to move from lnx201 to another node on the cluster:

  • For viewing data only, no batch processing, this command is enough: qrsh -q interactive.q

  • If you will want to batch process, choose a node with parallel cores by running: qrsh -q interactive.q -pe sge_pe 32

• To make file finding easier, change directories to your analysis area

  • cd /nfs/chess/auxiliary/reduced_data/cycles/YYYY-#/id3b/username-id/
  • where YYYY-# is the year and cycle number, e.g. 2022-2
  • username-id is PI name and project number, e.g. smieska-3222-A
• Activate the python environment you set up in the .bashrc by entering:
• • activate_msnc
• No need to export a path if you have edited your .bashrc as described above.
  • Note: It's possible to open InstantPlot without editing the .bashrc by running "/nfs/chess/user/chess_id3b/git_repos/saxswaxs-viewer/InstantPlot" in a terminal window
  • However, editing the .bashrc is the recommended method
  • Obsolete as of 2/22: export PATH="/nfs/chess/sw/anaconda3_ls/bin:$PATH"

InstantPlot inputs

• Required: PONI files via pyFAI calibration tool
• Optional: background snapshots, mask files from pyFAI calibration tool

InstantPlot operations

Option 1: Open InstantPlot with an existing config file specified

> InstantPlot configfilename.txt

• Either change directories to your "setup" area, where config files are stored, or specify the full file path

Option 2: Open InstantPlot without specifying config file

Option 2A: Manual setup (or first time setup)

• MUST HAVE: poni files for each detector (click PIL5, PIL9, PIL11 to locate poni files for each detector, usually in setup area). Highlight the needed .poni file and "open," or double click the needed file. When all paths are updated, click "load PONI files"

• OPTIONAL: background tiff snapshots (locate background snapshot .tiffs for each detector); click "load background files"

• OPTIONAL: mask files (tiff) from the calibration gui; click "load mask files"

Use the “save config” button at the bottom right to save as a text file in your setup area when finished.

Option 2B: Load an existing config file from inside InstantPlot

• Press “clear config file”

• Press “read config” and navigate to your config file (e.g., /nfs/chess/aux/reduced_data/cycles/2021-3/id3b/[username-BTR-ID]/setup)

• Press “load config file”

• Press “load PONI files”

• Press “load background files” if desired

• Press "load mask files" if desired

To view 2D data:

• Press “select path” at bottom left and navigate to scan directory on /nfs/chess/raw/[YYYY-X]/id3b/[username-BTR-ID]/
  • YYYY-X = Year-cycle number, e.g. 2021-3 or 2022-2, etc.
  • You need to open the actual scan folder (e.g. scanname_001/ ) by double clicking on it. It will look empty. Click "OK." This should populate the “tiff files” window for you to view the individual snapshots
  • If you click somewhere outside the "Choose directory" window it can get lost and InstantPlot can appear frozen- check the bar at the top of the window to locate it again.
• “Plot2D” tab on the right has options for changing the data display here

To view 1D data:

• Choose 1D SAXS or 1D WAXS tabs at top left. “Plot 1D” tab at top right has options for viewing these plots.
• Good to know: if you change the q/chi ranges in the Plot 1D area, you can see the result (what will be integrated) on the reciprocal space SAXS/WAXS tabs at top left.

Batch processing

Batch processing tools are available on the "batch processing" tab.

If you're running on the cluster, we recommend starting InstantPlot from a specialized node:

• In a new terminal window, hop to a higher computing power node with parallel processing environment invoked:
  • qrsh -q interactive.q -pe sge_pe 40
    • OR: qrsh -q interactive.q -pe sge_pe 32 (fewer cores but more of these nodes are available)
  • Note: if you're on the station analysis computer lnx-id3b-1, you'll need to first ssh to lnx201 as yourself
    • > ssh -Y your_CLASSE_ID@lnx201.classe.cornell.edu

• cd to your auxiliary area (/nfs/chess/auxiliary/reduced_data/cycles/YYYY-N/id3b/userid-BTR-ID)

Open InstantPlot as normal, described above (> InstantPlot configfilename.txt)

Load your config file and make any desired adjustments in the Plot 1D views (q ranges, number of points, etc).

When you're happy with the integration, go to the Batch Processing tab and choose your options:

• check the boxes for integrating saxs, waxs, or both
• choose your plot type (I vs. q, I vs. chi, I vs. q vs. chi)
• choose your correction type (flux, absorption, background correction options)
• choose to overwrite or skip existing output files
• check the box for "Traverse child directories" if you want to process multiple folders with this configuration
• check the box for "process in parallel" if you're running on a specialized computing node and you have mapping data
• click "Generate output path" and then edit if desired

When you're happy with all of the above, then click "Begin Batch Processing"

• If you chose "Traverse child directories," you should now see a pop-up window asking you to select directories. Choose the ones you want to process with the current configuration and press "continue"
• There may be a delay before you see output in the terminal window where InstantPlot is running, but give it a minute or two and you should start seeing results being written to text files

-- LouisaSmieska - 4 May 2022