page updated by Jelena 16:44 08Mar2019

List of fixes and updates for CM5 :

1) catch the most common errors for user main log file :
  - run into /t1-data quota
  - run into cluster area size
  - run into situation of BOTH flashed and nonflashed bams (of an oligo) being empty in end of folderC
  - run into situation of BOTH flashed and nonflashed bams (of an oligo) being empty in end of folderD
  - other common cases ?

2) generalise the above (1) : to yield "fool-proof" error reporter, to keep users in map what actually happened in crash.
   - make a generalised "error catcher" to bring users a better idea of why the run actually crashed

3) helping users to understand run logs and errors
   - better documentation of the error log locations and contents
   - better organisation of the error log files
   - new parsers and tables to integrate the situation over all parallel runs
 
4) better re-run instructions (linking crashes to rerun flags in illustrative manner)
   adding "common troubles" section to find and fix the most common crashes ( --useClusterDiskArea , running into quota, crashing with non-read-containing oligos)

5) improving the general run instructions
  - consolidating the current user instructions to something more useful
  - extending the user instructions to become a real web site + user manual + vignette/hands-on-tutorial

6) checking that the newest basic user cases work in expected manner
   - onlyHub situation where folderD crashed (should force generation of intact hubs even though crashed D folder)
   - deleting wig and bam files in ALL common combinations of run modes ( --wholeNode24 (yes/no) --useClusterDiskArea (yes/no)  )
   
7) building the --onlyBlat user case (currently --onlyBlat will be ran in CS5, and only the main run in CM5)
   - and its fully parallel "CM5" style running logic
   
8) improving the data hubs
   - explaining the data hub tracks somewhere (the new normalised tracks)
   - adding the bigbed into the hubs (to always have the "color key" in the hub)
   - pondering in which order to give the hubs in the E_dataHubs.txt : potentially making this into a html link to a html page where
     important parts could have highlights etc. Generally : E_dataHubs.txt is too long and generates "blah blah blah" feeling in users,
     and they tend to just "pick a random hub" without knowing if that's the right one.
   - bringing the blat-filtered fragments track back (how exactly ?), and having it in "dense" as default.
   
9) this one is wishful thinking (i.e. "time permitting")
   - building a true auto-restarter for crashed oligos (if oligos crash for innocent reasons like disk quota and thus can be readily restarted)
     : restarting folderD would KNOW which folderD oligos already completed fine, and would rerun only the oligos which crashed.
   - building a true auto-restarter for crashed fastqs (if fastqs crash for innocent reasons like disk quota and thus can be readily restarted)
     : restarting folderB would KNOW which folderB fastqs already completed fine, and would rerun only the fastqs which crashed.
   - to make this happen : using the pre-built (not properly tested) flags and subroutines for this (if ever time to complete this properly)