CHANGE LOG

Version:

apt-1.14.4

Date:

Fri Jan 26, 2011

This release focusses on improvements to the apt-copynumber-cyto pipeline for processing of CytoScanHD and CytoScan750K arrays. Highlights include:

• Support for dropped probesets expected in na33 annotation libraries.
• Covariate adjusters to adjust for modulation in Allele Peaks data in certain rare samples.
• Improved performance.

Bug fixes and new features

• [APT-960] - Improve confusing help text for apt-copynumber-cyto
• [APT-965] - Some CNProbeSet state is passed to the next probe set when a bad annotation is found.
• [APT-981] - Implement Allele Peaks covariate adjuster.
• [APT-1025] - Add vignette for CytoScan750K support

Version:

apt-1.14.3.1

Date:

Tue Nov 1, 2011

This is a minor release in support of Axiom avian arrays. Added support for ZW gender calling (ZW female, ZZ male).

Improvements and New Features

• [APT-535] - apt-probeset-genotype - added support for gender calling on avian arrays. Added options --chrZprobes, --chrWprobes and --zw-gender-calling.

Version:

apt-1.14.3

Date:

Fri July 15, 2011

The following highlights the changes enhancements in this version of the software.

• Support for CytoScanHD array analysis using apt-copynumber-cyto
• Various new and improved algorithms for CytoScanHD analysis
• Various performance improvements
• apt-midas was restored to the release package

Details below:

Bug fixes

• [APT-345] - CNAnalysisMethodGaussianSmooth - improved error messages and added a check for the minimum value of the smooth_bw parameter
• [APT-434] - apt-probeset-summarize - improved checks and error messages for .bgp files lacking probes with necessary representative GC counts
• [APT-517] - apt-probeset-genotype - added tests to verify the file format of priors files
• [APT-727] - apt-copynumber-cyto - improved performance for SSA processing for CytoScanHD CEL files
• [APT-728] - apt-copynumber-cyto - added "uncomputed gender" boolean flag to cychp file header to indicate when no gender calling was enabled
• [APT-822] - apt-copynumber-cyto - updated the calvin file writer to not include empty segments tables
• [APT-871] - apt-copynumber-cyto - standardized the naming of text files created by geno-qc
• [APT-887] - apt-copynumber-cyto - fixed a bug causing LOH segments that extended across long gaps in marker coverage to be appropriately identified to gap boundaries
• [APT-888] - apt-copynumber-cyto - fixed a bug where the LOH Segments algorithm did not always utilize respecting the minGenomicSpan parameter resulting in some LOH segments being miscalled
• [APT-930] - Updated -help text in apt-copynumber-cyto
• [APT-932] - Fixed problems compiling hdf5 libraries from source on Ubuntu

Improvements and New Features

• [APT-40] - docs - The file format document has been updated
• [APT-587] - apt-cdf-export - added the "--file-md5" option to compute the md5 of an entire file.
• [APT-645] - CNReference api - updated the module to support CytoScanHD
• [APT-763] - apt-copynumber-cyto - added a log message indicating if snp-qc-snp-list could not be read
• [APT-789] - apt-copynumber-cyto - changed the default behaviour to require modules to be invoked explicitly. The following modules must now be explicitly called to use them as shown before:
  • local-gc-background-correction-reference-method pdnn-reference-method
  • wave-correction-reference-method wave-correction-reference-method
  • local-gc-background-intensity-adjustment-method pdnn-intensity-adjustment-method
  • image-correction-intensity-adjustment-method high-pass-filter-intensity-adjustment-method
  • wave-correction-log2ratio-adjustment-method wave-correction-log2ratio-adjustment-method
• [APT-810] - apt-copynumber-cyto - improved performance of wave correction
• [APT-817] - apt-copynumber-cyto - Genotyping and LOH analysis on the X chromosome are now based on CN state and not on the presence/absence of the Y chromosome for the CytoScanHD pipeline
• [APT-858] - apt-copynumber-cyto - made the additional-waves-reference-method module selected-qc parameter more generic
• [APT-933] - Improved readme guidance for windows builds
• [APT-534] - apt-copynumber-cyto - support for processing flags for CytoScanHD array annotations file
• [APT-733] - apt-copynumber-cyto - implemented Generic Signal Covariate Adjuster
• [APT-734] - apt-copynumber-cyto - implemented Generic log2 ratio Covariate Adjuster
• [APT-752] - apt-copynumber-cyto - implemented high pass filtering
• [APT-773] - apt-copynumber-cyto - implemented gradient correction applicable for all chips with copy number probes
• [APT-776] - apt-copynumber-cyto - added Computed Call Rate to Report file and CYCHP header
• [APT-778] - apt-copynumber-cyto - the proportion of the autosomes that exhibit LOH is now in the CYCHP header
• [APT-785] - apt-copynumber-cyto - added support for separate HMM parameters for Autosomes, ChrX and ChrY
• [APT-809] - apt-copynumber-cyto - added support for external covariates file
• [APT-843] - apt-copynumber-cyto - changed name of the log2ratio-adjustment-high-pass-filter to log2ratio-adjustment-method
• [APT-825] - apt-copynumber-cyto - implemented wavelet shrinkage estimator (signal restoration) for log2 ratios
• [APT-884] - apt-copynumber-workflow - implemented a check to ensure the minimum number of cel files were provided for reference file creation
• [APT-912] - Removed obsolete applications apt-copynumber-analysis, apt-copynumber-reference, and apt-copynumber-log2ratio from release. Restored apt-midas.
• [APT-923] - Incorporated changes in Fusion SDK into APT package
• [APT-928] - Removed apt-dmet-copynumber from 1.14.3 release

Version:

apt-1.14.2

Date:

Fri March 18, 2011

The following highlights the changes enhancements in this version of the software.

• Support for the new CDF file format version described in the section on the Fusion SDK Version 1.11.1 above.
• Support for various arrays as indicated above in the section on the new features included in GTC 4.1.
• New call rate metrics calculation using autosomal SNPs only
• Support for 3000 SNPQC probesets during genotyping
• New artifact reduction support in the genotyping workflow to support Axiom arrays with new design
• Performance improvements

Details below:

Improvements and New Features

• Genotyping
  • [APT-228] - apt-geno-qc aborts improved error reporting for corrupted cel files.
  • [APT-252] - apt-probeset-genotype - fix description of --disk-cache option
  • [APT-104] - apt-probeset-genotype - Report sample call rate, sample percentage of AB calls (het_rate) and sample percentage of AA/AB calls (hom_rate)
  • [APT-148] - apt-snp-summary - break out snp summary table calculations in GTC into stand-alone app.
  • [APT-159] - apt-data-subset - allow sample multichannel cel files for Axiom testing.
  • [APT-213] - apt-probeset-genotype - More compact data output for genotyping for A5 and txt formats.
  • [APT-314] - apt-probeset-genotype - API for reading cluster location information from models files.
  • [APT-297] - apt-probeset-summarize - Remove mas5 signal and detection.
  • [APT-467] - apt-probeset-genotype - Implemented new QC and overall Call Rate metrics for Axiom Genotyping Runs.
  • [APT-501] - apt-probeset-genotype - Implemented new Artifact Reduction algorithms.
  • [APT-518] - apt-probeset-genotype - add option to strip off allele suffix from probeset id when writing feature effects file
  • [APT-535] - apt-probeset-genotype - add gender flip to handle avian arrays (Z/W chromosomes).`
  • [APT-568] - apt-geno-qc Implemented reagent version 1 and 2 discriminator.
  • [APT-659] - Improved genotyping performance change from GTC 4.0 / APT 1.12 to GTC 4.1 / APT 1.14.2

• Cyto
  • [APT-158] - Cannot specify waviness-seg-count-cutoff=0 and use-high-waviness-seg-count=false for the same run.
  • [APT-238] - Cyto enhancement: write SCARS even when info is 0
  • [APT-339] - Replace guid value from .Report.txt file of the apt-copynumber-cyto application by something that keeps the same value for gold and test runs or delete it.
  • [APT-380] - Improve handling of output directory creation for Windows
  • [APT-150] - Change apt-copynumber-cyto wave correction code to default to the number of waves found in the reference if none defined on the command line. Also report the number of waves used in the cychp file.
  • [APT-183] - Add median log2 ratio per chromosome to chromosome summary table in cychp files.
  • [APT-237] - Add ability to configure cychp file name
  • [APT-250] - Support for new parameter for ref mode file creation in cyto that indicates if CEL files based on ATP or an manual process.
  • [APT-336] - Add Meta-data support to Reference Model file creation and chp files for Cyto
  • [APT-353] - Creation details in model files (genotyping specifically)
  • [APT-180] - Adding waveSmooth analysis string option for cyto
  • [APT-267] - Cyto enhancement memory issue for more large numbers of cel files
  • [APT-309] - For APS/APG, when processing cel files, print the name rather than a dot to the log file.
  • [APT-275] - Warning message change when number of waves is not specified
  • [APT-266] - Change apt implementation of SCAR values so that no more nan values for the 1498 SNP markers with Information Content = 0
  • [APT-636] - Check probeset lists for errors and report meaningful message if found
  • [APT-740] - apt-geno-qc handling of avian array QC (Z/W chromosomes).
  • [APT-344] - LOH parameters reported in cychp file.
  • [APT-563] - for apt-copynumber-cyto need to be able to specify sketch size (override default) for SNP6 data where --cyto2 is false
  • [APT-182] - Cyto image correction to use single blocking
  • [APT-516] - Introduction of copy number calling module for CytoScanHD.
  • [APT-528] - Enable Cyto HMM call for CytoScanHD arrays under apt-copynumber-cyto
  • [APT-529] - Gender calling issues for CytoScanHD array from apt-copynumber-cyto
  • [APT-530] - Enable confidence score calculation in apt-copynumber-cyto for CytoScanHD array
  • [APT-531] - Enable LOH result from the Chromosome summary table in apt-copynumber-cyto for CytoScanHD
  • [APT-544] - Add processed allelic difference to apt-workflow-cyto.
  • [APT-652] - Implement new QC metric for SNP markers
  • [APT-775] - Implement a quantile normaliztion methodology which respects differences between autosomes and chromosome X/Y
  • [APT-546] - Add command line control of BRLMM-P parameters in cyto
  • [APT-656] - SNP6 CN performance improvement from GTC 4.0/APT 1.12 to GTC 4.1/APT Head

• General
  • [APT-55] - apt-cel-transform: work with multi channel cel files
  • [APT-56] - apt-cel-transform: generate new GUID and propagate old in headers
  • [APT-57] - apt-cel-transform: include command line in cel header
  • [APT-121] - Reduce disk usage by recycling intesity marts
  • [APT-122] - Fix DiskIntensityMart to allow arbitrary data names.
  • [APT-127] - Remove AnalysisStream function calls.
  • [APT-129] - Store probe level data outside of ChipLayout.
  • [APT-140] - Remove references to raw_intensity_mean and raw_intensity_stddev from reports.
  • [APT-155] - Write only analysis probe intensities instead of all intensities for final diskmart
  • [APT-156] - Initialize diskmart with CEL order instead of layout order
  • [APT-193] - IntensityMart::getCelData that accepts a channel designation
  • [APT-206] - Add apt-copynumber-correlation a new app to generate a correlation matrix of log2 ratios using cychp files as input.
  • [APT-216] - The --analysis-files-path search path is Unix-style colon-separated and it gets confused by Windows absolute paths.
  • [APT-220] - Improve version reporting
  • [APT-224] - Do not include NaNs in correlation calculations for Calvin::equivalent(...) function.
  • [APT-253] - Allow hints for a subset of CEL files/samples in the hints file
  • [APT-262] - apt-cel-extract error message when --target-sketch-file is given, but not --analysis
  • [APT-279] - Replace calls to quickSort and binarySearch by STL equivalents.
  • [APT-350] - Allow brlmm-p posteriors to be used as priors without edits
  • [APT-358] - Configure - add example of setting sdk_output_top
  • [APT-10] - Allow use of sparse-mart (in-memory mart) with chipstream code
  • [APT-138] - File5 - Add char/short datatypes
  • [APT-198] - SummaryGenotypeEngine can input and output file5 format
  • [APT-212] - Output summaries, calls and confidences in sqlite format for Simon's group
  • [APT-234] - Support for providing metadata to write into the CHP file header
  • [APT-245] - Calvin comparison
  • [APT-293] - Engines listen and prints on sockets for Windows integration
  • [APT-273] - Handle long filenames on windows
  • [APT-284] - apt-cel-transformer - add --cel-files option.
  • [APT-360] - Consolidate file functions from Util into calvin_files FileUtils
  • [APT-686] - Use environment variables/macros for documentation version numbers, etc.
  • [APT-436] - Converted all of APT to use FsPath libraries. (Consistent fix for MAX_PATH issues).
  • [APT-246] - Tsv support for shorts
  • [APT-688] - Added CDF GUID to the CHP header
  • [APT-685] - Remove in-place updates of source code from doxygen make process
  • [APT-696] - Fix formatting of html docs
  • [APT-697] - Move help docs out of .cpp files into separate .dox files
  • [APT-699] - Clean-up/simplify make dox task
  • [APT-458] - apt-cdf-to-spf - modify to handle multi-channel cdf input (CDF file format version 4)
  • [APT-485] - calvinlite should check for other celfile versions.
  • [APT-731] - calvinlite should be able to select keys to print.
  • [APT-482] - Added md5 values to CHP file headers.
  • [APT-548] - Replaced comman and tab delimited file access with file/TsvFile. (Removed util/AffxFile).
  • [APT-672] - apt-annotation-converter added for all platforms.

Bugs

• Genotyping
  • [APT-124] - Genotype chp check reports errors with confidences match instead of when they don't match
  • [APT-178] - Canary parameter help info printed on stdout when it should be on stderr
  • [APT-202] - Calvin GenericFileWriter generates empty error message if it cannot open the file.
  • [APT-225] - Uncaught exception in apt-probeset-genotype if cel file does not exist.
  • [APT-261] - apt-probeset-summarize unable to remove temp dir
  • [APT-370] - Application apt-probeset-summarize memory issue for PM only array.
  • [APT-388] - apt-probset-summarize fails when running Adapter Type Normalization analysis method - Allowing annotation-file to be specified for apt-probeset-summarize.
  • [APT-398] - apt-probeset-genotype - added a minimum value to CSepThr.
  • [APT-393] - Using SNP list for genotyping produces correct txt files but CHP file does not contain calls only header
  • [APT-415] - QC performance improvements.
  • [APT-488] - Error when insufficient space to run genotyping
  • [APT-512] - Rat array genotyping fails using APT windows (head and 1.12), ok in Linux (head and 1.12)
  • [APT-550] - Reading a poorly formed models file causes the application to abruptly exit
  • [APT-632] - Added error if user tries to specify both --read-genders and --set-gender-method on the commandline.
  • [APT-664] - SNP6 copynumber - Gender column in 4.1 changed to "Unknown" for all CNCHP files
  • [APT-682] - apt-probeset-genotype fails to run for axiom array when library files are in E or F drive (rather than C)
  • [APT-714] - apt-geno-qc memory leak when reading cel files fixed.
  • [APT-722] - Memory ios_base::failbit set error shows when using .cel files on network paths.
  • [APT-799] - snpQC call rates show as percentage instead of fraction.
  • [APT-207] - APT High Pass Filtering never runs when PDNN Background Correction is turned off
  • [APT-445] - SpectClust::findNLargestEvals always returns "false" for "converged" (even if did converge)

• Cyto
  • [APT-154] - CNReferenceMethodWaveCorrection opens reference files as read/write when should be read only
  • [APT-162] - Cyto legacy bug fixes: util/AffyConvgetDouble(...) - Rounding errors on floating point output.
  • [APT-164] - Cyto legacy bug fixes: CNAnalysisMethodChipstream::getSampleId(...) - No longer parsing "_NA" from sample name.
  • [APT-165] - Cyto legacy bug fixes: CNAnalysisMethodChipstream Missing Last Genotype Call Override Sample
  • [APT-166] - Cyto legacy bug fixes: Median == 0. (Very rarely occuring bug, seen in some cancer samples only).
  • [APT-167] - Cyto legacy bug fixes: Median Intensity
  • [APT-169] - Cyto legacy bug fixes: CNLog2RatioAdjustmentMethodWaveCorrection - Application always uses the maximum number of waves in CN reference regardless of what user specifies for wave-count.
  • [APT-170] - Cyto legacy bug fixes: CNProbeSet Underflow in GC Binning.
  • [APT-171] - Cyto legacy bug fixes: Probe Set Name to Genomic Order.
  • [APT-172] - Cyto legacy bug fixes: CNSmoother Module - remove short segments in CN run time encoding.
  • [APT-175] - CEL file index numbers are being printed to report files instead of CEL file names
  • [APT-241] - Cyto performance improvements.
  • [APT-215] - Parameter --analysis-files-path is being ignored for the --snp-qc-snp-list file in certain cases
  • [APT-276] - Cyto CNAnalysis for reference model file uses wrong extension if not set in job file
  • [APT-294] - Improved CN/HMM
  • [APT-295] - Improve AllelePeaks detection
  • [APT-299] - Improve LOH calling for chromosomes 19 and 22
  • [APT-321] - Change in Behavior for --cels Argument: errAbort CNCytoEngine if CEL files are specified both as command line arguments and cel-file or cels list(s).
  • [APT-354] - Added probe-file option to the analysis-files-path prepend list in CNCytoEngine
  • [APT-391] - CN/SNP processing flags need to be handled properly - handling of flag 2.
  • [APT-419] - apt-copynumber-cyto --explain unexpected should throw warning message.
  • [APT-456] - Marker Names Unavailable for SCAR Values from APT - now output all markers used in the reference creation to the ProbeSet section of the cychp file.
  • [APT-466] - apt-calvinlite-util does not display values of boolean header parameters, only an empty string.
  • [APT-489] - Allele peaks data differences between Cyto SSA runs
  • [APT-502] - Marker count of Mosaicism counts the SNPs that do not have a log2ratio
  • [APT-519] - apt-copynumber-cyto vs workflow Chr X LOH differences for Female samples
  • [APT-520] - apt-copynumber-cyto vs workflow extra records in reference feature_response and snp_posteriors table
  • [APT-522] - Numerical differences between estimated waves from apt-copynumber-workflow and apt-copynumber-cyto
  • [APT-523] - AGR probe id was not generated in CN reference build for SNP6 data using apt-copynumber-cyto
  • [APT-525] - LOH calls problem comparing results from apt-copynumber-cyto and apt-copynumber-workflow
  • [APT-540] - Scrambled CNReferenceHeader Header Values in Cyto and Workflow
  • [APT-541] - cyto generates single-sample mode output with rounding differences vs workflow when wave correction enabled
  • [APT-615] - apt-copynumber-cyto AllelePeaks algorithm 6X duplicate rows in PekSets dataset

• General
  • [APT-142] - File5 - size not correct after write_array.
  • [APT-260] - apt-chp-to-txt documentation bug
  • [APT-281] - apt-file5-util: no help reported when no args on command line
  • [APT-337] - Util::memInfo reports swapAvail=0 on Darwin OS-X - Documented.
  • [APT-363] - CHP file headers slow to read - improved.
  • [APT-369] - Fix bug in apt-mas5 where .cel was included in output chp file name
  • [APT-374] - TmpFileFactory ignoring user-defined file pre/suffixes
  • [APT-510] - APT will fail if batch name has "=" sign
  • [APT-612] - apt-calvinlite-util unable read short .chp file with missing data which Calvin Viewer can read
  • [APT-634] - Fixed Windows 260 character MAX_PATH limit. Consolidated all file access methods and routines in util/Util and util/AffxFile respectively to util/Fs to support windows MAX_PATH.
  • [APT-669] - Function SnpModelConverter::getTsvArrayType fails to get chip type of posterior models - added substring matching to check chip types.
  • [APT-674] - Added exception handling for SQLite libraries.
  • [APT-701] - FsPath constructor doesn't set internal path name.

Version:

apt-1.12.a

Date:

Tue Sept 23 2010

• Incremental release containing apt-geno-qc and apt-probeset-genotype only.

Version:

apt-1.12.0

Date:

Monday Oct 12 2009

• Full Axiom ™ support
• Added support for multi channel cel files to apt-cel-extract

Version:

apt-1.11.0 (Internal Release Only)

Date:

Sat Aug 29 2009

• Fixed bug in Windows x64 binary release where executables depended on debug runtime libraries
• Fixed a bug in AdapterTypeNormalization where transformed intensities were not passed through to next chipstream when initializing.
• Partial release to support QC and genotype calling on the Axiom ™ platform. We recomend Staying with 1.10.2 for other applications of APT until the next full release (1.12.0).
• Fixed a problem with the segments in a CYCHP file. The segment_id should be a UINT, not a STRING.
• Cytogenetics Genome Wide Array Support
• Added apt-summary-genotype to windows binary releases
• Added apt-mas5
• Added support for SQLite annotation files to apt-copynumber-workflow
• Added artifact reduction chipstream node.
• Inbred tweaks to apt-probeset-genotype
• Added waviness amplitude metric to cychp summary metrics
• Added segments for CNNeutralLOH and NormalDiploid. [experimental]
• Added antigenomic ratio to CNExperiment [experimental]
• Added Mosaicism. [experimental]
• enable wave correction for SNP6 [experimental]
• refactor how feature effects are handled. Now match by probe ID AND probeset ID
• Switch Windows builds to use VS2008
• Added XML output options for unit and regression tests
• Added support for multiple intensity channels (eg multi color CEL files)
• Refactor BaseEngine and handling of option and state.
• Commandline programs which use a BaseEngine derived analysis, now accept a set of shared options
• remove use of tmp chp file when outputing CHP files
• removed in-memory mode for apt-probeset-genotype and apt-probeset-summarize
• reconcilled differences in same options between apt-probeset-genotype and apt-probeset-summarize

Version:

apt-1.10.2

Date:

Tue Jan 27 2009

• Added support for DMET Plus:
  • apt-dmet-genotype will generate DMET CHP files from DMET CEL files. The CHP files contain genotyping and copynumber results.
  • apt-dmet-translation will generate DMET star allele translation reports from CHP files.
• Fix bug in apt-probeset-summarize --probeset-ids option when processing genotype probesets using the expr.genotype=true modifier in the analysis string.
• Improve error handling when required columns in TSV files are missing.
• Fix CNCHP header so that files can be loaded into GTC 3.x
• Fixed problem in CN reference file generation so that APT generated reference files are backwards compatabile with GTC 3.x
• reconcilled differences in same options between apt-probeset-genotype and apt-probeset-summarize

Version:

apt-1.10.2

Date:

Tue Jan 27 2009

• Added support for DMET Plus:
  • apt-dmet-genotype will generate DMET CHP files from DMET CEL files. The CHP files contain genotyping and copynumber results.
  • apt-dmet-translation will generate DMET star allele translation reports from CHP files.
• Fix bug in apt-probeset-summarize --probeset-ids option when processing genotype probesets using the expr.genotype=true modifier in the analysis string.
• Improve error handling when required columns in TSV files are missing.
• Fix CNCHP header so that files can be loaded into GTC 3.x
• Fixed problem in CN reference file generation so that APT generated reference files are backwards compatabile with GTC 3.x
• Fix quantile normalization code so that values outside a predefined target sketch are capped. Without this fix infinite values (and negative values) were possible.
• Fix makeDir error in apt-geno-qc when output file specification (from --out-dir option) did not include a directory portion.
• Accomidate genotype probesets which do not have equal numbers of allele A and B probes.
• In apt-cel-extract, fix the ordering of probes in the on-disk intensity mart. Previous version was very slow due to non-optimal ordering of probe intensities on disk. Related to this fix is that the output order is different when using a probeset list. (It is now in the same order as the provided probeset list).
• Fixed a bug where target sketches provided to apt-cel-transformer where assumed to be in a5 format.
• Fixed a bug where apt-probeset-summarize would fail if an entire probeset was removed via the probe kill list when using a CDF file.
• Added utility apt-engine-wrapper which allows one to directly call analysis engines and query for what options an engine will accept.
• Added apt-file5-util which allows one to convert between a5 and text/tsv file formats.
• Drop binaries for Solaris/SPARC
• Change in OS-X binaries:
  • dropped "un-tested" universal binaries
  • providing stand alone 64-bit PowerPC binaries
  • providing stand alone 64-bit Intel binaries
• Changed APT guids to be RFC-4122 format guids

Version:

apt-1.10.1

Date:

Mon Nov 3 2008

• Fixed bug in MAPD calculation by copynumber workflow. In 1.10.0, the MAPD values were computed before putting probesets in genome order. The implication is that MAPDs reported by 1.10.0 were inflated relative to the real MAPD values.
• Add genome GC content correction to copynumber workflow.
• Add a log file to apt-summary-genotype
• Add on-disk mode to apt-cel-extract (on by default)
• Add progress meters for chip layout file reading (ie CDF)
• Fixed bug where block size and memory check code was still being run even when on-disk mode was set. The implication in 1.10.0 was that multiple iterations (library file reads) were being done unnecessarily. Results were still correct.
• Add copynumber binaries to windows installer (missing in 1.10.0 release)
• Changed the code to only read lib files only once when doing one iteration in apt-probeset-genotype and apt-probeset-summarize.
• Clean up of a5 options for apt-probeset-genotype
• Add a5 options to apt-probeset-summarize
• Refactored the analysis engines to inherit from a common engine base class.

Version:

apt-1.10.0

Date:

Wed Jul 9 2008

• Fixed bug where pm-sum method was reporting values equivalent to pm-only when using SPF file rather than a CDF file. This is in the context of using apt-probeset-summarize with genotyping array via the "expr" addition to the analysis string.
• Fixed bug where incorrect "x" coordinate was reported in feature effect output files from apt-probeset-summarize and apt-probeset-genotype (all releases) and in apt-cel-extract (releases newer than 1.8.0) output. Incorrect values only show up when working with arrays where the number of rows and columns in the CEL file are not equal.
• Fixed bug in PLIER where feature effects were updated after the last update to target responses
• Added copynumber functionality for SNP6 (see apt-copynumber-workflow).
• Added Canary copynumber calling for SNP6 (see apt-canary).
• Complete refactor of apt-chp-to-txt. Now handles most chp file formats and types. Resequencing and Tag chp files are not handled by this tool.
• Added Vignettes:
  • SNP6 Copynumber analysis using individual components
  • Single sample genotyping (ie for QC purposes)
• Removed apt-cel-to-pq. Supplanted by apt-cel-extract.
• Misc
  • Added experimental HDF5 output file formats to apt-probeset-genotype
  • Added on-disk intensity mart. Set as the default for apt-probeset-summarize and apt-probeset-genotype.
  • Fixed memory capping on x64 windows and 64-bit OS-X.
  • Added the ability to apply pre-computed feature effects when using median polish in apt-probeset-summarize and apt-probeset-genotype.
  • Added an adaptor type normalization (used by copynumber workflow)
  • apt-probeset-genotype will now report signal estimates for copy number probesets when the --all-types and --summaries option is given

Version:

apt-1.8.6

Date:

Mon Feb 3 2008

• Fix bug in IterPlier where one would get a segfault if the number of chips exceeded 100.
• Change from float to double in QuantBirdseed methods to improve cross platform portability. This may result in slight differences in the confidence value and in rare cases a change between making a genotype call versus a no-call.
• order of results in apt-probeset-genotype report file have changed due to some refactoring of the code.
• added birdseed-v2
• added ContrastQC qc metric to apt-geno-qc
• fix memory estimation (don't cap at 2G on Windows (x64) and OS-X (ppc64, x86_64)
• apt-cel-transformer: remove ability to change formats. add ability to handle agcc files
• Improvements to command line option handling
• Allow med-norm to be followed by other chip stream components when using a fixed target
• added a target and doavg options to quant-norm
• fix bug in --select-probes option of apt-probeset-genotype where probe IDs were not reported correctly

Version:

apt-1.8.5

Date:

Wed Oct 31 2007

• Change to the DM Implementation DM calls/confidences can be reported by apt-geno-qc and are generated and used by apt-probeset-genotype to generate seed calls for BRLMM on 500k data. Differences were observed between different platforms (particularly 64-bit vs 32-bit) in a small number of SNPs (ie 0-5 SNPs out of 250,000 on 500K Sty Chip). Code changes were made which rectify this, however you may see some some changes in DM results because of this.

• Binary Releases
  • Added 64-bit Windows Binaries (x64)

• New documentation
  • Vignettes
    • Masking Out Probes in apt-probeset-summarize
    • Use of APT to Analyze WT Based Exon and Gene Expression Arrays
  • Added new top level pages and new navigation for end user (binary release) documentation
    • Change Log
    • Vignettes
    • FAQs
    • License
    • File Formats
    • Platforms
  • Updated Manuals
    • Added new FAQ items to application specific manuals

• New Applications:
  • apt-cel-convert: a tool for converting between various CEL file formats.

• Changed Applications:
  • Most Applications
    • log messages on stderr rather than stdout
  • apt-cel-transformer:
    • support PGF/CLF and SPF in addition to CDF
  • apt-cel-to-pq:
    • support AGCC CEL files
  • apt-cel-extract:
    • added support for AGCC CEL files
    • complete rewrite of this application to use the core APT SparseMart framework. As a result this is really a new tool as far as command line arguements and behavior of the application are concerned.
    • the tool can now scale to a large number of cel files (although you will need to set the --block-size manually)
    • block number is reported (critical for assigning genotype probes to alleles)
    • you can provide an analysis string (including pm-adjustor) to manipulate probe intensities
    • you can report just the pm values, the pm and background (ie mm, gcbg, ...) values together (ie 132,22), or the result of the pm minus background
    • you can provide both a probeset list and probe list (intersection will be extracted)
    • probeset and probe lists now work with CDF files
    • the probeset type is reported (useful for multi-type CDF files like SNP6)
    • the tool will work without any library files (with reduced functionality of course)
    • enable use of experimental spf library files
  • apt-geno-qc:
    • added --chrX-probe and --chrY-probe options which enable CN probe gender calling
    • added --probe-class-file option which will turn on the CelStatListener and report probe intensity means by group
    • new column headers -- consistency with apt-probeset-genotype report.
    • new ordering of columns -- this will continue to vary over time. You should rely on column labels rather than column position to parse out specific information. Same holds true for apt-probeset-genotype report file
    • dropped --rpt/-r option, (replaced with --dm-out option)
    • Added --dm-out, --dm-thresh, and --dm-het-mult options for enabling DM calls and confidence output and setting DM options. Note that the output is now in TSV format.
    • complete refactor of the source code
  • apt-probeset-summarize:
    • Fixed a bug in MAS5 background correction routine when using pgf file. In short, the number of probe sets was set to 0 when using a pgf file.
    • quality assessment metrics and QC probeset values are now written to the AGCC CHP file headers
    • probesets which failed to be quantified (ie certain controls) are no longer written to the AGCC CHP file. In previous versions such probesets would typically show up with a 0.0 quantification value.
    • changes to how the md5sum (gc background probes and normalization probes) is computed to ensure cross platform consistency
    • syncronized meta header data between apt-probeset-genotype and apt-probeset summarize. IE, there will be some changes to the meta info going into CHP files and other output files.
    • added support for SPF files
    • fixed bug where mas50-bg chipstream failed if there were additional chip streams after it
    • added additional boundary conditions in RMA::findBandWidth in order to match behavior of BioConductor. May fix errors with certain chips with low overall intensity.
    • previous behavior was to not output mm_* quality metrics when bgrd probes were provided (via --bgp option). Now mm_* is reported if there are more than 5000 MM probes listed in the library file(s). Note that for the Human, Mouse, and Rat Exon 1.0 ST arrays, this will be dominated by the "genomic background" probes which are flagged as MM probes in the PGF file.
    • removed the path from the cel file names listed in the report file.
    • improved masking of probes (still experimental)
      • --rm-probes to --kill-list (sync w/ apt-probeset-genotype)
      • take into account probeset when masking probes -- ie you can now mask out a probe that is in more than one probeset
      • masking file can have probe_id (1 base) or x/y position. if both are present it will check to see that they are consistent.
      • when probesets and meta probesets are completely masked out, a warning will be issued rather than having the software exit with an error message.
    • added --mem-usage option to set memory use in MB as alternative to setting block-size
    • added --chip-type option. can be specified multiple times. This option will dictate the chip type in the output files and it will also specify what chip types the cel and lib files are checked against. Note that this will override the typcial check between chip types in lib files versus those in the cel files.
    • When doing CDF based analysis, chip types based on substrings ending with "." are also considered a match. IE, CDF file foo.bar.v1.r10.cdf will be detected as matching the possible chip types foo.bar.v1.r10, foo.bar.v1, foo.bar, and foo.
    • new --cc-md-chp-output option will generate experimental AGCC Multi Data CHP files.
    • added a --set-analysis-name option which will override the name computed from the analysis string. This option will affect the prefix used for output file names and the labels in certain meta fields and/or report file metrics.
  • apt-probeset-genotype:
    • fixed bug where chip-type in CHP file header was not set when using --force option
    • removed --txt-output option, which had a bug where calls were not thresholded in text output file for birdseed
    • fixed bug on solaris where use of birdseed method would cause a segfault
    • Added documentation on --write-models option of apt-probeset-genotype
    • fix reported analysis spec for all genotype calling methods. The genotype calling method (Quant Method) was not reported and in some cases incorrect algorithm parameters were reported.
    • for BRLMM-P, cluster_distance_mean metric is now correctly reported. Previously it was just set to zero.
    • fixed bug where -s option with --cc-chp-output would fail
    • added a number of additional metrics to the report file:
      • DM call rate (when DM listener is run -- ie BRLMM)
      • Various gender calls based on methods which are suitable to the chips/analysis being run
      • Various metrics underlying the gender calls
    • added a --probe-class-file option which will read in a file grouping probes into classes for use by the CelStatListener.
    • added --read-genders option to read genders from a file.
    • implement --set-gender-method option to force use of a particular gender method. Current options are:
      • "cn-probe-chrXY-ratio"
      • "dm-chrX-het-rate"
      • "em-cluster-chrX-het-contrast"
      • "user-supplied"
      • "none"
      • "supplied-genotypes-chrX-het-rate"
    • change chrXForce to --no-gender-force: ie run even if no suitable gender method is available
    • removed labelz (brlmm-p) KXY option which did nothing.
    • added birdseed-dev genotype calling method. "birdseed-v1" (alias for full analysis, and quant method name) will return the original birdseed implementation. "birdseed-dev" (alias for full analysis, and quant method name) will return the latest integrated method for birdseed from the Broad. This implementation will change over time. "birdseed" (alias for full analysis, and quant method name) will return the original birdseed-v1 implementation.
    • An option, --set-analysis-name, will be added to apt-probeset-genotype. This will allow one to set the analysis name irregardless of what analysis method is used. IE, "--set-analysis-name %birdseed" would allow birdseed-dev to masquerade as birdseed. Note that the analysis spec field would still enumerate the exact method used.
    • birdseed-dev has been updated to reflect birdseed version 2.6 from the Broad.
    • a column header for the cel file names is now present in report file output. Meta information is now included in the report file header. Additional column (call_rate) was added which matches that in the AGCC CHP file header. Also note that only the filename is now listed. The path, if present is removed.
    • changes to how the md5sum (gc background probes and normalization probes) is computed to ensure cross platform consistency
    • syncronized meta header data between apt-probeset-genotype and apt-probeset summarize. IE, there will be some changes to the meta info going into CHP files and other output files.
    • added support for SPF files
    • improved masking of probes (still experimental)
      • see details under apt-probeset-summarize
      • currently cannot remove an entire probeset -- will result in an error. use probeset list with -s option instead.
    • added --chip-type option. can be specified multiple times. This option will dictate the chip type in the output files and it will also specify what chip types the cel and lib files are checked against. Note that this will override the typcial check between chip types in lib files versus those in the cel files.
    • When doing CDF based analysis, chip types based on substrings ending with "." are also considered a match. IE, CDF file foo.bar.v1.r10.cdf will be detected as matching the possible chip types foo.bar.v1.r10, foo.bar.v1, foo.bar, and foo.
    • added a --set-analysis-name option which will override the name computed from the analysis string. This option will affect the prefix used for output file names and the labels in certain meta fields and/or report file metrics.

• Source Code
  • refactor how genders are handled. Now GenderCalls is then generated and passed around in the genotype engine.
    • refactor EM gender calling to implement ChipSummary into reporters
    • refactor DM listener to implement ChipSummary into reporters
  • refactor PgOptions class to dynamicly allocate options. (See apt-probeset-genotype for example of the new interface.)
  • integrated birdseed 2.4 from the Broad
    • QuantMethods: birdseed-v1, birdseed-dev, birdseed (which is birdseed-v1)
    • Analysis Strings: birdseed-v1, birdseed-dev, birdseed (which is birdseed-v1)
    • refactor the CN probe gender calling method as a ChipSummary into reporters.
    • change eps/epsilon alg params to use full number, not scientific notation which breaks the state propigation in analysis-spec
    • moved SpecialSnps into chipstream folder
    • fixed propigation of user set values into reported analysis spec for birdseed
    • current birdseed folder renamed birdseed-dev
    • created birdseed-v1 folder with previous birdseed implementation [GenderCaller and FitSNPGaussiansPriors3]
    • The code in birdseed-v1 moved into a C++ namespace "birdseed::v1".
    • The code in birdseed-dev moved into a C++ namespace "birdseed::dev".
    • New analysis spec alias: birdseed-v1, birdseed-dev. birdseed-dev will always point to the latest Broad recomendations. birdseed-v2 will be added when the birdseed v2 method is frozen.
    • Analysis spec "birdseed" would remain as a pointer to "birdseed-v1".
    • birdseed correction-factor was locked at 1. can now be set by user.
  • implement FileGenders for reading genotype genders -- also checks against cel file names in opts.celFiles (previous stub did not).
  • removed unused legacy code in ProbesetGenotypeEngine.cpp
  • minor clean up of some memory de-allocation in main compute method
  • complete refactor of ChipLayout and CelListeners.
  • The regression tests have been reconfigured to allow for running the regression tests outside of Affymetrix. Regression test data is now provided as a separate download from the APT source code.
  • Refactored windows builds to use static libraries for core components (ie calvin, file, chipstream).
  • Lots more regression tests
  • Support for expression Chp file checking in CalvinChpCheck.h
  • fix memory allocation in Pg::Options
  • More checks regarding GC count info when using gc-bg, pm-gcbg, and dabg
  • More use of Verbose class for log output (rather than using cout or cerr directly.
  • More use of Err class (rather than exit() or throwing exceptions)
  • Verbose::warn no default to cerr rather than cout.
  • Moved chip-type checks out of the command line wrappers and into the engine code. (ProbesetSummarize and ProbesetGenotype)
    • generally you do not want to fill in the chipType and chipTypeSupplied members of the engine options class.
    • setting chipType will force all output to be reported as that chip type.
    • setting chipTypeSupplied will force chip type checking against that list of chip types. If chipType is not set, it will be set to the first entry in chipTypeSupplied.
  • Refactored how chip summary metrics are propigated into output files. There is now a ChipSummary interface which various classes can implement. These classes can then be registered with certain reporters (ie Run Report and AGCC CHP reporters) as sources of chip summary info.

Version:

apt-1.8.0

Date:

Tue May 29 2007

• New Applications:
  • apt-geno-qc: a tool for computing single chip QC metrics for WGSA (500k, Genome Wide SNP5, and Genome Wide SNP6) genotyping chips

• Changed Applications:
  • apt-probeset-summarize:
    • Fixed overflow error in accumulator for DABG values (previous behavior affected probesets with lots of probes where the DABG p-value with go to 1 rather than 0.
    • fixed bug related to non-square CEL files where the mapping between x/y coordinates and probe IDs was incorrect
    • atom_id field in the feature response and feature residual files now contain "0". Atom IDs are no longer retained in the analysis software in order to reduce the memory requirements. As a result, the information is not available for reporting into this output file.
    • added ability to treat genotyping and copy number probesets as expression probesets. See "--explain expr".
    • added pca-select and spect-select feature selection methods (intended for use in computing gene level estimates from WT Gene and Exon arrays). See "--explain pca-select" and "--explain spect-select".
    • enable use of meta probeset files (mps) with cdf files
    • added PM+MM (pm-sum) adjustor. See "--explain pm-sum".
    • added median and mean summarization methods. See "--explain median".
  • apt-probeset-genotype:
    • fixed bug related to non-square CEL files where the mapping between x/y coordinates and probe IDs was incorrect
    • added birdseed method for genotyping calling (associated with SNP6)
    • added brlmm-p-plus method for genotyping calling (associated with SNP6)
    • added brlmm-p method for genotyping calling (associated with SNP5)

• Source Code
  • reduce memory requirements of apt-probeset-summarize and apt-probeset-genotype by using a new internal structures for chip layout information
  • added support for an experimental chip layout format (SPF)
  • new build system with multiple static libraries

Version:

apt-1.6.0

Date:

Fri Sep 29 2006

• General
  • Upgraded to Newmat 10D
  • Added preliminary support for Command Console Files (CEL and CHP) note that not all applications support CC CEL and CHP files yet. See application specific notes below for which applications now support CC files.

• Change to the PLIER algorithm
  • The PLIER source code contained in this release is replacing the separate stand alone PLIER SDK.
  • The PLIER code was changed by the addition of three new options:
    • SafetyZero: Within the PLIER code zero values are now adjusted by a slight positive bias (default 0.000001) to prevent errors during log transformation. This is particularly relevant when feature response values are rounded to zero and feed into PLIER as precomputed feature effects.
    • NumericalTolerance: Sets how hard the PLIER algorithm will work to resolve small differences between results when using precomputed vs non-precomputed feature responses. (Default is 0.1)
    • FixPrecomputed: This option causes the PLIER code to recompute the target response after fitting the feature response. This improves concordance between PLIER results generated de novo versus those generated using a precomputed feature response.
  • These changes improve the stability of PLIER estimates when comparing those generated using precomputed feature responses versus those generated using full PLIER model fit. Note that in this release of APT you may still see some significant differences in cases where the likelihood space is relatively flat combined with rounding errors caused by writing out floating point values to the text feature response file.

• Change sketch normalization default sketch size
  • Sketch normalization (ie when you request quant-norm and do not specify a sketch size) will now default to a sketch size of the 1% of the features or 50,000 - which ever is higher. If there are fewer than 50,000 features, then all the features will be used (ie full quantile normalization).

• New Applications:
  • apt-cel-extract: extract intensities from CEL files
  • apt-chp-to-txt: extract values from CHP files
  • apt-cel-to-pq: dump out mapping genotype probeset intensities as quartet
  • apt-dump-pgf: extract/dump out information in pgf and clf library files
  • apt-summary-vis: create IGB and BED visualization files from summary.txt files
  • apt-cdf-export: extract/dump out information in the cdf library files
  • apt-tsv-join: merge text files
  • apt-matrix-diff: compare two text files w/ matrix of values

• Changed Applications:
  • apt-midas:
    • Changed -c option to --cel-files for consistency with other APT apps.
    • Changed header in file grouping CEL files from cel_file to cel_files for consistency with other APT apps.
  • apt-probeset-summarize:
    • meta probeset files are now specified using a separate option, -m or --meta-probesets rather than by using the -s option as before.
    • change --chp-output option to --xda-chp-output
    • change output/input files to use probe_id rather than feature_id for improved consistency with terminology used in various library files
    • apt-probeset-summarize will now try and guess an appropriate block size given the amount of available memory to reduce the change of out of memory errors and disk swap
    • support for Comand Console (CC) CHP file output and CEL file input
    • rm-probe option for masking out probes
    • transposed report file contents to be consistent with apt-probeset-genotype
    • reports associated with each quantification method such that the use of multiple analysis parameters (-a) will result in multiple reports
    • reports are always generated -- removed --report option
    • added an avgdiff and sea methods
    • added md5sum of probes used for GC background correction
    • added md5sum of probes used for computing quantile normalization sketch
    • refactored memory allocation to improve memory use on Windows where the address space is more fragmented
    • reduced memory used by the quantile normalization (including sketch) code

• apt-probeset-genotype:
  • change --chp-output option to --xda-chp-output
  • rm-probe option for masking out probes
  • automatic blocking of probesets to improve usability on low memory machines
  • support for Comand Console (CC) CHP file output and CEL file input (CC genotype CHP file content is currently experimental and will change)
  • changed Unknown to UnknownGender
  • for xda CHP output, report median raw PM values for controls (in order to be consistent with CHP files generated by GTYPE)
  • the report file now had a column header for the cel file name, "cel_files"

• Binary Releases
  • Added Mac OS-X Universal/Fat Binaries for i386/ppc/ppc64
  • Removed dependency on .NET for windows binaries

• Source Code
  • draft file format documentation
  • refactor common code out of the application specific engines
  • refactor the QC reports to allow for probe and probeset level metrics
  • formalizing structure of options that are used by chipstream via SelfDoc::Opt class which contains a name, description, default and range values for each available option.
  • include Command Console file parsing code
  • allow for serialization of engine parameters
  • clean up code to quell gcc warnings
  • enable building fat/universal binaries on OS-X
  • cancel engines is now done through the ProgressHandlers
  • refactor general enumeration and types not specific to any one class or method

Version:

apt-1.4.0

Date:

Wed Apr 26 2006

This is the first stand alone release of the Affymetrix Power Tools (APT). Previous versions were released as part of the Exon Array Computational Tool (ExACT).

• General
  • The default APT license is now GPL. For commercial override contact devnet@affymetrix.com.

• New Applications
  • apt-cel-transformer: create new CEL files with various data transformations applied (ie normalization)
  • apt-snp-compare: simple tool for comparing matrix of SNP calls

• Changed Applications
  • apt-probeset-genotype: implements the BRLMM genotype calling algorithm for mapping arrays
  • apt-probeset-summarize: generate probeset signal estimates from expression and exon arrays
    • changed name to include apt prefix
    • change feature response and residual output (and input) files now have probeset_id, atom_id, feature_id (aka probe_id), x, and y values.
    • change default precision of output (0 places for most files, 5 digits for dabg p values)
  • Broken out main probeset-genotype and probeset-summarize into a core engine and a separate thin command line wrapper.
  • Refactor status and log reporting to facilitate GUI integration
  • Migrate windows builds to Visual Studio 2005
  • New build system for *NIX systems
  • More doxygen high level and API documentation

Version:

apt-1.2.1 (as part of ExACT 1.2.1)

Date:

Mon Jan 30 2006

Minor changes on the C++ side to support re-applying a multi-chip PLIER analysis to new chips.

• Changed Applications
  • probeset-summarize
    • added option to use pre-computed normalization sketches and precomputed feature responses
    • added option to save sketches and precomputed feature responses
    • WARNING, because of the way the PLIER algorithm alternates optimizing the signal estimate and the feature responses, using precomputed feature responses will result in a slightly different answer when the same CEL files are used.

Version:

apt-1.2.0 (as part of ExACT 1.2.0)

Date:

Mon Jan 02 2006

This was essentially the first release of APT. Most of the C++ code was experimental and thus subject to change. You may try the devnet@affymetrix.com mail address or the DevNet ExACT message forum on Affymetrix.com for unofficial support.

• New Applications
  • midas: compute the MiDAS F stat and P values for alternative splicing, the MiDAS method is described in a whitepaper available here.
  • apt-probeset-summarize: generate probeset signal estimates from expression and exon arrays
    • does normalization and signal summarization in a single step
    • includes RMA, IterPLIER, and PLIER
    • experimental

• Source Code
  • Added C++ DABG implementation. See the source code under sdk/dabg.
  • Added experimental C++ implementation of the MiDAS altsplice detection method. See the source code under sdk/midas.
  • Added experimental C++ ChipStream code. This code provides a framework to apply multiple transformations in a multichip analysis. See code under sdk/chipstream.
    • A command line program using RMA, ...) in a single go and can do so with either a CDF file or a PGF/CLF files. See code under sdk/chipstream/probeset-summarize.
  • Added a separate build system for the C++ code. The makefile will build a simple Affymetrix SDK static library and the chip stream (probeset-summarize) and midas executables. If you have CPP Unit installed, some basic unit testing will also be done.

Affymetrix Power Tools (APT) Release 1.14.4