needle tools changes

BMERC : needle tools : Appendices : needle tools changes

For a detailed chronology of changes, see the CHANGES.text file that came with the needle tools distribution. Most bug fixes are mentioned in the bug list for the affected program. This page only summarizes other user-visible changes.

Table of contents

1. needle tools changes
   1. Table of contents
   2. Changes from Release 1.5 to Release 1.6
   3. Changes from Release 1.4 to Release 1.5
   4. Changes from Release 1.3 to Release 1.4
   5. Changes from Release 1.2 to Release 1.3
   6. Changes from Release 1.1 to Release 1.2
   7. Changes from Release 1.0 to Release 1.1
   8. Changes in Release 1.0

1. make-domain-core.pl and pdb-domain-seq.pl now use the "new" SCOP domain indexing scheme: PDB sequence/insertion code instead of sequential indices that SCOP used to use. This is a major incompatible change, but since we've been using the new SCOP databases for roughly a year now, this is also an important bug fix.

Other changes in Release 1.6:

1. filter-pdb-atoms.pl and efa.pl and now support full chain subrange syntax, selecting the specified subsets of residues from a PDB entry. Plus a minor tweak to fix filter-pdb-atoms.pl chain continuity checking.
2. Fixed some obscure barrel-related secondary structure designation assignment bugs in guess-ss-designations.lisp (internal to make-ss-designations).
3. Made diff-fields.pl check the return codes from cut invocations.
4. Fixed install.pl so it runs on Linux, added a -quiet option for consistency, and put the destination directory at the start of @INC for perl scripts installed with the -inc switch.

Incompatible changes in Release 1.5:

1. Removed scripts/old-make-core-depends.pl, scripts/old-make-mrf-depends.pl, scripts/make-lib.pm, and scripts/mrf-make-lib.pm from the distribution; these are now officially obsolete.

Other changes in Release 1.5:

1. Added a number of tools for manipulating alignments for use in grading needle threading results. These are the fssp-core-corr.pl, correct-alignment.pl, align.pl, pima-multi-align.pl, pima2fssp.pl, and make-align-depends.pl scripts. See also the Threading alignment experiments section.
2. dssp-ss-states.pl is now much more robust about incomplete DSSP sequences or misplaced gaps, because it now uses global alignment to compare the DSSP sequence to the "correct" sequence, and outputs secondary structure states only for residues in the given sequence.
3. Fixed two bugs in make-ss-designations in dealing with semi-pathological cases, one where it turned helices into strands if they were H-bonded to strand residues, and another where it died on strands H-bonded to themselves.
4. Made add-pdb-radii.pl and efa.pl (and other abbreviated-DSSP-reading programs) more robust about detecting bogus multi-model abreviated DSSP input files.
5. Added Yahoo-style navigation bars to all HTML documentation pages, split the detailed table of contents into its own page, and reorganized the needle tools home page accordingly.

Incompatible changes in Release 1.4:

1. There are no incompatible changes in this release.

Other changes in Release 1.4:

1. Added the -model and -no-missing-atoms arguments to the filter-pdb-atoms.pl script, made it ignore deuterium atoms as well as hydrogens, and suppressed all "Undefined AA" messages unconditionally for nucleotides.
2. Fixed a variable name in hydro2 (internal to efa.pl) and a subscripting error in the make-ss-designations script, both of which broke under Solaris.
3. Fixed two bugs in mrf-scores, one whereby it insisted on always producing at least 10 singleton environments, even if fewer counts were defined, and a separate poisson bug in the residue-dependent exposure code (an undocumented feature).
4. Added the (undocumented) aa-thresholds-to-envs.pl script to the distribution.
5. Reinstated -no-xval support in the make-mrf-depends.pl script.

In addition to the code changes outlined below, a number of the documentation pages were split up in order to make them load faster. This was done by refining the program categories under "needle tools program descriptions" from eight categories in Release 1.2 to the present 12 categories.

Incompatible changes in Release 1.3:

1. There are believed to be no incompatible changes in this release.

Other changes in Release 1.3:

1. filter-pdb-atoms.pl now supports the -include-hetatm and -exclude-hetatm args for filtering HETATM records, and the -include-unk argument for including UNK residues in ATOM records.
2. Also added an -include-unk arg to the pdb-to-seq.pl script, in addition to bug fixes.
3. Added a -chain arg to efa.pl.
4. The dependency file generators now include a "backtrace" of target files being sought/constructed in warnings and verbose information messages.
5. dssp-ss-states.pl now takes input on stdin by default, and checks for dssp4.pl errors.
6. Most scripts that accept a chain identifier argument now consider a chain ID of "_" (underscore) to be a synonym for " ".
7. make-ss-designations now attempts to use Allegro Common Lisp if it can't find cmu-cl, and dies if it can't find either.
8. install.pl no longer depends on the existence of a native "install" program.
9. Minor improvements to dssp4.pl -verbose messages.
10. Introduced the compare-nexps.pl script (not documented).

1. Most significantly, Release 1.2 incorporates a new "rule-based" implementations of make-core-depends.pl and make-mrf-depends.pl; the "Dependency files for make" section has been greatly revised to accomodate this. For the most part, these programs are compatible with previous releases, with the principal exception that the new versions do not support the -no-xval arguments yet. For this reason. the original implementations are still available as old-make-core-depends.pl and old-make-mrf-depends.pl (though they are not installed by default). Also, make-core-depends.pl now makes sequence files as well as core files by default.

Other changes in Release 1.2:

1. Added the pdb-domain-seq.pl and make-domain-core.pl programs to the repertoire. In conjunction with the new makefile generators mentioned above, these programs allow one to make cores from one or more chains or portions of chains. See the description of the new core definition file format (an upward-compatible extension to the old core list and naming convention) for details.

1. Added two enhanced versions of existing programs: efa.pl is an enhanced version of the generate-exposure script, and pdb-to-seq.pl program is an enhanced version of make-seq-file.pl. generate-exposure and make-seq-file.pl are both still in Release 1.1 (and up to 1.3 so far), but they may be made obsolete in the future.
2. Added four "BLAST" interface scripts; see the "BLAST programs" page for details.
3. core-ss-states is another new program that produces a description of secondary structure based on an existing core file. And the new fa2tbl.pl script does the obvious conversion.
4. filter-pdb-atoms.pl now ignores Q atoms representing hydrogens (e.g. in 1acp; see remark 10 there).
5. make-core.pl has an -hcb-all option, and warns if too few segments (default is 4).
6. dssp4.pl has a new -turns-are-loops option.
7. A major bug was fixed: make-mrf-depends.pl xval singleton cores were not included.
8. mrf-scores now has -loop=1 and -loop=2 arguments to control explicitly the number of loop environments. [We hope to generalized this soon. -- rgr, 2-Jul-98.]
9. A number of bugs and other compatibility problems on the Alphas were fixed.

1. mrf-scores now takes a -min-pair-count parameter; the default is zero. You must now supply "-min-pair-count 4" to get the original behavior.
2. make-mrf-depends.pl now uses different macros to invoke programs. In particular, make-mrf-depends.pl no longer forces the -poisson arg to mrf-scores; this must be supplied via the macro.
3. Similarly, make-core-dependencies.pl is now called make-core-depends.pl, and it also uses macros to invoke programs. Furthermore, the make-dssp-segs script has been replaced with an explicit pipe between the secondary structure generation program (usually dssp4.pl) and the segment labelling program (usually make-ss-designations) when building secondary structure definition (".dssp") files. This allows arguments to be passed to dssp4.pl. The make-dssp-segs script is now obsolescent on that account.

In any event, if you use makefiles, you will need to edit your makefiles and remake dependencies (see the example makefile for hints). If you use scripts, you will need to change the arguments to the mrf-scores program.

Other changes in Release 1.0:

1. New sing-envs.pl script, with much more flexible exposure definition.
2. init-needle-tools allows needle tools to be installed on multiple incompatible systems at a site; see the "Multiple system installation" section for details.
3. make-ss-designations is now smarter about sheets and barrels, and is documented (but not very well).
4. make-mrf-depends.pl now accepts the -xval-sets argument, which can be used to implement a variety of cross-validation strategies; see the "Types of cross-validation" section for details. GMT environments are now always made as a separate target. Non-cross-validated scores can be made on request. If the -warn-if-no-hlg-file argument is given, make-mrf-depends.pl will warn if no homolog alignment file is found for the -alignment-file argument to mrf-counts.
5. Both make-mrf-depends.pl and make-core-depends.pl name the loaded perl libraries in the makefile header, pretty-print the command lines, and support parameters to change all prefixes and suffixes (see the "File naming options" section).
6. mrf-envs now only requires the core file in order to generate a GMT environment file.
7. mrf-counts no longer requires that both the -sing-env-file and -pair-env-file parameters be specified. If either or both is omitted, those counts will not appear in the output.
8. Added the -symmetric-pair-envs argument to mrf-counts and mrf-scores (see the mrf-counts discussion of pair counting for details). [this was actually in Release 0.1. -- rgr, 10-Dec-97.]
9. mrf-scores has a new -gmt-marginal-file argument allows GMT marginals to be specified in great detail; see the "GMT marginal sets" section for specifics. mrf-scores now always warns about core positions with no pairwise edges when computing GMT scores. (Before you had to say -verbose to find this out.) Consecutive score-writing options are now collected together and executed in the order (singleton, loop, pair) for efficiency. The -write-pair-sing argument has been split into -write-pair and -write-gmt args; -write-pair-sing is still accepted for backward compatibility.
10. dssp4.pl now has -locus, -min-helix-length, -dont-fill-e-gaps, -dont-fill-h-gaps, and -min-strand-length args, and accepts PDB-format input from the standard input (by omitting the PDB file name). Specifying -verbose causes warnings to be generated for each change of secondary structure assignment as a consequence of smoothing.
11. The tbl2fa.pl, diff-fields.pl, and dssp-ss-states.pl scripts have been added to the repertoire. install.pl is an old script that is now documented.
12. make-seq-file.pl now takes a -format argument, and is capable of producing FA, IG, TBL, or "just-seq" (raw sequence) format output.
13. filter-pdb-atoms.pl now handles PDB MODEL and ENDMDL records by arbitrarily taking the first model, and detects and reports on missing sidechain atoms (but only those that is has been requested to output).