Molecular replacement
Molecular replacement serves to position one or more known structures (models) in the unit cell of the unknown crystal structure with the goal of providing preliminary phase information.
For the positioning of a single molecule in the asymmetric unit (ASU), there are 6 degrees of freedom: 3 d.o.f. for rotation (around e.g. the x, y, and z axes), and 3 d.o.f. for translation along (e.g.) x,y,z.
A 6-dimensional search is computationally very demanding if it is performed on a suitable grid. As an example, rotating on a 3-dimensional grid with 10° spacing means almost 14000 unique possibilities (using Lattman angles); a more exhaustive 5° search would mean 8 times as many possibilities. Translating on a 3-dimensional grid with 1 Å spacing in a 100 * 100 * 100 Å^3 unit cell means 1.000.000 translations. Doing both (a translation search for each rotation, or vice versa) then would lead to a combinatorial explosion of the number of possibilites that have to be evaluated.
It was recognized by Rossmann and Blow (around 1962) (FIXME: give correct reference) that a Patterson function can conceptually be decomposed into those vectors arising from intra-molecular vectors, and those from inter-molecular vectors. The intra-molecular vectors would change their orientation (but not their length) when the molecule of the ASU would rotate. For a given rotation of that molecule, only the inter-molecular vectors would change upon translation of the molecule through the ASU. It is thus clear that it is advantageous to first determine the rotation of the molecule, by restricting the comparison between model patterson and crystal patterson to the intra-molecular vectors. When the correct rotation is found, all vectors may then be used to find the correct translation.
Self-rotation function
A self-rotation function may be calculated to find the relative rotation between two molecules in the asymmetric unit. It is often calculated in polar coordinates (theta, phi, kappa), because these separate the amount of rotation (kappa) from the axis around which the rotation takes place (theta, phi) (called omega, phi in the polarrfn documentation).
For visualization often a stereographic projection is used.
A self-rotation function may be calculated and visualized with the help of molrep or polarrfn.
Details can be found at [1] (and the following section 2.9) and at [2] (see section 5.2).