From the Unreal Tech Page:
A detail texture is a very small, fine pattern which is faded in as you approach a surface, for example wood grain, or imperfections in stone. When you import a detail texture, you need to set its Scale value to a small number (typically 0.25nm) to specify its scaling relative to whatever texture it applies to.
Detail textures and macrotextures modulate (multiply) the surface they're applied to. By modulating, they have the ability to scale the surface's brightness up or down. Colors with RGB brightness values from 0-127 darken the surface; 128 has no effect; and 129-255 brighten the surface. Therefore, when drawing microtextures and macrotextures, it is important that you design them with their brightnesses in the proper range so they affect surfaces naturally. If a detail texture's average brightness deviates too far from 128, then surfaces will appear to brighten or darken as you approach them.
See color blending for more on modulation.
Setting detail textures
Setting detail textures for a new texture is really simple. After importing the texture into the texture browser right click on it and goto Properties -> Texture -> Detail texture (just use the appropriate tex from detail.utx) . See also Texture Properties.
No other setting really except to turn on detail texture for d3d. At the console (~) type Preferences goto rendering -> d3d -> detail texture set this to true.
Comment by Luggage: I noticed that the above doesn't always apply when I was messing around with detail textures for my own pack... very bright textures are darkened by a 128 brightness detail texture, while dark textures are brightened up a bit. I then converted the original texture to greyscale to determine the average brightness, and adapted my detail textures to that. Worked fine for me, although I don't know if it's really true.