Q: How should I set the compressor for vocals in Nuendo?
The principles of setting the compressor are the same in all DAWs, and hardware compressors too. Whether you use Nuendo, Cubase, Logic, Pro Tools, Sonar or any other recording system, you will make the same decisions when you set the compressor.
There are two reasons why you might want to compress. One is to keep the vocal at a steady level so that it is easier to mix with the instrumental backing track. If it continually rises and falls in level, then sometimes it will be overly prominent, at other times it will drop back below the level of the instruments.
The other reason to compress is because you want to achieve the compressed sound, regardless of variations in level.
Either way, it is good to have a starting point. So do this...
Set the Ratio control to 3:1
Set the Attack control to 1 millisecond and the Release control to 100 milliseconds.
If you have a Knee control, set it to its minimum value ('hard knee').
Set the Gain control to +6 dB.
Now that you have the basic settings in place, play the track and experiment with the Threshold control.
Find a setting that sounds good to you. There are no rights or wrongs and you do have to make your own decision rather than follow rules that might not apply to your particular circumstances.
Look carefully at the Gain Reduction meter. It should be dancing up and down. If it isn't then your threshold is too low, or your attack and release controls are not set correctly.
Notice how a lower threshold gives you more gain reduction; a higher threshold gives you less.
Now experiment with the Ratio control. Once again, find a sound that you like.
When you are happy, experiment further with the Release control. You will find that this acts subjectively as a 'More' or 'Less' control.
The setting of the Gain control isn't important, although some engineers like to set it so that the input and output signals are approximately the same level. This makes comparison between uncompressed and compressed easier, so that you know you are making improvements.
Once you have the hang of the controls, experiment, experiment and experiment. The only true path to mastery of compression is to learn through as much experience as you can possibly get.