This is one of those topics that can fill a book, so let me try to break this down to some simple ideas. Consider an oscillator that you build to produce one single frequency; perhaps a crystal oscillator, perhaps LC; the technology doesn’t matter. Ideally, the oscillator would produce just that one frequency you’ve designed it to produce. It would output a voltage, V(t) = A * sin(2*pi*f*t) where A is the amplitude, f is the frequency, t the time the voltage occurs.
As I harp on all the time, nature is a bitch and the reality we get is not that ideal value. What we get is something like V(t) = [A+E(t)] * sin(2*pi*f*t+p(t)). That’s a fancy way of saying that in addition to the desired output we get fluctuations in amplitude E(t) in time, and fluctuations in phase p(t). As a general rule, the effects of the phase fluctuations dominate the amplitude noise. Perhaps you’ve seen a signal on an oscilloscope where you can see the signal width (time) varying slightly, perhaps an edge of the square wave is jumping around slightly on the screen. That’s called jitter and jitter is just another way of describing phase noise (you can convert jitter in time to phase noise and vice versa). If this signal is listened to on a good receiver, or observed on a spectrum analyzer, it goes from being the ideal situation in which all the power appears on one frequency to a signal with a noise around it that drops off as the frequency offset increases. This is called a noise pedestal. The higher the phase noise from the oscillator, the higher that pedestal is.
This is frequency in the WiFi band, and shows noise in a 10 MHz span. I’ll get back to some descriptions of what you’re seeing after some additional points.
One of the most important things to know about noise is that the way we see it depends on how we look at it. (Hmm… almost sounds like a zen saying) By that I mean that if it’s truly random noise, the power goes up as the measurement bandwidth goes up. Because of that relationship and the ease of converting the effects in different bandwidths, noise is usually specified as a power in decibels below the desired signal in a 1 Hz bandwidth, or dBc/Hz. To convert the levels seen, simply use 10*log(bandwidth ratio). If we double the bandwidth, the noise power is 3 dB higher; if we use 100 times the bandwidth, the noise power is 20 dB higher and so on. This allows easy, direct calculation of the effects of the noise on our system.
In that scope picture, it says along the bottom left that the “resolution bandwidth” (RBW) is 100 kHz; 10*log(100,000) is 50, so the noise in 1 Hz is 50 dB lower or -50dBc. Look at the point 2 MHz above the signal that I circled. That noise is about 50 dB below the desired signal, so -50dBc -50 dB says it’s -100dBc/Hz. By the way, if you’re intending to measure noise, it’s good practice to make the video bandwidth (VBW) 1/10 of the RBW all the way down to 1/100 of the RBW. This makes for long sweep times, but video averages out much of the variation in the noise level.
What difference does this make? First off, let’s look at the transmitter side. Say you’re operating ham radio field day and you’re at a club station that wants one station on CW and another on phone on some band at all times. Consider 20 meters, and say the CW guy is at 14.025 MHz while the phone guys are 14.225. A good transmitter could have the phase noise down 145 dB 200 kHz away, right on the CW guy’s frequency. You’re putting out 100W, which is +50 dBm. Noise 145 below that is -95 dBm. In the 500 Hz bandwidth the CW guy is using that adds back up to -68 dBm. Sure that’s a tiny fraction of a billionth of a watt (1nW = -60 dBm), but it’s a very big signal to a receiver. It’s 89 microvolts in 50 ohms when the radio can copy under 1 microvolt easily – less than ¼ of a microvolt.
In reality, that -68 dBm only exists at the transmitter output and what the receiver hears depends on how much loss there is between the transmit and receive antenna. The only thing that can help here is having their antennas as far apart as possible to add path loss that would lower the noise.
Phase noise is also a problem for receivers, but in other ways than just hearing another transmitter. The main issue is that phase noise on the receiver local oscillator can mix with undesired signals out of band and translate noise or spurious onto the desired channel as interfering signals. For reasons I’ve never quite grasped, this is referred to as “reciprocal mixing” – it seems like straight up, normal mixing to me. In effect, your receiver’s local oscillator becomes self-jamming.
This drawing depicts the LO with noise dropping off in the adjacent channel, mixing noise into desired IF. In ham radio, where most services are not channelized, the offset can be tiny. They’re attempting to show that the noise mixed onto the undesired signal is degrading the signal to noise ratio of the desired signal. If there are spurious signals on the LO, they'll mix into the channel, too. If the desired signal is weaker than pictured, you can imagine it completely under the noise pedestal mixed on top of it.
Exactly how to design a low phase noise local oscillator is so far beyond the scope of what I can do here that I can barely address it. In a PLL, the phase noise of the voltage controlled oscillator (VCO) affects the noise more farther from the carrier, and a VCO with bigger components, higher Q, will be lower noise (rule of engineering: Q comes by the cubic yard). The amount of division in the frequency synthesizer affects the noise closer to the carrier than the VCO. Direct digital synthesizers are typically lower phase noise than a synthesizer, but much richer in spurious output signals. Those spurs also mix in to the IF by reciprocal mixing.
The newer, higher-end ham radios on the market have eliminated the receiver LO by doing RF direct sampling; they bandpass filter the signals being received and then convert them to digital without mixing in the analog world. Likewise, the transmit LO can be produced by a Direct Digital Synthesizer and the modulation performed in digital signal processing, before converting the modulated signals to RF to amplify and transmit.