Some CPUs, such as the original AMD Athlon and Intel Pentium II were manufactured with L2 cache on a circuit board instead of physically on the CPU itself. This was done because the manufacturing process at the time couldn't produce CPUs with on die L2 cache in high enough quantities to be profitable. This off-die L2 cache typically ran at a fraction of the core clock speed of the CPU, which resulted in a loss of performance due to higher latency and lower bandwidth.
Before the Athlon and Pentium II, processors had no local L2 cache at all, and instead relied on cache on the motherboard that was accessed over the frontside bus. The Intel 486, Pentium, AMD K5 and K6 are examples of frontside bus based L2 cache architectures.
Over time, as manufacturing processes have advanced, the cache has moved onto the CPU package, and then onto the die itself. There have also been hybrid designs with a seperate L2 cache die within the same physical package (called an MCM), such as the Intel Pentium Pro, which was extremely expensive to produce as it could not be fault tested until it was assembled. A defective cache or CPU meant throwing both away. The Pentium II was largely based on the Pentium Pro, but moved the L2 cache off of the CPU package and onto a circuit board. This way, faulty cache could be replaced without destroying the processor.
Off-die cache still exists in some extremely high end hardware, such as the IBM POWER5, which has an L3 cache of 36MB.