I wonder does it take away of the cogeners that we can actually taste in the first place?
Yes, the difference is very tangible and noticeable.
To novices, it's not always so noticeable in the taste, but it's clearly identifiable in the mouthfeel. The fats and congeners that are removed by chillfiltering are actually oils.
So NCF whiskies typically have a much oiler, juicer mouthfeel, and usually a stronger flavour too.
I demonstrate this at every tasting I host. If you pour water very slowly and carefully down the side of your glass, you can actually get the water to "lift" and separate the oils from the whisky. Dip your finger into the glass so that it doesn't penetrate the oily layer. Then remove and rub your fingers together and feel the slippery, greasy, oily texture. That's the stuff that contributes to the mouthfeel and contains a lot of flavour.
Now imagine that your dram had all those oils and flavours removed!
The vast majority of whisky I drink these days is cask strength and NCF. I'm afraid that when I taste a chill-filtered whisky, it usually fails to impress me, because I'm just used to more flavour and a better mouthfeel.