colour blind

i was wondering, if every one saw colours differently, like my blue was your yellow and your pink was my green, would anyone ever notice.
if we wanted to test it, lets say i showed you a blue colour pencil. To me it looks blue, to you it looks blue too, except you blue is my purple, so you see it purple. but since from young you've called purple blue, you'd say thanks for the blue pencil Miriam! and i wouldn't suspect a thing.